The 512K route issue


I was first made aware of an issue when the hosting provider where I host this blog at were tweeting apologies on 12/08/14 for an interrupted service and I later received an excellently worded apology and explanation from them. A couple of colleagues also got in touch later that evening with reports from further afield.

The facts in no particular order

  1. Essentially, routers and switches either make the decision to forward packets in hardware using a special type of very fast memory called TCAM or more software based, using cheaper and somewhat slower RAM. The advantage of TCAM is its speed and its ability to provide an output with a single CPU cycle but it is costly and also a finite resource. RAM on the other hand is slower, but you can usually throw more of it at a problem. Depending on which model of router/switch you have depends on which forwarding method is used
  2. The number of IPv4 routes on the Internet has been growing steadily and increasingly since its creation. Back in early May 2014, this global routing table hit 500K routes
  3. The devices that use TCAM are not only restricted by the finite size available to it, but this TCAM is used for other things besides IPv4 routing information, e.g. access lists (ACLs), QoS policy information, IPv6 routes, MPLS information, multicast routes. So in effect, TCAM is partitioned according to the use the device is being put to. Cisco’s 6500 and 7600 switch and router platforms (respectively) have a default setting for each of these. On many of the devices, the limit for IPv4 routes is set to 512K
  4. Verizon have a big block of IP addresses that they advertise as an aggregated prefix
  5. On Tuesday, for some reason, Verizon started advertising a large amount of subnets within their block as /24 networks instead, to the tune of several thousand, causing the global routing table to exceed the 512K limit on those devices configured as such
  6. This had the impact that those affected devices did not have enough TCAM to hold the full Internet routing table and so the prefixes that didn’t make it in to the table would not be reachable. As prefixes come up and down on the Internet all the time, these routes would have been random in nature throughout the issue i.e. it would not have just been the Verizon routes affected

Are you affected?

If you have Cisco 6500 or 7600 devices running full BGP tables, you need to run the following command:

If the IPv4 line of output is 512k or lower, you are in a pickle and will need to change the settings by entering the command below:

Where the 1000 is the number of 1K entries i.e. the setting as shown in the first output would be 512. Typing a ‘?’ instead of the number will return the maximum available on your platform, so you could in theory be requiring a hardware refresh to add to your woes.

If you have an ASR9K, follow the instructions here to get to your happy place:

Most other router platforms use RAM and so the more you have, the more routes it can handle. The performance varies widely from platform to platform. You should check the vendor’s documentation for specifics e.g. the Cisco ASR1002-X will do 500,000 IPv4 routes with 4GB of RAM and 1,000,000 with 8GB RAM

Who is to blame?

There is an ongoing debate at the moment about whether Cisco are liable or the service providers. I would argue that it is predominantly the latter but Cisco could have done a better job of advising their customers. Cisco did post an announcement about this on their website a number of months ago, but I didn’t spot it so I’m assuming many other customers didn’t also.


Having said that, if you buy a bit of kit to do something, you need to take some responsibility for failing to include capacity planning in to your operational strategy.

Till the next time. (#768K!)

Wireshark 2 preview


I recently updated my Wireshark installation to version 1.12.0 and during my normal happy-clicky install process, noticed one of the options to install was something called ‘Wireshark 2 Preview’. Intrigued, I carried on clicking, making sure there were no further boxes wanting to install the Ask Toolbar. (Die Java, just die!)

Wireshark 2 preview

As those of you who use Wireshark regularly will probably know, the developers announced a big change that was on its way back with the release of 1.11.0 in October 2013, that change being they were switching the user interface library from GTK+ to Qt. I believe this decision was arrived at to provide a more standardised feel for the app across multiple platforms. Also, support for GTK+ was waning.

First thoughts

When you install version 1.12.0, you will also get the option to add start menu and desktop icons for the version 2 preview. Upon opening the preview, the word that immediately sprung to mind was ‘clean’. It’s much less cluttered than the current version. In fact, it takes  a little getting used to, but that’s change for you.

I encourage you to go and try it out for yourself but a couple of things that I have noticed from playing with it that I like are:

  • The interface selection screen shows a mini utilisation graph so you can see at a glance which interfaces have traffic going over them. Useful if you have many NICs on your machine e.g. VMware installed
  • The IO graphs seem to be better scaled without any tinkering, plus have guidelines that make reading graphs easier. As these are exportable also, it makes reporting look prettier


Overall, I like the new version. As expected, there are a couple of bugs I’ve found that I’ll be feeding back to Gerald and his gang, but this definitely feels like a step in the right direction.

Till the next time.

Technical rewind


I was recently thinking about the future of this blog and had been considering whether to bin it or come back to it with renewed enthusiasm. After all, there are thousands of other blogs out there that cover similar topics, ranging in quality from barely readable to excellent. Whilst I hope that mine falls no further down than the middle of that scale, I asked myself what value do people get from my own posts.

When I logged on to the admin portal for the first time in a while, I  saw two key things that made me realise that I should continue writing, perhaps not as frequently as some other bloggers, but with more posts that are close to my heart and hopefully that will shine through in my writing. The first was that, whilst my viewing figures are not particularly spectacular, they have been constant throughout my recent absence so people are still coming over, both to check out what is on offer and also from search results. The second thing I noticed was that there were almost a dozen updates for WordPress itself, the theme and some plugins and I found myself feeling quite protective and applied the relevant TLC.

Technical rewind

I’ve worked in IT for well over 10 years, achieved my CCNA back in 2009 and my CCNP in about 2012. I got past the half way point towards my CCNP Security and then something dawned on me. Something that made me down my certification tools and take a long look at myself. My appointment to a management role in the last year has only cemented my thinking.

The quest I was on to further my knowledge according to Cisco’s road map in addition to my new, less hands on role had left my foundational routing and switching knowledge less polished than I would have liked. I still function as a good network engineer, but I get a certain satisfaction from having nuts and bolts knowledge at my fingertips and I’ve been aware that this has slipped since the new year.

Regarding the certification path, the blueprints for most of the exams never match the on the job knowledge requirements. So in a busy world, you spend huge amounts of time learning about things that Cisco want you to learn, but your boss isn’t bothered about and quite often, nor should you be. They are just not relevant for the day to day or even tomorrow.

With that in mind and with the time that I am currently able to commit to studying, I am going to aim for the CCIE R&S Written as a way of refreshing my current certs but more importantly, I will deep dive in to all the relevant topics to give that much needed polish. Those studies will hopefully provide me with some good topics on which to blog too.


As I recently tweeted, I find that knowledge is a foundation to build upon rather than a skip to fill up. Being self aware of when that knowledge needs some maintenance is a key skill for any engineer to prevent it all falling down about them. Do your core skills need brushing up on?

Till the next time.

Back in the game

Remember me? It’s been a while since I last posted. Coincidentally, it was a few days before I stepped in to the Acting Head of Networks role at my current employer whilst the guy in that role was temporarily unavailable.

I’d felt in the months up to that point that I’d started to lose focus on my career. I’d lost the drive to study as much, I was enjoying my day to day role less and less. Then it all changed on the Monday I came in to the office in my temporary role. There were deadlines to hit, projects to complete and bridges to build. With little time to waste on trivial matters, I sat the team down and we discussed what steps lay ahead of us.

I was very pleased, and no less proud, to say that we all came together as a team and hit our targets. At the start of the New Year, my team leader decided to move on to pastures new permanently and so I was offered the role permanently which I accepted.

Whilst the responsibilities of the new role means that I need to do far less support work (save for when the crap hits the fan, when I can’t help myself), it does mean I get to spend more time reviewing the company strategy and looking at solutions that can deliver it. I can now start looking at ways to be more innovative, increase productivity and cut deployment/troubleshooting times as much as possible. I’m still a couple of exams away from my CCNP Security and to be honest, I’m in two minds as to whether I should complete it or focus my attention elsewhere.

So there you have it. Still working at a great company but in a role with the right people that makes me much happier. Focus has returned. There are a few things I want to carry out in the next couple of months or so to tidy things up and then its innovation central.

Till the next time. is moving home

I am currently in the process of moving my domain to a new provider. Obviously, I am hoping that this will all go exactly to plan without any downtime, but please bear with me should the site go down at all.

Till the next time.


Site has now been migrated with only a couple of minor hiccups. If you find any issues with the way the site displays, please contact me at and I’ll get straight on it.

The ten commandments of networking


Earlier this year, I posted a quasi-zen tweet (@vegaskid1973) in jest which seemed to tickle the fancy of a few of you based on replies and retweets and so I thought I’d use the rare free ten minutes I find myself with to flesh out the idea, which I light-heartedly present here as the ten commandments of networking:

  1. You shall not trust a Visio diagram, lest you bring the customer site down
  2. You shall not covet a colleague’s serial cable. Get your own and hands off mine!
  3. You will backup and protect your configs like they were your first-born
  4. You shall not bear false witness against a network incident. Unless explaining it to management
  5. You shall have no other gods, but feel free to revere unicorns
  6. You shall not murder a good TCP joke unless you are sure they will get it
  7. You shall write the customer SLA/contract in such general terms so as never to breach it
  8. Remember the 7th layer of the OSI model. On it you shall not do any work, leave that to the devs
  9. You shall not commit config until you are confident your CV/resume is up to date
  10. Whilst you may be vendor agnostic, you must believe in intelligent design


Please add your own suggestions in the comments below.

Till the next time.

Are you a lion or a gazelle?


There is an old fable that has been attributed to various sources, which I’m not concerned about verifying but it goes something like this:

Every morning in Africa, a gazelle wakes up knowing it must outrun the fastest lion, or it will be killed and eaten.  Every morning a lion wakes up knowing it must outrun the slowest gazelle, or it will starve  to death.  It does not matter if you are a lion or a gazelle…when the sun comes up each morning, you’d better be running.

Face value

The message here is clear. To survive, you have to keep moving, else become extinct. This is so applicable to the world of IT. Things change so quickly. Of course dinosaurs in IT do exist but in today’s climate more than ever, they are struggling to avoid being relegated to irrelevancy.

Reading between the lions (sic)

In my opinion, the fable offers far more value if you ask yourself whether you would rather be a lion or a gazelle, figuratively speaking, from the point of view of an IT professional and the information explosion we face on most days.  How best to deal with it?

Would you rather be a gazelle, trying to be ahead of the curve, having to keep up with every new technology, every vendor’s new product release, every new protocol, read every blog post, twitter feed, RFC, book, listen to every podcast, lab every scenario, attend every event, etc., fearful that you may be gobbled up if you stop?

Or would you rather be a lion and filter out the noise, focus on what is relevant, feast on the juiciest knowledge, that which will sustain you, make you stronger and still give you time to spend with your pride, comfortable in the knowledge that you are at the upper end of the food chain?


The art of survival is not just about making it through the day. It’s about focussing your efforts in the right place at the right time so you can keep enough energy for the other important things in your life. Be sure to refocus on whatever you are currently doing. It’s less about what you can achieve on a day to day basis but rather what you can sustain throughout your career and life.

Till the next time.

10 tenets of working in IT – Tenet 7, Honesty


This isn’t a post about stealing your colleague’s lunch from the fridge in the kitchen. You will also be disappointed if you came here for advice on what to do about people who park in disabled parking spaces without a permit. Rather, it discusses being honest with yourself and with people you have real, direct interactions with.


There are many related words/phrases I could have chosen to base a blog post on in lieu of honesty. Courtesy, integrity, morality, etc. They are all worthy attributes but I somehow feel that honesty encompasses all of them. Rather than get into a  deep philosophical discussion on truth and the ways of the world, I’d prefer to use some simple bullet points from the original 10 tenets post to keep things simple:

  • Be honest with yourself in the first instance
  • Only then can you be honest with colleagues, customers, friends and family
  • Know when to put your hands up and say “I don’t know”
  • Don’t bury things when you get something wrong, get it out in the open
  • Know when it is time to change job
  • Know when it is time to change career
  • Ask for the same level of honesty from the people you deal with (this needs to be addressed differently depending on who we are talking about!)
  • Ask for feedback about yourself from those people you deal with
  • Make sure you get your yearly appraisal. This is the ideal opportunity for you and your line manager to align your goals with that of the company


In an era when people are all too keen to splash details of their personal life online, discussing what they’ve had for dinner, who they were out with the night before and what they think of their boss, many people are still unable to be as honest with themselves or with people when face to face and not hiding behind ‘the net’.

It is human nature for people to build walls to hide behind and sadly, the first casualty is often truth. I’ve found that my career has taken the biggest leaps forward when I’ve been honest with both myself and those around me.

Try being more honest with yourself and with the people you deal with. You may find it  very liberating.

Till the next time.

Poll #1: what blog topics would you like to see more of on

Over the last few months, my blog hits have steadily increased so I must have been doing something right. Blogging is a rewarding task beyond reader numbers though as anybody who blogs themselves will hopefully know. Despite the fact that I have a lengthy list of blog post ideas, some already in draft form, I thought it would be an interesting exercise to ask my readers what sort of topics they would like to see here.

With that in mind, I would really appreciate a few moments of your time. Please select up to three topics below that you would like to see more posts on here at

Poll results

After a few weeks, the poll has finally closed. It seems that most people wanted a combination of lifestyle\career\studying type posts and R&S\Security. Nobody was interested in vendor posts and only three votes were cast for SDN\DC topics.

Thank you again for your time.

SDN: a vendor’s dream, simplicity’s nightmare


You may have heard some rumblings over the last couple of years regarding Software Defined Networking, known better to save you time, as SDN. I’ve listened to countless podcasts, read dozens of blog posts and scoured the Internet trying to make sense of it in that time period and the conclusion I’ve come to is this:

Currently, there isn’t enough joint momentum and focus in the industry around SDN to make this something I need to care about. (12/09/13)

Note that I’ve dated my quote and with a very good reason i.e. I am hopeful that my opinion will change and sooner rather than later.

I liken the recent explosion around SDN to a volcanic eruption. It started off with some gentle rumblings beneath our feet. There were then a series of exciting tremors. Right now, there is a lot of hot air, dangerous fallout and toxic gases, some spectacular fireworks with several ‘oooo’ and ‘ahhh’ moments but it will be some time before things cool down and I’ll want to commit to walking out amongst it.  At that time, I’ll almost certainly blog again about how SDN helps me and my organisation.

In the beginning

When I first started learning about SDN, it was primarily about separating the control plane from the data plane. Centralised, policy based networking. One place to tell your nice GUI what you wanted and a controller that pushed that desire out to your estate. No more logging on to 100 devices and configuring each on a one by one basis. End to end control.

It sounded sweet, made perfect sense to me and was a vast improvement over having a centralised management system that still had to go out to each device that was autonomous and configure them one by one, albeit automatically. It is better still than manually configuring each device which is where many of us are and have been since time began.


From intellectual discussions about overlays versus tunnels to ‘cute’ terms like ‘the only way is the overlay’, there is no denying that overlays are a hot topic at the moment. Speak to somebody who is buzzing about SDN and they’ll tell you that you need to make your physical underlay network solid, so you can overlay all of the unicorn goodness etc.

I’ve sat quietly back thinking all the while, why is this new? Surely you always want your underlying network infrastructure to be stable? It allows you to add the overlay networks on top more reliably and helps troubleshooting to boot. Then I saw a tweet from Ethan Banks referring to exactly what I had been thinking:

It seems we are getting excessively excited, not about brand new ways of doing things, but of different ways of doing things that we’ve done before, for many years. Sure, there is some innovative tech hitting the market place but it’s undermined by a deluge of ‘polished turd’ marketing and a lack of standards as each vendor tries to do best by themselves and not the industry.

OK, VMware’s NSX looks like it is willing to take the bull by the horns. Open Daylight looks promising and addresses some of my concerns around a collaborative effort. I do see huge potential with SDN but much more needs to be done.


It happened with virtualisation, cloud, automation and orchestration. Terms that got molested by sales and marketing folk across the globe. Perhaps ‘cloud’ is the only one in that list that truly got mistreated to the same degree that is happening to SDN right now. It seems that barely a product release goes by without the mention of SDN. It’s actually insulting to me as an IT professional  that these vendors expect me to lap up their wares because I should be caught up in the hype. Let’s calm it down a bit people and start challenging the claims being made. Forget the shiny shiny and let’s start telling, nay, demanding what we want from the vendors.

What question is SDN trying to answer?

This is the key question for me. It’s why I’m not handing out leaflets in my local city centre with the virtues of SDN printed on it, or going to my CTO demanding budget to implement it. It’s also why, until now, I’ve not blogged about it. From clearer beginnings, I am now unsure what exactly SDN is trying to achieve. The waters have been muddied. Or to return to my initial metaphor, there is a lava misinformation regarding SDN.


I’ve seen the buzz around SDN grow exponentially over the last couple of years but my need hasn’t grown at the same rate. SDN, or rather the industry as a whole, needs to mature considerably before I’ll take it seriously. It needs to become more relevant. Until that time, I’m going to continue building solid networks that run the services that my customers are asking for.

Till the next time.

10 tenets of working in IT – Tenet 6, Share


Let me share a little secret with you (wow, a post about sharing and I’m diving straight in with a share). People almost always get more done when they come together as a team, working towards a common goal. That doesn’t have to, and indeed should not mean continuous meetings. We live in a time where collaboration can be a far simpler task than it once was. Email, instant messaging, video conferencing, collaborative web portals and interactive whiteboards can all facilitate teamwork. Having said that, it all depends on how the team interacts with each other. If you are pulling in different directions, forget it. You’ll do more damage than good. But if you aren’t sharing, you are going to damage productivity. This post looks at how to share more effectively but makes the assumption that team frictions are at a minimum and that you are all aiming for the same goals.


There are three things, the lack of which are guaranteed to make me speak out. Communication, common sense and documentation. If you have knowledge of a customer infrastructure, write it up in an email, wiki article, Word document or whatever works for your organisation. Use standardised templates. Create document sets. Send links to colleagues. Use version control. Knowledge is power – sharing knowledge is the real power. Pass on tips. Give praise when receiving knowledge. Don’t assume people’s skills. Drop useful links on your blog\social sites. In a word, participate.


Despite having worked with some incredibly talented people in the field of IT, I’ve yet to meet one who couldn’t learn something from another person. Sharing your skills with somebody else has a number of benefits:

  • It empowers the person you are teaching to go away and do something they were previously unable to
  • It gives you an opportunity to clarify the knowledge in your own head
  • It is often helpful to bounce knowledge back and forth. This can lead to a mutual knowledge transfer that benefits all parties
  • It frees you up to do other work
  • On a more selfish note, it can be very satisfying to teach others
  • You remove the single point of knowledge failure syndrome, allowing you to sleep better when you are away on holiday, comfortable that  the knowledge isn’t also getting a suntan


Beyond sharing knowledge and skills, it is important to share workloads. Quite often, one or more people on the team will seem to be busier than others. This can happen for a number of reasons e.g. the person can’t help themselves volunteering for tasks, they are seen as an easy touch for dumping work on or they are renowned as the subject matter expert.

By documenting the things you and your team are responsible for and sharing your skills, you free yourselves up to share workloads, meaning that the people who were  busy before won’t get burned out (or at least not so quickly) and the people who were previously not so busy won’t have as much chance to get bored.


The concept behind this the tenet is sharing knowledge, skills and workloads. When that happens, we all grow as individuals, as teams and as organisations. The whole most certainly is larger than the sum of the parts. Failing to do so stunts growth and we fail to realise our true potential. Don’t let that happen.

Till the next time.

Running ASDM and WebVPN on the same interface


So you are thinking of running ASDM and WebVPN on the same interface? This is quite a rare configuration for the simple reason that ASDM is a management tool and WebVPN is usually enabled on the outside interface and best practice would dictate using an internal or even dedicated management interface to allow ASDM\CLI connections to. However, in a lab environment, this isn’t such an issue. In fact, in my labs, the machine I manage the ASA from is also the machine I test VPN connectivity from so this is a requirement for me.

Running ASDM and WebVPN on same interface

You basically have two options. You can change the port that ASDM runs on, or change the port that WebVPN runs on. As stated, this is mostly seen in a non-production environment so it probably doesn’t matter too much which way you do it but if for any reason you had to use this configuration in production, you would probably want to change the ASDM port so your remote users don’t have to worry about changing ports.

Both options are very simple to implement. To change the ASDM port, you enter a modified version of the command you enter to enable ASDM:

This changes the ASDM port to 4343. As stated, missing out ‘4343’ still enables ASDM but on the default port of 443.

To change the WebVPN port only requires an extra line:

Of course, both services can be run on the same port if required, but you need to know the URL to access ASDM. (The WebVPN URL is the default and so will load with just the IP address\hostname). The ASDM URL at time of writing on software version 9.1(2) is:


Once you have downloaded and installed the ASDM launcher, you again don’t need to worry about having different ports as the launcher itself connects to the correct URL automatically.


There may not be many situations in which you would consider running ASDM and WebVPN on the same interface but it’s good to know it can be done from both a port and URL point of view.

Till the next time.

ASDM inaccessible on Cisco ASA 9.1(1)


I’ll keep this short but sweet and hopefully this will save somebody a lot of head scratching out there. I unwrapped a couple of brand new ASA5512X firewalls but found the ASDM inaccessible.

ASDM inaccessible

All the standard stuff was in there, entered in global configuration mode:

I had enabled the http server and told the ASA which host address to accept connections from. I had enabled local authentication and a user name. When I connected, I got a “This webpage is not available” message.

After some sniffing around, I found a line of config that is critical:

This is the default on the two ASA devices I received. A security device. That has DES encryption as the default setting. Not very good Cisco. Not only is it weak encryption, but it stops my Chrome connecting to ASDM. Funnily enough, IE8, which was another browser on the jump box I was using allows the connection but I missed this until after I fixed it due to being a convert to Chrome for quite some time. So one option is to use an older, less secure browser. Or…


The right fix would be to change the default ssl encryption as below:

This now allows more recent (and secure) browsers to connect to ASDM. The command above is shown in the default config in version 9.1(1). In older versions, you would need to type:

I also believe the default in older versions would be to enable pretty much all levels of SSL encryption:


The key point to remember here is, when you use a new version of software that you have become familiar with, try and find out what the differences are! I’ve not checked at which point this change was made and whilst it’s not a show stopper it is annoying.

By the way, you may have noticed I set the encryption at AES128 and some of you may be aware that AES256 is an option. The reason I currently choose AES128 over the 192 or 256 bit versions is I’ve read about vulnerabilities (albeit non-critical) with those key lengths. I’d be interested in anybody else’s take on this.

Till the next time.

Why is IPv6 adoption so slow?


My, how time flies. I completed my CCNP a year ago and as part of the ROUTE and TSHOOT exams, there was coverage of IPv6 but it really felt more like a bolt on, rather than an equivalent study of IPv4. The last IPv4 /8 range has finally been allocated and yet there still appears to be a resistance to transition to the new protocol, instead using black magic to squeeze ever more out of the depleted IPv4 space. In this post, I look at some of the reasons I feel are behind the slow adoption of IPv6.

IPv6 adoption

So has IPv6 adoption exploded in the last 12 months? The answer is a definite no. I came back from Cisco Live London 2012 having sat in on a few IPv6 sessions and a four hour hands-on lab that, combined, gave me a very comfortable grasp of the protocol. I keenly wrote an implementation plan for the service provider I worked at upon my return. Whilst the number of people in the industry who report they are planning on implementing IPv6 in the next 12 months has increased many fold, the actual number who have implemented it remains a fairly steady and linear growth.

Demanding priorities

I’m not going to kid on that rolling out IPv6 is a trivial task that everybody should just do, by the end of next week if at all possible, thank you very much. It takes time and planning. Cisco’s own internal IT team, Cisco IT, took over five years to deploy IPv6 across their global estate and they are still working on it. These boys should grasp IPv6 as well as anybody yet they understand that you don’t just slap IPv6 in without some serious thought.

When I presented my implementation plan to the business, it was prioritised and promptly not implemented. Why? Simply because none of our customers were asking for it. Nobody was saying ‘we need services that are only being served up on IPv6’ or ‘we want to offer services to IPv6 only clients’. I look at why this is the wrong perspective later on. So the plan went to the bottom of the priority queue. There were other things we could be doing that visibly brought in money. I suppose in one respect, the good news is that the effort required to implement it was at least understood by the business. Despite my earlier jokey comment, there were no ‘can you have it ready by next week’ expectations.

Ignorance and fear

I think the biggest factor holding up adoption is a combination of both ignorance and fear. People are naturally fearful of change. We aren’t talking about a minor change here either. Add in herd behaviour and you get people avoiding the topic at all costs because they don’t want to be the one that pushed IPv6 on to other people’s already busy schedules.

If you ask anybody who has a certain level of technical knowledge about the biggest risks around IPv6 deployment, you might get some responses back that mention security. Whilst it’s true that IPv6 brings new attack vectors to the party, I don’t believe that it is any more or less secure than IPv4. It all boils down to risk management. If you currently don’t protect your IPv4 infrastructure, then you’ll be no worse off, you’re already doing it wrong. If you do, then you just need to be aware of the newer vulnerabilities and be as vigilant as you currently are.

If you ‘dig’ IPv6, have read up on it and are comfortable working with it, you might possibly forget what it was like before you tackled it so this might all seem like ludicrous behaviour, yet I see it quite often (IPv6 isn’t a unique topic in that respect either). What is the root of this fear? There are a number of factors as I see it and if I was pointing fingers, I’d be pointing them, with various degrees of wagging at:

  • The press. How many times have you read a comparison of the IPv6 address space to the number of grains of sand multiplied by the total number of freckles on all the ginger people in the world? Perhaps just once for that specific example, but it seems there are now as many comparisons as there are addresses. We get it. IPv6 has a ridiculously large address space. Let’s not forget the 128 bit length too. Big numbers can carry the wow factor, which is why the press regurgitate them with such vigour, but they can also scare people, for example when somebody gets given a /32 IPv6 address and told to create an addressing scheme from it for their global network
  • IETF. I understand that protocols need to adapt but therein lies the rub. I can honestly say that since I first started studying IPv6, and that goes back to my sysadmin days, IPv6 has changed several times over. There have been changes to the address scopes, the usage of /126 – /128 networks, the ICMPv6 implementation, etc. Whilst I would argue that all of the changes I’ve seen have been for the greater good, it strikes fear in to some people, particularly those who are risk averse. In IT, there is an eternal struggle between stability and agility but I don’t think I’d get much argument from people when I say that I like my lower level protocols to be stable. Rightly or wrongly, the transformation that IPv6 has gone through, even in the last three years, let alone since inception, is enough to scare a lot of people away from tackling it. Heck, I know some people who avoid learning about it because ‘it will have changed again in a few months’
  • Vendors. We need to cast at least a couple of dirty looks across to the vendors. Even in June 2013, some are not implementing all of the relevant pieces of the IPv6 puzzle so IT houses that utilise that vendor are being forced to hold off for the time being. I’m not just talking about hardware here. There needs to be full interoperability across the board
  • Service providers. Quite possibly one of the biggest culprits in my opinion. Build it and they will come. Start coming up with crazy ideas like CGN (Carrier Grade NAT) and you’ll just create a world of pain further down the road. That’s the equivalent of taking a rug from a dump site and trying to brush the rest of the dump under it. Taking my analogy further, I guess I’m proposing building a brand new dump, which doesn’t really sell it…but regardless, service providers need to start pulling their fingers out. Not only do the big boys need to be running it across their core, which I’m confident most already do, but companies who offer Internet services to consumers should have IPv6 as a highly visible and preferred option
  • You and me. By sheer size of numbers, I place a fair chunk of responsibility with the people working at the coal face. The network engineers, the developers, the sysadmins. Yes, it might seem like a scary proposition but it isn’t going to implement itself and the requirement isn’t going to go away. So even if you have no demand at the moment, believe me, you need to get planning regardless. When that demand comes in, you need to be able to deliver and support or else you will be left behind. There is lots of talk about the industry holding up IPv6 deployment. Well guess what? You and I are a part of that industry so let us all accept some responsibility and look at how we can overcome the obstacles. I don’t think anybody can afford to sit back and think the problem will resolve itself and please don’t be of the mindset that this isn’t going to happen in your working career


Earlier I talked about deploying IPv6 to get to IPv6 content and mentioned this is the wrong way of looking at things, certainly at this time. The current need to move to IPv6 is due to the lack of available IPv4 addresses, pure and simple. As far as I’m aware, there isn’t a whole lot of IPv6 exclusive content out there. However, the sooner we can get everything dual stacked, running IPv4 and IPv6, the sooner we can retire IPv4 and then IPv6 really comes in to it’s own.


IPv6 isn’t a flash in the pan. It’s a protocol borne out of necessity but it offers so much more than what IPv4 currently does. Burying your head in the sand won’t make it go away. There is no better time than now to get familiar with it. Watch this space for a high level planning post that will hopefully give you a starting point. After all, they say that a journey of 340 undecillion miles begins with a single step.

Till the next time.

Trouble Ticket #3: Cisco Secure Desktop, ASDM not reflecting CLI


Just in case anybody was thinking ‘wow, this guy must have a cushy job, only three trouble tickets over the last 18 months’, then think again. The purpose of these trouble ticket posts is to write about those issues that don’t occur every day, the crafty quirks that have you scratching your head. In this case, I was witnessing something with the Cisco Secure Desktop feature that didn’t make sense and was frustrating to say the least.

Cisco Secure Desktop

For those not in the know, Cisco Secure Desktop (CSD) is a feature of the Cisco ASA platform that allows remote access VPN clients to go through a number of health checks before being allowed to connect. In the scenario discussed here, I had configured a couple of client certificate checks and wanted to add a third for a specific client type.



The diagram above shows the third certificate check added at the bottom right. If this third check fails, the client is denied login.


When I saved the changes, everything looked as it should. The policy in the diagram above is not saved in the normal configuration file on the ASA but in this file: ‘flash:/sdesktop/data.xml’. As you can see from the snippet below, immediately after entering the third certificate check, the GUI and CLI agree on the configuration (certain details changed\removed):

The problem reared it’s ugly head when I closed ASDM and reloaded it. The XML file always shows correctly what the ASA is doing, but ASDM was now missing that third check in the GUI so now it appeared it was only carrying out the original certificate checks. That meant that to delete the third check, or add yet another check, I would need to hack the XML file.

Troubleshooting steps

I tried reloading the ASA. That made the GUI update to reflect the current XML file, but again, as soon as I made any changes and reloaded ASDM, the GUI would revert back to the two certificate check, regardless of what other checks I added. It was as if there was a cache somewhere.

I upgraded CSD to the latest version and still no joy. At that time, I got Cisco TAC involved. The person who I was dealing with was helpful and took all the details so he could lab it up. At first he reported he couldn’t replicate the issue but a few days later got back to me to report it was occurring but intermittently.

He requested some Java debug logs, which I duly gave and then waited for about a week while he escalated it to the development team.


Finally, he came back to me to advise that this was being listed as a new bug:


Lucky me, I got my first bug listed. Something tells me it won’t be my last. There is a better workaround than rebooting the ASA each time though:


Basically, disabling and re-enabling CSD re-syncs the ASDM with the XML file, so until the bug is fixed, if you see this issue, I’d suggest doing this workaround before you make any changes to your CSD policies. FYI, I was using version 1.7 of Java and apparently 1.6 doesn’t have the same problem. So, to manage your firewall security device, feel free to use a less secure Java client. Sigh…

Till the next time.