Posted on

Trouble Ticket #4: Unable to download NAT policy for ACE

Introduction

This was a fun one. Coming in to work on a Friday morning for what you hope is an uneventful segue in to the weekend and your colleague looks up from his fast scrolling terminal screen and says “we may have a problem”.

Unable to download NAT policy for ACE

The change he had implemented was simple enough. Add a couple of new sub-interfaces to the Cisco ASA firewall, add the required security ACLs and configure the NAT and no-NAT (NAT0) rules. The firewall code was still pre 8.3 on 8.0(4), so used the older NAT syntax.

The problem arose when the no-NAT config was applied, specifically adding ACE entries to the ACL that the no-NAT applied to the new interfaces referenced. The firewall threw up the following message:

[sourcecode language=”plain”]
Unable to download NAT policy for ACE
[/sourcecode]

In the context of the above sequence of events, this message isn’t actually that obscure. Pre version 8.3, the Cisco ASA uses policy based NAT. For the no-NAT, it uses an ACL to decide which traffic should not be NAT’d as it comes in to an interface. As the new ACEs were being put in to the firewall, the above message is effectively telling us that the firewall was unable to apply this to the no-NAT policy. So the ACE shows up in the config, but it isn’t having any effect.

In addition to this, we had also lost management access to certain networks through the firewall as part of this change.

The fix

The config was rolled back as a matter of course but the issue remained. Running packet tracer on the firewall showed that the issue was down to the no-NAT, although comparing the config with a backup showed no differences.

Based on our gut feelings and the message we saw, the NAT0 statement was removed and re-added and the issue vanished. Searching Cisco.com brought up this bug (CSCsl46310). Cisco recommend reloading the firewall as a workaround prior to reapplying the NAT0 statement, but that wasn’t required in our case.

Known fixed releases are supposedly 8.2(0.79), 8.0(3.2) and 8.1(0.130), although on the download site, 8.2.5 is a recommended version so I think that will be my first stop.

Summary

It is actually a pleasant surprise when a bug at least produces behaviour and a system message that can be used to troubleshoot without too much effort.

Till the next time.

Posted on

Trouble Ticket #3: Cisco Secure Desktop, ASDM not reflecting CLI

Introduction

Just in case anybody was thinking ‘wow, this guy must have a cushy job, only three trouble tickets over the last 18 months’, then think again. The purpose of these trouble ticket posts is to write about those issues that don’t occur every day, the crafty quirks that have you scratching your head. In this case, I was witnessing something with the Cisco Secure Desktop feature that didn’t make sense and was frustrating to say the least.

Cisco Secure Desktop

For those not in the know, Cisco Secure Desktop (CSD) is a feature of the Cisco ASA platform that allows remote access VPN clients to go through a number of health checks before being allowed to connect. In the scenario discussed here, I had configured a couple of client certificate checks and wanted to add a third for a specific client type.

Diagram

CSD_ASDM

The diagram above shows the third certificate check added at the bottom right. If this third check fails, the client is denied login.

Problem

When I saved the changes, everything looked as it should. The policy in the diagram above is not saved in the normal configuration file on the ASA but in this file: ‘flash:/sdesktop/data.xml’. As you can see from the snippet below, immediately after entering the third certificate check, the GUI and CLI agree on the configuration (certain details changed\removed):

The problem reared it’s ugly head when I closed ASDM and reloaded it. The XML file always shows correctly what the ASA is doing, but ASDM was now missing that third check in the GUI so now it appeared it was only carrying out the original certificate checks. That meant that to delete the third check, or add yet another check, I would need to hack the XML file.

Troubleshooting steps

I tried reloading the ASA. That made the GUI update to reflect the current XML file, but again, as soon as I made any changes and reloaded ASDM, the GUI would revert back to the two certificate check, regardless of what other checks I added. It was as if there was a cache somewhere.

I upgraded CSD to the latest version and still no joy. At that time, I got Cisco TAC involved. The person who I was dealing with was helpful and took all the details so he could lab it up. At first he reported he couldn’t replicate the issue but a few days later got back to me to report it was occurring but intermittently.

He requested some Java debug logs, which I duly gave and then waited for about a week while he escalated it to the development team.

Workaround

Finally, he came back to me to advise that this was being listed as a new bug:

CSCuh08360

Lucky me, I got my first bug listed. Something tells me it won’t be my last. There is a better workaround than rebooting the ASA each time though:

Summary

Basically, disabling and re-enabling CSD re-syncs the ASDM with the XML file, so until the bug is fixed, if you see this issue, I’d suggest doing this workaround before you make any changes to your CSD policies. FYI, I was using version 1.7 of Java and apparently 1.6 doesn’t have the same problem. So, to manage your firewall security device, feel free to use a less secure Java client. Sigh…

Till the next time.