Using ICMP packets can be a great troubleshooting tool in a network setting and is probably one of the most commonly used tools by any network admin. Recently I came across an issue where I would run a repeated ping with a count of lets say 1000. Over the course of the one thousand pings or more, I would notice random, dropped packets. All of the other packets would be transmitted with zero issue.
In my case, this was due to control plan policing. This feature is called copp for short. Nexus 7000 copp, it is a tool to protect the backplane of your system from possible attacks or resource usage that could cause instability. That being said, one of the things that is monitored is ICMP traffic. That is where my problem was coming from.
The copp policy looks a bit like this:
IP access list copp-system-acl-icmp
10 permit icmp any any echo
20 permit icmp any any echo-reply
class-map type control-plane match-any copp-system-class-monitoring
match access-group name copp-system-acl-icmp
match access-group name copp-system-acl-icmp6
This “monitoring” section of the copp policy is where the issue came from. There are a couple of way to get around this problem. The first is a drastic approach and that is just turning off copp altogether. In the Admin VDC, you use the command:
N7K(config)# no copp profile strict
(sub in any other profile you might be using)
This will turn off copp and you will notice immediate resolution of the issue and pings will no longer drop. That being said, this is not the best way to fix the issue and the syslog prompts you will get when you turn off the nx-os copp profile will let you know that.
Instead of that though, maybe try this command:
N7K# show policy-map interface control-plane | b monitoring
You will see the settings listed around the amount of bandwidth that is allowed to be used before ICMP packets are policed. You can edit the lines of config that you see to increase this bandwidth amount.
Here is my resolution to the issue for the time being. I understand as a Network Engineer that if I am running an extended ping that a packet may drop here and there. I am not going to jump and change a policing policy that is there to protect the stability of my system. The end user in this specific case is not affected negatively, so I can live with the issue and choose to keep the heightened security that comes with this against a flood of ICMP traffic. It’s a weighted situation in which the admin needs to make the decision to change the settings of not. Obviously if the end user becomes affected or if you have an application that relies on ICMP traffic, then you’d definitely want to look into changing the policing limits and or methods on the Nexus 7000 copp policy.
As with many things in networking, it all relies on each specific installation.