Recently, out of nowhere, I had a 3850 stack that spiked in terms of CPU usage. This particular stack had no recent changes in configuration and had been up and running for around 20 weeks since the last reboot. Some of the symptoms were high ping times for anything passing through this switch and a significant amount of lag when trying to SSH into this stack. The average CPU for each core on switch 1 (the master switch) was in the 90th percentile. It was pretty much pegged there with little change throughout the day.
After checking the processes and the CPU they were using, I noticed that the “fed” and “iosd” processes were using a large amount of CPU. I started diving deeper into this to try to resolve the issue. I then ran the following commands:
- show process cpu detail process fed sorted | ex 0.0
- show process cpu detail process iosd sorted | ex 0.0
That gave me some more insight as to what was happening. That is where something big jumped out at me. There was a single process that was running at 97%. It was “NGWC l2m”. I consulted with TAC and they mentioned that this was from IPv6 requests being flooded to this stack. Since I don’t use IPv6, we ran a command to disable these requests from affecting the system software. Enabling snooping allowed these requests to be handled by the hardware. This is enabled by entering the following command:
- ipv6 mld snooping
This gave us IMMEDIATE relief in terms of CPU. We went from the 90’s down to the 30’s. If you happen to come across a similar issue, give this a try and see if you are affected by the same thing I was.
Hope that helps!