snmptrapd is dropping packets when under load

Issues related to configuring your network
chelomm@gmail.com
Posts: 4
Joined: 2017/04/18 07:00:23

snmptrapd is dropping packets when under load

Postby chelomm@gmail.com » 2017/09/02 14:06:30

Hi

This is regarding snmptrapd.

I have noticed that when there are considerable amount of traps (1000+/min) arrive at the interface some of them are dropped, or not logged into /var/log/trapd.log

Before making this post I have search the web quite a bit for this issue but could not find any relevant info.

To simulate it I have simple script on server A sending traps to server B every second.

I can see these traps on the B server tcpdump and they all arrive OK.

But when I search for them in /var/log/snmp/trap.log I see some gaps.

This is quite important for me to make sure that no traps are lost.

When I run the same test in the lab for few hours with no load, I see ZERO packets lost, everything matches perfectly, the number of sent traps, the number of received traps in tcpdump and number of traps in /var/log/snmp/trapd.log

but when under load in the prod environment, which runs RHEL6.8 with around 1500 traps per min, I see some gaps.

the same happens in another prod environment running CentsOS6.5

I would really appreciate your help here.

Here is the:
----------------------------------------------------------------------
# ps -ef | grep snmptrapd
root 21077 1 0 Sep01 ? 00:02:19 /usr/sbin/snmptrapd -Lf /var/log/snmp/trapd.log -p /var/run/snmptrapd.pid
----------------------------------------------------------------------
and
----------------------------------------------------------------------
# cat /etc/snmp/snmptrapd.conf
# Example configuration file for snmptrapd
#
# No traps are handled by default, you must edit this file!
#
# authCommunity log,execute,net public
# traphandle SNMPv2-MIB::coldStart /usr/bin/bin/my_great_script cold
disableAuthorization yes
ignoreAuthFailure yes
----------------------------------------------------------------------

and

----------------------------------------------------------------------
# cat /etc/sysconfig/snmptrapd
# snmptrapd command line options
# OPTIONS="-Lsd -p /var/run/snmptrapd.pid"
OPTIONS="-Lf /var/log/snmp/trapd.log -p /var/run/snmptrapd.pid"
----------------------------------------------------------------------

Any advice will be much appreciated.

Thank you

Mike