Hello,
I am facing a strange issue with syslog-ng v3.5.2 (also faced same issue with v3.9) running on centos 6.10.
After a few hours of running syslog-ng the machine becomes unresponsive, we cannot pass the ssh login screen, it accepts username but there's no password prompt. I cannot run any commands (crontab etc).
The only way to get into the machine, is to always keep an extra ssh session and login into that to kill syslog-ng. After killing syslog-ng machine becomes responsive again.
This machine accepts on average a couple of thousants packets per second from 300-400 hosts.
$ ss -s
Total: 330 (kernel 1024)
TCP: 1124 (estab 136, closed 958, orphaned 10, synrecv 0, timewait 948/0), ports 988
Transport Total IP IPv6
* 1024 - -
RAW 0 0 0
UDP 30 18 12
TCP 166 97 69
INET 196 115 81
FRAG 0 0 0
Anybody has faced a similar behaviour before?
Thanks
Machine hangs with syslog-ng after a while
Re: Machine hangs with syslog-ng after a while
Did you look at `top` at the time? Or if you have sysstat installed then you could look at the output from `sar` to see some system stats. Sounds to me most likely you have something with a memory leak and it's using up all RAM and swapping like crazy at which point the machine will slow to a crawl...
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
-
- Posts: 2
- Joined: 2018/09/18 06:11:47
Re: Machine hangs with syslog-ng after a while
Thanks for replying, it seems that cpu and memory is quite low. running syslog-ng in verbose mode it shows this error at some point:TrevorH wrote: ↑2018/09/18 06:54:53Did you look at `top` at the time? Or if you have sysstat installed then you could look at the output from `sar` to see some system stats. Sounds to me most likely you have something with a memory leak and it's using up all RAM and swapping like crazy at which point the machine will slow to a crawl...
Code: Select all
iv_fd_epoll_timerfd_create: got error 24[Too many open files]
Re: Machine hangs with syslog-ng after a while
That could be the problem. You could increase the maximum that it's allowed to use from limits.conf perhaps but I would guess it's running as root anyway?
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke