ssh and console access hang after random amounts of time

Installing, Configuring, Troubleshooting server daemons such as Web and Mail
Post Reply
memblin
Posts: 2
Joined: 2008/10/20 21:39:26

ssh and console access hang after random amounts of time

Post by memblin » 2008/10/20 21:56:17

I've done lots of searching and I keep finding references to auditd causing this if the filesystem gets
to X% full.

Here is one place I see it referenced.
http://www.centos.org/modules/newbb/viewtopic.php?topic_id=4312

I did some checking on our boxes, we've got five of them running CentOS 4, each has heavy inbound
scp traffic.

Randomly it seems they will just hang now and then requiring a reboot to get them
back up and running. The /var/log/auditd directory has nothing in it, chkconfig --list auditd shows
auditd disabled at every run level. A quick /sbin/service auditd status also says it is disabled.

When I try to ssh -v this is where I get hung

*snip from ssh -v output*
chris:~ crow$ ssh -v server4
OpenSSH_5.1p1, OpenSSL 0.9.7l 28 Sep 2006
debug1: Reading configuration data /etc/ssh_config
debug1: Connecting to server4 [] port 22.
debug1: Connection established.
debug1: identity file /Users/crow/.ssh/identity type -1
debug1: identity file /Users/crow/.ssh/id_rsa type -1
debug1: identity file /Users/crow/.ssh/id_dsa type -1
debug1: Remote protocol version 1.99, remote software version OpenSSH_3.9p1
debug1: match: OpenSSH_3.9p1 pat OpenSSH_3.*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-cbc hmac-md5 none
debug1: kex: client->server aes128-cbc hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'server4' is known and matches the RSA host key.
debug1: Found key in /Users/crow/.ssh/known_hosts:74
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/crow/.ssh/identity
debug1: Trying private key: /Users/crow/.ssh/id_rsa
debug1: Trying private key: /Users/crow/.ssh/id_dsa
debug1: Next authentication method: password
crow@server4's password:
debug1: Authentication succeeded (password).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.

That's all I get. It'll sit there until I close the terminal and go hard boot the server.

A box that is having the problem still responds to ping. I see no error messages in the logs once
the system comes back up. None of the file systems are full either.

I've never seen anything like this before. The audit system was already disabled when I started
researching the problem. I do see [kauditd] in the process list and on boot I do see auditd initialize
but it shows disabled.

*snip from messages*
messages.1:Oct 17 13:18:53 asset2 kernel: audit: initializing netlink socket (disabled)
messages.1:Oct 17 13:18:53 asset2 kernel: audit(1224249443.850:1): initialized


Anyone have any ideas?

Post Reply

Return to “CentOS 4 - Server Support”