Corrupting ext3 filesystem

If it doesn't fit in another category, ask it here.
Post Reply
justus5
Posts: 2
Joined: 2005/09/30 07:40:27

Corrupting ext3 filesystem

Post by justus5 » 2005/09/30 08:34:06

This probably isn't exactly the right forum to ask, but I thought to give it a try since our systems are running CentOS ;)

We have two identical AMD64 Opteron production servers, running CentOS4.1. Both systems have a 2.7TB SATA RAID5
configured with Areca ARC-1160 card and mounted as a ext3 filesystem.

Everything has run smoothly with the above configuration - until last week. The other server had suffered a hdd failure during
the night and for some reason the whole OS had crashed. To make a long story short I booted it up and noticed that the RAID were
rebuilding. I also checked and monitored the syslog for a while and didn't notice anything unusual.
Unfortunately after a few hours syslog was populating with filesystem errors and at the end of the day it seemed that the
filesystem were wasted.

After rebuilding the system in addition with newer kernel and RAID driver and some stress tests we returned the server
back to prodution. After few hours syslog were showing disk errors again:

EXT3-fs error (device sdb1): ext3_new_block. Allocating block in system zone - block: 718242306
Aborting journal on device sdb1

So, the filesystem were corrupting again.

Any ideas what could cause this? Disks should be ok. Areca card tells that the RAID set should be ok. Memory tests pass clean.
Apparently this is a hardware problem, since the other server is running ok (or are we waiting the lightning to strike twice?).

Post Reply

Return to “CentOS 4 - Miscellaneous Questions”