Judging from the 'cciss'-bit it would appear your disks are on an HP SmartArray (just guessing, I only have HP boxes here), which begs the question whether you run that in RAID or not.
Not sure you can request SMART data from hardware raid, or even from a SmartArray. I had a go on one of my boxes but got 'A mandatory SMART command failed' all the time.
If my assessment that you're on an HP SMartArray (ProLiant server) is correct, I suggest you have a look at DMESG and search for any complaints about the cciss driver; you may need to update the SmartArray firmware. After upgrading a couple of problem-free boxes from RHEL 5.2 / 5.4 to RHEL 5.6 I had three of them failing (after some hours or days) on cciss I/O; writing to the disks timed out, causing infinite hang . One of these boxes had a line in DMESG
- Code: Select all
cciss 0000:04:01.0: Controller reports max supported commands of 0, an obvious lie. Using 16. Ensure that firmware is up to date.
but the others did not. All of them were fixed by updating the SmartArray firmware to the latest level, though.
Somewhere during startup you must see messages as to why the filesystem is placed in read-only mode. Although your symptoms are not the same (and you're not on 5.6 but 5.5), you may be suffering from similar firmware issues, and it doesn't hurt to check/try. You may also get some indications when you go into the SmartArray's BIOS on boot, and see if it has any issues with drive statuses.
However, if you haven't done so yet, before anything else, my first line of action would be to backup any valuable data from the still readable disk, as it may still be the disk that's failing.
I found that HP SmartAray identifications are very confusing, and the same controller can be referred to under at least three different names. Updating the firmware from a running box (running the downloaded .scexe program) failed, so I had to boot from the SmartUpdate iso, through ILO Virtual Media, as getting physical access to the box is somewhat of a hassle. SmartUpdate ISOs (firmware cd's) can be downloaded from the HP support site.
Hope this helps.