Centos 5 x86_64 raid install

[LowWalker]

So, I am building a new mail/ftp server. I have 4 hd's, my mobo has a raid controller on it, which works awesome for windows... but you have to run an adaptec driver to get it working with centos. The controller is the intel ESB2 and it can do raid 1.

So basically i want to do this, create array1 with boot/OS. Create array2 with FTP/Mailstore space. I am using vsftpd and zimbra for the mail/ftp.

When I try to install, I just get a black screen when I try to let centos installer default the layout. Same thing if I choose any specific mounts etc.

Any suggestions on resources to check out or mount point settings during install? I have the server up right now with just 1 array loaded to test some things and work on my zimbra config builds etc.

Thanks!

* Edit. I am using raid 1 for each of the pairs of arrays.

[pschaff]

If you search the forum for "ESB2" you will find a lot of relevant posts. Seems like the best bet is to use software RAID. Please do the "Required reading" and search for an answer.

[LowWalker]

Yeah, I did that after :(

I think I figured it out anyway. Grub was choosing my swap as the default boot partition... reinstalling now. This post is embarrassing now lol.

[LowWalker]

Ok, so here is the break down of what I just did and it failed. I will look for some grub threads but at least I don't feel so dumb anymore.

I created 2 arrays with the adaptec config,

system sd0/sd1
data sd2/sd3

In the centos install i chose custom layout,

/dev/mapper/dev_system
/dev_system1 - /boot 100mb
/dev_system2 - / (fill to max size)
/dev_system3 - swap 16G

/dev/mapper/dev_data
/dev_data1 - /opt (fill to max size)


On the grub install screen, it chose /dev_system3 as the boot default, I changed that to /dev_system1
Also it chose /dev_data1 for the MBR, I changed that to /dev_system1 also. (I assumed the MBR would need to be on the boot drive, but meh you know assuming.)

Finished install....

Reboot, and I get black screen with

GRUB _

:-?

Anyway, I know the install works with the hardware drivers, because if I do it with just the defaults loaded or my above layout for partitions without the other array on the system it boots fine. I know I am messing up somewhere in the grub probably.

[AlanBartlett]

Correct me if I'm wrong but it looks like you are still using the BIOS (software) fakeraid rather than the kernel (software) raid.

Go back into your BIOS and disable its raid, then re-install using the kernel raid (mdadm).

I'm sure both [b]Gerald[/b] and [b]Markuk[/b] will have something to say about the use of BIOS fakeraids . . .

[LowWalker]

Yes I am trying to use the adaptec still. Obviously.. I am not a linux pro, so the filesystem is not exactly a firm thing in my head.

I just tried to do another similar install, but with the difference that i removed the other adaptec array (my /opt) and then set the /dev_system array like above. I then used the software raid in the partition screen to make /dev/md0 out of the other 2 disks hoping that my system would like that more. Blank screen reboot....

Going back after that into the bios, no drives are listed under sata, so i configured the 3 and 4 HDs as individual volumes in the adaptec config.... trying again.

If someone has a better idea about the file system basically i want to have OS/FTP array 1, in raid1 mirror. I am trying to mount /opt on the second array, raid1 mirror because that will be my main mail store for zimbra. I will look into mdadm now.

Thanks for the help!

[gerald_clark]

Since you created the BIOS RAID, the drives have metadata on them that can interfere with normal operation.
You can add nodmraid boot option, or erase it with
dd if=/dev/zero bs=1M count=10 of=/dev/XXX
where XXX is the hardware device.

[jlehtone]

[code]dmraid -r --erase_metadata [device-path...][/code]
should be possible too.

[RobTheAdmin]

LowWalker, I had this same exact problem about a year ago when starting my new job. I was able to get it resolved. You *must* disable hardware RAID in BIOS or in the controller's boot menu (wherever it may be). In my case, my RAID controller had its own menu at boot time. Then you *must* reinstall CentOS with software RAID. Otherwise when you update the kernel after a new install, you'll always be greeted with Grub. This happened to me about 20 times before I found some help in the CentOS IRC channel.

I have the documentation around here somewhere to implement the software RAID correctly. Let me dig it up. I should find it soon and I'll post it here.

[RobTheAdmin]

Section One

1. Disable hardware-based RAID
2. Begin graphical installation of CentOS
3. Customize partitions and remove *all* partitions, including LVMs (we're going to start from scratch)
4. Create 100MB /boot partition on sda (your 1st drive), make it primary, ext3 is fine
5. Create 100MB /boot2 partition on sdb (your 2nd drive), make it primary, ext3 is fine
6. Create a software RAID partition on sda and sdb for 1GB. (This partition is based on your RAM and personal preferences. This is going to be RAIDed swap. I have 4GB RAM and chose to use 1GB swap. Works great so far. You may want a larger or smaller swap...it's up to you.)
7. Create a software RAID partition on sda to fill to maximum allowable size, do the same on sdb
8. Click the RAID button, create a RAID device
9. Pick the two 1GB partitions and choose File System Type swap, choose RAID 1 (default RAID device is fine, probably md0), hit OK
10. Click RAID again, create RAID device
11. Mount point is /, file type ext3, default RAID device (probably md1), choose RAID1
12. Your partitions are complete. Your screen should look similar to this:
RAID Devices
--- /dev/md0 swap [check mark] 1024
--- /dev/md1 ext3 [check mark] [lots of GBs]

Hard Drives
--- /dev/sda
-------- /dev/sda1 /boot ext3 [check mark] 100
-------- /dev/sda2 /dev/md0 software RAID [no check mark] 1024
-------- /dev/sda3 /dev/md1 software RAID [no check mark] [lots of GBs]

--- /dev/sdb
-------- /dev/sdb1 /boot2 ext3 [check mark] 100
-------- /dev/sdb2 /dev/md0 software RAID [no check mark] 1024
-------- /dev/sdb3 /dev/md1 software RAID [no check mark] [lots of GBs]

13. Click next to go to the Grub bootloader screen.
14. It should default to installing Grub to /dev/sda, click Next
15. Continue installing CentOS on your own as you normally would.
16. Proceed to Section Two when CentOS is installed and you're sitting at the command prompt as root. Do not update CentOS, do not do anything!


Section Two

1. Make sure you're root.
2. Input this command "umount /boot2", hit Enter
3. Edit /etc/fstab with a text editor. (I like to use the nano text editor. Depending on the packages you installed, it may not be on your system. "yum install nano" will do the trick.)
4. Remove the entire line that contains /boot2 from it. Save and exit.
5. Input this command "dd if=/dev/sda1 of=/dev/sdb1", hit Enter (your drives should now grind a bit)
6. Edit /boot/grub/device.map with a text editor.
7. You should see something like this:
(hd0) /dev/sda
(hd1) /dev/sdb
8. Now edit device.map so that it looks like this:
(hd0) /dev/sdb
(hd1) /dev/sda
9. Save and exit device.map
10. Input this command "/sbin/grub-install /dev/sdb", hit Enter
11. Edit device.map and put it in its original state. Save and exit.
12. Cross your fingers. Type in "reboot", hit Enter.
13. Once you're booted up, do a system update. Do not reboot after the update.
14. Go to Section Three.


Section Three

1. Input this command "umount /boot"
2. Input this command "dd if=/dev/sda1 of=/dev/sdb1"
3. Reboot.
4. Check that you're actually running the updated kernel by this command "uname -a"
5. Done!


Special thanks for this guide go to timverhoeven of #centos on irc.freenode.net. Timverhoeven told me to remember this procedure. You must do it each time you update the kernel or else you might end up at the dreaded Grub prompt. I'm not sure if he meant I needed to repeat Section Two and Three, or just Section Three when I upgraded the kernel. Smart minds reading this thread might clue us in.