[SOLVED] Problems with RAID 1 GRUB setup

General support questions
dejanst
Posts: 124
Joined: 2011/09/20 14:13:13

[SOLVED] Problems with RAID 1 GRUB setup

Postby dejanst » 2011/09/20 14:57:32

I recently upgraded from CentOS 5.7 to CentOS 6 (32 bit) and I have some issues with setting up GRUB on both HDDs.

The server is HP Proliant Microserver that is running 2 HDD's that have the same capacity. Both HDDs have the same partition layout, the partitions are properly combined in 8 RAID devices.

Code: Select all


== BEGIN df -h ==
Filesystem            Size  Used Avail Use% Mounted on
/dev/md2              9.7G  349M  8.8G   4% /
tmpfs                 440M     0  440M   0% /dev/shm
/dev/md0              291M   38M  239M  14% /boot
/dev/md7               73G  180M   69G   1% /komna
/dev/md6               49G  180M   46G   1% /telos
/dev/md5              2.0G   35M  1.8G   2% /tmp
/dev/md3              7.7G  883M  6.5G  12% /usr
/dev/md4              4.9G  215M  4.4G   5% /var

== BEGIN fdisk -l ==
Disk /dev/md0 doesn't contain a valid partition table
Disk /dev/md1 doesn't contain a valid partition table
Disk /dev/md2 doesn't contain a valid partition table
Disk /dev/md3 doesn't contain a valid partition table
Disk /dev/md4 doesn't contain a valid partition table
Disk /dev/md5 doesn't contain a valid partition table
Disk /dev/md6 doesn't contain a valid partition table

Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f15fe

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1               1        1275    10240000   fd  Linux raid autodetect
/dev/sda2            1275        1530     2048000   fd  Linux raid autodetect
/dev/sda3   *        1530        1569      307200   fd  Linux raid autodetect
/dev/sda4            1569       19458   143693824    5  Extended
/dev/sda5            1569        7943    51200000   fd  Linux raid autodetect
/dev/sda6            7943        8963     8192000   fd  Linux raid autodetect
/dev/sda7            8963        9600     5120000   fd  Linux raid autodetect
/dev/sda8            9601        9856     2048000   fd  Linux raid autodetect
/dev/sda9            9856       19458    77128704   fd  Linux raid autodetect

Disk /dev/sdb: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0003b78a

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1               1        1275    10240000   fd  Linux raid autodetect
/dev/sdb2            1275        1530     2048000   fd  Linux raid autodetect
/dev/sdb3   *        1530        1569      307200   fd  Linux raid autodetect
/dev/sdb4            1569       19458   143693824    5  Extended
/dev/sdb5            1569        7943    51200000   fd  Linux raid autodetect
/dev/sdb6            7943        8963     8192000   fd  Linux raid autodetect
/dev/sdb7            8963        9600     5120000   fd  Linux raid autodetect
/dev/sdb8            9601        9856     2048000   fd  Linux raid autodetect
/dev/sdb9            9856       19458    77128704   fd  Linux raid autodetect

Disk /dev/md0: 314 MB, 314560512 bytes
2 heads, 4 sectors/track, 76797 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md1: 2096 MB, 2096095232 bytes
2 heads, 4 sectors/track, 511742 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md2: 10.5 GB, 10484641792 bytes
2 heads, 4 sectors/track, 2559727 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md3: 8387 MB, 8387551232 bytes
2 heads, 4 sectors/track, 2047742 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md4: 5241 MB, 5241823232 bytes
2 heads, 4 sectors/track, 1279742 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md5: 2096 MB, 2096095232 bytes
2 heads, 4 sectors/track, 511742 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md6: 52.4 GB, 52427681792 bytes
2 heads, 4 sectors/track, 12799727 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000


Disk /dev/md7: 79.0 GB, 78978674688 bytes
2 heads, 4 sectors/track, 19281903 cylinders
Units = cylinders of 8 * 512 = 4096 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / Disk /dev/md7 doesn't contain a valid partition table
512 bytes
Disk identifier: 0x00000000

== END   fdisk -l ==

== BEGIN blkid ==
/dev/sdb1: UUID="0b26d410-68bf-3813-7439-722b9944c54a" UUID_SUB="b4f29272-c47f-a723-0927-8c2d671b8a0d" LABEL="localhost.localdomain:2" TYPE="linux_raid_member"
/dev/sdb2: UUID="2c222a2b-0319-14d6-ff37-e9de8d67b86d" UUID_SUB="ea656011-3919-0b97-12ca-a5d931225e4b" LABEL="localhost.localdomain:1" TYPE="linux_raid_member"
/dev/sdb3: UUID="038f3aa1-3903-75fb-711f-ce7eb3f93b4a" UUID_SUB="6747df7a-cd6f-c6d0-f594-ddd5c0b60e8e" LABEL="localhost.localdomain:0" TYPE="linux_raid_member"
/dev/sdb5: UUID="c1366613-ace9-499a-698d-9f0d70ba42c0" UUID_SUB="f0ae8820-880e-b096-8971-16bd335b0599" LABEL="localhost.localdomain:6" TYPE="linux_raid_member"
/dev/sdb6: UUID="ab331e81-c714-593c-c407-3f3e22088f8b" UUID_SUB="fc3a348a-9243-9a10-580d-9669f771e9c2" LABEL="localhost.localdomain:3" TYPE="linux_raid_member"
/dev/sdb7: UUID="e6ef64c5-bc3b-dcf9-80dd-1dd292102140" UUID_SUB="be72b5f3-5210-6c6a-8078-4d38207423f5" LABEL="localhost.localdomain:4" TYPE="linux_raid_member"
/dev/sdb8: UUID="a5bf1ee9-904d-7510-9d41-34d86e5d2420" UUID_SUB="31f0aa42-4290-2dd7-9887-28621a205855" LABEL="localhost.localdomain:5" TYPE="linux_raid_member"
/dev/sdb9: UUID="309aafb0-e7fb-05b2-1b68-ef692ae37a2a" UUID_SUB="2ba852f3-156b-5633-1896-a6160f6cfea5" LABEL="localhost.localdomain:7" TYPE="linux_raid_member"
/dev/sda1: UUID="0b26d410-68bf-3813-7439-722b9944c54a" UUID_SUB="e7a528fe-d81b-1880-602f-78c1d0cb96dd" LABEL="localhost.localdomain:2" TYPE="linux_raid_member"
/dev/sda2: UUID="2c222a2b-0319-14d6-ff37-e9de8d67b86d" UUID_SUB="ae524eb7-2625-748c-acd9-2897d932fbeb" LABEL="localhost.localdomain:1" TYPE="linux_raid_member"
/dev/sda3: UUID="038f3aa1-3903-75fb-711f-ce7eb3f93b4a" UUID_SUB="4bfaeff8-6e21-2015-c3a2-9d216f8e46ed" LABEL="localhost.localdomain:0" TYPE="linux_raid_member"
/dev/sda5: UUID="c1366613-ace9-499a-698d-9f0d70ba42c0" UUID_SUB="7db77c08-0ccd-9ddf-a548-8ee9d963c16d" LABEL="localhost.localdomain:6" TYPE="linux_raid_member"
/dev/sda6: UUID="ab331e81-c714-593c-c407-3f3e22088f8b" UUID_SUB="29131f4a-9f8e-7769-86df-8613048bd747" LABEL="localhost.localdomain:3" TYPE="linux_raid_member"
/dev/sda7: UUID="e6ef64c5-bc3b-dcf9-80dd-1dd292102140" UUID_SUB="fd4d1009-468f-14db-9b15-7527a3d44849" LABEL="localhost.localdomain:4" TYPE="linux_raid_member"
/dev/sda8: UUID="a5bf1ee9-904d-7510-9d41-34d86e5d2420" UUID_SUB="445be3e3-bd74-b70d-d06d-3a35b76c8661" LABEL="localhost.localdomain:5" TYPE="linux_raid_member"
/dev/sda9: UUID="309aafb0-e7fb-05b2-1b68-ef692ae37a2a" UUID_SUB="85b5ac0c-5992-cc94-186a-8e03b5d9a17f" LABEL="localhost.localdomain:7" TYPE="linux_raid_member"
/dev/md0: UUID="72cf5870-d603-4d24-899a-976a4290b7f5" TYPE="ext2"
/dev/md1: UUID="e19e1ca8-ea57-44d0-99ca-c116b87b6b29" TYPE="swap"
/dev/md2: UUID="e7959d18-e82b-46a9-a8d4-0ebdf315869e" TYPE="ext4"
/dev/md3: UUID="ccbba718-54ee-437c-9d0d-3ab650c6b2ba" TYPE="ext4"
/dev/md4: UUID="3b2359e1-dc0a-491c-887e-91900e696d5a" TYPE="ext4"
/dev/md5: UUID="4c02597d-cb5b-4620-b55a-3d4c22e87e53" TYPE="ext4"
/dev/md6: UUID="43e63578-26fb-4019-bf46-cbbb05817d6e" TYPE="ext4"
/dev/md7: UUID="439ddd09-b01a-4144-b7f2-7f6ac4f186b6" TYPE="ext4"
== END   blkid ==

== BEGIN cat /proc/mdstat ==
Personalities : [raid1]
md7 : active raid1 sda9[0] sdb9[1]
      77127612 blocks super 1.1 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md6 : active raid1 sda5[0] sdb5[1]
      51198908 blocks super 1.1 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md5 : active raid1 sda8[0] sdb8[1]
      2046968 blocks super 1.1 [2/2] [UU]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md4 : active raid1 sda7[0] sdb7[1]
      5118968 blocks super 1.1 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md3 : active raid1 sda6[0] sdb6[1]
      8190968 blocks super 1.1 [2/2] [UU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md2 : active raid1 sda1[0] sdb1[1]
      10238908 blocks super 1.1 [2/2] [UU]
      bitmap: 1/1 pages [4KB], 65536KB chunk

md1 : active raid1 sda2[0] sdb2[1]
      2046968 blocks super 1.1 [2/2] [UU]
     
md0 : active raid1 sda3[0] sdb3[1]
      307188 blocks super 1.0 [2/2] [UU]
     
unused devices: <none>
== END   cat /proc/mdstat ==



I set up the RAID array during the installation procedure and everything is OK.

The second step for me was to make both members of the RAID1 /boot bootable. For this I followed this tutorial that works perfectly on CentOS 5.x:

http://wiki.centos.org/HowTos/SoftwareRAIDonCentOS5

But saddly this is not the case if I try to do the same thing on CentOS 6.

If I try the method where I set up the GRUB during boot (pres c when the GRUB splashchreen appers), I get the following error:

Code: Select all

grub> root (hd0,0)
grub> setup (hd0)
Error 17: Cannot mount selected partition

grub> root (hd1,0)
grub> setup (hd1)
Error 17: Cannot mount selected partition


If I try the method when I set up GRUB from a running machine, I get the following error:

Code: Select all

grub> device (hd0) /dev/sda
device (hd0) /dev/sda
grub> device (hd1) /dev/sdb
device (hd1) /dev/sdb

grub> root (hd0,0)
root (hd0,0)
 Filesystem type unknown, partition type 0xfd

grub> setup (hd0)
setup (hd0)
Error 1 : Cannot mount selected partition

grub> root (hd1,0)
root (hd1,0)
 Filesystem type unknown, partition type 0xfd

grub> setup (hd1)
setup (hd1)
Error 1 : Cannot mount selected partition

grub> quit
quit


I would appreciate a lot if someone takes the time to help me out with this problem. A big thanks in advance.

dejanst
Posts: 124
Joined: 2011/09/20 14:13:13

Re: Problems with RAID 1 GRUB setup

Postby dejanst » 2011/09/22 17:30:27

If this is a bug that can not be fixed eazily - does anybody know of a different way to get the both members of the RAID1 /boot bootable. Because that is the main thing that is bothering me at the moment.

User avatar
WhatsHisName
Posts: 1537
Joined: 2005/12/19 20:21:43
Location: /earth/usa/nj

[SOLVED] Problems with RAID 1 GRUB setup

Postby WhatsHisName » 2011/09/22 18:03:09

It would help to associate root with the correct partitions.

== BEGIN df -h ==
Filesystem Size Used Avail Use% Mounted on
...
/dev/md0 291M 38M 239M 14% /boot
...

== BEGIN cat /proc/mdstat ==
...
md0 : active raid1 sda3[0] sdb3[1]
307188 blocks super 1.0 [2/2] [UU]
...

The assigned root should be (hd0,2) and (hd1,2), not (hd0,0) and (hd1,0).


You can also "find" the correct partitions. With separate /boot partitions:

Code: Select all

# grub

grub> find /grub/grub.conf
find /grub/grub.conf
 (hd0,2)
 (hd1,2)

grub> quit

dejanst
Posts: 124
Joined: 2011/09/20 14:13:13

Re: Problems with RAID 1 GRUB setup

Postby dejanst » 2011/09/23 13:18:54

WhatsHisName - thank you very much, you were spot on.

It looks like the CentOS 6 installer uses a bit different md device layout than CentOS 5 and that caused a problem for a user that does not know much about GRUB and follows the online tutorials blindly ;)

Thank you very much for all the help. The issue is resolved.

User avatar
WhatsHisName
Posts: 1537
Joined: 2005/12/19 20:21:43
Location: /earth/usa/nj

Re: Problems with RAID 1 GRUB setup

Postby WhatsHisName » 2011/09/23 13:37:53

You are welcome.

The howto should probably include the use of "grub> find ...", but even that will lead to some confusion. You really should do searches for both /boot/grub/grub.conf and /grub/grub.conf (no separete /boot vs. separate /boot partition, respectively) to cover all the bases.

Code: Select all

# grub

grub> find /boot/grub/grub.conf
find /boot/grub/grub.conf
 (hd0,4)
 (hd1,4)

grub> find /grub/grub.conf
find /grub/grub.conf
 (hd0,5)
 (hd0,6)
 (hd0,7)
 (hd1,5)
 (hd1,6)
 (hd1,7)

grub> quit

...for a multiboot system.

You can "find" almost anything present. I think most howtos "find" stage1 or stage2 instead of grub.conf, which is somewhat unique to rpm-based systems (i.e., Debian/Ubuntu have menu.lst (old) or grub.cfg (new)).

User avatar
AlanBartlett
Forum Moderator
Posts: 9311
Joined: 2007/10/22 11:30:09
Location: ~/Earth/UK/England/Suffolk
Contact:

Re: [SOLVED] Problems with RAID 1 GRUB setup

Postby AlanBartlett » 2011/09/23 19:38:49

Thank you for reporting back.

For posterity (and on your behalf), this thread is marked [SOLVED].