[RESOLVED] Raid 5 Linux Software Raid incorrectly assembled at boot

[raphk]

Hi All - Hope someone can help.

The plan was to move from Centos 5.6 to 6, with a brand new system drive (Centos 5.6 Install still intact and bootable), and 'import' a fully functional 4 disk linux software raid 5 to CentOS 6.
Working Raid 5 in CentOS 5.6 info:
[code]
/dev/md0:
Version : 0.90
Creation Time : Tue Sep 28 11:06:44 2010
Raid Level : raid5
Array Size : 5860535808 (5589.04 GiB 6001.19 GB)
Used Dev Size : 1953511936 (1863.01 GiB 2000.40 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Sun Jul 24 16:54:12 2011
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 256K


UUID : xxxxx
Events : 0.21655

Number Major Minor RaidDevice State
0 8 97 0 active sync /dev/sdg1
1 8 81 1 active sync /dev/sdf1
2 8 65 2 active sync /dev/sde1
4 8 49 3 active sync /dev/sdd1
[/code]

If i boot into CentOS with the 4 Raid 5 drives plugged in, CentOS 6 assembles the software raid 5 volume as degraded. The odd thing is it removes one of the drives as active sync "/dev/sdd1" and labels it as a "spare",
In CentOS 6, issuing mdadm --detail --scan --verbose displays:
[code]
ARRAY /dev/md0 level=raid5 num-devices=4 metadata=0.90 spares=1 UUID=xxxxx
devices=/dev/sdg1,/dev/sdf1,/dev/sde1,/dev/sdd1,[b]/dev/sdd[/b]
[/code]
for some reason it's displaying both "/dev/sdd" & "/dev/sdd1" as raid members ... which makes no sense to me

If i force the raid 5 assemble in CentOS6:
[code]
mdadm --assemble --run --force /dev/md0 /dev/sd[defg]1
[/code]
everything comes up fine; shows 4 active devices & all data is accessible. If i reboot CentOS 6, it auto assembles the raid, & we're back to square 1 (degraded volume, where 1 of the drives is set as a spare and not "active sync").

I tried to remove this phantom "/dev/sdd" from the array, then re added "/dev/sdd1", which forced a rebuild. I let it rebuild over night, hoping this would fix it - but no, rebooting CentOS 6 again causes the raid to be detected incorrectly.

If I boot into Centos 5.6 again, the raid 5 volume is detected with no issues.

any ideas on why this is happening? what can I do to fix this? any help would be greatly appreciated

[raphk]

still having this problem.. any idea why /dev/sdg AND /dev/sdg1 are showing up here as part of my software raid?
[code]

[root@masterworld ~]# mdadm --examine --scan --verbose
ARRAY /dev/md0 level=raid5 num-devices=4 UUID=xxxx
spares=1 devices=[size=120]dev/sdg1,/dev/sdg[/size],/dev/sdf1,/dev/sde1,/dev/sdd1
[/code]


I have no spares even though it says there is 1 above..

[code]

# mdadm --detail /dev/md0
/dev/md0:
Version : 0.90
Creation Time : Tue Sep 28 11:06:44 2010
Raid Level : raid5
Array Size : 5860535808 (5589.04 GiB 6001.19 GB)
Used Dev Size : 1953511936 (1863.01 GiB 2000.40 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0
Persistence : Superblock is persistent

Update Time : Mon Jul 25 09:18:05 2011
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0

Layout : left-symmetric
Chunk Size : 256K

UUID : xxxx
Events : 0.22525

Number Major Minor RaidDevice State
0 8 97 0 active sync /dev/sdg1
1 8 81 1 active sync /dev/sdf1
2 8 65 2 active sync /dev/sde1
3 8 49 3 active sync /dev/sdd1
[/code]

after examining all the raid members, i noticed:

[code]
mdadm --examine /dev/sdg
/dev/sdg:
Magic : a92b4efc
Version : 0.90.00
UUID : 41c31797:3a18fc36:7973c7dd:f58d82e7
Creation Time : Tue Sep 28 11:06:44 2010
Raid Level : raid5
Used Dev Size : 1953511936 (1863.01 GiB 2000.40 GB)
Array Size : 5860535808 (5589.04 GiB 6001.19 GB)
Raid Devices : 4
Total Devices : 5
Preferred Minor : 0

Update Time : Wed Mar 30 15:49:07 2011
State : clean
Active Devices : 4
Working Devices : 5
Failed Devices : 0
Spare Devices : 1
Checksum : a2b31834 - correct
Events : 20584

Layout : left-symmetric
Chunk Size : 256K

Number Major Minor RaidDevice State
this 4 8 80 4 spare /dev/sdf

0 0 8 65 0 active sync /dev/sde1
1 1 8 17 1 active sync /dev/sdb1
2 2 8 49 2 active sync /dev/sdd1
3 3 8 33 3 active sync /dev/sdc1
4 4 8 80 4 spare /dev/sdf[/code]

[code]

# mdadm --examine /dev/sdg1
/dev/sdg1:
Magic : a92b4efc
Version : 0.90.00
UUID : 41c31797:3a18fc36:7973c7dd:f58d82e7
Creation Time : Tue Sep 28 11:06:44 2010
Raid Level : raid5
Used Dev Size : 1953511936 (1863.01 GiB 2000.40 GB)
Array Size : 5860535808 (5589.04 GiB 6001.19 GB)
Raid Devices : 4
Total Devices : 4
Preferred Minor : 0

Update Time : Mon Jul 25 09:19:33 2011
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Checksum : a34d0bbc - correct
Events : 22525

Layout : left-symmetric
Chunk Size : 256K

Number Major Minor RaidDevice State
this 0 8 97 0 active sync /dev/sdg1

0 0 8 97 0 active sync /dev/sdg1
1 1 8 81 1 active sync /dev/sdf1
2 2 8 65 2 active sync /dev/sde1
3 3 8 49 3 active sync /dev/sdd1
[/code]

both /dev/sdg & /dev/sdg1 have raid information for some reason - how do I clear the information in /dev/sdg so it is not detected as part of the raid (it should be /dev/sdg1)?

[raphk]

well this has been resolved, removing the linux software raid data from /dev/sdg fixed this issue.