grub2-install on nvme (not UEFI), first 512 bytes different on 2 disks

General support questions
Post Reply
antits
Posts: 1
Joined: 2018/10/02 10:44:49

grub2-install on nvme (not UEFI), first 512 bytes different on 2 disks

Post by antits » 2018/10/02 11:12:57

Hello,

on a system with 2 same-capacity nvme drives,one of the disks was replaced.
The drives are part on mdadm RAID-1. No LVM in use.

After the drive replacement, I did copy the partition table using sfdisk from one disk to the other and run grub2-install on both.

fdisk reports the partition table on both drives as expected.

I run:

Code: Select all

grub2-install /dev/nvme0n1
grub2-install /dev/nvme1n1
but the first 512 written are quite different after that.

It is the first time I see this.
I have done it many times with non-nvme disks and the first 512 bytes were the same (with only difference the disk identifier bytes 1B8, 1B9, 1BA, 1BB).


Below I show some extra info.

Code: Select all

# file -s /dev/nvme0n1
/dev/nvme0n1: x86 boot sector; partition 1: ID=0xfd, starthead 32, startsector
2048, 16777216 sectors; partition 2: ID=0xfd, starthead 254, startsector 16779264,
1310720 sectors; partition 3: ID=0xfd, starthead 254, startsector 18089984,
33554432 sectors; partition 4: ID=0xf, starthead 254, startsector 51644416,
948570800 sectors, code offset 0x63

# file -s /dev/nvme1n1
/dev/nvme1n1: x86 boot sector; partition 1: ID=0xfd, starthead 32, startsector
2048, 16777216 sectors; partition 2: ID=0xfd, starthead 254, startsector 16779264,
1310720 sectors; partition 3: ID=0xfd, starthead 254, startsector 18089984,
33554432 sectors; partition 4: ID=0xf, starthead 254, startsector 51644416,
948570800 sectors, code offset 0x63

Code: Select all

# fdisk /dev/nvme1n1
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): u
Changing display/entry units to cylinders (DEPRECATED!).

Command (m for help): p

Disk /dev/nvme1n1: 512.1 GB, 512110190592 bytes, 1000215216 sectors
Units = cylinders of 2048 * 512 = 1048576 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x4d9789da

         Device Boot      Start         End      Blocks   Id  System
/dev/nvme1n1p1               2        8193     8388608   fd  Linux raid autodetect
/dev/nvme1n1p2            8194        8833      655360   fd  Linux raid autodetect
/dev/nvme1n1p3            8834       25217    16777216   fd  Linux raid autodetect
/dev/nvme1n1p4           25218      488387   474285400    f  W95 Ext'd (LBA)
/dev/nvme1n1p5           25219      488386   474283352   fd  Linux raid autodetect


# fdisk /dev/nvme0n1
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): u
Changing display/entry units to cylinders (DEPRECATED!).

Command (m for help): p

Disk /dev/nvme0n1: 512.1 GB, 512110190592 bytes, 1000215216 sectors
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xaf7a16ad

         Device Boot      Start         End      Blocks   Id  System
/dev/nvme0n1p1               1        1045     8388608   fd  Linux raid autodetect
/dev/nvme0n1p2            1045        1127      655360   fd  Linux raid autodetect
/dev/nvme0n1p3            1127        3215    16777216   fd  Linux raid autodetect
/dev/nvme0n1p4            3215       62261   474285400    f  W95 Ext'd (LBA)
/dev/nvme0n1p5            3215       62261   474283352   fd  Linux raid autodetect 

Code: Select all

#  dd if=/dev/nvme0n1 bs=512 count=1 status=noxfer |hexdump -C
1+0 records in
1+0 records out
00000000  eb 63 90 10 8e d0 bc 00  b0 b8 00 00 8e d8 8e c0 |.c..............|
00000010  fb be 00 7c bf 00 06 b9  00 02 f3 a4 ea 21 06 00 |...|.........!..|
00000020  00 be be 07 38 04 75 0b  83 c6 10 81 fe fe 07 75 |....8.u........u|
00000030  f3 eb 16 b4 02 b0 01 bb  00 7c b2 80 8a 74 01 8b |.........|...t..|
00000040  4c 02 cd 13 ea 00 7c 00  00 eb fe 00 00 00 00 00 |L.....|.........|
00000050  00 00 00 00 00 00 00 00  00 00 00 80 01 00 00 00 |................|
00000060  00 00 00 00 ff fa 90 90  f6 c2 80 74 05 f6 c2 70 |...........t...p|
00000070  74 02 b2 80 ea 79 7c 00  00 31 c0 8e d8 8e d0 bc |t....y|..1......|
00000080  00 20 fb a0 64 7c 3c ff  74 02 88 c2 52 be 05 7c  |. ..d|<.t...R..||
00000090  b4 41 bb aa 55 cd 13 5a  52 72 3d 81 fb 55 aa 75 |.A..U..ZRr=..U.u|
000000a0  37 83 e1 01 74 32 31 c0  89 44 04 40 88 44 ff 89 |7...t21..D.@.D..|
000000b0  44 02 c7 04 10 00 66 8b  1e 5c 7c 66 89 5c 08 66 |D.....f..\|f.\.f|
000000c0  8b 1e 60 7c 66 89 5c 0c  c7 44 06 00 70 b4 42 cd |..`|f.\..D..p.B.|
000000d0  13 72 05 bb 00 70 eb 76  b4 08 cd 13 73 0d 5a 84 |.r...p.v....s.Z.|
000000e0  d2 0f 83 de 00 be 85 7d  e9 82 00 66 0f b6 c6 88 |.......}...f....|
000000f0  64 ff 40 66 89 44 04 0f  b6 d1 c1 e2 02 88 e8 88 |d.@f.D..........|
00000100  f4 40 89 44 08 0f b6 c2  c0 e8 02 66 89 04 66 a1 |.@.D.......f..f.|
00000110  60 7c 66 09 c0 75 4e 66  a1 5c 7c 66 31 d2 66 f7 |`|f..uNf.\|f1.f.|
00000120  34 88 d1 31 d2 66 f7 74  04 3b 44 08 7d 37 fe c1 |4..1.f.t.;D.}7..|
00000130  88 c5 30 c0 c1 e8 02 08  c1 88 d0 5a 88 c6 bb 00 |..0........Z....|
00000140  70 8e c3 31 db b8 01 02  cd 13 72 1e 8c c3 60 1e |p..1......r...`.|
00000150  b9 00 01 8e db 31 f6 bf  00 80 8e c6 fc f3 a5 1f |.....1..........|
00000160  61 ff 26 5a 7c be 80 7d  eb 03 be 8f 7d e8 34 00 |a.&Z|..}....}.4.|
00000170  be 94 7d e8 2e 00 cd 18  eb fe 47 52 55 42 20 00 |..}.......GRUB .|
00000180  47 65 6f 6d 00 48 61 72  64 20 44 69 73 6b 00 52 |Geom.Hard Disk.R|
00000190  65 61 64 00 20 45 72 72  6f 72 0d 0a 00 bb 01 00  |ead. Error......|
000001a0  b4 0e cd 10 ac 3c 00 75  f4 c3 00 00 00 00 00 00 |.....<.u........|
000001b0  00 00 00 00 00 00 00 00  ad 16 7a af 00 00 00 20 |..........z.... |
000001c0  21 00 fd fe ff ff 00 08  00 00 00 00 00 01 00 fe |!...............|
000001d0  ff ff fd fe ff ff 00 08  00 01 00 00 14 00 00 fe |................|
000001e0  ff ff fd fe ff ff 00 08  14 01 00 00 00 02 00 fe |................|
000001f0  ff ff 0f fe ff ff 00 08  14 03 b0 0a 8a 38 55 aa |.............8U.|
00000200


* New disk is having many zeros up to 00000050 and at the end (the partition table) is so different than the dump above:

Code: Select all

#  dd if=/dev/nvme1n1 bs=512 count=1 status=noxfer |hexdump -C
1+0 records in
1+0 records out
00000000  eb 63 90 00 00 00 00 00  00 00 00 00 00 00 00 00 |.c..............|
00000010  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 |................|
*
00000050  00 00 00 00 00 00 00 00  00 00 00 80 01 00 00 00 |................|
00000060  00 00 00 00 ff fa 90 90  f6 c2 80 74 05 f6 c2 70 |...........t...p|
00000070  74 02 b2 80 ea 79 7c 00  00 31 c0 8e d8 8e d0 bc |t....y|..1......|
00000080  00 20 fb a0 64 7c 3c ff  74 02 88 c2 52 be 05 7c  |. ..d|<.t...R..||
00000090  b4 41 bb aa 55 cd 13 5a  52 72 3d 81 fb 55 aa 75 |.A..U..ZRr=..U.u|
000000a0  37 83 e1 01 74 32 31 c0  89 44 04 40 88 44 ff 89 |7...t21..D.@.D..|
000000b0  44 02 c7 04 10 00 66 8b  1e 5c 7c 66 89 5c 08 66 |D.....f..\|f.\.f|
000000c0  8b 1e 60 7c 66 89 5c 0c  c7 44 06 00 70 b4 42 cd |..`|f.\..D..p.B.|
000000d0  13 72 05 bb 00 70 eb 76  b4 08 cd 13 73 0d 5a 84 |.r...p.v....s.Z.|
000000e0  d2 0f 83 de 00 be 85 7d  e9 82 00 66 0f b6 c6 88 |.......}...f....|
000000f0  64 ff 40 66 89 44 04 0f  b6 d1 c1 e2 02 88 e8 88 |d.@f.D..........|
00000100  f4 40 89 44 08 0f b6 c2  c0 e8 02 66 89 04 66 a1 |.@.D.......f..f.|
00000110  60 7c 66 09 c0 75 4e 66  a1 5c 7c 66 31 d2 66 f7 |`|f..uNf.\|f1.f.|
00000120  34 88 d1 31 d2 66 f7 74  04 3b 44 08 7d 37 fe c1 |4..1.f.t.;D.}7..|
00000130  88 c5 30 c0 c1 e8 02 08  c1 88 d0 5a 88 c6 bb 00 |..0........Z....|
00000140  70 8e c3 31 db b8 01 02  cd 13 72 1e 8c c3 60 1e |p..1......r...`.|
00000150  b9 00 01 8e db 31 f6 bf  00 80 8e c6 fc f3 a5 1f |.....1..........|
00000160  61 ff 26 5a 7c be 80 7d  eb 03 be 8f 7d e8 34 00 |a.&Z|..}....}.4.|
00000170  be 94 7d e8 2e 00 cd 18  eb fe 47 52 55 42 20 00 |..}.......GRUB .|
00000180  47 65 6f 6d 00 48 61 72  64 20 44 69 73 6b 00 52 |Geom.Hard Disk.R|
00000190  65 61 64 00 20 45 72 72  6f 72 0d 0a 00 bb 01 00  |ead. Error......|
000001a0  b4 0e cd 10 ac 3c 00 75  f4 c3 00 00 00 00 00 00 |.....<.u........|
000001b0  00 00 00 00 00 00 00 00  da 89 97 4d 00 00 00 00 |...........M....|
000001c0  01 01 fd 3f e0 ff 00 08  00 00 00 00 00 01 00 3f |...?...........?|
000001d0  e0 ff fd 3f e0 ff 00 08  00 01 00 00 14 00 00 3f |...?...........?|
000001e0  e0 ff fd 3f e0 ff 00 08  14 01 00 00 00 02 00 3f |...?...........?|
000001f0  e0 ff 0f 3f e0 ff 00 08  14 03 b0 0a 8a 38 55 aa |...?.........8U.|
00000200 

Code: Select all

# fdisk -l /dev/nvme0n1

Disk /dev/nvme0n1: 512.1 GB, 512110190592 bytes, 1000215216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xaf7a16ad

        Device Boot      Start         End      Blocks   Id  System
/dev/nvme0n1p1            2048    16779263     8388608   fd  Linux raid autodetect
/dev/nvme0n1p2        16779264    18089983      655360   fd  Linux raid autodetect
/dev/nvme0n1p3        18089984    51644415    16777216   fd  Linux raid autodetect
/dev/nvme0n1p4        51644416  1000215215   474285400    f  W95 Ext'd (LBA)
/dev/nvme0n1p5        51646464  1000213167   474283352   fd  Linux raid autodetect
# fdisk -l /dev/nvme1n1

Disk /dev/nvme1n1: 512.1 GB, 512110190592 bytes, 1000215216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x4d9789da

        Device Boot      Start         End      Blocks   Id  System
/dev/nvme1n1p1            2048    16779263     8388608   fd  Linux raid autodetect
/dev/nvme1n1p2        16779264    18089983      655360   fd  Linux raid autodetect
/dev/nvme1n1p3        18089984    51644415    16777216   fd  Linux raid autodetect
/dev/nvme1n1p4        51644416  1000215215   474285400    f  W95 Ext'd (LBA)
/dev/nvme1n1p5        51646464  1000213167   474283352   fd  Linux raid autodetect

Code: Select all

#  mdadm -Evvs
/dev/nvme1n1p5:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 766dbafc:83f21e38:d0e0e375:cbb2cec5
           Name : rescue:3
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 948304560 (452.19 GiB 485.53 GB)
     Array Size : 474152256 (452.19 GiB 485.53 GB)
  Used Dev Size : 948304512 (452.19 GiB 485.53 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262064 sectors, after=48 sectors
          State : clean
    Device UUID : 31ffa102:90578852:c8898aaa:14210774

Internal Bitmap : 8 sectors from superblock
    Update Time : Tue Oct  2 13:11:19 2018
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 4c4cbd81 - correct
         Events : 25125


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme1n1p4:
   MBR Magic : aa55
Partition[0] :    948566704 sectors at         2048 (type fd)
/dev/nvme1n1p3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 64c3281f:3679a41e:5326edd1:1184d9a1
           Name : rescue:2
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 33521664 (15.98 GiB 17.16 GB)
     Array Size : 16760832 (15.98 GiB 17.16 GB)
    Data Offset : 32768 sectors
   Super Offset : 8 sectors
   Unused Space : before=32616 sectors, after=0 sectors
          State : clean
    Device UUID : bbefa52f:c2133ff2:172b82a1:a4be2e6c

    Update Time : Tue Oct  2 13:11:16 2018
  Bad Block Log : 512 entries available at offset 136 sectors
       Checksum : db1dd203 - correct
         Events : 3175


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme1n1p2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 74db73d4:0984ad60:05c3e608:996208dd
           Name : rescue:1
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 1309664 (639.48 MiB 670.55 MB)
     Array Size : 654784 (639.44 MiB 670.50 MB)
  Used Dev Size : 1309568 (639.44 MiB 670.50 MB)
    Data Offset : 1056 sectors
   Super Offset : 8 sectors
   Unused Space : before=976 sectors, after=96 sectors
          State : clean
    Device UUID : 14c97cf0:9ea17862:77d7ccae:5169a07f

    Update Time : Tue Oct  2 11:10:22 2018
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 9192814f - correct
         Events : 164


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme1n1p1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : b03146da:3928c165:ec53c4f3:83051fff
           Name : rescue:0
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 16760832 (7.99 GiB 8.58 GB)
     Array Size : 8380416 (7.99 GiB 8.58 GB)
    Data Offset : 16384 sectors
   Super Offset : 8 sectors
   Unused Space : before=16304 sectors, after=0 sectors
          State : clean
    Device UUID : b54c886b:8ee38475:0c55c812:d016c055

    Update Time : Tue Oct  2 04:16:27 2018
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 7277037e - correct
         Events : 116


   Device Role : Active device 1
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme1n1:
   MBR Magic : aa55
Partition[0] :     16777216 sectors at         2048 (type fd)
Partition[1] :      1310720 sectors at     16779264 (type fd)
Partition[2] :     33554432 sectors at     18089984 (type fd)
Partition[3] :    948570800 sectors at     51644416 (type 0f)
mdadm: No md superblock detected on /dev/md/0.
mdadm: No md superblock detected on /dev/md/3.
mdadm: No md superblock detected on /dev/md/2.
mdadm: No md superblock detected on /dev/md/1.
/dev/nvme0n1p5:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x1
     Array UUID : 766dbafc:83f21e38:d0e0e375:cbb2cec5
           Name : rescue:3
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 948304560 (452.19 GiB 485.53 GB)
     Array Size : 474152256 (452.19 GiB 485.53 GB)
  Used Dev Size : 948304512 (452.19 GiB 485.53 GB)
    Data Offset : 262144 sectors
   Super Offset : 8 sectors
   Unused Space : before=262056 sectors, after=48 sectors
          State : clean
    Device UUID : eb3522cc:b72b4343:303da9bd:4cd553d0

Internal Bitmap : 8 sectors from superblock
    Update Time : Tue Oct  2 13:11:19 2018
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 75f33039 - correct
         Events : 25125


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme0n1p4:
   MBR Magic : aa55
Partition[0] :    948566704 sectors at         2048 (type fd)
/dev/nvme0n1p3:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 64c3281f:3679a41e:5326edd1:1184d9a1
           Name : rescue:2
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 33521664 (15.98 GiB 17.16 GB)
     Array Size : 16760832 (15.98 GiB 17.16 GB)
    Data Offset : 32768 sectors
   Super Offset : 8 sectors
   Unused Space : before=32680 sectors, after=0 sectors
          State : clean
    Device UUID : 0ad2b3a7:3290bde4:eccf1900:715dbb55

    Update Time : Tue Oct  2 13:11:16 2018
  Bad Block Log : 512 entries available at offset 72 sectors
       Checksum : 8dce7422 - correct
         Events : 3175


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme0n1p2:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : 74db73d4:0984ad60:05c3e608:996208dd
           Name : rescue:1
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 1309664 (639.48 MiB 670.55 MB)
     Array Size : 654784 (639.44 MiB 670.50 MB)
  Used Dev Size : 1309568 (639.44 MiB 670.50 MB)
    Data Offset : 1056 sectors
   Super Offset : 8 sectors
   Unused Space : before=976 sectors, after=96 sectors
          State : clean
    Device UUID : af89a9cf:f0e5df4c:fd1ee8f9:deb664fc

    Update Time : Tue Oct  2 11:10:22 2018
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 23061b51 - correct
         Events : 164


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme0n1p1:
          Magic : a92b4efc
        Version : 1.2
    Feature Map : 0x0
     Array UUID : b03146da:3928c165:ec53c4f3:83051fff
           Name : rescue:0
  Creation Time : Wed Sep 19 10:09:51 2018
     Raid Level : raid1
   Raid Devices : 2

 Avail Dev Size : 16760832 (7.99 GiB 8.58 GB)
     Array Size : 8380416 (7.99 GiB 8.58 GB)
    Data Offset : 16384 sectors
   Super Offset : 8 sectors
   Unused Space : before=16304 sectors, after=0 sectors
          State : clean
    Device UUID : 66d10326:ede26db3:3da7d573:3e24cab9

    Update Time : Tue Oct  2 04:16:27 2018
  Bad Block Log : 512 entries available at offset 16 sectors
       Checksum : 2ff2e72f - correct
         Events : 116


   Device Role : Active device 0
   Array State : AA ('A' == active, '.' == missing, 'R' == replacing)
/dev/nvme0n1:
   MBR Magic : aa55
Partition[0] :     16777216 sectors at         2048 (type fd)
Partition[1] :      1310720 sectors at     16779264 (type fd)
Partition[2] :     33554432 sectors at     18089984 (type fd)
Partition[3] :    948570800 sectors at     51644416 (type 0f)

Is this a bug with nvme disks? Am I forgeting anything?


Really thank you in advance for any help.

Post Reply