diskless booting using ipxe+iscsi hangs with dracut error

General support questions
Post Reply
iamauser
Posts: 24
Joined: 2015/12/10 14:43:48

diskless booting using ipxe+iscsi hangs with dracut error

Post by iamauser » 2017/04/20 16:55:52

Hi Experts,

I am trying to boot a diskless CentOS-7 VM using iPXE from a remote iSCSI target. This is how I am booting from iPXE command line,

Code: Select all

iPXE> sanboot iscsi:10.10.201.235::3260:1:iqn.2017-04.net.timbuktu.vms01:ipxe-centos
The boot hangs midway with the following dracut error. Any suggestion is appreciated.

Code: Select all

[  OK  ] Reached target Paths.
[  OK  ] Reached target Basic System.
[    1.879252] 8139cp 0000:00:03.0 ibft0: link up, 100Mbps, full-duplex, lpa 0x05E1
[    2.005564] tsc: Refined TSC clocksource calibration: 2399.985 MHz
[    4.524074] dracut: FATAL: No handler for netroot type 'iscsi:10.10.201.235::3260:1:iqn.2017-04.net.timbuktu.vms01:ipxe-centos'
[    4.527404] dracut: Refusing to continue
[    4.538742] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
[    4.545350] systemd-journald[97]: Received SIGTERM from PID 1 (systemd-shutdow).
[    4.552599] systemd-shutdown[1]: Sending SIGKILL to remaining processes...
[    4.555682] systemd-shutdown[1]: Unmounting file systems.
[    4.557112] systemd-shutdown[1]: All filesystems unmounted.
[    4.558437] systemd-shutdown[1]: Deactivating swaps.
[    4.559660] systemd-shutdown[1]: All swaps deactivated.
[    4.560910] systemd-shutdown[1]: Detaching loop devices.
[    4.562229] systemd-shutdown[1]: All loop devices detached.
[    4.563520] systemd-shutdown[1]: Detaching DM devices.
[    4.564787] systemd-shutdown[1]: All DM devices detached.
[    4.566254] systemd-shutdown[1]: Halting system.
[    4.572123] System halted.
This is how the kernel line looks like in grub.cfg

Code: Select all

linux16 /vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto nomodeset rd.lvm.lv=centos/root rd.lvm.lv=centos/swap console=tty0 console=ttyS0,115200n8 netroot=iscsi:10.10.201.235::3260:1:iqn.2017-04.net.timbuktu.vms01:ipxe-centos iscsi_initiator=iqn.2017-04.net.timbuktu.vms01:ipxe-centos rd.iscsi.ibft=1 rd.iscsi.waitnet=0 rd.iscsi.testroute=0

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by aks » 2017/04/21 16:09:59

Perhaps your initrd doesn't contain the iscsi software? COnfirm that and if it is rebuild it (just Google for that).

iamauser
Posts: 24
Joined: 2015/12/10 14:43:48

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by iamauser » 2017/04/21 18:13:32

Thanks. I will look into this for sure. Right now, I installed CentOS-7 VM through virt-install locally on the iSCSI server.
Follow up question, does the netroot and initrd setup for a network boot happens automagically when the OS installation is done via iPXE+iSCSI ?

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by aks » 2017/04/24 16:08:45

Am not sure what you are asking. Generally the initrd for install from CD/USB/Network is actually the same. It doesn't really hurt if an (un-used) module is there or not. It only matter if a needed module is not there (at this point in the boot process, the local disk is not available that's the point of initrd - to provide like a local "disk" to load modules from).

iamauser
Posts: 24
Joined: 2015/12/10 14:43:48

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by iamauser » 2017/04/24 20:22:47

I gave up trying to install CentOS on a diskless VM over iPXE+iSCSI. It fails with 'failing to mount /sysroot', when given a iscsi target that is pointing to a raw qemu-kvm image.

Coming back to my original question and error for booting an already installed CentOS-7 using iPXE+iSCSI, I see the following four lines in /sbin/netroot in initramfs-3.10.0-514....img which seems to be causing the error I see:

Code: Select all

    handler=${netroot%%:*}
    handler=${handler%%4}
    handler=$(command -v ${handler}root)
    if [ -z "$netroot" ] || [ ! -e "$handler" ] ; then
        die "No handler for netroot type '$netroot'"
    fi
Given in the command line, netroot=iscsi:10.10.201.235:::1:net.timbuktu.vms01:ipxe-centos, this logic should return handler=iscsiroot, but I don't see any command like that in the OS. Am I reading this correctly ?

iamauser
Posts: 24
Joined: 2015/12/10 14:43:48

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by iamauser » 2017/04/24 21:05:18

I added a few available iscsi modules, named: iscsi_ibft, iscsi_boot_sysfs, iscsi_tcp, iscsi_target_mod into the initramfs and rebuild it. I also installed missing iscsi-initiator package.

This time, I kept grub kernel line simple,

Code: Select all

linux16 ...... root=/dev/mapper/centos-root ro console=tty0 console=ttyS0,115200n8 netroot=iscsi:10.10.201.235:::1:net.timbuktu.vms01:ipxe-centos
This didn't end up in the earlier error 'No Handler', but instead, went into dracut-initqueue timeout. In the midway, it seems to have found the remote FS, but later it kept going for eternity with this message. See the attached screen shot for more info.

Code: Select all

[   ***] A start job is running for dev-mapp...oot.device (xx min yy s/ no limit )
This seems to be related to the bug reported on centos here :
https://bugs.centos.org/view.php?id=10084

Thanks for any suggestion.
Attachments
Screen Shot 2017-04-24 at 3.58.03 PM.png
Screen Shot 2017-04-24 at 3.58.03 PM.png (46.15 KiB) Viewed 5934 times

iamauser
Posts: 24
Joined: 2015/12/10 14:43:48

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by iamauser » 2017/04/25 17:30:50

Update:

It turned out 'add_driver' is not enought, the initramfs was missing the dracut module iscsi.

Added the iscsi module and rebuilt the initramfs

Code: Select all

~]# echo 'add_dracutmodules+="iscsi"' >> /etc/dracut.conf.d/iscsi.conf
~]# cd /boot && dracut -f initramfs-3.10.0-514.10.2.el7.x86_64.img 3.10.0-514.10.2.el7.x86_64
~]# lsinitrd | grep -i iscsi
Now, while trying to boot over iPXE, it asks for 'Enter username' right at the grub menu page. It doesn't even let me edit the grub menuentry.

I have no username or password set for the iscsi target. Any ideas how to get pass this ?

Code: Select all

ACL information says : ALL

aks
Posts: 3073
Joined: 2014/09/20 11:22:14

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by aks » 2017/04/27 18:17:03

I guess you need a an iscsi config file too (it's been a while since I've done iSCSI), if you're prompted for a login, it could be chap (or something similar) is required - check the target. There is/was a "send targets" command to allow a user (initiator) to get a list of available LUNs. That too could be hidden behind authentication. Once again that'll be on the target.

iamauser
Posts: 24
Joined: 2015/12/10 14:43:48

Re: diskless booting using ipxe+iscsi hangs with dracut error

Post by iamauser » 2017/04/27 21:35:21

my iSCSI server somehow entered into a strange state and needed a reboot. Things came back fine. With that addition I can boot fine. This can be marked as solved. Finally for later references,

- If one has installed a minimal CentOS - 7 on a VM and wants to boot from an iSCSI target using iPXE (assuming a functional dhcp server sending root-path to iPXE),

- Make sure the initramfs has iscsi module loaded.

Code: Select all

     lsinitrd | grep iscsi
     
If not, check these packages are there,

Code: Select all

    iscis-initiator-utils
    dracut-network
    
Add iscsi module to dracut and rebuild initramfs

Code: Select all

   echo 'add_dracutmodules+="iscsi"' > /etc/dracut.conf.d/iscsi.conf
   dracut -f initramfs-3.10.....x86_64.img 3.10....x86_64
   lsinitrd | grep iscsi
   
Add netroot to /etc/default/grub and update grub.cfg

Code: Select all

   GRUB_CMDLINE_LINUX="..... netroot=iscsi:<iscsi-server-ip>::::<iscsi server iqn>:<iscsi target name>
   # Rebuild
    grub2-mkconfig -o /boot/grub2/grub.cfg
    grub2-install /dev/sda # whichever is your device
   
Boot should work now.

Post Reply