KVM guest does not start - 100% CPU

General support questions
Post Reply
foxb
Posts: 1927
Joined: 2006/04/20 19:03:33
Location: Montreal/QC

KVM guest does not start - 100% CPU

Post by foxb » 2020/05/20 22:38:54

After I've updated my host KVM/QEMU guests using br0 do not start.
Actually start, but are not responsive and use 100% CPU

Guest using virbr0 (DHCP) works

Vibvirt version 4.5.0-33.el7_8.1
qemy-kvm version 1.5.3-173.el7_8.3

I did not see anything obvious in the logs
qemu log

Code: Select all

2020-05-20 22:01:55.970+0000: starting up libvirt version: 4.5.0, package: 33.el7_8.1 (CentOS BuildSystem <http://bugs.centos.org>, 2020-05-12-16:25:35, x86-01.bsys.centos.org), qemu version: 1.5.3 (qemu-kvm-1.5.3-173.el7_8.3), kernel: 3.10.0-1127.8.2.el7.x86_64, hostname: host
LC_ALL=C \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \
QEMU_AUDIO_DRV=none \
/usr/libexec/qemu-kvm \
-name guest \
-S \
-machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off,dump-guest-core=off \
-cpu IvyBridge-IBRS,+md-clear,+spec-ctrl,+ssbd \
-m 4096 \
-realtime mlock=off \
-smp 1,sockets=1,cores=1,threads=1 \
-uuid b930d708-eec8-4ad5-95f8-94c6dae06ddb \
-display none \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-1-guest/monitor.sock,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=delay \
-no-hpet \
-no-shutdown \
-global PIIX4_PM.disable_s3=1 \
-global PIIX4_PM.disable_s4=1 \
-boot strict=on \
-device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x4.0x7 \
-device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x4 \
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x4.0x1 \
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x4.0x2 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 \
-drive file=/var/lib/libvirt/images/guest.qcow2,format=qcow2,if=none,id=drive-virtio-disk0 \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x6,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-drive if=none,id=drive-ide0-0-0,readonly=on \
-device ide-cd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 \
-netdev tap,fd=26,id=hostnet0,vhost=on,vhostfd=28 \
-device virtio-net-pci,netdev=hostnet0,id=net0,mac=xx:xxx:xx:xx,bus=pci.0,addr=0x3 \
-chardev pty,id=charserial0 \
-device isa-serial,chardev=charserial0,id=serial0 \
-chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-1-guest/org.qemu.guest_agent.0,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 \
-object rng-random,id=objrng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x8 \
-msg timestamp=on
char device redirected to /dev/pts/2 (label charserial0)
qemu: terminating on signal 15 from pid 2406
2020-05-20 22:04:34.902+0000: shutting down, reason=shutdown
messages log

Code: Select all

May 20 18:01:56 host systemd-machined: New machine qemu-1-guest.
May 20 18:01:56 host systemd: Started Virtual Machine qemu-1-guest.
May 20 18:01:56 host kvm: 1 guest now active
May 20 18:04:34 host kernel: br0: port 2(vnet0) entered disabled state
May 20 18:04:34 host kernel: device vnet0 left promiscuous mode
May 20 18:04:34 host kernel: br0: port 2(vnet0) entered disabled state
May 20 18:04:34 host libvirtd: 2020-05-20 22:04:34.689+0000: 1166: info : libvirt version: 4.5.0, package: 33.el7_8.1 (CentOS BuildSystem <http://bugs.centos.org>, 2020-05-12-16:25:35, x86-01.bsys.centos.org)
May 20 18:04:34 host libvirtd: 2020-05-20 22:04:34.689+0000: 1166: info : hostname: host
May 20 18:04:34 host libvirtd: 2020-05-20 22:04:34.689+0000: 1166: error : qemuMonitorIO:718 : internal error: End of file from qemu monitor
May 20 18:04:34 host kvm: 0 guests now active
May 20 18:04:34 host systemd-machined: Machine qemu-1-guest terminated.
my last action was to kill qemu process of the guest.

Any ideas where to look?

foxb
Posts: 1927
Joined: 2006/04/20 19:03:33
Location: Montreal/QC

Re: KVM guest does not start - 100% CPU

Post by foxb » 2020/05/22 13:54:47

Looking further I've enavled VNC console.

It looks something went wrong and the guest is stuck in

Code: Select all

 grub>
Trying to boot manually fails on

Code: Select all

dracut-initqueue timeout

foxb
Posts: 1927
Joined: 2006/04/20 19:03:33
Location: Montreal/QC

Re: KVM guest does not start - 100% CPU

Post by foxb » 2020/05/22 17:47:51

Next update

I was able to boot in rescue mode and install grub.
Noticed that /boot/grub2/grub.cfg is empty and recreated it.

Direct boot did not work.

Only way I was ablr to get it work is to boot in anaconda and quit install.

Not ideal, but will do the job until fix is discovered.

Post Reply