Hi There
Hope you are all well. We got this weird issue at the moment. Let me explain.
We are on ovirt 4.3.5. We use Centos 7.6 and when we do an update on the VM's the VM's can not boot into multi user mode. It complains that the root mount can't be found. When you boot using ISO and rescue mode you can see the disks. When you chroot into the VM and run pvscan twice your VM can be rebooted and it is fixed. We don't know if this is Centos based or maybe something on ovirt. Screenshots below. Any help would be appreciated.
Should I log a bug request ?
Thanks
Nardus
Inconsistent metadata on VM's VG'ss
Inconsistent metadata on VM's VG'ss
- Attachments
-
- Screenshot from 2019-08-12 14-18-33.png (29.03 KiB) Viewed 1616 times
-
- Screenshot from 2019-08-12 15-10-53.png (25.52 KiB) Viewed 1616 times
-
- Screenshot from 2019-08-11 18-15-14.png (41.66 KiB) Viewed 1617 times
Last edited by nardusg on 2019/08/13 09:53:43, edited 2 times in total.
Re: Inconsistent metadata on VM's VG'ss
So what is updated when you "do an update"? I presume the kernel must be involved? Does it create the initramfs file for it in /boot correctly? Does it create the entry in /boot/grub2/grub.cfg correctly? I'd suggest scrolling back through the console messages to find out the actual source of the problem.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Re: Inconsistent metadata on VM's VG'ss
Yes, all goes fine, no errors that is visible. On reboot it just take long and then dracut times out.
- Attachments
-
- Screenshot from 2019-08-09 21-54-44.png (167.8 KiB) Viewed 1617 times
Re: Inconsistent metadata on VM's VG'ss
Hi
So this seems to be related to lvmetad and the last upgrade that is in the above screenshot. If disable lvmetad in /etc/lvm/lvm.conf and systemctl, I then run pvscan and rebuild using dracut. My VM's don't display inconsistent metadata anymore. Also appear to happen only to more than one disk. Still investigating.
So solutions is for someone that get the same type of issue. Will open a bug if there is more people with the same issue.
Ciao
Nar
So this seems to be related to lvmetad and the last upgrade that is in the above screenshot. If disable lvmetad in /etc/lvm/lvm.conf and systemctl, I then run pvscan and rebuild using dracut. My VM's don't display inconsistent metadata anymore. Also appear to happen only to more than one disk. Still investigating.
So solutions is for someone that get the same type of issue. Will open a bug if there is more people with the same issue.
Ciao
Nar
-
- Posts: 83
- Joined: 2019/08/07 01:50:24
- Location: Perth, Australia but originally from Carshalton, Surrey
Re: Inconsistent metadata on VM's VG'ss
Are the VMs clones? What is your storage?
-
- Posts: 83
- Joined: 2019/08/07 01:50:24
- Location: Perth, Australia but originally from Carshalton, Surrey
Re: Inconsistent metadata on VM's VG'ss
I post my questions because I think you are hitting issues due to this https://blogs.ovirt.org/2017/12/lvm-con ... -easy-way/
The host scans and mounts the volumes of the guest within itself by default. This has caused me corruption and outages, because it is/was not included in the documentation. So I had to find out the hard way.
The fix back then was to configure LVM filter manually with Red Hats help. There is now a CLI that will discover and configure the filter for you. I suggest to use that because after they introduced the tool, my first host that went down for yum update did not boot due to the filter but was fine up until that point which was one or two years.
The host scans and mounts the volumes of the guest within itself by default. This has caused me corruption and outages, because it is/was not included in the documentation. So I had to find out the hard way.
The fix back then was to configure LVM filter manually with Red Hats help. There is now a CLI that will discover and configure the filter for you. I suggest to use that because after they introduced the tool, my first host that went down for yum update did not boot due to the filter but was fine up until that point which was one or two years.
Re: Inconsistent metadata on VM's VG'ss
Thanks, will check it out