how to re-align glibc packages after crash

[paperoga]

Hi all,
I was yum updating my centos7 system while a power outage occurred
As a result, my glibc packages (i686 and x86_64) went misaligned, and now yum complains upon updating:

[root@jlb217 ~]# yum update
Loaded plugins: fastestmirror, langpacks, priorities
Loading mirror speeds from cached hostfile
* base: it.centos.contactlab.it
* epel: pkg.adfinis-sygroup.ch
* extras: it.centos.contactlab.it
* nux-dextop: mirror.li.nux.ro
* updates: it.centos.contactlab.it
Resolving Dependencies
--> Running transaction check
---> Package glibc.i686 0:2.17-260.el7 will be updated
--> Processing Dependency: glibc = 2.17-260.el7 for package: glibc-common-2.17-260.el7.x86_64
---> Package glibc.i686 0:2.17-260.el7_6.5 will be an update
--> Finished Dependency Resolution
Error: Package: glibc-common-2.17-260.el7.x86_64 (@base)
Requires: glibc = 2.17-260.el7
Removing: glibc-2.17-260.el7.i686 (@base)
glibc = 2.17-260.el7
Updated By: glibc-2.17-260.el7_6.5.i686 (updates)
glibc = 2.17-260.el7_6.5
Available: glibc-2.17-260.el7_6.3.i686 (updates)
glibc = 2.17-260.el7_6.3
Available: glibc-2.17-260.el7_6.4.i686 (updates)
glibc = 2.17-260.el7_6.4
You could try using --skip-broken to work around the problem
** Found 2 pre-existing rpmdb problem(s), 'yum check' output follows:
glibc-2.17-260.el7_6.5.x86_64 is a duplicate with glibc-2.17-260.el7.i686
glibc-common-2.17-260.el7_6.5.x86_64 is a duplicate with glibc-common-2.17-260.el7.x86_64

indeed, glibc-common.x86_64 exists as duplicates (as a base version 2.17-260.el7 and the most updated one 2.17-260.el7_6.5 exist), while glibc.i686 is
only available as base version 2.17-260.el7 and glibc.x86_64 as 2.17-260.el7_6.5 updated version:

yum list --showduplicates glibc
Loaded plugins: fastestmirror, langpacks, priorities
Loading mirror speeds from cached hostfile
* base: it.centos.contactlab.it
* epel: pkg.adfinis-sygroup.ch
* extras: it.centos.contactlab.it
* nux-dextop: mirror.li.nux.ro
* updates: it.centos.contactlab.it
Installed Packages
glibc.i686 2.17-260.el7 @base
glibc.x86_64 2.17-260.el7_6.5 installed
Available Packages
glibc.i686 2.17-260.el7 base
glibc.x86_64 2.17-260.el7 base
glibc.i686 2.17-260.el7_6.3 updates
glibc.x86_64 2.17-260.el7_6.3 updates
glibc.i686 2.17-260.el7_6.4 updates
glibc.x86_64 2.17-260.el7_6.4 updates
glibc.i686 2.17-260.el7_6.5 updates
glibc.x86_64 2.17-260.el7_6.5 updates

[root@jlb217 ~]# yum list --showduplicates glibc-common
Loaded plugins: fastestmirror, langpacks, priorities
Loading mirror speeds from cached hostfile
* base: it.centos.contactlab.it
* epel: pkg.adfinis-sygroup.ch
* extras: it.centos.contactlab.it
* nux-dextop: mirror.li.nux.ro
* updates: it.centos.contactlab.it
Installed Packages
glibc-common.x86_64 2.17-260.el7 @base
glibc-common.x86_64 2.17-260.el7_6.5 installed
Available Packages
glibc-common.x86_64 2.17-260.el7 base
glibc-common.x86_64 2.17-260.el7_6.3 updates
glibc-common.x86_64 2.17-260.el7_6.4 updates
glibc-common.x86_64 2.17-260.el7_6.5 updates

If I try to remove any of these packages, there are a lot of dependencies which follow so it doesn't look a good idea, at least as a first attempt.
If I try to update glibc.i686 alone, I end up with the same yum update complaints reported above,
If I try to update bot glibc and glibc-common, transaction checks break, telling that glibc-common already exists:

root@jlb217 ~]# yum update glibc.i686 glibc-common.x86_64
Loaded plugins: fastestmirror, langpacks, priorities
Loading mirror speeds from cached hostfile
* base: it.centos.contactlab.it
* epel: pkg.adfinis-sygroup.ch
* extras: it.centos.contactlab.it
* nux-dextop: mirror.li.nux.ro
* updates: it.centos.contactlab.it
Resolving Dependencies
--> Running transaction check
---> Package glibc.i686 0:2.17-260.el7 will be updated
---> Package glibc.i686 0:2.17-260.el7_6.5 will be an update
---> Package glibc-common.x86_64 0:2.17-260.el7 will be updated
---> Package glibc-common.x86_64 0:2.17-260.el7_6.5 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================================
Package Arch Version Repository Size
====================================================================================================================================================
Updating:
glibc i686 2.17-260.el7_6.5 updates 4.3 M
glibc-common x86_64 2.17-260.el7_6.5 updates 12 M

Transaction Summary
====================================================================================================================================================
Upgrade 2 Packages

Total size: 16 M
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test


Transaction check error:
package glibc-common-2.17-260.el7_6.5.x86_64 is already installed

Error Summary
-------------

I am stuck...
any idea? thanks!

[TrevorH]

You need to remove the newer versions of all duplicates so that a new `yum update` will re-install those packages and make sure they are correct.

Please see https://wiki.centos.org/HowTos/Cleaning ... tePackages for a way to recover. The first 2 paragraphs should be enough to et you going as long as the system has the newer package-cleanup installed (and if it doesn't then update yum-utils).

Do make sure you back up /var/lib/rpm and /var/lib/yum first.

[paperoga]

Thanks for your answer!
Unfortunately, I am not able to remove the latest libc-common-2.17-260.el7_6.5.x86_64:

[root@jlb217 ~]# package-cleanup --cleandupes --removenewestdupes

Loaded plugins: fastestmirror, langpacks, priorities
Not removing glibc-common-2.17-260.el7_6.5.x86_64 because it is required by 1 installed package
No duplicates to remove
Warning: Some duplicates were not removed because they are required by installed packages.
You can try --cleandupes without --removenewestdupes, or review them with --dupes and remove manually.

in fact, if I try to remove the package by hand, it has not just one, but hundreds of dependencies... I am going to erase almost everything! actually, there is no such a risk as at the end I get errors due to the impossiblity to remove "systemd" and "yum", so it stops.. phew...
anyway, I am too scared at this point

Would it be dangerous to try a "rpm remove " with --nodeps? Is there a chance to restore the same package in exactly the same place being all the rest untouched? Is it a reckless operation?

[TrevorH]

Post the output from rpm -qa glibc\* | sort so we can see everything that is installed.

Most likely you will need to use the following on each one of the newer of each of the duplicates, leaving the old one alone:

rpm -e --justdb --nodeps $package

But this is dangerous so do not forget your backups!

[paperoga]

Hello, thanks for all suggestions. This is the output from rpm -qa glibc:

[root@jlb217 ~]# rpm -qa glibc\* | sort
glibc-2.17-260.el7_6.5.x86_64
glibc-2.17-260.el7.i686
glibc-common-2.17-260.el7_6.5.x86_64
glibc-common-2.17-260.el7.x86_64
glibc-devel-2.17-260.el7_6.5.x86_64
glibc-headers-2.17-260.el7_6.5.x86_64

it looks like the only package which was not updated to el7_6.5 is the i686 glibc one. Indeed it would like to get updated, but together with glibc-common-2.17-260.el7_6.5.x86_64... which is already up-to-date, so nothing happens.
Since somehow the system works (as far as I can tell, after the removal of other duplicates which however were not to crucial for the system), what would you think about keeping everything on hold for the moment and waiting for a new version (6.6 or beyond) to appear, and then try again with the update, so that something newer is also found for glibc-common? do you think it could work?
I am really scared of removing glibc-common-2.17-260.el7_6.5.x86_64 (even if without dependences) since all other x86_64 libraries are aligned to this version...

[paperoga]

SOLVED
I waited for the new release of all glibc packages, and updated all of them at a time. No complain, and luckily no blackout this time.