Kernel not upgraded to 3.10.0-957.1.3.el7.x86_64 after upgrading to CentOS 7.6.1810

General support questions
Post Reply
soonyee
Posts: 3
Joined: 2018/12/17 07:08:24
Location: Asia/Singapore

Kernel not upgraded to 3.10.0-957.1.3.el7.x86_64 after upgrading to CentOS 7.6.1810

Post by soonyee » 2018/12/17 08:09:04

I just upgraded 2 CentOS 7.5 servers (VMs running on VMware ESXi 5.5) to CentOS 7.6.1810 over the weekend (Dec 15) but one of the servers' kernel was not upgraded to the latest kernel 3.10.0-957.1.3.el7.x86_64 after the upgrade. It still stays at 3.10.0-862.14.4.el7.x86_64.
I am puzzled as I upgraded both servers in exactly the same way.
I tried redoing the upgrade (after reverting to a previous snapshot) thinking I might have done something wrong but still had no luck.

Both servers have SELinux in enforcing mode and the only message i get in the server having problem in /var/log/audit/audit.log is
type=AVC msg=audit(1544845370.261:321): avc: denied { connectto } for pid=2593 comm="plymouth" path=002F6F72672F667265656465736B746F702F706C796D6F75746864 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:system_r:plymouthd_t:s0 tclass=unix_stream_socket

I don't see why plymouth being denied would cause kernel upgrade to fail, nor did I see any error messages during the upgrade process.

This is the getinfo.sh output for the server having problem (still at older kernel 3.10.0-862.14.4.el7.x86_64 :
basedata-web-proxy
https://paste.centos.org/view/embed/31f50617

----------------------------------------------------------------------------------------------------------------------------------------

This is the getinfo.sh output for the other server in which the kernel was upgraded to the latest 3.10.0-957.1.3.el7.x86_64 :
basedata-apigateway
https://paste.centos.org/view/embed/ed45b409

I have tried google search on the web and in this forum but so far haven't found anything of help.
Any help would be appreciated. Thanks.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Kernel not upgraded to 3.10.0-957.1.3.el7.x86_64 after upgrading to CentOS 7.6.1810

Post by TrevorH » 2018/12/17 08:28:31

== BEGIN rpm -qa kernel\* | sort ==
== END rpm -qa kernel\* | sort ==
If that's right then that would certainly explain it. Can you run that command manually and confirm it's correct that there are no packages that match? rpm -qa kernel\*

If the output is empty, does just plain rpm -qa return lots of output (like 100's of lines)? Or is it (more or less) empty?
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

soonyee
Posts: 3
Joined: 2018/12/17 07:08:24
Location: Asia/Singapore

Re: Kernel not upgraded to 3.10.0-957.1.3.el7.x86_64 after upgrading to CentOS 7.6.1810

Post by soonyee » 2018/12/18 01:53:07

[root@web-proxy ~]#rpm -qa kernel\*
The above command returns nothing

For rpm -qa :

root@web-proxy ~]# rpm -qa
chkconfig-1.7.4-1.el7.x86_64
basesystem-10.0-7.el7.centos.noarch
libattr-2.4.46-13.el7.x86_64
info-5.1-5.el7.x86_64
p11-kit-0.23.5-3.el7.x86_64
libcap-2.22-9.el7.x86_64
gmp-6.0.0-15.el7.x86_64
popt-1.13-16.el7.x86_64
libtasn1-4.10-1.el7.x86_64
libgcrypt-1.5.3-14.el7.x86_64
libcap-ng-0.7.5-4.el7.x86_64
aide-0.15.1-13.el7.x86_64
nss-util-3.36.0-1.el7_5.x86_64
ca-certificates-2018.2.22-70.0.el7_5.noarch
tcp_wrappers-libs-7.6-77.el7.x86_64
libgcc-4.8.5-36.el7.x86_64
glibc-common-2.17-260.el7.x86_64
bash-4.2.46-31.el7.x86_64
libselinux-2.5-14.1.el7.x86_64
zlib-1.2.7-18.el7.x86_64
openssl-libs-1.0.2k-16.el7.x86_64
centos-release-7-6.1810.2.el7.centos.x86_64
elfutils-libelf-0.172-2.el7.x86_64
audit-libs-2.8.4-4.el7.x86_64
gpg-pubkey-f4a80eb5-53a7ff4b
ncurses-base-5.9-14.20130511.el7_4.noarch
ncurses-5.9-14.20130511.el7_4.x86_64
filesystem-3.2-25.el7.x86_64
ncurses-libs-5.9-14.20130511.el7_4.x86_64
pcre-8.32-17.el7.x86_64
grep-2.20-3.el7.x86_64
libffi-3.0.13-18.el7.x86_64
sed-4.2.2-5.el7.x86_64
gawk-4.0.2-4.el7_3.1.x86_64
libacl-2.2.51-14.el7.x86_64
keyutils-libs-1.5.8-3.el7.x86_64
libverto-0.2.5-4.el7.x86_64
p11-kit-trust-0.23.5-3.el7.x86_64
libgpg-error-1.12-3.el7.x86_64
nspr-4.19.0-1.el7_5.x86_64
nss-softokn-freebl-3.36.0-5.el7_5.x86_64
tcp_wrappers-7.6-77.el7.x86_64
tzdata-2018g-1.el7.noarch
glibc-2.17-260.el7.x86_64
libsepol-2.5-10.el7.x86_64
libcom_err-1.42.9-13.el7.x86_64
coreutils-8.22-23.el7.x86_64
krb5-libs-1.15.1-34.el7.x86_64
setup-2.8.71-10.el7.noarch
e2fsprogs-libs-1.42.9-13.el7.x86_64
libstdc++-4.8.5-36.el7.x86_64


kernel 3.10.0-957.1.3.el7.x86_64 not found in /boot :

[root@web-proxy boot]# ls /boot
config-3.10.0-693.21.1.el7.x86_64
config-3.10.0-862.11.6.el7.x86_64
config-3.10.0-862.14.4.el7.x86_64
config-3.10.0-862.3.2.el7.x86_64
config-3.10.0-862.9.1.el7.x86_64
efi
grub
grub2
initramfs-0-rescue-b0cbe21d63111d44b76fa95a2b474506.img
initramfs-3.10.0-693.21.1.el7.x86_64.img
initramfs-3.10.0-693.21.1.el7.x86_64kdump.img
initramfs-3.10.0-862.11.6.el7.x86_64.img
initramfs-3.10.0-862.11.6.el7.x86_64kdump.img
initramfs-3.10.0-862.14.4.el7.x86_64.img
initramfs-3.10.0-862.14.4.el7.x86_64kdump.img
initramfs-3.10.0-862.3.2.el7.x86_64.img
initramfs-3.10.0-862.3.2.el7.x86_64kdump.img
initramfs-3.10.0-862.9.1.el7.x86_64.img
initramfs-3.10.0-862.9.1.el7.x86_64kdump.img
initrd-plymouth.img
symvers-3.10.0-693.21.1.el7.x86_64.gz
symvers-3.10.0-862.11.6.el7.x86_64.gz
symvers-3.10.0-862.14.4.el7.x86_64.gz
symvers-3.10.0-862.3.2.el7.x86_64.gz
symvers-3.10.0-862.9.1.el7.x86_64.gz
System.map-3.10.0-693.21.1.el7.x86_64
System.map-3.10.0-862.11.6.el7.x86_64
System.map-3.10.0-862.14.4.el7.x86_64
System.map-3.10.0-862.3.2.el7.x86_64
System.map-3.10.0-862.9.1.el7.x86_64
vmlinuz-0-rescue-b0cbe21d63111d44b76fa95a2b474506
vmlinuz-3.10.0-693.21.1.el7.x86_64
vmlinuz-3.10.0-862.11.6.el7.x86_64
vmlinuz-3.10.0-862.14.4.el7.x86_64
vmlinuz-3.10.0-862.3.2.el7.x86_64
vmlinuz-3.10.0-862.9.1.el7.x86_64

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Kernel not upgraded to 3.10.0-957.1.3.el7.x86_64 after upgrading to CentOS 7.6.1810

Post by TrevorH » 2018/12/18 09:25:17

I fear that something nasty has happened to your rpm database or your system itself. The rpm database is most likely since the files that belong to the required kernel package are still present and if someone had erased the kernel rpm then those would have been deleted too. Your rpm -qa output appears to contain only 51 packages - and since a CentOS install done from the "Minimal" CD contains over 220, that's only 25% of the bare minimum required to run a system.

The reason that the kernel isn't being updated is that you don't have one installed at the moment so `yum update` doesn't have anything to update.

Personally, I'd go see if you have backups of /var/lib/{rpm,yum} and restore them after backing up what's there now in the broken version. Otherwise, given the scale of the damage and the lack of known reason how/why it happened, I'd look to reinstalling.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

soonyee
Posts: 3
Joined: 2018/12/17 07:08:24
Location: Asia/Singapore

Re: Kernel not upgraded to 3.10.0-957.1.3.el7.x86_64 after upgrading to CentOS 7.6.1810

Post by soonyee » 2018/12/18 12:40:22

Thanks Trevor for your reply.
I suspect the cause is corruption of files in /var after moving it to a separate partition following a recent hardening process required by my organization.
I shall attempt a re-install.

tunk
Posts: 1205
Joined: 2017/02/22 15:08:17

Re: Kernel not upgraded to 3.10.0-957.1.3.el7.x86_64 after upgrading to CentOS 7.6.1810

Post by tunk » 2018/12/19 22:07:31

I also had a problem with a new 7.6 install when moving /var and /tmp to new partitions.
In the end I did a touch /.autorelabel and reboot which seems to have fixed the problem.
It could be worth a try.

Post Reply