hi,
After last kernel upgrade on several cPanel servers using NVMe SSD, we get a high disk utilization according to iostat output, confirmed also by dstat -cd --disk-util (in case iostat would have a caveat : https://brooker.co.za/blog/2014/07/04/iostat-pct.html)
uname -a
3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Initial investigation :
- hardware is in perfect state in all servers, they use NVMe SSD for ages.
- such high level of disk utilization has no justification in term of running processes from any user, or any scripts, no iowait also at all
- what is reported by linux may be wrong, but this lets us a visible symptom that we can't trust, neither ignoring it util we have a realiable explanation.
we observed that :
- an upgrade centos on a basic server (with mechanic hdd in this case) is not causing this at reboot the following
- an excessive update of file /.readahead, this is not happening on other cpanel server using cloudlinux by ie
(our servers use no hard raid card)
We tried to disable readahead and reboot in case, but no change at all :
systemctl disable systemd-readahead-collect.service
systemctl disable systemd-readahead-replay.service
We contacted cPanel support, investigated deeply with 2 sysadmin, it appears that the answer will only come from centos 7 kernel developer
Thanks for your attention,
John
kernel upgrade giving high disk utilization
Re: kernel upgrade giving high disk utilization
I now upgraded to latest centos 7
uname -a
Linux rep4.hosting1976.fr 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Issue is still there, does someone has an explanation ?
uname -a
Linux rep4.hosting1976.fr 3.10.0-957.1.3.el7.x86_64 #1 SMP Thu Nov 29 14:49:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Issue is still there, does someone has an explanation ?
-
- Posts: 2019
- Joined: 2015/02/17 15:14:33
- Location: Bulgaria
- Contact:
Re: kernel upgrade giving high disk utilization
Open a bug in bugs.centos.org and then open another bug on bugzilla.redhat.com and relate that bug to the centos counterpart.
Re: kernel upgrade giving high disk utilization
Thanks for advice, done :
https://bugs.centos.org/view.php?id=15723
https://bugs.centos.org/view.php?id=15723