Dramaticlly slow down after patch of Meltdown

Support for security such as Firewalls and securing linux
Post Reply
mdalex
Posts: 3
Joined: 2018/01/18 02:37:09

Dramaticlly slow down after patch of Meltdown

Post by mdalex » 2018/01/18 03:08:45

Hello,

I have a X99 system with i7 5820K, which is used to run a emulator on X86; In the emulator the EPT is used to accelerate the address translating.
After switch to "3.10.0-693.11.6.el7.x86_64", the performance dropped 60-80% without any error from the EPT kernel module; If switch back to old kernel everything is back normal; And I tried to repalce the i7 5820K with i7 6800K, the problem is gone, there is no performance drop.

What could be the cause of the problem? Where should I look to solve it?

Thank you.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Dramaticlly slow down after patch of Meltdown

Post by TrevorH » 2018/01/18 07:39:04

Try toggling the 3 switches that control the new code and see which one makes the most difference. Look at the files /sys/kernel/debug/x86/*enabled - should be 3 of them, ibpb, ibrs and pti_enabled. First run cat /sys/kernel/debug/x86/*enabled and see what the current values are. You can change them by using e.g. echo 0 > /sys/kernel/debug/x86/pti_enabled

If ibrs_enabled=1 on my work machines then it doubles cpu usage.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

mdalex
Posts: 3
Joined: 2018/01/18 02:37:09

Re: Dramaticlly slow down after patch of Meltdown

Post by mdalex » 2018/01/18 09:19:31

TrevorH wrote:Try toggling the 3 switches that control the new code and see which one makes the most difference. Look at the files /sys/kernel/debug/x86/*enabled - should be 3 of them, ibpb, ibrs and pti_enabled. First run cat /sys/kernel/debug/x86/*enabled and see what the current values are. You can change them by using e.g. echo 0 > /sys/kernel/debug/x86/pti_enabled

If ibrs_enabled=1 on my work machines then it doubles cpu usage.
Thanks a lot, finally find the real culprit. the performance drop is due to the ibrs_enable, disable it turns the performance back.
Tried 4790/4790K/5820K/6700K/6800K/E5 2690, only 5820K is affected, it is strange.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Dramaticlly slow down after patch of Meltdown

Post by TrevorH » 2018/01/18 09:45:16

Check the output from dmesg on each of those processors and see which ones have microcode that support IBRS. I suspect that only the one that slows down is using the latest. Like this

Code: Select all

[root@centos7 ~]# dmesg | grep FEAT
[    0.116718] FEATURE SPEC_CTRL Not Present
[    0.117258] FEATURE IBPB_SUPPORT Not Present
That's on a VM and shows that there is no microcode that supports the IBRS mitigation. On a machine with fixed microcode it should say

[339488.642229] FEATURE SPEC_CTRL Present
[339488.642231] FEATURE IBPB_SUPPORT Present
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

mdalex
Posts: 3
Joined: 2018/01/18 02:37:09

Re: Dramaticlly slow down after patch of Meltdown

Post by mdalex » 2018/01/18 10:14:10

TrevorH wrote: That's on a VM and shows that there is no microcode that supports the IBRS mitigation. On a machine with fixed microcode it should say
Yes, you are right, the 5820K is the only one with updated microcode, we are doomed. Updating the microcode will affect all systems.

User avatar
TrevorH
Site Admin
Posts: 33202
Joined: 2009/09/24 10:40:56
Location: Brighton, UK

Re: Dramaticlly slow down after patch of Meltdown

Post by TrevorH » 2018/01/18 10:18:21

The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke

Post Reply