[SOLVED] high system interrupts
[SOLVED] high system interrupts
Hello,
i have centos system that hosts some web site, using apache + suphp + qmail.
From few days i have high cpu interrupts time, that cause high system load:
[code]
top - 10:41:03 up 18:09, 2 users, load average: 12.12, 11.90, 10.75
Tasks: 432 total, 4 running, 424 sleeping, 0 stopped, 4 zombie
Cpu(s): 1.5%us, 4.8%sy, 0.0%ni, 2.9%id, 0.9%wa, 0.8%hi, 89.0%si, 0.0%st
Mem: 3083884k total, 2846228k used, 237656k free, 142496k buffers
Swap: 2031608k total, 200k used, 2031408k free, 930236k cached
[/code]
You can see cpu spends 89.0%si
The system is under the KVM hypervisor, kernel 2.6.18 x86_64 and 6 core assigned.
This is the /proc/interrupts output:
[code]
[root@xp03 sbin]# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
0: 44065585 0 0 0 0 0 IO-APIC-edge timer
1: 18 10 5 4 2 7 IO-APIC-edge i8042
6: 0 0 0 1 1 0 IO-APIC-edge floppy
8: 0 0 0 0 0 0 IO-APIC-edge rtc
9: 0 0 0 0 0 0 IO-APIC-level acpi
10: 0 0 0 0 0 0 IO-APIC-level virtio0
11: 85 85 89 99 99 22437192 IO-APIC-level uhci_hcd:usb1, eth0, eth1
12: 46 47 45 45 45 34 IO-APIC-edge i8042
14: 798 795 792 781 2675375 798 IO-APIC-edge ide0
15: 20 21 20 642239 21 21 IO-APIC-edge ide1
NMI: 0 0 0 0 0 0
LOC: 65532418 65531344 65530593 65531579 65531966 65531978
RES: 3335059 2130455 2268401 2033564 1628958 789841
ERR: 0
MIS: 0[/code]
How can I debug the si% load?
thank you
[Moderator edit: Added [i]code[/i] tags to preserve formatting.]
i have centos system that hosts some web site, using apache + suphp + qmail.
From few days i have high cpu interrupts time, that cause high system load:
[code]
top - 10:41:03 up 18:09, 2 users, load average: 12.12, 11.90, 10.75
Tasks: 432 total, 4 running, 424 sleeping, 0 stopped, 4 zombie
Cpu(s): 1.5%us, 4.8%sy, 0.0%ni, 2.9%id, 0.9%wa, 0.8%hi, 89.0%si, 0.0%st
Mem: 3083884k total, 2846228k used, 237656k free, 142496k buffers
Swap: 2031608k total, 200k used, 2031408k free, 930236k cached
[/code]
You can see cpu spends 89.0%si
The system is under the KVM hypervisor, kernel 2.6.18 x86_64 and 6 core assigned.
This is the /proc/interrupts output:
[code]
[root@xp03 sbin]# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
0: 44065585 0 0 0 0 0 IO-APIC-edge timer
1: 18 10 5 4 2 7 IO-APIC-edge i8042
6: 0 0 0 1 1 0 IO-APIC-edge floppy
8: 0 0 0 0 0 0 IO-APIC-edge rtc
9: 0 0 0 0 0 0 IO-APIC-level acpi
10: 0 0 0 0 0 0 IO-APIC-level virtio0
11: 85 85 89 99 99 22437192 IO-APIC-level uhci_hcd:usb1, eth0, eth1
12: 46 47 45 45 45 34 IO-APIC-edge i8042
14: 798 795 792 781 2675375 798 IO-APIC-edge ide0
15: 20 21 20 642239 21 21 IO-APIC-edge ide1
NMI: 0 0 0 0 0 0
LOC: 65532418 65531344 65530593 65531579 65531966 65531978
RES: 3335059 2130455 2268401 2033564 1628958 789841
ERR: 0
MIS: 0[/code]
How can I debug the si% load?
thank you
[Moderator edit: Added [i]code[/i] tags to preserve formatting.]
-
- Retired Moderator
- Posts: 18276
- Joined: 2006/12/13 20:15:34
- Location: Tidewater, Virginia, North America
- Contact:
[SOLVED] high system interrupts
Welcome to the CentOS fora. Please see the recommended reading for new users linked in my signature.
Please [url=http://www.centos.org/modules/newbb/viewtopic.php?topic_id=28723&forum=54]provide more information about your system[/url] by running "./getinfo.sh basic" and showing us the output file.
Please [url=http://www.centos.org/modules/newbb/viewtopic.php?topic_id=28723&forum=54]provide more information about your system[/url] by running "./getinfo.sh basic" and showing us the output file.
Re: high system interrupts
thank you Phil.
Basic system information.
[code]
== BEGIN uname -rmi ==
2.6.18-238.12.1.el5 x86_64 x86_64
== END uname -rmi ==
== BEGIN rpm -qa \*-release\* ==
rpmforge-release-0.5.2-2.el5.rf
centos-release-notes-5.6-0
atomic-release-1.0-13.el5.art
centos-release-5-6.el5.centos.1
== END rpm -qa \*-release\* ==
== BEGIN cat /etc/redhat-release ==
CentOS release 5.6 (Final)
== END cat /etc/redhat-release ==
== BEGIN getenforce ==
Disabled
== END getenforce ==
== BEGIN free -m ==
total used free shared buffers cached
Mem: 3017 2448 568 0 117 907
-/+ buffers/cache: 1424 1592
Swap: 5023 86 4937
== END free -m ==
[/code]
Basic system information.
[code]
== BEGIN uname -rmi ==
2.6.18-238.12.1.el5 x86_64 x86_64
== END uname -rmi ==
== BEGIN rpm -qa \*-release\* ==
rpmforge-release-0.5.2-2.el5.rf
centos-release-notes-5.6-0
atomic-release-1.0-13.el5.art
centos-release-5-6.el5.centos.1
== END rpm -qa \*-release\* ==
== BEGIN cat /etc/redhat-release ==
CentOS release 5.6 (Final)
== END cat /etc/redhat-release ==
== BEGIN getenforce ==
Disabled
== END getenforce ==
== BEGIN free -m ==
total used free shared buffers cached
Mem: 3017 2448 568 0 117 907
-/+ buffers/cache: 1424 1592
Swap: 5023 86 4937
== END free -m ==
[/code]
-
- Retired Moderator
- Posts: 18276
- Joined: 2006/12/13 20:15:34
- Location: Tidewater, Virginia, North America
- Contact:
Re: high system interrupts
Strictly a guess, but Atomic is known for replacing large parts of CentOS with their own packages. It's not clear what KVM may have to do with the issue though. Is the information you show on a KVM guest or the host?
Re: high system interrupts
from Atomic repos i installed only the PHP 5.2.17 packages. The informations is from the KVM guest.
Maybe KVM in not related to the issue, I says that is under KVM to let you know about the entire environment.
Maybe KVM in not related to the issue, I says that is under KVM to let you know about the entire environment.
-
- Retired Moderator
- Posts: 18276
- Joined: 2006/12/13 20:15:34
- Location: Tidewater, Virginia, North America
- Contact:
Re: high system interrupts
I must confess ignorance about whether or not this applies to KVM, but on some other virtualization methods it may help to put "divider=10" on the kernel boot line to decrease the clock rate.
Re: high system interrupts
thank you Phil, I'm going to add the parameter, I will update you about.
Re: high system interrupts
Phil,
thank you very much! I added the divider kernel's parameter, and the CPU load has been dropped!
thank you very much! I added the divider kernel's parameter, and the CPU load has been dropped!
-
- Retired Moderator
- Posts: 18276
- Joined: 2006/12/13 20:15:34
- Location: Tidewater, Virginia, North America
- Contact:
Re: [SOLVED] high system interrupts
You are welcome, and thanks for reporting back. Marking this thread [SOLVED] for posterity.