SATA drive causes high iowait

[GOAway]

I installed CentOS 6 x86_64 on a no-name machine with Asus P5G41T-M LX Motherboard with 4GB RAM and E5700 CPU.
It is intended to replace an old machine which is P4 1GB RAM and 40GB HD.
The old machine work with load avg of ~0.5 during peak times while the new machine goes to 10-12 with the same amount of work with avg iowait of 50%.

My best guess is that the HD is not well configured or a wrong driver is used in the kernel.

I already read hundreds of web pages but could not find a solution to that problem.

2 posts from Asus support forums that confirm P5G41T-M LX has no AHCI support:
http://vip.asus.com/forum/view.aspx?board_id=1&model=P5G41-M+LX&id=20101102234219793&page=1&SLanguage=en-us
http://vip.asus.com/forum/view.aspx?board_id=1&model=P5G41T-M+LX&id=20100923144334393&page=1&SLanguage=en-us
While the user's manual states that the SATA ports are all 3Gb/s :
http://www.asus.com/Motherboards/Intel_Socket_775/P5G41TM_LX/

The BIOS has option to configure ATA/IDE as Compatible or Enhanced. Both values produce the same IO wait values.

Any idea how to sove this?

Thank you all!

Some details from the system:

[code]
[root@newmachine ~]# lsmod
Module Size Used by
hwmon_vid 3098 0
coretemp 5742 0
ipv6 321509 7325
ipt_REJECT 2349 2
nf_conntrack_ipv4 9440 2
nf_defrag_ipv4 1449 1 nf_conntrack_ipv4
xt_state 1458 2
nf_conntrack 79643 2 nf_conntrack_ipv4,xt_state
iptable_filter 2759 1
ip_tables 17733 1 iptable_filter
ppdev 8695 0
parport_pc 22944 0
parport 37327 2 ppdev,parport_pc
asus_atk0110 14470 0
hwmon 2464 2 coretemp,asus_atk0110
e100 33019 0
mii 5205 1 e100
sg 30186 0
iTCO_wdt 11708 0
iTCO_vendor_support 3022 1 iTCO_wdt
snd_hda_codec_realtek 297377 1
snd_hda_intel 25194 0
snd_hda_codec 82097 2 snd_hda_codec_realtek,snd_hda_intel
snd_hwdep 6714 1 snd_hda_codec
snd_seq 56557 0
snd_seq_device 6626 1 snd_seq
snd_pcm 84700 2 snd_hda_intel,snd_hda_codec
snd_timer 23087 2 snd_seq,snd_pcm
snd 70053 8 snd_hda_codec_realtek,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_seq,snd_seq_device,snd_pcm,snd_timer
soundcore 8052 1 snd
snd_page_alloc 8628 2 snd_hda_intel,snd_pcm
ext4 353851 2
mbcache 7918 1 ext4
jbd2 89033 1 ext4
sd_mod 37221 3
crc_t10dif 1507 1 sd_mod
sr_mod 16162 0
cdrom 39769 1 sr_mod
ata_generic 3611 0
pata_acpi 3667 0
ata_piix 22588 2
i915 329477 1
drm_kms_helper 32113 1 i915
drm 202185 2 i915,drm_kms_helper
i2c_algo_bit 5664 1 i915
i2c_core 31274 4 i915,drm_kms_helper,drm,i2c_algo_bit
video 20966 1 i915
output 2471 1 video
dm_mod 76824 6
[root@newmachine ~]#
[/code]

[code]
[root@newmachine ~]# cat /proc/scsi/scsi
Attached devices:
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: Hitachi HDS72105 Rev: JP2O
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi2 Channel: 00 Id: 01 Lun: 00
Vendor: Optiarc Model: DVD RW AD-7280S Rev: 1.01
Type: CD-ROM ANSI SCSI revision: 05
[/code]

dmesg output, relevant parts only
[quote]
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.32-71.29.1.el6.x86_64 (mockbuild@c6b5.bsys.dev.centos.org) (gcc version 4.4.4 20100726 (Red Hat 4.4.4-13) (GCC) ) #1 SMP Mon Jun 27 19:49:27 BST 2011
Command line: ro root=/dev/mapper/vg_vg1-lv_root rd_LVM_LV=vg_vg1/lv_root rd_LVM_LV=vg_vg1/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=auto rhgb quiet
KERNEL supported cpus:
Intel GenuineIntel
AMD AuthenticAMD
Centaur CentaurHauls
DMI present.
AMI BIOS detected: BIOS may corrupt low RAM, working around it.
PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Memory: 3874976k/4718592k available (4939k kernel code, 560068k absent, 283548k reserved, 3931k data, 1220k init)
Hierarchical RCU implementation.
CPU0: Intel Pentium(R) Dual-Core CPU E5700 @ 3.00GHz stepping 0a
Brought up 2 CPUs
Total of 2 processors activated (12002.60 BogoMIPS).
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
agpgart-intel 0000:00:00.0: Intel G41 Chipset
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
alloc irq_desc for 23 on node -1
alloc kstat_irqs on node -1
dracut: Starting plymouth daemon
ata_piix 0000:00:1f.1: version 2.13
ata_piix 0000:00:1f.1: PCI INT A -> GSI 18 (level, low) -> IRQ 18
ata_piix 0000:00:1f.1: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xffa0 irq 14
ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xffa8 irq 15
alloc irq_desc for 22 on node -1
alloc kstat_irqs on node -1
ata_piix 0000:00:1f.2: PCI INT B -> GSI 22 (level, low) -> IRQ 22
ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
ata_piix 0000:00:1f.2: setting latency timer to 64
scsi2 : ata_piix
scsi3 : ata_piix
ata3: SATA max UDMA/133 cmd 0xd080 ctl 0xd000 bmdma 0xc800 irq 22
ata4: SATA max UDMA/133 cmd 0xcc00 ctl 0xc880 bmdma 0xc808 irq 22
ata3.00: ATA-8: Hitachi HDS721050CLA362, JP2OA50E, max UDMA/133
ata3.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata3.01: ATAPI: Optiarc DVD RW AD-7280S, 1.01, max UDMA/100
ata3.00: configured for UDMA/133
ata3.01: configured for UDMA/100
scsi 2:0:0:0: Direct-Access ATA Hitachi HDS72105 JP2O PQ: 0 ANSI: 5
scsi 2:0:1:0: CD-ROM Optiarc DVD RW AD-7280S 1.01 PQ: 0 ANSI: 5
sr0: scsi3-mmc drive: 48x/48x writer dvd-ram cd/rw xa/form2 cdda tray
Uniform CD-ROM driver Revision: 3.20
sr 2:0:1:0: Attached scsi CD-ROM sr0
sd 2:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sda: sda1 sda2
sd 2:0:0:0: [sda] Attached SCSI disk
EXT4-fs (dm-0): mounted filesystem with ordered data mode
udev: starting version 147
alloc irq_desc for 21 on node -1
alloc kstat_irqs on node -1
HDA Intel 0000:00:1b.0: PCI INT A -> GSI 21 (level, low) -> IRQ 21
alloc irq_desc for 26 on node -1
alloc kstat_irqs on node -1
HDA Intel 0000:00:1b.0: irq 26 for MSI/MSI-X
HDA Intel 0000:00:1b.0: setting latency timer to 64
hda_codec: ALC887: BIOS auto-probing.
intel_rng: FWH not detected
iTCO_vendor_support: vendor-support=0
iTCO_wdt: Intel TCO WatchDog Timer Driver v1.05
iTCO_wdt: Found a ICH7 or ICH7R TCO device (Version=2, TCOBASE=0x0860)
iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
sd 2:0:0:0: Attached scsi generic sg0 type 0
sr 2:0:1:0: Attached scsi generic sg1 type 5
EXT4-fs (sda1): mounted filesystem with ordered data mode
eth0: no IPv6 routers present
eth1: no IPv6 routers present
i2c /dev entries driver
w83627ehf: Found W83627DHG-P chip at 0x290
ACPI: I/O resource w83627ehf [0x295-0x296] conflicts with ACPI region HWRE [0x290-0x299]
ACPI: If an ACPI driver is available for this device, you should use it instead of the native driver
[/quote]

[code]
[root@newmachine ~]# lspci
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:02.0 VGA compatible controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03)
00:1b.0 Audio device: Intel Corporation N10/ICH 7 Family High Definition Audio Controller (rev 01)
00:1c.0 PCI bridge: Intel Corporation N10/ICH 7 Family PCI Express Port 1 (rev 01)
00:1d.0 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #1 (rev 01)
00:1d.1 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #2 (rev 01)
00:1d.2 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #3 (rev 01)
00:1d.3 USB Controller: Intel Corporation N10/ICH 7 Family USB UHCI Controller #4 (rev 01)
00:1d.7 USB Controller: Intel Corporation N10/ICH 7 Family USB2 EHCI Controller (rev 01)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev e1)
00:1f.0 ISA bridge: Intel Corporation 82801GB/GR (ICH7 Family) LPC Interface Bridge (rev 01)
00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01)
00:1f.2 IDE interface: Intel Corporation N10/ICH7 Family SATA IDE Controller (rev 01)
02:00.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 (rev 08)
02:01.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 (rev 0c)
[/code]

[WhatsHisName]

[quote]2 posts from Asus support forums that confirm P5G41T-M LX has no AHCI support[/quote]
Not a good sign.

Usually, complaints about poor drive performance are linked either to (1) hardware issues (e.g., WD Green drives) or (2) the use of legacy/SATA instead of AHCI.

Without the option to select AHCI at the BIOS level, your options may be very limited.


Sorry could not be more helpful...

...Oh, sorry, the answer is "try a different motherboard", but I don't think that is the answer you were looking for. :-D

[GOAway]

Thank you for the reply. I have replaced the MB with a brand new one, this time with ICH10 chipset that has AHCI support:
Gigabyte P45T-ES3G

In the bios level i had the option "IDE" or "AHCI" and changed it from the default (IDE) into AHCI.
Rebooted the machine - load stays very high with high iowait.
So i thought it might be better to re-install the OS to make sure things work as they should with the correct drivers.

After re-install and reboot all goes to the same status - high load with high iowait.
The only thing to check now is the hard-drive. I already ordered a new WD Caviar and hopefully i will install it tomorrow.
Any ideas, besides the HD change?

Thanks ones again,

Here are some parameters from the system with it's new hardware and fresh installation:

[code]
[root@newmobo ~]# lsmod
Module Size Used by
sit 10313 0
tunnel4 2877 1 sit
nf_conntrack_ipv4 9440 2
nf_defrag_ipv4 1449 1 nf_conntrack_ipv4
iptable_filter 2759 1
ip_tables 17733 1 iptable_filter
autofs4 27683 3
sunrpc 241630 1
ipt_REJECT 2349 2
ip6t_REJECT 4562 2
nf_conntrack_ipv6 19655 2
xt_state 1458 4
nf_conntrack 79643 3 nf_conntrack_ipv4,nf_conntrack_ipv6,xt_state
ip6table_filter 2855 1
ip6_tables 19392 1 ip6table_filter
ipv6 321509 5774 sit,ip6t_REJECT,nf_conntrack_ipv6
dm_mirror 14003 0
dm_region_hash 12200 1 dm_mirror
dm_log 10088 2 dm_mirror,dm_region_hash
ppdev 8695 0
parport_pc 22944 0
parport 37327 2 ppdev,parport_pc
e100 33019 0
mii 5205 1 e100
i2c_i801 11190 0
sg 30186 0
iTCO_wdt 11708 0
iTCO_vendor_support 3022 1 iTCO_wdt
snd_hda_codec_nvhdmi 15159 4
snd_hda_codec_intelhdmi 18011 1 snd_hda_codec_nvhdmi
snd_hda_codec_realtek 297377 1
snd_hda_intel 25194 0
snd_hda_codec 82097 4 snd_hda_codec_nvhdmi,snd_hda_codec_intelhdmi,snd_hda_codec_realtek,snd_hda_intel
snd_hwdep 6714 1 snd_hda_codec
snd_seq 56557 0
snd_seq_device 6626 1 snd_seq
snd_pcm 84700 2 snd_hda_intel,snd_hda_codec
snd_timer 23087 2 snd_seq,snd_pcm
snd 70053 9 snd_hda_codec_intelhdmi,snd_hda_codec_realtek,snd_hda_intel,snd_hda_codec,snd_hwdep,snd_seq,snd_seq_device,snd_pcm,snd_timer
soundcore 8052 1 snd
snd_page_alloc 8628 2 snd_hda_intel,snd_pcm
ext4 353851 2
mbcache 7918 1 ext4
jbd2 89033 1 ext4
sr_mod 16162 0
cdrom 39769 1 sr_mod
sd_mod 37221 3
crc_t10dif 1507 1 sd_mod
pata_acpi 3667 0
ata_generic 3611 0
pata_jmicron 2747 0
ahci 39105 2
nouveau 539005 1
ttm 47134 1 nouveau
drm_kms_helper 32113 1 nouveau
drm 202185 3 nouveau,ttm,drm_kms_helper
i2c_algo_bit 5664 1 nouveau
video 20966 1 nouveau
output 2471 1 video
i2c_core 31274 5 i2c_i801,nouveau,drm_kms_helper,drm,i2c_algo_bit
dm_mod
[/code]

dmesg output:
[code]
ahci 0000:00:1f.2: version 3.0
ahci 0000:00:1f.2: PCI INT B -> GSI 19 (level, low) -> IRQ 19
alloc irq_desc for 27 on node -1
alloc kstat_irqs on node -1
ahci 0000:00:1f.2: irq 27 for MSI/MSI-X
ahci: SSS flag set, parallel bus scan disabled
ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3 Gbps 0x3f impl SATA mode
ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led clo pmp pio slum part ccc ems
ahci 0000:00:1f.2: setting latency timer to 64
scsi0 : ahci
scsi1 : ahci
scsi2 : ahci
scsi3 : ahci
scsi4 : ahci
scsi5 : ahci
ata1: SATA max UDMA/133 abar m2048@0xfdffd000 port 0xfdffd100 irq 27
ata2: SATA max UDMA/133 irq_stat 0x00400040, connection status changed irq 27
ata3: SATA max UDMA/133 abar m2048@0xfdffd000 port 0xfdffd200 irq 27
ata4: SATA max UDMA/133 abar m2048@0xfdffd000 port 0xfdffd280 irq 27
ata5: SATA max UDMA/133 abar m2048@0xfdffd000 port 0xfdffd300 irq 27
ata6: SATA max UDMA/133 abar m2048@0xfdffd000 port 0xfdffd380 irq 27
ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
ata1.00: ATA-8: Hitachi HDS721050CLA362, JP2OA50E, max UDMA/133
ata1.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32), AA
ata1.00: configured for UDMA/133
scsi 0:0:0:0: Direct-Access ATA Hitachi HDS72105 JP2O PQ: 0 ANSI: 5
ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata2.00: ATAPI: Optiarc DVD RW AD-7280S, 1.01, max UDMA/100, ATAPI AN
ata2.00: configured for UDMA/100
scsi 1:0:0:0: CD-ROM Optiarc DVD RW AD-7280S 1.01 PQ: 0 ANSI: 5
ata3: SATA link down (SStatus 0 SControl 300)
ata4: SATA link down (SStatus 0 SControl 300)
ata5: SATA link down (SStatus 0 SControl 300)
ata6: SATA link down (SStatus 0 SControl 300)
pata_jmicron 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
pata_jmicron 0000:03:00.0: setting latency timer to 64
scsi6 : pata_jmicron
scsi7 : pata_jmicron
ata7: PATA max UDMA/100 cmd 0xef00 ctl 0xee00 bmdma 0xeb00 irq 16
ata8: PATA max UDMA/100 cmd 0xed00 ctl 0xec00 bmdma 0xeb08 irq 16
sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sda: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
[/code]

lspci output:
[code]
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.4 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 5
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
00:1f.2 SATA controller: Intel Corporation 82801JI (ICH10 Family) SATA AHCI Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
01:00.0 VGA compatible controller: nVidia Corporation GT218 [GeForce 210] (rev a2)
01:00.1 Audio device: nVidia Corporation High Definition Audio Controller (rev a1)
03:00.0 IDE interface: JMicron Technology Corp. JMB368 IDE controller
04:00.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 (rev 08)
04:01.0 Ethernet controller: Intel Corporation 82557/8/9/0/1 Ethernet Pro 100 (rev 0c)
[/code]

sar output:
[code]
06:37:01 PM all 1.94 0.00 6.13 63.56 0.00 28.37
06:38:01 PM all 2.03 0.00 6.20 64.58 0.00 27.19
06:39:01 PM all 1.75 0.00 5.51 60.95 0.00 31.78
06:40:01 PM all 2.24 0.00 6.52 65.18 0.00 26.06
06:41:01 PM all 2.00 0.00 6.14 63.64 0.00 28.23
06:42:01 PM all 1.93 0.00 5.86 60.65 0.00 31.57
06:43:01 PM all 2.25 0.00 7.02 63.20 0.00 27.53
06:44:04 PM all 1.64 0.00 7.64 66.88 0.00 23.83
06:45:01 PM all 1.90 0.00 5.94 64.52 0.00 27.63
06:46:01 PM all 2.09 0.00 6.44 62.88 0.00 28.59
06:47:01 PM all 2.04 0.00 6.24 63.06 0.00 28.67
06:48:01 PM all 2.30 0.00 7.10 63.23 0.00 27.37
06:49:01 PM all 2.03 0.00 8.13 63.85 0.00 25.98
06:50:02 PM all 1.97 0.00 6.50 63.89 0.00 27.63
06:51:01 PM all 1.90 0.00 7.19 66.80 0.00 24.11
06:52:01 PM all 1.72 0.00 5.41 63.69 0.00 29.18
06:53:01 PM all 2.05 0.00 6.34 65.05 0.00 26.57
06:54:03 PM all 1.85 0.00 7.57 68.87 0.00 21.72
06:55:01 PM all 1.77 0.00 5.88 63.51 0.00 28.83
06:56:01 PM all 2.17 0.00 6.65 64.00 0.00 27.19
06:57:01 PM all 2.07 0.00 6.24 64.27 0.00 27.42
06:58:01 PM all 2.08 0.00 6.41 63.15 0.00 28.36
06:59:01 PM all 1.89 0.00 5.98 60.80 0.00 31.34
07:00:01 PM all 1.91 0.00 5.91 62.30 0.00 29.88
07:01:01 PM all 2.05 0.00 6.53 62.45 0.00 28.97
07:02:01 PM all 1.86 0.00 5.97 64.42 0.00 27.75
07:03:01 PM all 2.18 0.00 7.98 63.30 0.00 26.54
07:04:03 PM all 2.00 0.00 7.57 67.20 0.00 23.23
07:05:01 PM all 1.86 0.00 6.04 64.16 0.00 27.95
07:06:01 PM all 1.88 0.00 5.89 60.00 0.00 32.23
07:07:01 PM all 1.56 0.00 4.88 60.25 0.00 33.31
07:08:02 PM all 1.90 0.00 7.85 64.41 0.00 25.84
07:09:03 PM all 1.64 0.00 7.17 68.87 0.00 22.33

07:09:03 PM CPU %user %nice %system %iowait %steal %idle
07:10:02 PM all 1.83 0.00 6.08 68.69 0.00 23.40
07:11:01 PM all 1.68 0.00 5.45 72.49 0.00 20.38
07:12:01 PM all 1.67 0.00 6.75 69.74 0.00 21.84
07:13:01 PM all 1.92 0.00 5.70 65.59 0.00 26.79
07:14:01 PM all 1.92 0.00 5.81 65.73 0.00 26.54
07:15:01 PM all 1.92 0.00 5.86 65.54 0.00 26.68
07:16:01 PM all 1.76 0.00 5.48 62.86 0.00 29.90
07:17:01 PM all 1.98 0.00 5.83 63.24 0.00 28.95
07:18:01 PM all 2.02 0.00 6.31 63.33 0.00 28.34
07:19:01 PM all 2.34 0.00 7.13 61.05 0.00 29.48
Average: all 1.90 0.00 6.20 62.05 0.00 29.86
[/code]

[GOAway]

Updates:

After replacing the MB i also replaced the hard-drive, the RAM and the SATA cables.
Then tried both CentOS 6 and 5.7 (x86_64) and all combinations resulted the same - high iowait, high CPU utilization, slow service.
I must say that temperature in the servers room is reasonable and i also monitor internal PC temps with lm_sensors+cacti without any special issues monitored.

The only thing left to try is to replace the CPU. I will do that in few days but i doubt it will help.

If someone look at the logs i provided here and see any problem, i'll be glad to hear.