Random pauses at bootup and shutdown

[AlanBartlett]

If performing a fresh install, I would advise that you start the installer with [b]linux nodmraid[/b] at the [b]boot:[/b] prompt --

[code]
[b]boot:[/b] linux nodmraid
[/code]

[pschaff]

For CentOS-6 things have changed a bit. Hit to edit the boot line and append [b]nodmraid[/b].

[pjc123]

Well thanks for all the help, but I hate wasting people's time. I guess I have to wait until a bug fix is issued (What for I still don't know), version 6.2 comes out, a new Intel BIOS is issued, or I upgrade to a version 6 certified motherboard some time in the future. At least I can try out some of the new features until it crashes again.

[AlanBartlett]

[quote]
but I hate wasting people's time.
[/quote]
I would not say that you have been. We are all puzzled and would like to see this issue resolved. :-?

[ServerCent]

WAIT.. case not closed. :)

I have seen the exact same thing. It's not a matter though of boot up and power down pauses, I am getting random pausing occurring while running the system. I should say systemS.

I thought at 1st it was just maybe because of the system I was running with the huge .img files it was handling, until I installed another system today with totally different hardware and a different configuration, and saw the exact same thing happen again.

This is a hard pause where everything, and I mean EVERYTHING just stops for a around 20-30 seconds.. then suddenly it takes off again like nothing happened. This not only happens in SSH, but happens at the console as well, meaning its system wide.

The only commonality between these two systems, both were installed with Centos6 64bit and were fully updated from Centos default repo. One has cpanel installed to it, while the other just has centos with a raid controller for storage.

I am convinced this has absolutely nothing to do with softraid, since my storage system doesn't contain a softraid in it. The second cpanel system does, however, the pausing is completely irrelevant to this.

Also not running LVM on one system, and running LVM on another... I thought the pauses were occurring due to high IOs, but they can occur even when there isn't any high IO going on.

I hoped to find some kind of reference to a bug like the guy that started this thread, but I think this warrants some investigation.

One thing that was common with the installs, was that they were both minimal. Oh, and both are running dual Xeon E5520 CPUs. Otherwise the rest of the hardware is all different.

In the attachment is a txt file of the dmesg output of both these systems. For some reason I couldn't get a cut/paste of them to post here properly.

Any help/direction on this would be very appreciated!

EDIT: I attempted to upload a zip only to have an error occur. All kinds of attempts to include my dmesg in this post failed. Any suggestions as to how to share this info and/or other info is also appreciated.

[pschaff]

Welcome to the CentOS fora. Please see the recommended reading for new users linked in my signature.

[quote]
ServerCent wrote:
WAIT.. case not closed. :)
[/quote]
Unfortunately, nobody said it was.

Your issue does sound similar; however, reading the recommended links should make it clear that you should not hijack a thread asking for help with a similar issue. If you need attention to your specific issues it would be best to start a new Topic to get the help you need, providing a [url=https://www.centos.org/modules/newbb/viewtopic.php?viewmode=flat&topic_id=32523&forum=55]link[/url] to this one if required for context.

[pjc123]

[quote]
AlanBartlett wrote:
[quote]
but I hate wasting people's time.
[/quote]
I would not say that you have been. We are all puzzled and would like to see this issue resolved. :-?[/quote]

Fair enough. I just want to let you know that this is not a corporate server, but a desktop that I put together a year ago to keep myself sharp with the latest hardware and Red Hat based Linux technology until there are ever any jobs again (Yeah right). So no urgency. I will say that this is the first version of Linux that has given me a major problem (I have installed various versions of Centos, Fedora, Ubuntu, etc. without any major issues).

[pjc123]

So I wiped out CentOS 6 again, did a clean wipe of the drive, and installed Fedora 15 so I could use some of the tools that are available to examine the time lags in the boot process. Well, it wasn't even necessary to use the tools to debug because bootups and shutdowns screamed by at an unbelievable speed without any pauses or errors. I kept rebooting the computer to make sure it was consistent.

Although I don't know if it is related to the pauses in the operating system, the only consistent problem that shows up on all three operating systems are HDMI/sound errors. The only part with HDMI is the Radeon video card. On CentOS and SL6 there is an warning that quickly pops up in KDE about the HDMI port not being detected. In Fedora 15 there are more detailed errors in the /var/log/message files. After updating all packages in Fedora 15, it became the first time that the HDMI errors disappeared.


-----------------------------------------------------------------------------------------------------------------------------------------------

FEDORA 15 /var/log/messages immediately after install from DVD with no updated packages (There were 128 HDMI: invalid ELD messages so I shortened the listing)


Aug 9 18:58:48 lin1 kernel: [ 8.407158] hda_codec: ALC889: BIOS auto-probing.
Aug 9 18:58:48 lin1 kernel: [ 8.407164] hda_codec: ALC889: SKU not ready 0x411111f0
Aug 9 18:58:48 lin1 kernel: [ 8.417491] input: HDA Intel Headphone as /devices/pci0000:00/0000:00:1b.0/sound/card0/input5
Aug 9 18:58:48 lin1 kernel: [ 8.417761] HDA Intel 0000:01:00.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17
Aug 9 18:58:48 lin1 kernel: [ 8.448813] ALSA sound/pci/hda/hda_eld.c:352: HDMI: ELD buf size is 0, force 128
Aug 9 18:58:48 lin1 kernel: [ 8.448871] ALSA sound/pci/hda/hda_eld.c:161: HDMI: invalid ELD data byte 0
Aug 9 18:58:48 lin1 kernel: [ 8.448940] ALSA sound/pci/hda/hda_eld.c:161: HDMI: invalid ELD data byte 1
Aug 9 18:58:48 lin1 kernel: [ 8.448987] ALSA sound/pci/hda/hda_eld.c:161: HDMI: invalid ELD data byte 2
Aug 9 18:58:48 lin1 kernel: [ 8.449000] ALSA sound/pci/hda/hda_eld.c:161: HDMI: invalid ELD data byte 3
Aug 9 18:58:48 lin1 kernel: [ 8.449014] ALSA sound/pci/hda/hda_eld.c:161: HDMI: invalid ELD data byte 4
.
.
.
These errors continue until byte 127
Aug 9 18:58:48 lin1 kernel: [ 8.450648] ALSA sound/pci/hda/hda_eld.c:267: HDMI: Unknown ELD version 0

-----------------------------------------------------------------------------------------------------------------------------------------------

FEDORA 15 with all packages updated to latest software ( I am not sure how to read this, but it appears as if the HDMI port is detected properly)

Aug 9 19:42:50 lin1 kernel: [ 8.604554] HDA Intel 0000:00:1b.0: PCI INT A -> GSI 22 (level, low) -> IRQ 22
Aug 9 19:42:50 lin1 kernel: [ 8.652892] hda_codec: ALC889: BIOS auto-probing.
Aug 9 19:42:50 lin1 kernel: [ 8.654532] hda_codec: ALC889: SKU not ready 0x411111f0
Aug 9 19:42:50 lin1 kernel: [ 8.663855] input: HDA Intel Headphone as /devices/pci0000:00/0000:00:1b.0/sound/card0/input5
Aug 9 19:42:50 lin1 kernel: [ 8.665675] HDA Intel 0000:01:00.1: PCI INT B -> GSI 17 (level, low) -> IRQ 17
Aug 9 19:42:50 lin1 kernel: [ 8.688768] e1000e 0000:02:00.0: eth1: (PCI Express:2.5GT/s:Width x1) 00:1b:21:44:2f:8b
Aug 9 19:42:50 lin1 kernel: [ 8.690465] e1000e 0000:02:00.0: eth1: Intel(R) PRO/1000 Network Connection
Aug 9 19:42:50 lin1 kernel: [ 8.692571] e1000e 0000:02:00.0: eth1: MAC: 3, PHY: 8, PBA No: E46981-003
Aug 9 19:42:50 lin1 kernel: [ 8.737946] HDMI status: Pin=3 Presence_Detect=0 ELD_Valid=0
Aug 9 19:42:50 lin1 kernel: [ 8.740290] input: HDA ATI HDMI HDMI/DP as /devices/pci0000:00/0000:00:03.0/0000:01:00.1/sound/card1/input6

[pjc123]

Really, double sure, finally solved the problem this time !

You just gotta love intermittent problems. I was trying various kernel clock parameters when I came upon one that seemed to work (nohz=off). I confirmed this after performing many, many successful reboots. Why it works I do not know, nor do I know what the implications are, but now I can consistently boot both CentOS 6 and SL 6 without any random pause or "FAIL" messages during the bootup or shutdown process, or random errors in the logs. Googling around, I noticed a couple of others who have been using the following combination of two parameters to fix a similar problem as mine, problems with commercially installed software, and hibernation/resume issues specifically with the Toshiba NB series of laptops. I tested this combination and it also works for me, so that is what I am currently using.


----------------------------------------------------------------------------------

Kernel parameters that were added to fix the problem

nohz=off highres=off

-----------------------------------------------------------------------------------


Here are the descriptions of these items per the kernel documentation:


nohz= [KNL] Boottime enable/disable dynamic ticks
Valid arguments: on, off
Default: on

highres= [KNL] Enable/disable high resolution timer mode.
Valid parameters: "on", "off"
Default: "on"

I also experimented with the following, but either the problem was not fixed or new problems were created. I discovered that Centos 5.6 uses the jiffies clock source (In fact it is the only one available). CentOS 6 uses the tsc clock source (Available sources are tsc, hpet, acpi_pm and jiffies). I am guessing that the use of the jiffies clock source is why I never had any problems with CentOS 5.6.


clocksource= [GENERIC_TIME] Override the default clocksource
Format:
Override the default clocksource and use the clocksource
with the name specified.
Some clocksource names to choose from, depending on
the platform:
[all] jiffies (this is the base, fallback clocksource)
[ACPI] acpi_pm
[ARM] imx_timer1,OSTS,netx_timer,mpu_timer2,
pxa_timer,timer3,32k_counter,timer0_1
[AVR32] avr32
[X86-32] pit,hpet,tsc,vmi-timer;
scx200_hrt on Geode; cyclone on IBM x440
[MIPS] MIPS
[PARISC] cr16
[S390] tod
[SH] SuperH
[SPARC64] tick
[X86-64] hpet,tsc


In case someone else comes across this problem, my motherboard is an Intel DP55KG.

[pjc123]

So, I did one final install and everything is still booting up and shutting down properly with "nohz=off highres=off" added to the kernel, however cpu usage can get very high at times on multiple cores, and I can hear the processor fan spin up when that happens (One example is when the fan spun up multiple times during the 300+ file initial CentOS 6 yum update), so obviously that's not good. So not knowing enough how the kernel clock system works, I am not sure how to modify the kernel parameters to keep the fixes for the pausing problem and at the same time eliminate the high cpu usage. Any ideas?