Dear All,
Today i'm facing kernel panic and what i'm trace from vmcore-dmesg.txt as below:
<4>[ 1029.785631] ------------[ cut here ]------------
<4>[ 1029.785884] WARNING: at include/net/sock.h:492 netlink_release+0x283/0x290() (Not tainted)
<4>[ 1029.786350] Hardware name: S2600CP
<4>[ 1029.786565] Modules linked in: xt_owner tcp_diag inet_diag autofs4 xt_state ip6t_LOG ip6table_filter ip6_tables nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT ipt_LOG xt_limit iptable_filter xt_multiport iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 ip_tables cpufreq_ondemand acpi_cpufreq freq_table mperf ipv6 ext3 jbd microcode iTCO_wdt iTCO_vendor_support joydev sb_edac edac_core sg lpc_ich mfd_core igb i2c_algo_bit ptp pps_core i2c_i801 i2c_core ioatdma dca ext4 jbd2 mbcache sd_mod crc_t10dif sr_mod cdrom ahci isci libsas scsi_transport_sas wmi dm_mirror dm_region_hash dm_log dm_mod [last unloaded: scsi_wait_scan]
<4>[ 1029.789008] Pid: 52063, comm: government Not tainted 2.6.32-642.6.1.el6.x86_64 #1
<4>[ 1029.789464] Call Trace:
<4>[ 1029.789686] [<ffffffff8107c6f1>] ? warn_slowpath_common+0x91/0xe0
<4>[ 1029.789916] [<ffffffff8107c75a>] ? warn_slowpath_null+0x1a/0x20
<4>[ 1029.790168] [<ffffffff814a6223>] ? netlink_release+0x283/0x290
<4>[ 1029.790395] [<ffffffff8154c03b>] ? _spin_unlock_bh+0x1b/0x20
<4>[ 1029.790622] [<ffffffff81463bd9>] ? sock_release+0x29/0x90
<4>[ 1029.790847] [<ffffffff81463c57>] ? sock_close+0x17/0x30
<4>[ 1029.791073] [<ffffffff8119b5b5>] ? __fput+0xf5/0x210
<4>[ 1029.791328] [<ffffffff8119b6f5>] ? fput+0x25/0x30
<4>[ 1029.791554] [<ffffffff8119681d>] ? filp_close+0x5d/0x90
<4>[ 1029.791780] [<ffffffff811968f5>] ? sys_close+0xa5/0x100
<4>[ 1029.792007] [<ffffffff8100b0d2>] ? system_call_fastpath+0x16/0x1b
<4>[ 1029.792263] ---[ end trace 2afd564966ab50ef ]---
May i know what may cause of that problem ? Please help. Thank you.
Kernel Panic Centos 6.8
Re: Kernel Panic Centos 6.8
Dear All,
Facing more kernel panic and this time happen on different machine i have, all related with kernel 2.6.32-642.11.1.el6.x86_64:
<6>[15029.234760] imap[5069] trap stack segment ip:7f0ee156a770 sp:7ffcca880720 error:0 in libdovecot.so.0.0.0[7f0ee14c5000+10c000]
<4>[15029.539333] ------------[ cut here ]------------
<4>[15029.549601] WARNING: at mm/mmap.c:2472 exit_mmap+0x165/0x180() (Not tainted)
<4>[15029.559636] Hardware name: S5000VSA
<4>[15029.569521] Modules linked in: xt_owner tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT $
<4>[15029.630622] Pid: 5069, comm: imap Not tainted 2.6.32-642.11.1.el6.x86_64 #1
<4>[15029.640484] Call Trace:
<4>[15029.650249] [<ffffffff8107c6f1>] ? warn_slowpath_common+0x91/0xe0
<4>[15029.659945] [<ffffffff8107c75a>] ? warn_slowpath_null+0x1a/0x20
<4>[15029.669448] [<ffffffff8115d775>] ? exit_mmap+0x165/0x180
<4>[15029.678826] [<ffffffff810799bc>] ? mmput+0x6c/0x120
<4>[15029.688073] [<ffffffff810814db>] ? exit_mm+0x12b/0x180
<4>[15029.697145] [<ffffffff8108188f>] ? do_exit+0x15f/0x870
<4>[15029.706071] [<ffffffff810a68a0>] ? autoremove_wake_function+0x0/0x40
<4>[15029.714863] [<ffffffff81081ff8>] ? do_group_exit+0x58/0xd0
<4>[15029.723536] [<ffffffff81097e06>] ? get_signal_to_deliver+0x1f6/0x460
<4>[15029.732125] [<ffffffff8100a285>] ? do_signal+0x75/0x870
<4>[15029.740599] [<ffffffff81095f6e>] ? send_signal+0x3e/0x90
<4>[15029.748904] [<ffffffff810970e6>] ? force_sig+0x16/0x20
<4>[15029.757039] [<ffffffff8154c9fb>] ? do_trap+0x15b/0x160
<4>[15029.765043] [<ffffffff8100ab10>] ? do_notify_resume+0x90/0xc0
<4>[15029.772933] [<ffffffff8100bb5c>] ? retint_signal+0x48/0x8c
<4>[15029.780668] ---[ end trace eda3d6917f0fcfca ]---
<4>[15041.670167] Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=00:15:17:60:fa:9c:3c:e5:a6:55:87:d8:08:00 SRC=92.2.151.85 DST=202.75.41.63 LEN=40 TOS=0x00 PREC=0x00 TTL=52 ID=49042 PROTO=TCP SPT=52187 DPT=7547 $
<1>[15046.381725] BUG: unable to handle kernel NULL pointer dereference at (null)
<1>[15046.382606] IP: [<ffffffff8117fde4>] s_show+0xe4/0x330
<4>[15046.382606] PGD 3bac51067 PUD 31128b067 PMD 0
<4>[15046.382606] Oops: 0000 [#1] SMP
<4>[15046.382606] last sysfs file: /sys/module/inet_diag/refcnt
<4>[15046.382606] CPU 7
<4>[15046.382606] Modules linked in: xt_owner tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT $
<4>[15046.382606]
<4>[15046.382606] Pid: 5586, comm: sosreport Tainted: G W -- ------------ 2.6.32-642.11.1.el6.x86_64 #1 Intel S5000VSA/S5000VSA
<4>[15046.382606] RIP: 0010:[<ffffffff8117fde4>] [<ffffffff8117fde4>] s_show+0xe4/0x330
<4>[15046.382606] RSP: 0018:ffff880330067d98 EFLAGS: 00010086
<4>[15046.382606] RAX: ffff88045fc21140 RBX: 0000000000000282 RCX: ffff88045fc21180
<4>[15046.382606] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88045fc21150
<4>[15046.382606] RBP: ffff880330067e18 R08: 000000000000003b R09: 000000000000003b
<4>[15046.382606] R10: ffffffff817cf439 R11: 0000000000000004 R12: 00000000000093f6
<4>[15046.382606] R13: ffffffff817cf439 R14: 0000000000000b49 R15: 0000000000000000
<4>[15046.382606] FS: 00007f88b0cc7700(0000) GS:ffff8800283c0000(0000) knlGS:0000000000000000
<4>[15046.382606] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[15046.382606] CR2: 0000000000000000 CR3: 00000003a9532000 CR4: 00000000000007e0
<4>[15046.382606] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>[15046.382606] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>[15046.382606] Process sosreport (pid: 5586, threadinfo ffff880330064000, task ffff880448d59520)
<4>[15046.382606] Stack:
<4>[15046.382606] ffff88030000003b ffff880300000001 ffff880300000000 ffff88045fc21180
<4>[15046.382606] <d> ffff88045fc21140 ffff8802434c79c0 ffff88045fc48158 0000000000000000
<4>[15046.382606] <d> 0000000000000000 ffff88045fc40100 ffff880330067e18 ffff880245835cc0
<4>[15046.382606] Call Trace:
<4>[15046.382606] [<ffffffff811bfdc9>] seq_read+0x289/0x400
<4>[15046.382606] [<ffffffff8120583e>] proc_reg_read+0x7e/0xc0
<4>[15046.382606] [<ffffffff8119a3a5>] vfs_read+0xb5/0x1a0
<4>[15046.382606] [<ffffffff8119b156>] ? fget_light_pos+0x16/0x50
<4>[15046.382606] [<ffffffff8119a6f1>] sys_read+0x51/0xb0
<4>[15046.382606] [<ffffffff810ee25e>] ? __audit_syscall_exit+0x25e/0x290
<4>[15046.382606] [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
<4>[15046.382606] Code: 10 48 39 fe 74 34 4c 8b 45 c8 45 8b 88 18 80 00 00 45 89 c8 0f 1f 00 4d 85 ed 75 0f 44 39 4e 20 49 c7 c2 39 f4 7c 81 4d 0f 45 ea <48> 8b 36 4d 01 c4 48 83 c3 01 48 39 fe 75 dd 48 8b 30 48 39 f0
<1>[15046.382606] RIP [<ffffffff8117fde4>] s_show+0xe4/0x330
<4>[15046.382606] RSP <ffff880330067d98>
<4>[15046.382606] CR2: 0000000000000000
What may cause of this ? TQ
Facing more kernel panic and this time happen on different machine i have, all related with kernel 2.6.32-642.11.1.el6.x86_64:
<6>[15029.234760] imap[5069] trap stack segment ip:7f0ee156a770 sp:7ffcca880720 error:0 in libdovecot.so.0.0.0[7f0ee14c5000+10c000]
<4>[15029.539333] ------------[ cut here ]------------
<4>[15029.549601] WARNING: at mm/mmap.c:2472 exit_mmap+0x165/0x180() (Not tainted)
<4>[15029.559636] Hardware name: S5000VSA
<4>[15029.569521] Modules linked in: xt_owner tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT $
<4>[15029.630622] Pid: 5069, comm: imap Not tainted 2.6.32-642.11.1.el6.x86_64 #1
<4>[15029.640484] Call Trace:
<4>[15029.650249] [<ffffffff8107c6f1>] ? warn_slowpath_common+0x91/0xe0
<4>[15029.659945] [<ffffffff8107c75a>] ? warn_slowpath_null+0x1a/0x20
<4>[15029.669448] [<ffffffff8115d775>] ? exit_mmap+0x165/0x180
<4>[15029.678826] [<ffffffff810799bc>] ? mmput+0x6c/0x120
<4>[15029.688073] [<ffffffff810814db>] ? exit_mm+0x12b/0x180
<4>[15029.697145] [<ffffffff8108188f>] ? do_exit+0x15f/0x870
<4>[15029.706071] [<ffffffff810a68a0>] ? autoremove_wake_function+0x0/0x40
<4>[15029.714863] [<ffffffff81081ff8>] ? do_group_exit+0x58/0xd0
<4>[15029.723536] [<ffffffff81097e06>] ? get_signal_to_deliver+0x1f6/0x460
<4>[15029.732125] [<ffffffff8100a285>] ? do_signal+0x75/0x870
<4>[15029.740599] [<ffffffff81095f6e>] ? send_signal+0x3e/0x90
<4>[15029.748904] [<ffffffff810970e6>] ? force_sig+0x16/0x20
<4>[15029.757039] [<ffffffff8154c9fb>] ? do_trap+0x15b/0x160
<4>[15029.765043] [<ffffffff8100ab10>] ? do_notify_resume+0x90/0xc0
<4>[15029.772933] [<ffffffff8100bb5c>] ? retint_signal+0x48/0x8c
<4>[15029.780668] ---[ end trace eda3d6917f0fcfca ]---
<4>[15041.670167] Firewall: *TCP_IN Blocked* IN=eth0 OUT= MAC=00:15:17:60:fa:9c:3c:e5:a6:55:87:d8:08:00 SRC=92.2.151.85 DST=202.75.41.63 LEN=40 TOS=0x00 PREC=0x00 TTL=52 ID=49042 PROTO=TCP SPT=52187 DPT=7547 $
<1>[15046.381725] BUG: unable to handle kernel NULL pointer dereference at (null)
<1>[15046.382606] IP: [<ffffffff8117fde4>] s_show+0xe4/0x330
<4>[15046.382606] PGD 3bac51067 PUD 31128b067 PMD 0
<4>[15046.382606] Oops: 0000 [#1] SMP
<4>[15046.382606] last sysfs file: /sys/module/inet_diag/refcnt
<4>[15046.382606] CPU 7
<4>[15046.382606] Modules linked in: xt_owner tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT $
<4>[15046.382606]
<4>[15046.382606] Pid: 5586, comm: sosreport Tainted: G W -- ------------ 2.6.32-642.11.1.el6.x86_64 #1 Intel S5000VSA/S5000VSA
<4>[15046.382606] RIP: 0010:[<ffffffff8117fde4>] [<ffffffff8117fde4>] s_show+0xe4/0x330
<4>[15046.382606] RSP: 0018:ffff880330067d98 EFLAGS: 00010086
<4>[15046.382606] RAX: ffff88045fc21140 RBX: 0000000000000282 RCX: ffff88045fc21180
<4>[15046.382606] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88045fc21150
<4>[15046.382606] RBP: ffff880330067e18 R08: 000000000000003b R09: 000000000000003b
<4>[15046.382606] R10: ffffffff817cf439 R11: 0000000000000004 R12: 00000000000093f6
<4>[15046.382606] R13: ffffffff817cf439 R14: 0000000000000b49 R15: 0000000000000000
<4>[15046.382606] FS: 00007f88b0cc7700(0000) GS:ffff8800283c0000(0000) knlGS:0000000000000000
<4>[15046.382606] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[15046.382606] CR2: 0000000000000000 CR3: 00000003a9532000 CR4: 00000000000007e0
<4>[15046.382606] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>[15046.382606] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>[15046.382606] Process sosreport (pid: 5586, threadinfo ffff880330064000, task ffff880448d59520)
<4>[15046.382606] Stack:
<4>[15046.382606] ffff88030000003b ffff880300000001 ffff880300000000 ffff88045fc21180
<4>[15046.382606] <d> ffff88045fc21140 ffff8802434c79c0 ffff88045fc48158 0000000000000000
<4>[15046.382606] <d> 0000000000000000 ffff88045fc40100 ffff880330067e18 ffff880245835cc0
<4>[15046.382606] Call Trace:
<4>[15046.382606] [<ffffffff811bfdc9>] seq_read+0x289/0x400
<4>[15046.382606] [<ffffffff8120583e>] proc_reg_read+0x7e/0xc0
<4>[15046.382606] [<ffffffff8119a3a5>] vfs_read+0xb5/0x1a0
<4>[15046.382606] [<ffffffff8119b156>] ? fget_light_pos+0x16/0x50
<4>[15046.382606] [<ffffffff8119a6f1>] sys_read+0x51/0xb0
<4>[15046.382606] [<ffffffff810ee25e>] ? __audit_syscall_exit+0x25e/0x290
<4>[15046.382606] [<ffffffff8100b0d2>] system_call_fastpath+0x16/0x1b
<4>[15046.382606] Code: 10 48 39 fe 74 34 4c 8b 45 c8 45 8b 88 18 80 00 00 45 89 c8 0f 1f 00 4d 85 ed 75 0f 44 39 4e 20 49 c7 c2 39 f4 7c 81 4d 0f 45 ea <48> 8b 36 4d 01 c4 48 83 c3 01 48 39 fe 75 dd 48 8b 30 48 39 f0
<1>[15046.382606] RIP [<ffffffff8117fde4>] s_show+0xe4/0x330
<4>[15046.382606] RSP <ffff880330067d98>
<4>[15046.382606] CR2: 0000000000000000
What may cause of this ? TQ
Re: Kernel Panic Centos 6.8
I'd take it offline and run memtest86+ for a day or two.
The future appears to be RHEL or Debian. I think I'm going Debian.
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Info for USB installs on http://wiki.centos.org/HowTos/InstallFromUSBkey
CentOS 5 and 6 are deadest, do not use them.
Use the FAQ Luke
Re: Kernel Panic Centos 6.8
Hi,
Thanks fo reply, but even i change the new ram, the problem still occured, i also check on temperature and all is fine, what i gonna face now the server will automatically reboot 2 - 3 times per day, /var/crash are reporting same thing as my previous post, this is very strange
Any advice please ?
Thanks fo reply, but even i change the new ram, the problem still occured, i also check on temperature and all is fine, what i gonna face now the server will automatically reboot 2 - 3 times per day, /var/crash are reporting same thing as my previous post, this is very strange
Any advice please ?
Re: Kernel Panic Centos 6.8
This is the latest vmcore-dmesg report:
<4>[45621.367287] ------------[ cut here ]------------
<4>[45621.368254] WARNING: at lib/list_debug.c:48 list_del+0x6e/0xa0() (Not tainted)
<4>[45621.368254] Hardware name: S5000VSA
<4>[45621.368254] list_del corruption. prev->next should be ffff88045623b000, but was (null)
<4>[45621.368254] Modules linked in: tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT ipt_LOG x$
<4>[45621.368254] Pid: 4900, comm: php Not tainted 2.6.32-642.11.1.el6.x86_64 #1
<4>[45621.368254] Call Trace:
<4>[45621.368254] <IRQ> [<ffffffff8107c6f1>] ? warn_slowpath_common+0x91/0xe0
<4>[45621.368254] [<ffffffff8107c7f6>] ? warn_slowpath_fmt+0x46/0x60
<4>[45621.368254] [<ffffffff812ae7ae>] ? list_del+0x6e/0xa0
<4>[45621.368254] [<ffffffff81181388>] ? free_block+0xc8/0x170
<4>[45621.368254] [<ffffffff810a9e9f>] ? run_posix_cpu_timers+0x4bf/0x8f0
<4>[45621.368254] [<ffffffff811811c8>] ? kmem_cache_free+0x248/0x2b0
<4>[45621.368254] [<ffffffff8119b083>] ? file_free_rcu+0x33/0x40
<4>[45621.368254] [<ffffffff810f9275>] ? __rcu_process_callbacks+0x135/0x350
<4>[45621.368254] [<ffffffff810b2a7d>] ? ktime_get+0x6d/0x100
<4>[45621.368254] [<ffffffff810f94bb>] ? rcu_process_callbacks+0x2b/0x50
<4>[45621.368254] [<ffffffff81085275>] ? __do_softirq+0xe5/0x230
<4>[45621.368254] [<ffffffff8100c38c>] ? call_softirq+0x1c/0x30
<4>[45621.368254] [<ffffffff8100fca5>] ? do_softirq+0x65/0xa0
<4>[45621.368254] [<ffffffff81085105>] ? irq_exit+0x85/0x90
<4>[45621.368254] [<ffffffff81552d2a>] ? smp_apic_timer_interrupt+0x4a/0x60
<4>[45621.368254] [<ffffffff8100bc13>] ? apic_timer_interrupt+0x13/0x20
<4>[45621.368254] <EOI>
<4>[45621.368254] ---[ end trace 254cc2419ab31e29 ]---
<3>[45651.424819] slab: cache filp error: slabs_full accounting error
<3>[45654.160279] slab: cache filp error: slabs_full accounting error
<4>[45697.466036] ------------[ cut here ]------------
<4>[45697.467004] WARNING: at lib/list_debug.c:51 list_del+0x8d/0xa0() (Tainted: G W -- ------------ )
<4>[45697.467004] Hardware name: S5000VSA
<4>[45697.467004] list_del corruption. next->prev should be ffff8803e06d3040, but was (null)
<4>[45697.467004] Modules linked in: tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT ipt_LOG x$
<4>[45697.467004] Pid: 6147, comm: mysqld Tainted: G W -- ------------ 2.6.32-642.11.1.el6.x86_64 #1
<4>[45697.467004] Call Trace:
<4>[45697.467004] <IRQ> [<ffffffff8107c6f1>] ? warn_slowpath_common+0x91/0xe0
<4>[45697.467004] [<ffffffff8107c7f6>] ? warn_slowpath_fmt+0x46/0x60
<4>[45697.467004] [<ffffffff812ae7cd>] ? list_del+0x8d/0xa0
<4>[45697.467004] [<ffffffff81181388>] ? free_block+0xc8/0x170
<4>[45697.467004] [<ffffffff811811c8>] ? kmem_cache_free+0x248/0x2b0
<4>[45697.467004] [<ffffffff8119b083>] ? file_free_rcu+0x33/0x40
<4>[45697.467004] [<ffffffff810f9275>] ? __rcu_process_callbacks+0x135/0x350
<4>[45697.467004] [<ffffffff810b2a7d>] ? ktime_get+0x6d/0x100
<4>[45697.467004] [<ffffffff810f94bb>] ? rcu_process_callbacks+0x2b/0x50
<4>[45697.467004] [<ffffffff81085275>] ? __do_softirq+0xe5/0x230
<4>[45697.467004] [<ffffffff8100c38c>] ? call_softirq+0x1c/0x30
<4>[45697.467004] [<ffffffff8100fca5>] ? do_softirq+0x65/0xa0
<4>[45697.467004] [<ffffffff81085105>] ? irq_exit+0x85/0x90
<4>[45697.467004] [<ffffffff81552d2a>] ? smp_apic_timer_interrupt+0x4a/0x60
<4>[45697.467004] [<ffffffff8100bc13>] ? apic_timer_interrupt+0x13/0x20
<4>[45697.467004] <EOI>
<4>[45697.467004] ---[ end trace 254cc2419ab31e2a ]---
<2>[45731.271130] kernel BUG at mm/slab.c:3069!
<4>[45731.271130] invalid opcode: 0000 [#1] SMP
<4>[45731.271130] last sysfs file: /sys/devices/platform/coretemp.1/temp2_input
<4>[45731.271130] CPU 2
<4>[45731.271130] Modules linked in: tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT ipt_LOG x$
<4>[45731.271130]
<4>[45731.271130] Pid: 14030, comm: php Tainted: G W -- ------------ 2.6.32-642.11.1.el6.x86_64 #1 Intel S5000VSA/S5000VSA
<4>[45731.271130] RIP: 0010:[<ffffffff8117f7f4>] [<ffffffff8117f7f4>] cache_alloc_refill+0x1e4/0x240
<4>[45731.359394] RSP: 0018:ffff880017a07c88 EFLAGS: 00010086
<4>[45731.359394] RAX: 000000000000001a RBX: ffff88045a081000 RCX: 00000000ffffffff
<4>[45731.359394] RDX: ffff88042b274000 RSI: ffff88045fef36d0 RDI: ffff880360b0c000
<4>[45731.359394] RBP: ffff880017a07ce8 R08: ffff88042b274000 R09: 0000000000000000
<4>[45731.359394] R10: 0000000000000001 R11: 0000000000000000 R12: ffff88045a2bb400
<4>[45731.359394] R13: ffff88045fef36c0 R14: 000000000000001a R15: ffff8803d0589000
<4>[45731.359394] FS: 0000000000000000(0000) GS:ffff880028280000(0000) knlGS:0000000000000000
<4>[45731.359394] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[45731.359394] CR2: 00000034956001b8 CR3: 0000000138b27000 CR4: 00000000000007e0
<4>[45731.359394] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>[45731.359394] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>[45731.359394] Process php (pid: 14030, threadinfo ffff880017a04000, task ffff880138ba8ab0)
<4>[45731.359394] Stack:
<4>[45731.359394] ffff880138ba8ab0 0000000038ba8ab0 ffff88045fef3700 000492d017a07e58
<4>[45731.359394] <d> ffff88045fef36e0 ffff88045fef36d0 0000000000000000 0000000000000000
<4>[45731.359394] <d> 00000000000080d0 ffff88045a081000 00000000000080d0 0000000000000246
<4>[45731.359394] Call Trace:
<4>[45731.359394] [<ffffffff81180737>] kmem_cache_alloc+0x167/0x190
<4>[45731.359394] [<ffffffff8119b2d9>] get_empty_filp+0x79/0x190
<4>[45731.359394] [<ffffffff811ac886>] do_filp_open+0xa6/0xd20
<4>[45621.367287] ------------[ cut here ]------------
<4>[45621.368254] WARNING: at lib/list_debug.c:48 list_del+0x6e/0xa0() (Not tainted)
<4>[45621.368254] Hardware name: S5000VSA
<4>[45621.368254] list_del corruption. prev->next should be ffff88045623b000, but was (null)
<4>[45621.368254] Modules linked in: tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT ipt_LOG x$
<4>[45621.368254] Pid: 4900, comm: php Not tainted 2.6.32-642.11.1.el6.x86_64 #1
<4>[45621.368254] Call Trace:
<4>[45621.368254] <IRQ> [<ffffffff8107c6f1>] ? warn_slowpath_common+0x91/0xe0
<4>[45621.368254] [<ffffffff8107c7f6>] ? warn_slowpath_fmt+0x46/0x60
<4>[45621.368254] [<ffffffff812ae7ae>] ? list_del+0x6e/0xa0
<4>[45621.368254] [<ffffffff81181388>] ? free_block+0xc8/0x170
<4>[45621.368254] [<ffffffff810a9e9f>] ? run_posix_cpu_timers+0x4bf/0x8f0
<4>[45621.368254] [<ffffffff811811c8>] ? kmem_cache_free+0x248/0x2b0
<4>[45621.368254] [<ffffffff8119b083>] ? file_free_rcu+0x33/0x40
<4>[45621.368254] [<ffffffff810f9275>] ? __rcu_process_callbacks+0x135/0x350
<4>[45621.368254] [<ffffffff810b2a7d>] ? ktime_get+0x6d/0x100
<4>[45621.368254] [<ffffffff810f94bb>] ? rcu_process_callbacks+0x2b/0x50
<4>[45621.368254] [<ffffffff81085275>] ? __do_softirq+0xe5/0x230
<4>[45621.368254] [<ffffffff8100c38c>] ? call_softirq+0x1c/0x30
<4>[45621.368254] [<ffffffff8100fca5>] ? do_softirq+0x65/0xa0
<4>[45621.368254] [<ffffffff81085105>] ? irq_exit+0x85/0x90
<4>[45621.368254] [<ffffffff81552d2a>] ? smp_apic_timer_interrupt+0x4a/0x60
<4>[45621.368254] [<ffffffff8100bc13>] ? apic_timer_interrupt+0x13/0x20
<4>[45621.368254] <EOI>
<4>[45621.368254] ---[ end trace 254cc2419ab31e29 ]---
<3>[45651.424819] slab: cache filp error: slabs_full accounting error
<3>[45654.160279] slab: cache filp error: slabs_full accounting error
<4>[45697.466036] ------------[ cut here ]------------
<4>[45697.467004] WARNING: at lib/list_debug.c:51 list_del+0x8d/0xa0() (Tainted: G W -- ------------ )
<4>[45697.467004] Hardware name: S5000VSA
<4>[45697.467004] list_del corruption. next->prev should be ffff8803e06d3040, but was (null)
<4>[45697.467004] Modules linked in: tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT ipt_LOG x$
<4>[45697.467004] Pid: 6147, comm: mysqld Tainted: G W -- ------------ 2.6.32-642.11.1.el6.x86_64 #1
<4>[45697.467004] Call Trace:
<4>[45697.467004] <IRQ> [<ffffffff8107c6f1>] ? warn_slowpath_common+0x91/0xe0
<4>[45697.467004] [<ffffffff8107c7f6>] ? warn_slowpath_fmt+0x46/0x60
<4>[45697.467004] [<ffffffff812ae7cd>] ? list_del+0x8d/0xa0
<4>[45697.467004] [<ffffffff81181388>] ? free_block+0xc8/0x170
<4>[45697.467004] [<ffffffff811811c8>] ? kmem_cache_free+0x248/0x2b0
<4>[45697.467004] [<ffffffff8119b083>] ? file_free_rcu+0x33/0x40
<4>[45697.467004] [<ffffffff810f9275>] ? __rcu_process_callbacks+0x135/0x350
<4>[45697.467004] [<ffffffff810b2a7d>] ? ktime_get+0x6d/0x100
<4>[45697.467004] [<ffffffff810f94bb>] ? rcu_process_callbacks+0x2b/0x50
<4>[45697.467004] [<ffffffff81085275>] ? __do_softirq+0xe5/0x230
<4>[45697.467004] [<ffffffff8100c38c>] ? call_softirq+0x1c/0x30
<4>[45697.467004] [<ffffffff8100fca5>] ? do_softirq+0x65/0xa0
<4>[45697.467004] [<ffffffff81085105>] ? irq_exit+0x85/0x90
<4>[45697.467004] [<ffffffff81552d2a>] ? smp_apic_timer_interrupt+0x4a/0x60
<4>[45697.467004] [<ffffffff8100bc13>] ? apic_timer_interrupt+0x13/0x20
<4>[45697.467004] <EOI>
<4>[45697.467004] ---[ end trace 254cc2419ab31e2a ]---
<2>[45731.271130] kernel BUG at mm/slab.c:3069!
<4>[45731.271130] invalid opcode: 0000 [#1] SMP
<4>[45731.271130] last sysfs file: /sys/devices/platform/coretemp.1/temp2_input
<4>[45731.271130] CPU 2
<4>[45731.271130] Modules linked in: tcp_diag inet_diag autofs4 ipmi_devintf coretemp ipmi_si ipmi_msghandler ip6t_LOG nf_nat_ftp ipt_REDIRECT xt_conntrack iptable_mangle nf_conntrack_ftp ipt_REJECT ipt_LOG x$
<4>[45731.271130]
<4>[45731.271130] Pid: 14030, comm: php Tainted: G W -- ------------ 2.6.32-642.11.1.el6.x86_64 #1 Intel S5000VSA/S5000VSA
<4>[45731.271130] RIP: 0010:[<ffffffff8117f7f4>] [<ffffffff8117f7f4>] cache_alloc_refill+0x1e4/0x240
<4>[45731.359394] RSP: 0018:ffff880017a07c88 EFLAGS: 00010086
<4>[45731.359394] RAX: 000000000000001a RBX: ffff88045a081000 RCX: 00000000ffffffff
<4>[45731.359394] RDX: ffff88042b274000 RSI: ffff88045fef36d0 RDI: ffff880360b0c000
<4>[45731.359394] RBP: ffff880017a07ce8 R08: ffff88042b274000 R09: 0000000000000000
<4>[45731.359394] R10: 0000000000000001 R11: 0000000000000000 R12: ffff88045a2bb400
<4>[45731.359394] R13: ffff88045fef36c0 R14: 000000000000001a R15: ffff8803d0589000
<4>[45731.359394] FS: 0000000000000000(0000) GS:ffff880028280000(0000) knlGS:0000000000000000
<4>[45731.359394] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[45731.359394] CR2: 00000034956001b8 CR3: 0000000138b27000 CR4: 00000000000007e0
<4>[45731.359394] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<4>[45731.359394] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<4>[45731.359394] Process php (pid: 14030, threadinfo ffff880017a04000, task ffff880138ba8ab0)
<4>[45731.359394] Stack:
<4>[45731.359394] ffff880138ba8ab0 0000000038ba8ab0 ffff88045fef3700 000492d017a07e58
<4>[45731.359394] <d> ffff88045fef36e0 ffff88045fef36d0 0000000000000000 0000000000000000
<4>[45731.359394] <d> 00000000000080d0 ffff88045a081000 00000000000080d0 0000000000000246
<4>[45731.359394] Call Trace:
<4>[45731.359394] [<ffffffff81180737>] kmem_cache_alloc+0x167/0x190
<4>[45731.359394] [<ffffffff8119b2d9>] get_empty_filp+0x79/0x190
<4>[45731.359394] [<ffffffff811ac886>] do_filp_open+0xa6/0xd20
Re: Kernel Panic Centos 6.8
halimzhz wrote:Hi,
Thanks fo reply, but even i change the new ram, the problem still occured, i also check on temperature and all is fine, what i gonna face now the server will automatically reboot 2 - 3 times per day, /var/crash are reporting same thing as my previous post, this is very strange
Any advice please ?
You have bad hardware.
Re: Kernel Panic Centos 6.8
Dear Whoever,
You think i should change the motherboard ?
TQ
You think i should change the motherboard ?
TQ
Re: Kernel Panic Centos 6.8
Run memtest on it first.