Kernel call trace

rutekp · Post by **rutekp** » 2012/07/18 09:09:26

Hello,

Can anyone help trace what hardware/process generate error paste below ? Those error create usually one time per 14 days - so no often. Server is Centos 6, kernel 3.3.7:
Linux zzzzzzzzzzzzzzz 3.3.7 #2 SMP Sun May 27 07:35:49 EDT 2012 x86_64 x86_64 x86_64 GNU/Linux

Thanks
Pawel R.

[quote]
php-cgi: page allocation failure: order:1, mode:0x20
Pid: 1472, comm: php-cgi Not tainted 3.3.7 #2
Call Trace:
[] warn_alloc_failed+0xf3/0x160
[] ? wakeup_kswapd+0x10c/0x140
[] __alloc_pages_slowpath+0x42a/0x710
[] ? zone_watermark_ok+0x1f/0x30
[] __alloc_pages_nodemask+0x1a4/0x1f0
[] kmem_getpages+0x6a/0x180
[] fallback_alloc+0x1b7/0x270
[] ? cache_grow+0x288/0x2b0
[] ____cache_alloc_node+0x96/0x150
[] kmem_cache_alloc+0xfd/0x190
[] sk_prot_alloc+0x48/0x190
[] sk_clone_lock+0x20/0x310
[] inet_csk_clone_lock+0x16/0xd0
[] tcp_create_openreq_child+0x29/0x560
[] tcp_v4_syn_recv_sock+0x41/0x300
[] tcp_check_req+0x226/0x470
[] tcp_v4_hnd_req+0x6e/0x110
[] tcp_v4_do_rcv+0x13b/0x210
[] tcp_v4_rcv+0x5a4/0x830
[] ? ip_rcv+0x300/0x300
[] ? ip_rcv+0x300/0x300
[] ip_local_deliver_finish+0xdd/0x290
[] ip_local_deliver+0x80/0x90
[] ? update_curr+0x18c/0x1d0
[] ip_rcv_finish+0x119/0x3b0
[] ip_rcv+0x238/0x300
[] __netif_receive_skb+0x214/0x4d0
[] process_backlog+0x102/0x1f0
[] net_rx_action+0x103/0x2a0
[] __do_softirq+0xb7/0x210
[] ? call_softirq+0x1c/0x30
[] call_softirq+0x1c/0x30
[] do_softirq+0x65/0xa0
[] local_bh_enable_ip+0x94/0xa0
[] _raw_spin_unlock_bh+0x16/0x20
[] release_sock+0xe3/0x110
[] inet_stream_connect+0x6e/0x200
[] sys_connect+0xe7/0x110
[] ? do_fcntl+0x290/0x3f0
[] ? sock_map_fd+0x2a/0x40
[] ? sys_fcntl+0x76/0xb0
[] system_call_fastpath+0x16/0x1b

Pid: 18517, comm: php-cgi Not tainted 3.3.7 #2
Call Trace:
[] warn_alloc_failed+0xf3/0x160
[] ? wakeup_kswapd+0x10c/0x140
[] __alloc_pages_slowpath+0x42a/0x710
[] ? zone_watermark_ok+0x1f/0x30
[] __alloc_pages_nodemask+0x1a4/0x1f0
[] kmem_getpages+0x6a/0x180
[] fallback_alloc+0x1b7/0x270
[] ? cache_grow+0x288/0x2b0
[] ____cache_alloc_node+0x96/0x150
[] kmem_cache_alloc+0xfd/0x190
[] sk_prot_alloc+0x48/0x190
[] sk_clone_lock+0x20/0x310
[] inet_csk_clone_lock+0x16/0xd0
[] tcp_create_openreq_child+0x29/0x560
[] tcp_v4_syn_recv_sock+0x41/0x300
[] tcp_check_req+0x226/0x470
[] tcp_v4_hnd_req+0x6e/0x110
[] tcp_v4_do_rcv+0x13b/0x210
[] tcp_v4_rcv+0x5a4/0x830
[] ? ip_rcv+0x300/0x300
[] ? ip_rcv+0x300/0x300
[] ip_local_deliver_finish+0xdd/0x290
[] ip_local_deliver+0x80/0x90
[] ? update_curr+0x18c/0x1d0
[] ip_rcv_finish+0x119/0x3b0
[] ip_rcv+0x238/0x300
[] __netif_receive_skb+0x214/0x4d0
[] process_backlog+0x102/0x1f0
[] net_rx_action+0x103/0x2a0
[] __do_softirq+0xb7/0x210
[] ? call_softirq+0x1c/0x30
[] call_softirq+0x1c/0x30
[] do_softirq+0x65/0xa0
[] local_bh_enable_ip+0x94/0xa0
[] _raw_spin_unlock_bh+0x16/0x20
[] release_sock+0xe3/0x110
[] inet_stream_connect+0x6e/0x200
[] sys_connect+0xe7/0x110
[] ? do_fcntl+0x290/0x3f0
[] ? sock_map_fd+0x2a/0x40
[] ? sys_fcntl+0x76/0xb0
[] system_call_fastpath+0x16/0x1b

exim: page allocation failure: order:1, mode:0x20
Pid: 23467, comm: exim Not tainted 3.3.7 #2
Call Trace:
[] warn_alloc_failed+0xf3/0x160
[] ? wakeup_kswapd+0x10c/0x140
[] __alloc_pages_slowpath+0x42a/0x710
[] ? zone_watermark_ok+0x1f/0x30
[] __alloc_pages_nodemask+0x1a4/0x1f0
[] kmem_getpages+0x6a/0x180
[] fallback_alloc+0x1b7/0x270
[] ? cache_grow+0x288/0x2b0
[] ____cache_alloc_node+0x96/0x150
[] kmem_cache_alloc+0xfd/0x190
[] sk_prot_alloc+0x48/0x190
[] sk_clone_lock+0x20/0x310
[] inet_csk_clone_lock+0x16/0xd0
[] tcp_create_openreq_child+0x29/0x560
[] tcp_v4_syn_recv_sock+0x41/0x300
[] tcp_check_req+0x226/0x470
[] tcp_v4_hnd_req+0x6e/0x110
[] tcp_v4_do_rcv+0x13b/0x210
[] tcp_v4_rcv+0x5a4/0x830
[] ? ip_rcv+0x300/0x300
[] ? ip_rcv+0x300/0x300
[] ip_local_deliver_finish+0xdd/0x290
[] ip_local_deliver+0x80/0x90
[] ip_rcv_finish+0x119/0x3b0
[] ip_rcv+0x238/0x300
[] __netif_receive_skb+0x214/0x4d0
[] ? rcu_start_gp+0x203/0x2b0
[] process_backlog+0x102/0x1f0
[] net_rx_action+0x103/0x2a0
[] __do_softirq+0xb7/0x210
[] ? call_softirq+0x1c/0x30
[] call_softirq+0x1c/0x30
[] do_softirq+0x65/0xa0
[] local_bh_enable_ip+0x94/0xa0
[] _raw_spin_unlock_bh+0x16/0x20
[] release_sock+0xe3/0x110
[] inet_wait_for_connect+0xa2/0xf0
[] ? wake_up_bit+0x40/0x40
[] inet_stream_connect+0x11e/0x200
[] sys_connect+0xe7/0x110
[] ? hrtimer_start+0x18/0x20
[] ? do_setitimer+0x1a8/0x220
[] ? alarm_setitimer+0x3a/0x60
[] system_call_fastpath+0x16/0x1b

exim: page allocation failure: order:1, mode:0x20
Pid: 23467, comm: exim Not tainted 3.3.7 #2
Call Trace:
[] warn_alloc_failed+0xf3/0x160
[] ? wakeup_kswapd+0x10c/0x140
[] __alloc_pages_slowpath+0x42a/0x710
[] ? zone_watermark_ok+0x1f/0x30
[] __alloc_pages_nodemask+0x1a4/0x1f0
[] kmem_getpages+0x6a/0x180
[] fallback_alloc+0x1b7/0x270
[] ? cache_grow+0x288/0x2b0
[] ____cache_alloc_node+0x96/0x150
[] kmem_cache_alloc+0xfd/0x190
[] sk_prot_alloc+0x48/0x190
[] sk_clone_lock+0x20/0x310
[] inet_csk_clone_lock+0x16/0xd0
[] tcp_create_openreq_child+0x29/0x560
[] tcp_v4_syn_recv_sock+0x41/0x300
[] tcp_check_req+0x226/0x470
[] tcp_v4_hnd_req+0x6e/0x110
[] ? __kfree_skb+0x47/0xa0
[] tcp_v4_do_rcv+0x13b/0x210
[] tcp_v4_rcv+0x5a4/0x830
[] ? ip_rcv+0x300/0x300
[] ? ip_rcv+0x300/0x300
[] ip_local_deliver_finish+0xdd/0x290
[] ip_local_deliver+0x80/0x90
[] ip_rcv_finish+0x119/0x3b0
[] ip_rcv+0x238/0x300
[] __netif_receive_skb+0x214/0x4d0
[] ? rcu_start_gp+0x203/0x2b0
[] process_backlog+0x102/0x1f0
[] net_rx_action+0x103/0x2a0
[] __do_softirq+0xb7/0x210
[] ? call_softirq+0x1c/0x30
[] call_softirq+0x1c/0x30
[] do_softirq+0x65/0xa0
[] local_bh_enable+0x94/0xa0
[] dev_queue_xmit+0x18b/0x3e0
[] ip_finish_output+0x183/0x310
[] ip_output+0x98/0xa0
[] ? __ip_local_out+0xa3/0xb0
[] ip_local_out+0x29/0x30
[] ip_queue_xmit+0x15a/0x3f0
[] tcp_transmit_skb+0x42a/0x730
[] tcp_write_xmit+0x1f9/0x500
[] ? ksize+0x19/0x80
[] __tcp_push_pending_frames+0x26/0xc0
[] tcp_push+0x69/0x90
[] tcp_sendmsg+0x5bf/0xc60
[] inet_sendmsg+0x48/0xb0
[] ? __schedule+0x168/0x730
[] sock_sendmsg+0xf8/0x130
[] ? schedule+0x3f/0x60
[] ? schedule_timeout+0x1fd/0x2e0
[] ? tcp_rcv_state_process+0x97/0x4b0
[] ? prepare_to_wait+0x5e/0x90
[] ? inet_wait_for_connect+0xdb/0xf0
[] sys_sendto+0x139/0x190
[] ? do_setitimer+0xd4/0x220
[] ? alarm_setitimer+0x3a/0x60
[] system_call_fastpath+0x16/0x1b
[/quote]

gerald_clark · Post by **gerald_clark** » 2012/07/18 16:07:32

That is not a CentOS kernel.

rutekp · Post by **rutekp** » 2012/07/19 09:43:27

Yes, i know, but maybe someone here who know who to read call trace ?