[Varnish] #1293: varnishd: page allocation failure

Varnish varnish-bugs at varnish-cache.org
Wed Apr 10 20:50:45 CEST 2013


#1293: varnishd: page allocation failure
------------------------+--------------------
 Reporter:  msallen333  |       Type:  defect
   Status:  new         |   Priority:  normal
Milestone:              |  Component:  build
  Version:  3.0.3       |   Severity:  normal
 Keywords:              |
------------------------+--------------------
 Varnishd recently "hung" on my system with the below error
 /var/log/messages and had to be restarted. Does this appear to be a
 varnish 3.0.3 defect?

 Apr  9 15:42:59 lx11 kernel: __ratelimit: 26 callbacks suppressed
 Apr  9 15:42:59 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:42:59 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:42:59 lx11 kernel: Call Trace:
 Apr  9 15:42:59 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:42:59 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:42:59 lx11 kernel: [<ffffffff810a7bda>] ?
 tick_program_event+0x2a/0x30
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100bcb3>] ?
 invalidate_interrupt7+0x13/0x20
 Apr  9 15:42:59 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:42:59 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:42:59 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:42:59 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:42:59 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:42:59 lx11 kernel: <EOI>
 Apr  9 15:42:59 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:42:59 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:42:59 lx11 kernel: Call Trace:
 Apr  9 15:42:59 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:42:59 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8149473b>] ?
 tcp_validate_incoming+0x30b/0x3a0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100bc53>] ?
 invalidate_interrupt4+0x13/0x20
 Apr  9 15:42:59 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:42:59 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:42:59 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:42:59 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:42:59 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:42:59 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:42:59 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:42:59 lx11 kernel: <EOI>
 Apr  9 15:42:59 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:42:59 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:42:59 lx11 kernel: Call Trace:
 Apr  9 15:43:02 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:02 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149473b>] ?
 tcp_validate_incoming+0x30b/0x3a0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0348557>] ?
 ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:43:02 lx11 kernel: <EOI>
 Apr  9 15:43:02 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:43:02 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:43:02 lx11 kernel: Call Trace:
 Apr  9 15:43:02 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:02 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0348557>] ?
 ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:43:02 lx11 kernel: <EOI>
 Apr  9 15:43:02 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:43:02 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:43:02 lx11 kernel: Call Trace:
 Apr  9 15:43:02 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:02 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0348557>] ?
 ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:43:02 lx11 kernel: <EOI>
 Apr  9 15:43:02 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:43:02 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:43:02 lx11 kernel: Call Trace:
 Apr  9 15:43:02 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:02 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149473b>] ?
 tcp_validate_incoming+0x30b/0x3a0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8104c950>] ?
 smp_invalidate_interrupt+0x60/0xc0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:43:02 lx11 kernel: <EOI>
 Apr  9 15:43:02 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:43:02 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:43:02 lx11 kernel: Call Trace:
 Apr  9 15:43:02 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:02 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa0348557>] ?
 ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:02 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:02 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:02 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:43:07 lx11 kernel: <EOI>
 Apr  9 15:43:07 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:43:07 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:43:07 lx11 kernel: Call Trace:
 Apr  9 15:43:07 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:07 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81494f1d>] ?
 tcp_data_queue+0x3ed/0xc70
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814946f0>] ?
 tcp_validate_incoming+0x2c0/0x3a0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa0348557>] ?
 ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:43:07 lx11 kernel: <EOI>
 Apr  9 15:43:07 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:43:07 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:43:07 lx11 kernel: Call Trace:
 Apr  9 15:43:07 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:07 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81494f1d>] ?
 tcp_data_queue+0x3ed/0xc70
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814946f0>] ?
 tcp_validate_incoming+0x2c0/0x3a0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa0348557>] ?
 ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa04adfb0>] ?
 death_by_timeout+0x0/0x160 [nf_conntrack]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 15:43:07 lx11 kernel: <EOI>
 Apr  9 15:43:07 lx11 kernel: varnishd: page allocation failure. order:0,
 mode:0x20
 Apr  9 15:43:07 lx11 kernel: Pid: 16489, comm: varnishd Not tainted
 2.6.32-358.2.1.el6.x86_64 #1
 Apr  9 15:43:07 lx11 kernel: Call Trace:
 Apr  9 15:43:07 lx11 kernel: <IRQ>  [<ffffffff8112c207>] ?
 __alloc_pages_nodemask+0x757/0x8d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81166ab2>] ?
 kmem_getpages+0x62/0x170
 Apr  9 15:43:07 lx11 kernel: [<ffffffff811676ca>] ?
 fallback_alloc+0x1ba/0x270
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81167449>] ?
 ____cache_alloc_node+0x99/0x160
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81168610>] ?
 kmem_cache_alloc_node_trace+0x90/0x200
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8116882d>] ?
 __kmalloc_node+0x4d/0x60
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8143d8cd>] ? __alloc_skb+0x6d/0x190
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81493672>] ?
 tcp_collapse+0x1a2/0x3f0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81494a15>] ?
 tcp_try_rmem_schedule+0x245/0x360
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81494cdb>] ?
 tcp_data_queue+0x1ab/0xc70
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81497e79>] ?
 tcp_rcv_established+0x369/0x800
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8149fe93>] ?
 tcp_v4_do_rcv+0x2e3/0x430
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa0348557>] ?
 ipv4_confirm+0x87/0x1d0 [nf_conntrack_ipv4]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814a171e>] ? tcp_v4_rcv+0x4fe/0x8d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f430>] ?
 ip_local_deliver_finish+0x0/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f50d>] ?
 ip_local_deliver_finish+0xdd/0x2d0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f798>] ?
 ip_local_deliver+0x98/0xa0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147ec5d>] ?
 ip_rcv_finish+0x12d/0x440
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8147f1e5>] ? ip_rcv+0x275/0x350
 Apr  9 15:43:07 lx11 kernel: [<ffffffff814483bb>] ?
 __netif_receive_skb+0x4ab/0x750
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8149ea3a>] ?
 tcp4_gro_receive+0x5a/0xd0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144a798>] ?
 netif_receive_skb+0x58/0x60
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144a8a0>] ?
 napi_skb_finish+0x50/0x70
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144ce49>] ?
 napi_gro_receive+0x39/0x50
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa039932f>] ?
 bnx2_poll_work+0xd4f/0x1270 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8104c950>] ?
 smp_invalidate_interrupt+0x60/0xc0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8128d2b0>] ?
 swiotlb_map_page+0x0/0x100
 Apr  9 15:43:07 lx11 kernel: [<ffffffffa0399979>] ? bnx2_poll+0x69/0x2d8
 [bnx2]
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8144cf63>] ?
 net_rx_action+0x103/0x2f0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076fb1>] ?
 __do_softirq+0xc1/0x1e0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8109b77b>] ?
 hrtimer_interrupt+0x14b/0x260
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100c1cc>] ? call_softirq+0x1c/0x30
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100de05>] ? do_softirq+0x65/0xa0
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81076d95>] ? irq_exit+0x85/0x90
 Apr  9 15:43:07 lx11 kernel: [<ffffffff81517000>] ?
 smp_apic_timer_interrupt+0x70/0x9b
 Apr  9 15:43:07 lx11 kernel: [<ffffffff8100bb93>] ?
 apic_timer_interrupt+0x13/0x20
 Apr  9 16:15:24 lx11 kernel: <EOI>
 Apr  9 16:15:24 lx11 kernel: usb 6-1: USB disconnect, device number 2
 Apr  9 16:15:44 lx11 kernel: usb 6-1: new full speed USB device number 3
 using uhci_hcd
 Apr  9 16:15:44 lx11 kernel: usb 6-1: New USB device found, idVendor=03f0,
 idProduct=1027
 Apr  9 16:15:44 lx11 kernel: usb 6-1: New USB device strings: Mfr=1,
 Product=2, SerialNumber=0
 Apr  9 16:15:44 lx11 kernel: usb 6-1: Product: Virtual Keyboard
 Apr  9 16:15:44 lx11 kernel: usb 6-1: Manufacturer: HP
 Apr  9 16:15:44 lx11 kernel: usb 6-1: configuration #1 chosen from 1
 choice
 Apr  9 16:15:44 lx11 kernel: input: HP Virtual Keyboard as
 /devices/pci0000:00/0000:00:1e.0/0000:01:04.4/usb6/6-1/6-1:1.0/input/input6
 Apr  9 16:15:44 lx11 kernel: generic-usb 0003:03F0:1027.0003:
 input,hidraw0: USB HID v1.01 Keyboard [HP Virtual Keyboard] on
 usb-0000:01:04.4-1/input0
 Apr  9 16:15:44 lx11 kernel: input: HP Virtual Keyboard as
 /devices/pci0000:00/0000:00:1e.0/0000:01:04.4/usb6/6-1/6-1:1.1/input/input7
 Apr  9 16:15:44 lx11 kernel: generic-usb 0003:03F0:1027.0004:
 input,hidraw1: USB HID v1.01 Mouse [HP Virtual Keyboard] on
 usb-0000:01:04.4-1/input1
 Apr  9 17:50:33 lx11 varnishd[24950]: Manager got SIGINT
 Apr  9 17:50:42 lx11 varnishd[18973]: Platform:
 Linux,2.6.32-358.2.1.el6.x86_64,x86_64,-sfile,-smalloc,-hcritbit
 Apr  9 17:50:42 lx11 varnishd[18973]: child (18974) Started
 Apr  9 17:50:42 lx11 varnishd[18973]: Child (18974) said Child starts
 Apr  9 17:50:42 lx11 varnishd[18973]: Child (18974) said SMF.s0 mmap'ed

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1293>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator




More information about the varnish-bugs mailing list