[Varnish] #1051: child process died
Varnish
varnish-bugs at varnish-cache.org
Tue Jan 17 16:16:40 CET 2012
#1051: child process died
-----------------------+----------------------------------------------------
Reporter: sreniaw | Type: defect
Status: reopened | Priority: normal
Milestone: | Component: varnishd
Version: 3.0.2 | Severity: normal
Resolution: | Keywords:
-----------------------+----------------------------------------------------
Changes (by sreniaw2):
* status: closed => reopened
* resolution: fixed =>
Comment:
I am having problem with persistent cache.
Silo file is created using dd command with initial size of 35GBs.
After two weeks varnish is starting crashing.
I am using varnish-3.0.2-1.el5 on Centos Linux 5 x86_64 with 32 GB of
physical memory and 33 GB of swap.
dm-7 is only for silo file
{{{
# iostat -x 1
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz
avgqu-sz await svctm %util
dm-7 0.00 0.00 1.29 163.92 111.77 1311.38 8.61
0.60 3.64 1.29 21.30
dm-7 0.00 0.00 0.00 662.00 0.00 5296.00 8.00
113.08 171.04 1.51 100.10
dm-7 0.00 0.00 2.00 646.00 256.00 5168.00 8.37
116.20 177.84 1.54 100.10
dm-7 0.00 0.00 0.00 647.52 0.00 5180.20 8.00
113.79 182.72 1.53 99.11
dm-7 0.00 0.00 4.00 640.00 312.00 5120.00 8.43
94.62 131.55 1.55 100.10
dm-7 0.00 0.00 2.00 620.00 216.00 4960.00 8.32
118.29 202.90 1.61 100.20
dm-7 0.00 0.00 4.00 645.00 416.00 5160.00 8.59
114.23 179.15 1.54 100.10
dm-7 0.00 0.00 0.00 611.00 0.00 4888.00 8.00
112.30 164.37 1.64 100.10
dm-7 0.00 0.00 3.00 581.00 424.00 4648.00 8.68
113.39 209.71 1.71 100.00
dm-7 0.00 0.00 4.00 594.00 512.00 4752.00 8.80
91.58 168.89 1.67 100.10
dm-7 0.00 0.00 2.00 708.00 112.00 5664.00 8.14
115.57 158.32 1.41 100.10
dm-7 0.00 0.00 4.00 658.00 376.00 5264.00 8.52
115.12 166.39 1.51 100.10
dm-7 0.00 0.00 2.00 527.00 256.00 4216.00 8.45
114.82 213.90 1.89 100.10
dm-7 0.00 0.00 0.00 680.00 0.00 5440.00 8.00
115.64 171.43 1.47 100.20
dm-7 0.00 0.00 19.80 599.01 633.66 4792.08 8.77
101.46 172.11 1.60 99.21
dm-7 0.00 0.00 1.00 664.00 128.00 5312.00 8.18
115.89 167.74 1.51 100.20
dm-7 0.00 0.00 5.00 622.00 360.00 4976.00 8.51
114.20 172.12 1.60 100.10
dm-7 0.00 0.00 3.00 582.00 224.00 4656.00 8.34
112.75 205.01 1.71 100.10
dm-7 0.00 0.00 0.00 667.00 0.00 5336.00 8.00
110.22 164.68 1.50 100.20
dm-7 0.00 0.00 1.00 669.00 128.00 5352.00 8.18
103.50 155.61 1.50 100.20
dm-7 0.00 0.00 3.00 596.00 240.00 4768.00 8.36
113.84 184.50 1.67 100.10
dm-7 0.00 0.00 10.89 564.36 807.92 4514.85 9.25
114.11 203.60 1.72 99.21
dm-7 0.00 0.00 4.00 617.00 264.00 4936.00 8.37
116.19 184.58 1.61 100.10
dm-7 0.00 0.00 3.00 607.00 224.00 4856.00 8.33
112.08 186.60 1.64 100.00
dm-7 0.00 0.00 0.00 698.00 0.00 5584.00 8.00
113.24 145.28 1.44 100.20
dm-7 0.00 0.00 0.00 6062.00 0.00 48496.00 8.00
1007.00 106.46 0.17 100.10
dm-7 0.00 0.00 0.00 750.00 0.00 6000.00 8.00
268.74 834.84 1.34 100.20
dm-7 0.00 0.00 0.00 656.00 0.00 5248.00 8.00
115.95 209.27 1.52 100.00
dm-7 0.00 0.00 0.00 730.00 0.00 5840.00 8.00
152.54 187.88 1.37 100.20
dm-7 0.00 0.00 15.84 55.45 2146.53 443.56 36.33
21.85 652.62 6.14 43.76
dm-7 0.00 0.00 4.00 0.00 512.00 0.00 128.00
0.03 6.75 6.75 2.70
dm-7 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00
dm-7 0.00 0.00 3.00 0.00 288.00 0.00 96.00
0.00 1.33 1.33 0.40
dm-7 0.00 0.00 3.00 456.00 384.00 3648.00 8.78
38.76 64.82 0.66 30.40
dm-7 0.00 0.00 0.00 991.00 0.00 7928.00 8.00
119.20 120.89 1.01 100.20
dm-7 0.00 0.00 2.00 945.00 112.00 7560.00 8.10
114.86 114.68 1.06 100.20
dm-7 0.00 0.00 4.00 834.00 216.00 6672.00 8.22
118.92 138.72 1.19 100.10
dm-7 0.00 0.00 2.97 743.56 205.94 5948.51 8.24
117.64 156.17 1.33 99.21
dm-7 0.00 0.00 11.00 681.00 592.00 5448.00 8.73
117.81 176.87 1.45 100.00
}}}
{{{
# ps axu | grep varnishd
root 6462 0.0 0.0 34716380 1392 ? Ss Jan02 0:00
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -T localhost:6082 -f
/etc/varnish/default.vcl -u varnish
+-g varnish -S /etc/varnish/secret -p lru_interval 20 -p cli_timeout 60 -h
classic,500009 -s malloc,30G -s
persistent,/var/lib/varnish/store/silo,35433480192 -p
+thread_pool_min 500 -p thread_pool_max 4000 -p thread_pools 4 -p
thread_pool_add_delay 2 -p session_linger 100 -p send_timeout 120 -p
sess_timeout 10 -p listen_depth 4096
+-p nuke_limit 50
varnish 29817 25.9 0.0 0 0 ? Zl 22:29 4:48
[varnishd] <defunct>
}}}
{{{
# ps axu | grep varnishd
varnish 1870 25.4 2.4 55950024 808356 ? Sl 22:48 0:09
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -T localhost:6082 -f
/etc/varnish/default.vcl -u varnish
+-g varnish -S /etc/varnish/secret -p lru_interval 20 -p cli_timeout 60 -h
classic,500009 -s malloc,30G -s
persistent,/var/lib/varnish/store/silo,35433480192 -p
+thread_pool_min 500 -p thread_pool_max 4000 -p thread_pools 4 -p
thread_pool_add_delay 2 -p session_linger 100 -p send_timeout 120 -p
sess_timeout 10 -p listen_depth 4096
+-p nuke_limit 50
root 3969 0.0 0.0 61172 728 pts/3 R+ 22:48 0:00 grep
varnishd
root 6462 0.0 0.0 34716380 1400 ? Ss Jan02 0:00
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -T localhost:6082 -f
/etc/varnish/default.vcl -u varnish
+-g varnish -S /etc/varnish/secret -p lru_interval 20 -p cli_timeout 60 -h
classic,500009 -s malloc,30G -s
persistent,/var/lib/varnish/store/silo,35433480192 -p
+thread_pool_min 500 -p thread_pool_max 4000 -p thread_pools 4 -p
thread_pool_add_delay 2 -p session_linger 100 -p send_timeout 120 -p
sess_timeout 10 -p listen_depth 4096
+-p nuke_limit 50
}}}
Next day:
{{{
# ps axu | grep varnishd
root 6462 0.0 0.0 34716380 1524 ? Ss Jan02 0:00
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -T localhost:6082 -f
/etc/varnish/default.vcl -u varnish -g varnish -S /etc/varnish/secret -p
lru_interval 20 -p cli_timeout 60 -h classic,500009 -s malloc,30G -s
persistent,/var/lib/varnish/store/silo,35433480192 -p thread_pool_min 500
-p thread_pool_max 4000 -p thread_pools 4 -p thread_pool_add_delay 2 -p
session_linger 100 -p send_timeout 120 -p sess_timeout 10 -p listen_depth
4096 -p nuke_limit 50
varnish 14624 11.4 17.9 56139464 5921276 ? Sl 15:14 4:29
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -T localhost:6082 -f
/etc/varnish/default.vcl -u varnish -g varnish -S /etc/varnish/secret -p
lru_interval 20 -p cli_timeout 60 -h classic,500009 -s malloc,30G -s
persistent,/var/lib/varnish/store/silo,35433480192 -p thread_pool_min 500
-p thread_pool_max 4000 -p thread_pools 4 -p thread_pool_add_delay 2 -p
session_linger 100 -p send_timeout 120 -p sess_timeout 10 -p listen_depth
4096 -p nuke_limit 50
}}}
{{{
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-u varnish -g varnish \
-S /etc/varnish/secret \
-p lru_interval=20 \
-p cli_timeout=60 \
-h classic,500009 \
-s malloc,30G -s
persistent,/var/lib/varnish/store/silo,35433480192 \
-p thread_pool_min=500 \
-p thread_pool_max=4000 \
-p thread_pools=4 \
-p thread_pool_add_delay=2 \
-p session_linger=100 \
-p send_timeout=120 \
-p sess_timeout=10 \
-p listen_depth=4096 \
-p nuke_limit=50"
}}}
Any clues ?
--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1051#comment:3>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator
More information about the varnish-bugs
mailing list