Varnish virtual memory usage
Henry Paulissen
h.paulissen at qbell.nl
Fri Oct 23 09:49:39 CEST 2009
Didnt did the trick though.
############################################################################
#####################
############################################################################
#####################
uptime 13662 . Child uptime
client_conn 959755 70.25 Client connections accepted
client_drop 0 0.00 Connection dropped, no sess
client_req 959747 70.25 Client requests received
cache_hit 470881 34.47 Cache hits
cache_hitpass 0 0.00 Cache hits for pass
cache_miss 91401 6.69 Cache misses
backend_conn 402367 29.45 Backend conn. success
backend_unhealthy 0 0.00 Backend conn. not attempted
backend_busy 0 0.00 Backend conn. too many
backend_fail 15 0.00 Backend conn. failures
backend_reuse 87098 6.38 Backend conn. reuses
backend_toolate 7263 0.53 Backend conn. was closed
backend_recycle 94363 6.91 Backend conn. recycles
backend_unused 0 0.00 Backend conn. unused
fetch_head 0 0.00 Fetch head
fetch_length 475447 34.80 Fetch with Length
fetch_chunked 5867 0.43 Fetch chunked
fetch_eof 0 0.00 Fetch EOF
fetch_bad 0 0.00 Fetch had bad headers
fetch_close 0 0.00 Fetch wanted close
fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed
fetch_zero 0 0.00 Fetch zero len
fetch_failed 1 0.00 Fetch failed
n_sess_mem 495 . N struct sess_mem
n_sess 46 . N struct sess
n_object 41675 . N struct object
n_vampireobject 0 . N unresurrected objects
n_objectcore 41902 . N struct objectcore
n_objecthead 35695 . N struct objecthead
n_smf 0 . N struct smf
n_smf_frag 0 . N small free smf
n_smf_large 0 . N large free smf
n_vbe_conn 8 . N struct vbe_conn
n_wrk 1600 . N worker threads
n_wrk_create 1600 0.12 N worker threads created
n_wrk_failed 0 0.00 N worker threads not created
n_wrk_max 0 0.00 N worker threads limited
n_wrk_queue 0 0.00 N queued work requests
n_wrk_overflow 809 0.06 N overflowed work requests
n_wrk_drop 0 0.00 N dropped work requests
n_backend 5 . N backends
n_expired 49304 . N expired objects
n_lru_nuked 0 . N LRU nuked objects
n_lru_saved 0 . N LRU saved objects
n_lru_moved 198381 . N LRU moved objects
n_deathrow 0 . N objects on deathrow
losthdr 21 0.00 HTTP header overflows
n_objsendfile 0 0.00 Objects sent with sendfile
n_objwrite 954443 69.86 Objects sent with write
n_objoverflow 0 0.00 Objects overflowing workspace
s_sess 959749 70.25 Total Sessions
s_req 959747 70.25 Total Requests
s_pipe 0 0.00 Total pipe
s_pass 398061 29.14 Total pass
s_fetch 481313 35.23 Total fetch
s_hdrbytes 327272320 23954.93 Total header bytes
s_bodybytes 1551538833 113566.01 Total body bytes
sess_closed 959740 70.25 Session Closed
sess_pipeline 0 0.00 Session Pipeline
sess_readahead 0 0.00 Session Read Ahead
sess_linger 0 0.00 Session Linger
sess_herd 9 0.00 Session herd
shm_records 64046389 4687.92 SHM records
shm_writes 4351501 318.51 SHM writes
shm_flushes 0 0.00 SHM flushes due to overflow
shm_cont 4212 0.31 SHM MTX contention
shm_cycles 26 0.00 SHM cycles through buffer
sm_nreq 0 0.00 allocator requests
sm_nobj 0 . outstanding allocations
sm_balloc 0 . bytes allocated
sm_bfree 0 . bytes free
sma_nreq 572071 41.87 SMA allocator requests
sma_nobj 83345 . SMA outstanding allocations
sma_nbytes 499243475 . SMA outstanding bytes
sma_balloc 1781391844 . SMA bytes allocated
sma_bfree 1282148369 . SMA bytes free
sms_nreq 8170 0.60 SMS allocator requests
sms_nobj 0 . SMS outstanding allocations
sms_nbytes 0 . SMS outstanding bytes
sms_balloc 4770776 . SMS bytes allocated
sms_bfree 4770776 . SMS bytes freed
backend_req 486293 35.59 Backend requests made
n_vcl 1 0.00 N vcl total
n_vcl_avail 1 0.00 N vcl available
n_vcl_discard 0 0.00 N vcl discarded
n_purge 1 . N total active purges
n_purge_add 1 0.00 N new purges added
n_purge_retire 0 0.00 N old purges deleted
n_purge_obj_test 0 0.00 N objects tested
n_purge_re_test 0 0.00 N regexps tested against
n_purge_dups 0 0.00 N duplicate purges removed
hcb_nolock 0 0.00 HCB Lookups without lock
hcb_lock 0 0.00 HCB Lookups with lock
hcb_insert 0 0.00 HCB Inserts
esi_parse 0 0.00 Objects ESI parsed (unlock)
esi_errors 0 0.00 ESI parse errors (unlock)
############################################################################
#####################
############################################################################
#####################
############################################################################
#####################
############################################################################
#####################
top - 03:35:50 up 23 days, 17:24, 1 user, load average: 0.44, 0.43, 0.43
Tasks: 377 total, 2 running, 374 sleeping, 0 stopped, 1 zombie
Cpu0 : 0.0%us, 0.3%sy, 0.0%ni, 99.7%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu1 : 0.6%us, 0.6%sy, 0.0%ni, 98.1%id, 0.0%wa, 0.0%hi, 0.6%si,
0.0%st
Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu3 : 1.2%us, 0.9%sy, 0.0%ni, 96.9%id, 0.0%wa, 0.0%hi, 0.9%si,
0.0%st
Cpu4 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu5 : 4.4%us, 0.9%sy, 0.0%ni, 94.4%id, 0.0%wa, 0.0%hi, 0.3%si,
0.0%st
Cpu6 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu7 : 0.3%us, 0.0%sy, 0.0%ni, 99.4%id, 0.3%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu8 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu9 : 2.5%us, 0.9%sy, 0.0%ni, 96.0%id, 0.0%wa, 0.0%hi, 0.6%si,
0.0%st
Cpu10 : 8.4%us, 0.9%sy, 0.0%ni, 90.7%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu11 : 2.2%us, 0.9%sy, 0.0%ni, 95.6%id, 0.3%wa, 0.0%hi, 0.9%si,
0.0%st
Cpu12 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu13 : 2.2%us, 0.9%sy, 0.0%ni, 96.0%id, 0.0%wa, 0.0%hi, 0.9%si,
0.0%st
Cpu14 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si,
0.0%st
Cpu15 : 0.6%us, 0.3%sy, 0.0%ni, 98.8%id, 0.0%wa, 0.0%hi, 0.3%si,
0.0%st
Mem: 24675740k total, 20720232k used, 3955508k free, 792060k buffers
Swap: 16779884k total, 70380k used, 16709504k free, 13771132k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7634 nobody 15 0 5929m 2.3g 81m S 2.2 9.6 4:56.76
/usr/local/varnish/sbin/varnishd -P /var/run/varnish_xxx.pid -a 0.0.0.0:607
-f /usr/local/varnish/etc/varnish/xxx.xxx.xxx.vcl -T 0.0.0.0:707 -s
malloc,10G -i xxx -n /usr/local/varnish/var/va
29144 nobody 15 0 2620m 486m 81m S 1.2 2.0 3:19.36
/usr/local/varnish/sbin/varnishd -P /var/run/varnish_yyy.pid -a 0.0.0.0:604
-f /usr/local/varnish/etc/varnish/yyy.yyy.yyy.vcl -T 0.0.0.0:704 -s
malloc,1G -i yyy -n /u
11229 nobody 15 0 2008m 221m 80m S 0.9 0.9 117:35.14
/usr/local/varnish/sbin/varnishd -P /var/run/varnish_zzz.pid -a 0.0.0.0:606
-f /usr/local/varnish/etc/varnish/zzz.zzz.zzz.vcl -T 0.0.0.0:706 -s
malloc,512m -i zzz -n /usr/loca
29308 nobody 15 0 1520m 195m 150m S 0.6 0.8 2:23.04
/usr/local/varnish/sbin/varnishd -P /var/run/varnish_aaa.pid -a 0.0.0.0:605
-f /usr/local/varnish/etc/varnish/aaa.aaa.aaa.vcl -T 0.0.0.0:705 -s
file,/usr/local/varnish/var/varnish/flv/cache.
7646 root 15 0 116m 96m 80m S 0.0 0.4 0:00.00
/usr/local/varnish/bin/varnishncsa -afcD -n www -w
/usr/local/varnish/var/varnish/xxx/access_log -P
/var/run/varnishnsca_xxx.pid
29312 root 15 0 101m 81m 80m S 0.0 0.3 0:00.00
/usr/local/varnish/bin/varnishncsa -afcD -n flv -w
/usr/local/varnish/var/varnish/yyy/access_log -P
/var/run/varnishnsca_yyy.pid
29150 root 15 0 101m 81m 80m S 0.0 0.3 0:00.00
/usr/local/varnish/bin/varnishncsa -afcD -n photos_html -w
/usr/local/varnish/var/varnish/zzz/access_log -P
/var/run/varnishnsca_zzz.pid
############################################################################
#####################
############################################################################
#####################
See varnish_yyy that one is configured without the purge and malloc is set
to use 1G max. I could except a usage from 1.5G to 2G max. But it`s using
2.6G within 4 hours and will not decrease over time (it will only grow).
My startup command:
############################################################################
#####################
############################################################################
#####################
$exec -P $pidfile -a 0.0.0.0:${proxy_port} -f ${configfile} -T
0.0.0.0:${management_port} -s ${cachefile} -i ${INSTANCE_NAME} -n
${programdir}/var/varnish/${INSTANCE_NAME} -p obj_workspace=8192 -p
sess_workspace=262144 -p listen_depth=8192 -p lru_interval=60 -p
sess_timeout=10 -p shm_workspace=32768 -p ping_interval=2 -p thread_pools=16
-p thread_pool_min=100 -p thread_pool_max=4000 -p esi_syntax=1 -p
overflow_max=10000 > /dev/null 2>&1
############################################################################
#####################
############################################################################
#####################
Note that adding the '-p lru_interval=60' did change the memory problem. The
memory usage shifted from reserved memory to the virtual memory balloon.
Anyway, it's a good tip to use the ttl=0 and restart instead of purge.
Regards,
More information about the varnish-misc
mailing list