[Varnish] #1374: High SWAP usage
Varnish
varnish-bugs at varnish-cache.org
Mon Nov 18 12:09:50 CET 2013
#1374: High SWAP usage
----------------------+--------------------
Reporter: jammy | Owner:
Type: defect | Status: new
Priority: high | Milestone:
Component: varnishd | Version: 3.0.3
Severity: critical | Resolution:
Keywords: |
----------------------+--------------------
Description changed by tfheen:
Old description:
> We are running varnish 3.0.3 in production. but recently, we are
> experiencing high SWAP usage recently. As you can see from the following
> information, we configured varnished to use 4G malloc storage, but not
> sure why it ate SWAP so heavily.
>
> I'm looking forward to hearing from the community.
>
> #1. uname -a
> Linux ip-10-36-1-238 2.6.32-358.14.1.el6.x86_64 #1 SMP Tue Jul 16
> 23:51:20 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
>
> #2.
> /usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f
> /etc/varnish/yottaa.vcl -T 127.0.0.1:6082 -t 120 -w 500,1000,360 -p
> thread_pool_add_delay 2 -p listen_depth 65535 -p ban_lurker_sleep 1 -p
> sess_workspace 204800 -p thread_pool_workspace 307200 -p http_req_size
> 102400 -p http_resp_size 102400 -p http_req_hdr_len 51200 -p
> http_resp_hdr_len 51200 -p http_max_hdr 500 -S /etc/varnish/secret -s
> malloc,4066M
>
> #3. cat /proc/14464/status
> VmPeak: 19214032 kB
> VmSize: 19211984 kB
> VmLck: 0 kB
> VmHWM: 5540916 kB
> VmRSS: 4494520 kB
> VmData: 19055864 kB
> VmStk: 152 kB
> VmExe: 492 kB
> VmLib: 5820 kB
> VmPTE: 33332 kB
> VmSwap: '''3383040''' kB
>
> #4. varnishstat -1
>
> n_sess_mem 919 . N struct sess_mem
> n_sess 12 . N struct sess
> n_object 94417 . N struct object
> n_vampireobject 0 . N unresurrected objects
> n_objectcore 94622 . N struct objectcore
> n_objecthead 95253 . N struct objecthead
> n_waitinglist 720 . N struct waitinglist
> n_vbc 23 . N struct vbc
> n_wrk 1000 . N worker threads
> n_wrk_create 1002 0.00 N worker threads created
>
> n_expired 8899069 . N expired objects
> n_lru_nuked 227237 . N LRU nuked objects
> n_lru_moved 4688309 . N LRU moved objects
> losthdr 0 0.00 HTTP header overflows
> n_objsendfile 0 0.00 Objects sent with sendfile
> n_objwrite 5670271 2.97 Objects sent with write
> n_objoverflow 0 0.00 Objects overflowing workspace
> s_sess 7342009 3.84 Total Sessions
> s_req 14921754 7.81 Total Requests
> s_pipe 1057 0.00 Total pipe
> s_pass 446621 0.23 Total pass
> s_fetch 10027602 5.25 Total fetch
> s_hdrbytes 9425706241 4932.10 Total header bytes
> s_bodybytes 168048482086 87933.18 Total body bytes
>
> n_ban 15133 . N total active bans
> n_ban_gone 10716 . N total gone bans
> n_ban_add 21766 0.01 N new bans added
> n_ban_retire 6633 0.00 N old bans deleted
> n_ban_obj_test 3386265 1.77 N objects tested
> n_ban_re_test 422921545 221.30 N regexps tested against
> n_ban_dups 5514 0.00 N duplicate bans removed
>
> SMA.s0.c_req 1662338 0.87 Allocator requests
> SMA.s0.c_fail 33874757152 17725.33 Allocator failures
> SMA.s0.c_bytes 70233567950 36750.47 Bytes allocated
> SMA.s0.c_freed 66209262888 34644.71 Bytes freed
> SMA.s0.g_alloc 195376 . Allocations outstanding
> SMA.s0.g_bytes 4024305062 . Bytes outstanding
> SMA.s0.g_space 239204954 . Bytes available
> SMA.Transient.c_req 106362180 55.66 Allocator requests
> SMA.Transient.c_fail 0 0.00 Allocator failures
> SMA.Transient.c_bytes 12698410805901 6644580.25 Bytes allocated
> SMA.Transient.c_freed 12698409165965 6644579.39 Bytes freed
> SMA.Transient.g_alloc 1139 . Allocations outstanding
> SMA.Transient.g_bytes 1639936 . Bytes outstanding
> SMA.Transient.g_space 0 . Bytes available
New description:
We are running varnish 3.0.3 in production. but recently, we are
experiencing high SWAP usage recently. As you can see from the following
information, we configured varnished to use 4G malloc storage, but not
sure why it ate SWAP so heavily.
I'm looking forward to hearing from the community.
{{{
#1. uname -a
Linux ip-10-36-1-238 2.6.32-358.14.1.el6.x86_64 #1 SMP Tue Jul 16 23:51:20
UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
}}}
{{{
#2.
/usr/sbin/varnishd -P /var/run/varnish.pid -a :80 -f
/etc/varnish/yottaa.vcl -T 127.0.0.1:6082 -t 120 -w 500,1000,360 -p
thread_pool_add_delay 2 -p listen_depth 65535 -p ban_lurker_sleep 1 -p
sess_workspace 204800 -p thread_pool_workspace 307200 -p http_req_size
102400 -p http_resp_size 102400 -p http_req_hdr_len 51200 -p
http_resp_hdr_len 51200 -p http_max_hdr 500 -S /etc/varnish/secret -s
malloc,4066M
}}}
{{{
#3. cat /proc/14464/status
VmPeak: 19214032 kB
VmSize: 19211984 kB
VmLck: 0 kB
VmHWM: 5540916 kB
VmRSS: 4494520 kB
VmData: 19055864 kB
VmStk: 152 kB
VmExe: 492 kB
VmLib: 5820 kB
VmPTE: 33332 kB
VmSwap: '''3383040''' kB
}}}
{{{
#4. varnishstat -1
n_sess_mem 919 . N struct sess_mem
n_sess 12 . N struct sess
n_object 94417 . N struct object
n_vampireobject 0 . N unresurrected objects
n_objectcore 94622 . N struct objectcore
n_objecthead 95253 . N struct objecthead
n_waitinglist 720 . N struct waitinglist
n_vbc 23 . N struct vbc
n_wrk 1000 . N worker threads
n_wrk_create 1002 0.00 N worker threads created
n_expired 8899069 . N expired objects
n_lru_nuked 227237 . N LRU nuked objects
n_lru_moved 4688309 . N LRU moved objects
losthdr 0 0.00 HTTP header overflows
n_objsendfile 0 0.00 Objects sent with sendfile
n_objwrite 5670271 2.97 Objects sent with write
n_objoverflow 0 0.00 Objects overflowing workspace
s_sess 7342009 3.84 Total Sessions
s_req 14921754 7.81 Total Requests
s_pipe 1057 0.00 Total pipe
s_pass 446621 0.23 Total pass
s_fetch 10027602 5.25 Total fetch
s_hdrbytes 9425706241 4932.10 Total header bytes
s_bodybytes 168048482086 87933.18 Total body bytes
n_ban 15133 . N total active bans
n_ban_gone 10716 . N total gone bans
n_ban_add 21766 0.01 N new bans added
n_ban_retire 6633 0.00 N old bans deleted
n_ban_obj_test 3386265 1.77 N objects tested
n_ban_re_test 422921545 221.30 N regexps tested against
n_ban_dups 5514 0.00 N duplicate bans removed
SMA.s0.c_req 1662338 0.87 Allocator requests
SMA.s0.c_fail 33874757152 17725.33 Allocator failures
SMA.s0.c_bytes 70233567950 36750.47 Bytes allocated
SMA.s0.c_freed 66209262888 34644.71 Bytes freed
SMA.s0.g_alloc 195376 . Allocations outstanding
SMA.s0.g_bytes 4024305062 . Bytes outstanding
SMA.s0.g_space 239204954 . Bytes available
SMA.Transient.c_req 106362180 55.66 Allocator requests
SMA.Transient.c_fail 0 0.00 Allocator failures
SMA.Transient.c_bytes 12698410805901 6644580.25 Bytes allocated
SMA.Transient.c_freed 12698409165965 6644579.39 Bytes freed
SMA.Transient.g_alloc 1139 . Allocations outstanding
SMA.Transient.g_bytes 1639936 . Bytes outstanding
SMA.Transient.g_space 0 . Bytes available
}}}
--
--
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1374#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator
More information about the varnish-bugs
mailing list