varnish-cache-3.0.3-plus increase %CPU and crashed by "errno = 12(Cannot allocate memory)"
Xianzhe Wang
wxz19861013 at gmail.com
Wed May 15 10:21:22 CEST 2013
Hi,
I use the order:
ps H -e -o pid,tid,pcpu,cmd --sort=pcpu |grep varnish
I get the message:
12255 12280 0.1 /opt/ace/cache/varnish-cache-3.0.3-plus/sbin/varnishd -f
/opt/ace/cache/varnish-stream.vcl -s
persistent,/opt/ace/file/varnish_cache_persistent.data,12G -T 127.0.0.1:2000-a
0.0.0.0:8080 -h classic,500009 -p thread_pool_min 200 -p thread_pool_max
2000 -p thread_pool_add_delay 2 -p session_linger 100 -p thread_pools 2 -p
http_req_hdr_len 32768 -p http_resp_hdr_len 32768 -p http_max_hdr 256 -p
lru_interval 3600
12255 30175 22.6 …… （omit startup command）
12255 26385 87.2 …… （omit startup command）
12255 30478 89.1 …… （omit startup command）
12255 21623 96.3 …… （omit startup command）
12255 12504 96.7 …… （omit startup command）
There are several threads consume about 100% CPU.
Then I debugged varnish, attach the threads above. I found they lingered
in in VGZ_WrwGunzip .
I think there may be some *endless loop* or *deadlock* issues.
There are some debug log about above threads:
attach 21623
...
inflate (strm=0x7fd5652680f8, flush=0) at inflate.c:1222
1222 RESTORE();
(gdb) info thread
* 1 process 21623 inflate (strm=0x7fd5652680f8, flush=0) at inflate.c:1222
(gdb) thread 1
[Switching to thread 1 (process 21623)]#0 inflate (strm=0x7fd5652680f8,
flush=0) at inflate.c:1222
1222 RESTORE();
(gdb) bt
#0 inflate (strm=0x7fd5652680f8, flush=0) at inflate.c:1222
#1 0x0000000000428068 in VGZ_Gunzip (vg=0x7fd5652680c0,
pptr=0x7fd55eddf458, plen=0x7fd55eddf460) at cache_gzip.c:290
#2 0x00000000004287e4 in VGZ_WrwGunzip (sp=0x7fd575a13008,
vg=0x7fd5652680c0, ibuf=<value optimized out>, ibufl=<value optimized out>,
obuf=0x7fd55eddf4f0 "<link rel=\"icon\" type=\"image/gif\"
href=\"favicon.gif\" />\n\n<script type=\"text/javascript\">\n\n var _gaq
= _gaq || [];\n _gaq.push(['_setAccount', 'UA-36075559-1']);\n
_gaq.push(['_trackPageview']);\n\n (f"..., obufl=32768,
obufp=0x7fd55ede7540) at cache_gzip.c:374
#3 0x0000000000434b66 in RES_StreamWrite (sp=0x7fd575a13008) at
cache_response.c:463
#4 0x0000000000435034 in RES_StreamBody (sp=0x7fd575a13008) at
cache_response.c:571
#5 0x00000000004182a8 in cnt_streamdeliver (sp=0x7fd575a13008) at
cache_center.c:935
#6 0x000000000041add5 in CNT_Session (sp=0x7fd575a13008) at steps.h:44
#7 0x0000000000433aca in wrk_thread_real (qp=0x7fd62590e240,
shm_workspace=<value optimized out>, sess_workspace=65536, nhttp=256,
http_space=<value optimized out>, siov=512) at cache_pool.c:186
#8 0x00007fd625de16a6 in start_thread () from /lib64/libpthread.so.0
#9 0x00007fd625b50f7d in clone () from /lib64/libc.so.6
#10 0x0000000000000000 in ?? ()
attach 12504
...
VGZ_WrwGunzip (sp=0x7fd598a1d008, vg=0x7fd612a19900, ibuf=<value optimized
out>, ibufl=<value optimized out>,
obuf=0x7fd5d7bcc4f0 "\342\340\355\350\377\" src=\"
http://www.yandex.ru/cycounter?www.alternativa7.ru/\" /> </a>\t
\n<!--Rating at Mail.ru COUNTEr-->\n <a href=\"
http://top.mail.ru/jump?from=1028785\" target=\"_top\" ><img width=\"88\"
height=\"31\" bor"..., obufl=32768, obufp=0x7fd5d7bd4540) at
cache_gzip.c:377
377 if (i < VGZ_OK) {
(gdb) info thread
* 1 process 12504 VGZ_WrwGunzip (sp=0x7fd598a1d008, vg=0x7fd612a19900,
ibuf=<value optimized out>, ibufl=<value optimized out>,
obuf=0x7fd5d7bcc4f0 "\342\340\355\350\377\" src=\"
http://www.yandex.ru/cycounter?www.alternativa7.ru/\" /> </a>\t
\n<!--Rating at Mail.ru COUNTEr-->\n <a href=\"
http://top.mail.ru/jump?from=1028785\" target=\"_top\" ><img width=\"88\"
height=\"31\" bor"..., obufl=32768, obufp=0x7fd5d7bd4540) at
cache_gzip.c:377
(gdb) thread 1
[Switching to thread 1 (process 12504)]#0 VGZ_WrwGunzip
(sp=0x7fd598a1d008, vg=0x7fd612a19900, ibuf=<value optimized out>,
ibufl=<value optimized out>,
obuf=0x7fd5d7bcc4f0 "\342\340\355\350\377\" src=\"
http://www.yandex.ru/cycounter?www.alternativa7.ru/\" /> </a>\t
\n<!--Rating at Mail.ru COUNTEr-->\n <a href=\"
http://top.mail.ru/jump?from=1028785\" target=\"_top\" ><img width=\"88\"
height=\"31\" bor"..., obufl=32768, obufp=0x7fd5d7bd4540) at
cache_gzip.c:377
377 if (i < VGZ_OK) {
(gdb) bt
#0 VGZ_WrwGunzip (sp=0x7fd598a1d008, vg=0x7fd612a19900, ibuf=<value
optimized out>, ibufl=<value optimized out>,
obuf=0x7fd5d7bcc4f0 "\342\340\355\350\377\" src=\"
http://www.yandex.ru/cycounter?www.alternativa7.ru/\" /> </a>\t
\n<!--Rating at Mail.ru COUNTEr-->\n <a href=\"
http://top.mail.ru/jump?from=1028785\" target=\"_top\" ><img width=\"88\"
height=\"31\" bor"..., obufl=32768, obufp=0x7fd5d7bd4540) at
cache_gzip.c:377
#1 0x0000000000434b66 in RES_StreamWrite (sp=0x7fd598a1d008) at
cache_response.c:463
#2 0x0000000000435034 in RES_StreamBody (sp=0x7fd598a1d008) at
cache_response.c:571
#3 0x00000000004182a8 in cnt_streamdeliver (sp=0x7fd598a1d008) at
cache_center.c:935
#4 0x000000000041add5 in CNT_Session (sp=0x7fd598a1d008) at steps.h:44
#5 0x0000000000433aca in wrk_thread_real (qp=0x7fd62590e240,
shm_workspace=<value optimized out>, sess_workspace=65536, nhttp=256,
http_space=<value optimized out>, siov=512) at cache_pool.c:186
#6 0x00007fd625de16a6 in start_thread () from /lib64/libpthread.so.0
#7 0x00007fd625b50f7d in clone () from /lib64/libc.so.6
#8 0x0000000000000000 in ?? ()
(gdb)
Could you tell me how it happened? And how to fix it?
Thank you again.
Regards,
Shawn.
2013/5/15 Xianzhe Wang <wxz19861013 at gmail.com>
> Hi,
> I used varnish-cache-3.0.3-plus(get from
> https://github.com/mbgrydeland/varnish-cache/tree/3.0.3-plus).
> I use persistent storage and "set beresp.do_stream = true".
> The %CPU is increased from 4% to 800% (8 cpu), and varnish crashed
> sometimes by errno = 12 (Cannot allocate memory).
>
> my startup command is:
> /opt/ace/cache/varnish-cache-3.0.3-plus/sbin/varnishd -f
> /opt/ace/cache/varnish-stream.vcl -s
> persistent,/opt/ace/file/varnish_cache_persistent.data,12G -T
> 127.0.0.1:2000 -a 0.0.0.0:8080 -p thread_pool_min 200 -p thread_pool_max
> 2000 -p thread_pool_add_delay 2 -p session_linger 100 -p thread_pools 2 -p
> http_req_hdr_len 32768 -p http_resp_hdr_len 32768 -p http_max_hdr 256
>
> Then I change the command to:
> /opt/ace/cache/varnish-cache-3.0.3-plus/sbin/varnishd -f
> /opt/ace/cache/varnish-stream.vcl -s
> persistent,/opt/ace/file/varnish_cache_persistent.data,12G -T
> 127.0.0.1:2000 -a 0.0.0.0:8080 *-h classic,500009* -p thread_pool_min 200
> -p thread_pool_max 2000 -p thread_pool_add_delay 2 -p session_linger 100 -p
> thread_pools 2 -p http_req_hdr_len 32768 -p http_resp_hdr_len 32768 -p
> http_max_hdr 256 *-p lru_interval 3600*
>
> The %CPU is increasing too, but varnish hasn't crashed for 16 hours.
>
> Is there somebody has experience about this problem?
> Any advice will be greatly appreciated.
>
> Here is the information about system and varnish.
>
> top:
> top - 09:37:36 up 50 days, 13:06, 3 users, load average: 5.00, 4.97, 4.73
> Tasks: 141 total, 1 running, 140 sleeping, 0 stopped, 0 zombie
> Cpu(s): 62.6%us, 0.0%sy, 0.0%ni, 37.4%id, 0.0%wa, 0.0%hi, 0.0%si,
> 0.0%st
> Mem: 20132M total, 19781M used, 351M free, 137M buffers
> Swap: 4102M total, 3M used, 4098M free, 18517M cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
>
>
> 12255 nobody 20 0 14.7g 11g 11g S 502 59.8 2280:28 varnishd
>
>
> 30934 acce 20 0 8772 1212 872 R 0 0.0 0:00.02 top
>
>
> free -m:
> total used free shared buffers cached
> Mem: 20132 19782 350 0 137 18517
> -/+ buffers/cache: 1127 19004
> Swap: 4102 3 4098
>
>
> ulimit -a:
> core file size (blocks, -c) 0
> data seg size (kbytes, -d) unlimited
> scheduling priority (-e) 0
> file size (blocks, -f) unlimited
> pending signals (-i) 156007
> max locked memory (kbytes, -l) 64
> max memory size (kbytes, -m) 17523512
> open files (-n) 65535
> pipe size (512 bytes, -p) 8
> POSIX message queues (bytes, -q) 819200
> real-time priority (-r) 0
> stack size (kbytes, -s) 4096
> cpu time (seconds, -t) unlimited
> max user processes (-u) 156007
> virtual memory (kbytes, -v) 19853520
> file locks (-x) unlimited
>
>
> varnish panic.show:
> Last panic at: Tue, 14 May 2013 02:56:50 GMT
> Assert error in CNT_Session(), cache_center.c line 1813:
> Condition((w->do_stream) == 0) not true.
> errno = 12 (Cannot allocate memory)
> thread = (cache-worker)
> ident =
> Linux,2.6.32.59-0.7-xen,x86_64,-spersistent,-smalloc,-hcritbit,epoll
> sp = 0x7f4dcd947008 {
> fd = 1435, id = 1435, xid = 1304392785,
> client = xx.xx.xx.xx 2396,
> step = STP_DONE,
> handling = hit_for_pass,
> err_code = 502, err_reason = (null),
> restarts = 0, esi_level = 0
> flags = do_stream is_gunzip
> bodystatus = 4
> ws = 0x7f4dcd947080 {
> id = "sess",
> {s,f,r,e} = {0x7f4dcd949610,+344,(nil),+65536},
> },
> http[req] = {
> ws = 0x7f4dcd947080[sess]
> "GET",
> "http://www.2xk.cn/index.asp",
> "HTTP/1.1",
> "Accept: */*",
> "Referer: http://www.2xk.cn/",
> "Accept-Language: zh-cn",
> "User-Agent:",
> "Host: www.2xk.cn",
> "Proxy-Connection: Keep-Alive",
> "Pragma: no-cache",
> "X-Forwarded-For: xx.xx.xx.xx",
> "Accept-Encoding: gzip",
> },
> worker = 0x7f4d83a1ca90 {
> ws = 0x7f4d83a1ccd8 {
> id = "wrk",
> {s,f,r,e} = {0x7f4d83a0aa20,+616,(nil),+65536},
> },
> },
> vcl = {
> srcname = {
> "input",
> "Default",
> },
> },
> },
>
>
> /opt/ace/cache/varnish-cache-3.0.3-plus/bin/varnishstat -1
> acce at sz-acce-varnish01:~>
> /opt/huawei/ace/cache/varnish-cache-3.0.3-plus/bin/varnishstat -1
> client_conn 2341640 40.03 Client connections accepted
> client_drop 8 0.00 Connection dropped, no sess/wrk
> client_req 3031369 51.82 Client requests received
> cache_hit 164025 2.80 Cache hits
> cache_hitpass 409263 7.00 Cache hits for pass
> cache_miss 1857720 31.75 Cache misses
> backend_conn 500716 8.56 Backend conn. success
> backend_unhealthy 0 0.00 Backend conn. not attempted
> backend_busy 0 0.00 Backend conn. too many
> backend_fail 8097 0.14 Backend conn. failures
> backend_reuse 2375031 40.60 Backend conn. reuses
> backend_toolate 34982 0.60 Backend conn. was closed
> backend_recycle 2410099 41.20 Backend conn. recycles
> backend_retry 15375 0.26 Backend conn. retry
> fetch_head 1051 0.02 Fetch head
> fetch_length 2185540 37.36 Fetch with Length
> fetch_chunked 600378 10.26 Fetch chunked
> fetch_eof 0 0.00 Fetch EOF
> fetch_streamed 2792280 47.73 Fetch streamed
> fetch_bad 0 0.00 Fetch had bad headers
> fetch_close 10117 0.17 Fetch wanted close
> fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed
> fetch_zero 0 0.00 Fetch zero len
> fetch_failed 37332 0.64 Fetch failed
> fetch_1xx 0 0.00 Fetch no body (1xx)
> fetch_204 2720 0.05 Fetch no body (204)
> fetch_304 12112 0.21 Fetch no body (304)
> n_sess_mem 7231 . N struct sess_mem
> n_sess 93 . N struct sess
> n_object 86410 . N struct object
> n_vampireobject 0 . N unresurrected objects
> n_objectcore 86666 . N struct objectcore
> n_objecthead 51679 . N struct objecthead
> n_waitinglist 22932 . N struct waitinglist
> n_vbc 24 . N struct vbc
> n_wrk 400 . N worker threads
> n_wrk_create 14003 0.24 N worker threads created
> n_wrk_failed 2553 0.04 N worker threads not created
> n_wrk_max 0 0.00 N worker threads limited
> n_wrk_lqueue 0 0.00 work request queue length
> n_wrk_queued 114487 1.96 N queued work requests
> n_wrk_drop 8 0.00 N dropped work requests
> n_backend 4 . N backends
> n_expired 1427930 . N expired objects
> n_lru_nuked 42056 . N LRU nuked objects
> n_lru_moved 0 . N LRU moved objects
> losthdr 112 0.00 HTTP header overflows
> n_objsendfile 0 0.00 Objects sent with sendfile
> n_objwrite 194021 3.32 Objects sent with write
> n_objoverflow 0 0.00 Objects overflowing workspace
> s_sess 2341612 40.03 Total Sessions
> s_req 3031369 51.82 Total Requests
> s_pipe 7436 0.13 Total pipe
> s_pass 1003423 17.15 Total pass
> s_fetch 17926 0.31 Total fetch
> s_stream 2787698 47.65 Total streamed requests
> s_hdrbytes 1048771245 17927.10 Total header bytes
> s_bodybytes 20286532441 346766.48 Total body bytes
> sess_closed 1781645 30.45 Session Closed
> sess_pipeline 11352 0.19 Session Pipeline
> sess_readahead 7111 0.12 Session Read Ahead
> sess_linger 1375398 23.51 Session Linger
> sess_herd 874711 14.95 Session herd
> shm_records 244018270 4171.11 SHM records
> shm_writes 19725032 337.17 SHM writes
> shm_flushes 3325 0.06 SHM flushes due to overflow
> shm_cont 33597 0.57 SHM MTX contention
> shm_cycles 104 0.00 SHM cycles through buffer
> sms_nreq 49339 0.84 SMS allocator requests
> sms_nobj 0 . SMS outstanding allocations
> sms_nbytes 0 . SMS outstanding bytes
> sms_balloc 23339027 . SMS bytes allocated
> sms_bfree 23339027 . SMS bytes freed
> backend_req 2867335 49.01 Backend requests made
> n_vcl 1 0.00 N vcl total
> n_vcl_avail 1 0.00 N vcl available
> n_vcl_discard 0 0.00 N vcl discarded
> n_ban 1 . N total active bans
> n_ban_gone 1 . N total gone bans
> n_ban_add 6 0.00 N new bans added
> n_ban_retire 5 0.00 N old bans deleted
> n_ban_obj_test 23688 0.40 N objects tested
> n_ban_re_test 0 0.00 N regexps tested against
> n_ban_dups 5 0.00 N duplicate bans removed
> hcb_nolock 0 0.00 HCB Lookups without lock
> hcb_lock 0 0.00 HCB Lookups with lock
> hcb_insert 0 0.00 HCB Inserts
> esi_errors 0 0.00 ESI parse errors (unlock)
> esi_warnings 0 0.00 ESI parse warnings (unlock)
> accept_fail 0 0.00 Accept failures
> client_drop_late 0 0.00 Connection dropped late
> uptime 58502 1.00 Client uptime
> dir_dns_lookups 0 0.00 DNS director lookups
> dir_dns_failed 0 0.00 DNS director failed lookups
> dir_dns_hit 0 0.00 DNS director cached lookups hit
> dir_dns_cache_full 0 0.00 DNS director full dnscache
> vmods 0 . Loaded VMODs
> n_gzip 0 0.00 Gzip operations
> n_gunzip 1266712 21.65 Gunzip operations
> LCK.sms.creat 1 0.00 Created locks
> LCK.sms.destroy 0 0.00 Destroyed locks
> LCK.sms.locks 148017 2.53 Lock Operations
> LCK.sms.colls 0 0.00 Collisions
> LCK.smp.creat 1 0.00 Created locks
> LCK.smp.destroy 0 0.00 Destroyed locks
> LCK.smp.locks 10868618 185.78 Lock Operations
> LCK.smp.colls 0 0.00 Collisions
> LCK.sma.creat 1 0.00 Created locks
> LCK.sma.destroy 0 0.00 Destroyed locks
> LCK.sma.locks 4945521 84.54 Lock Operations
> LCK.sma.colls 0 0.00 Collisions
> LCK.smf.creat 0 0.00 Created locks
> LCK.smf.destroy 0 0.00 Destroyed locks
> LCK.smf.locks 0 0.00 Lock Operations
> LCK.smf.colls 0 0.00 Collisions
> LCK.hsl.creat 0 0.00 Created locks
> LCK.hsl.destroy 0 0.00 Destroyed locks
> LCK.hsl.locks 0 0.00 Lock Operations
> LCK.hsl.colls 0 0.00 Collisions
> LCK.hcb.creat 0 0.00 Created locks
> LCK.hcb.destroy 0 0.00 Destroyed locks
> LCK.hcb.locks 0 0.00 Lock Operations
> LCK.hcb.colls 0 0.00 Collisions
> LCK.hcl.creat 500009 8.55 Created locks
> LCK.hcl.destroy 0 0.00 Destroyed locks
> LCK.hcl.locks 4918023 84.07 Lock Operations
> LCK.hcl.colls 0 0.00 Collisions
> LCK.vcl.creat 1 0.00 Created locks
> LCK.vcl.destroy 0 0.00 Destroyed locks
> LCK.vcl.locks 2573092 43.98 Lock Operations
> LCK.vcl.colls 0 0.00 Collisions
> LCK.stat.creat 1 0.00 Created locks
> LCK.stat.destroy 0 0.00 Destroyed locks
> LCK.stat.locks 7933355 135.61 Lock Operations
> LCK.stat.colls 0 0.00 Collisions
> LCK.sessmem.creat 1 0.00 Created locks
> LCK.sessmem.destroy 0 0.00 Destroyed locks
> LCK.sessmem.locks 7932823 135.60 Lock Operations
> LCK.sessmem.colls 0 0.00 Collisions
> LCK.wstat.creat 1 0.00 Created locks
> LCK.wstat.destroy 0 0.00 Destroyed locks
> LCK.wstat.locks 142801 2.44 Lock Operations
> LCK.wstat.colls 0 0.00 Collisions
> LCK.herder.creat 1 0.00 Created locks
> LCK.herder.destroy 0 0.00 Destroyed locks
> LCK.herder.locks 4032 0.07 Lock Operations
> LCK.herder.colls 0 0.00 Collisions
> LCK.wq.creat 2 0.00 Created locks
> LCK.wq.destroy 0 0.00 Destroyed locks
> LCK.wq.locks 11199698 191.44 Lock Operations
> LCK.wq.colls 0 0.00 Collisions
> LCK.objhdr.creat 800591 13.68 Created locks
> LCK.objhdr.destroy 748913 12.80 Destroyed locks
> LCK.objhdr.locks 15295089 261.45 Lock Operations
> LCK.objhdr.colls 0 0.00 Collisions
> LCK.exp.creat 1 0.00 Created locks
> LCK.exp.destroy 0 0.00 Destroyed locks
> LCK.exp.locks 3544111 60.58 Lock Operations
> LCK.exp.colls 0 0.00 Collisions
> LCK.lru.creat 9854 0.17 Created locks
> LCK.lru.destroy 8840 0.15 Destroyed locks
> LCK.lru.locks 2066424 35.32 Lock Operations
> LCK.lru.colls 0 0.00 Collisions
> LCK.cli.creat 1 0.00 Created locks
> LCK.cli.destroy 0 0.00 Destroyed locks
> LCK.cli.locks 19486 0.33 Lock Operations
> LCK.cli.colls 0 0.00 Collisions
> LCK.ban.creat 1 0.00 Created locks
> LCK.ban.destroy 0 0.00 Destroyed locks
> LCK.ban.locks 3842809 65.69 Lock Operations
> LCK.ban.colls 0 0.00 Collisions
> LCK.vbp.creat 1 0.00 Created locks
> LCK.vbp.destroy 0 0.00 Destroyed locks
> LCK.vbp.locks 7808 0.13 Lock Operations
> LCK.vbp.colls 0 0.00 Collisions
> LCK.vbe.creat 1 0.00 Created locks
> LCK.vbe.destroy 0 0.00 Destroyed locks
> LCK.vbe.locks 1017738 17.40 Lock Operations
> LCK.vbe.colls 0 0.00 Collisions
> LCK.backend.creat 4 0.00 Created locks
> LCK.backend.destroy 0 0.00 Destroyed locks
> LCK.backend.locks 6346813 108.49 Lock Operations
> LCK.backend.colls 0 0.00 Collisions
> LCK.busyobj.creat 13983 0.24 Created locks
> LCK.busyobj.destroy 13582 0.23 Destroyed locks
> LCK.busyobj.locks 47081281 804.78 Lock Operations
> LCK.busyobj.colls 0 0.00 Collisions
> SMP.s0.c_req 3025767 . Allocater requests
> SMP.s0.c_fail 352 . Allocater failures
> SMP.s0.c_bytes 115414281600 . Bytes allocated
> SMP.s0.c_freed 110868836208 . Bytes freed
> SMP.s0.g_alloc 158780 . Allocations outstanding
> SMP.s0.g_bytes 4545445392 . Bytes outstanding
> SMP.s0.g_free 138490880 . Free space
> SMP.s0.g_free_pending 0 . Free space pending
> SMP.s0.g_segments 1012 . In use segments
> SMP.s0.c_segments_nuked 8506 . Number of nuked segments
> SMP.s0.g_vampireobjects 0 . Vampire objects
> SMP.s0.c_resurrection_fail 46 . Failed resurrections
> SMA.Transient.c_req 2271927 38.84 Allocator requests
> SMA.Transient.c_fail 0 0.00 Allocator failures
> SMA.Transient.c_bytes 59014304554 1008757.04 Bytes allocated
> SMA.Transient.c_freed 58939691205 1007481.65 Bytes freed
> SMA.Transient.g_alloc 18069 . Allocations
> outstanding
> SMA.Transient.g_bytes 74613349 . Bytes outstanding
> SMA.Transient.g_space 0 . Bytes available
> VBE.proxy1(10.21.34.21,,8080).vcls 1 . VCL references
> VBE.proxy1(10.21.34.21,,8080).happy18446744073709551615 . Happy
> health probes
> VBE.proxy2(10.21.34.22,,8080).vcls 1 . VCL references
> VBE.proxy2(10.21.34.22,,8080).happy18446744056529682431 . Happy
> health probes
> VBE.proxy3(10.21.34.23,,8082).vcls 1 . VCL references
> VBE.proxy3(10.21.34.23,,8082).happy18446744073709551615 . Happy
> health probes
> VBE.proxy4(10.21.34.24,,8081).vcls 1 . VCL references
> VBE.proxy4(10.21.34.24,,8081).happy18446744073709551615 . Happy
> health probes
>
>
> varnish-stream.vcl:
> acce at sz-acce-varnish01:~> cat /opt/huawei/ace/cache/varnish-stream.vcl
> # This is a basic VCL configuration file for varnish. See the vcl(7)
> # man page for details on VCL syntax and semantics.
> #
> # Default backend definition. Set this to point to your content
> # server.
> #
>
> probe healthcheck {
> .url = "/";
> .interval = 30s;
> .timeout = 0.5s;
> .window = 8;
> .threshold = 3;
> .initial = 3;
> }
>
> backend proxy1 {
> .host = "10.21.34.21";
> .port = "8080";
> .probe = healthcheck;
> }
>
> backend proxy2 {
> .host = "10.21.34.22";
> .port = "8080";
> .probe = healthcheck;
> }
>
> backend proxy3 {
> .host = "10.21.34.23";
> .port = "8082";
> .probe = healthcheck;
> }
>
> backend proxy4 {
> .host = "10.21.34.24";
> .port = "8081";
> .probe = healthcheck;
> }
>
> director proxy client {
> {
> .backend = proxy1;
> .weight = 1;
> }
> {
> .backend = proxy2;
> .weight = 1;
> }
> # {
> # .backend = proxy3;
> # .weight = 1;
> # }
> # {
> # .backend = proxy4;
> # .weight = 1;
> # }
> }
>
>
> acl refresh {
> "10.21.34.31";
> }
>
>
> # Below is a commented-out copy of the default VCL logic. If you
> # redefine any of these subroutines, the built-in logic will be
> # appended to your code.
> sub vcl_recv {
>
> if(req.http.X-Real-IP){
> set client.identity = req.http.X-Real-IP;
> }else if (req.http.referer) {
> set client.identity = req.http.referer;
> }else{
> set client.identity = req.url;
> }
> # set req.backend = proxy;
> if (client.ip == "10.21.34.11" || client.ip == "10.21.34.12"){
>
> if (req.http.X-SZ-TEST && req.http.X-SZ-TEST == "1"){
> set req.backend = proxy3;
> } else {
> set req.backend = proxy;
> }
>
> } else {
> set req.backend = proxy4;
> if (client.ip ~ refresh) {
> set req.hash_always_miss = true;
> }
> }
>
> #set grace
> if (req.backend.healthy) {
> set req.grace = 30s;
> } else {
> set req.grace = 30m;
> }
>
> if (req.restarts == 0) {
> if (req.http.x-forwarded-for) {
> set req.http.X-Forwarded-For =
> req.http.X-Forwarded-For + ", " + client.ip;
> } else {
> set req.http.X-Forwarded-For = client.ip;
> }
> }
> if (req.request != "GET" &&
> req.request != "HEAD" &&
> req.request != "PUT" &&
> req.request != "POST" &&
> req.request != "TRACE" &&
> req.request != "OPTIONS" &&
> req.request != "DELETE") {
> # /* Non-RFC2616 or CONNECT which is weird. */
> return (pipe);
> }
>
> if (req.http.x-pipe && req.restarts > 0) {
> remove req.http.x-pipe;
> return (pipe);
> }
>
> if(req.request != "GET" && req.request != "HEAD") {
> # /* We only deal with GET and HEAD by default */
> return (pass);
> }
>
> if (req.http.Cache-Control ~ "no-cache") {
> return (pass);
> }
>
> if (req.http.Accept-Encoding) {
> if (req.url ~
> "\.(webp|jpeg|png|mid|mp3|gif|sql|jpg|nth|thm|utz|mtf|sdt|hme|tsk|zip|rar|sx|pxl|cab|mbm|app|exe|apk)$")
> {
> # No point in compressing these
> remove req.http.Accept-Encoding;
> } elsif (req.http.Accept-Encoding ~ "gzip") {
> set req.http.Accept-Encoding = "gzip";
> } elsif (req.http.Accept-Encoding ~ "deflate") {
> set req.http.Accept-Encoding = "deflate";
> } else {
> # unknown algorithm
> remove req.http.Accept-Encoding;
> }
> }
>
>
> # if (req.request == "GET" &&
> # req.url ~
> "\.(webp|jpeg|png|mid|mp3|gif|sql|jpg|nth|thm|utz|mtf|sdt|hme|tsk|zip|rar|sx|pxl|cab|mbm|app|exe|apk)$"){
> # unset req.http.Cookie;
> # return (lookup);
> # }
>
> # if (req.http.Authorization || req.http.Cookie) {
> # return (pass);
> # }
>
> if (req.http.Authorization) {
> return (pass);
> }
>
> return (lookup);
> }
>
> sub vcl_pipe {
> # # Note that only the first request to the backend will have
> # # X-Forwarded-For set. If you use X-Forwarded-For and want to
> # # have it set for all requests, make sure to have:
> set bereq.http.connection = "close";
> # # here. It is not set by default as it might break some broken web
> # # applications, like IIS with NTLM authentication.
> return (pipe);
> }
>
> sub vcl_pass {
>
> return (pass);
> }
>
> sub vcl_hash {
>
> if (req.url ~ ".(jpeg|jpg|png|gif|ico|js|css)\?.*") {
> hash_data(regsub(req.url, "\?[^\?]*$", ""));
> } else{
> hash_data(req.url);
> }
>
> # hash_data(req.url);
>
> if (req.http.host) {
> hash_data(req.http.host);
> } else {
> hash_data(server.ip);
> }
> return (hash);
> }
>
> sub vcl_hit {
>
> return (deliver);
> }
>
> sub vcl_miss {
>
> return (fetch);
> }
>
>
> sub vcl_fetch {
> set beresp.grace = 30m;
>
> set beresp.do_stream = true;
>
> if (beresp.http.Content-Length && beresp.http.Content-Length ~
> "[0-9]{8,}") {
> set req.http.x-pipe = "1";
> return (restart);
> }
>
> if (beresp.http.Pragma ~ "no-cache" || beresp.http.Cache-Control ~
> "no-cache" || beresp.http.Cache-Control ~ "private"){
> return (hit_for_pass);
> }
>
> if (beresp.ttl <= 0s ||
> beresp.http.Set-Cookie ||
> beresp.http.Vary == "*") {
>
> set beresp.ttl = 120 s;
> return (hit_for_pass);
> }
>
> if(beresp.status >=300 && beresp.status <= 307){
> set beresp.ttl = 120 s;
> return (hit_for_pass);
> }
>
> return (deliver);
> }
>
> sub vcl_deliver {
> if (obj.hits > 0) {
> set resp.http.X-Cache = "HIT from varnis";
> set resp.http.X-Hits = obj.hits;
> } else {
> set resp.http.X-Cache = "MISS from varnish";
> }
> remove resp.http.Via;
> remove resp.http.X-Varnish;
>
> return (deliver);
> }
>
> sub vcl_error {
> set obj.http.Content-Type = "text/html; charset=utf-8";
> set obj.http.Retry-After = "5";
> synthetic {"
> <?xml version="1.0" encoding="utf-8"?>
> <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
> "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
> <html>
> <head>
> <title>"} + obj.status + " " + obj.response + {"</title>
> </head>
> <body>
> <h1>Error "} + obj.status + " " + obj.response + {"</h1>
> <p>"} + obj.response + {"</p>
> <h3>Guru Meditation:</h3>
> <p>XID: "} + req.xid + {"</p>
> <hr>
> <p>Varnish cache server</p>
> <p>varnish</p>
> </body>
> </html>
> "};
> return (deliver);
> }
>
> sub vcl_init {
> return (ok);
> }
>
> sub vcl_fini {
> return (ok);
> }
>
>
> varnish param.show
> param.show
> 200
> acceptor_sleep_decay 0.900000 []
> acceptor_sleep_incr 0.001000 [s]
> acceptor_sleep_max 0.050000 [s]
> auto_restart on [bool]
> ban_dups on [bool]
> ban_lurker_sleep 0.010000 [s]
> between_bytes_timeout 60.000000 [s]
> cc_command "exec gcc -std=gnu99 -pthread -fpic -shared
> -Wl,-x -o %o %s"
> cli_buffer 8192 [bytes]
> cli_timeout 10 [seconds]
> clock_skew 10 [s]
> connect_timeout 0.700000 [s]
> critbit_cooloff 180.000000 [s]
> default_grace 10.000000 [seconds]
> default_keep 0.000000 [seconds]
> default_ttl 120.000000 [seconds]
> diag_bitmap 0x0 [bitmap]
> esi_syntax 0 [bitmap]
> expiry_sleep 1.000000 [seconds]
> fetch_chunksize 128 [kilobytes]
> fetch_maxchunksize 262144 [kilobytes]
> first_byte_timeout 60.000000 [s]
> group nobody (65533)
> gzip_level 6 []
> gzip_memlevel 8 []
> gzip_stack_buffer 32768 [Bytes]
> gzip_tmp_space 0 []
> gzip_window 15 []
> http_gzip_support on [bool]
> http_max_hdr 256 [header lines]
> http_range_support on [bool]
> http_req_hdr_len 32768 [bytes]
> http_req_size 32768 [bytes]
> http_resp_hdr_len 32768 [bytes]
> http_resp_size 32768 [bytes]
> idle_send_timeout 60 [seconds]
> listen_address 0.0.0.0:8080
> listen_depth 1024 [connections]
> log_hashstring on [bool]
> log_local_address off [bool]
> lru_interval 3600 [seconds]
> max_esi_depth 5 [levels]
> max_restarts 4 [restarts]
> nuke_limit 50 [allocations]
> pcre_match_limit 10000 []
> pcre_match_limit_recursion 10000 []
> ping_interval 3 [seconds]
> pipe_timeout 60 [seconds]
> prefer_ipv6 off [bool]
> queue_max 100 [%]
> rush_exponent 3 [requests per request]
> saintmode_threshold 10 [objects]
> send_timeout 600 [seconds]
> sess_timeout 5 [seconds]
> sess_workspace 65536 [bytes]
> session_linger 100 [ms]
> session_max 100000 [sessions]
> shm_reclen 255 [bytes]
> shm_workspace 8192 [bytes]
> shortlived 10.000000 [s]
> stream_maxchunksize 256 [kilobytes]
> stream_tokens 10 []
> syslog_cli_traffic on [bool]
> thread_pool_add_delay 2 [milliseconds]
> thread_pool_add_threshold 0 [requests]
> thread_pool_fail_delay 200 [milliseconds]
> thread_pool_max 2000 [threads]
> thread_pool_min 200 [threads]
> thread_pool_purge_delay 1000 [milliseconds]
> thread_pool_stack unlimited [bytes]
> thread_pool_timeout 300 [seconds]
> thread_pool_workspace 65536 [bytes]
> thread_pools 2 [pools]
> thread_stats_rate 10 [requests]
> user nobody (65534)
> vcc_err_unref on [bool]
> vcl_dir
> /opt//ace/cache/varnish-cache-3.0.3-plus/etc/varnish
> vcl_trace off [bool]
> vmod_dir
> /opt/ace/cache/varnish-cache-3.0.3-plus/lib/varnish/vmods
> waiter default (epoll, poll)
>
>
> Thank you again.
>
> Regards,
> Shawn
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20130515/89257953/attachment-0001.html>
More information about the varnish-misc
mailing list