varnish crash

[Γιάννης Καραγιαννίδης]

Dear Varnish List,

Today, we experienced a Varnish crash.
Just look at the syslog file. We did, and we found the following error message:

Dec  9 13:14:45 server1 varnishd[4347]: Child (22446) not responding to CLI, killing it.
Dec  9 13:14:45 server1 varnishd[4347]: Child (22446) not responding to CLI, killing it.
Dec  9 13:14:45 server1 varnishd[4347]: Child (22446) died signal=6
Dec  9 13:14:45 server1 varnishd[4347]: Child (22446) Panic message: Assert error in VRT_r_req_xid(), cache_vrt_var.c line 430:#012  Condition((p = WS_Alloc(sp->http->ws, size)) != 0) not true.#012thread = (cache-worker)#012ident = Linux,3.2.0-4-amd64,x86_64,-smalloc,-smalloc,-hcritbit,epoll#012Backtrace:#012  0x432215: pan_ic+0xd5#012  0x43f4f9: VRT_r_req_xid+0x109#012  0x7fe285462e1c: ./vcl.6EDYfe7E.so(VGC_function_vcl_error+0x813) [0x7fe285462e1c]#012  0x43986b: VCL_error_method+0x4b#012  0x41a260: CNT_Session+0x470#012  0x433f89: wrk_thread_real+0x489#012  0x7fe714a2ab50: /lib/x86_64-linux-gnu/libpthread.so.0(+0x6b50) [0x7fe714a2ab50]#012  0x7fe7147750ed: /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fe7147750ed]#012sp = 0x7fdc8f282008 {#012  fd = 4385, id = 4385, xid = 1916227992,#012  client = 66.249.78.214 64989,#012  step = STP_ERROR,#012  handling = deliver,#012  err_code = 503, err_reason = (null),#012  restarts = 4, esi_level = 0#012  flags = #012  bodystatus = 4#012  ws = 0x7fdc8f282080 { overflow#012    id = "sess",#012    {s,f,r,e} = {0x7fdc8f282c78,+65536,(nil),+65536},#012  },#012  http[req] = {#012    ws = 0x7fdc8f282080[sess]#012      "GET",#012      "/%CE%88%CF%84%CE%BF%CE%B9%CE%BC%CE%BF%CF%82%20%CE%BD%CE%B1%20%CF%83%CF%85%CE%BD%CE%B1%CE%BD%CF%84%CE%B7%CE%B8%CE%B5%CE%AF%20%CE%BC%CE%B5%20%CF%84%CE%BF%CE%BD%20%CF%80%CF%81%CF%8C%CE%B5%CE%B4%CF%81%CE%BF%20%CF%84%CE%BF%CF%85%20%CE%A3%CE%A5%CE%A1%CE%99%CE%96%CE%91,%20%CE%B1%CE%BB%CE%BB%CE%AC%20%CE%BA%CE%B1%CE%B9%20%CE%BC%CE%B5%20%CF%84%CE%BF%CF%85%CF%82%20%CE%B5%CF%80%CE%B9%CE%BA%CE%B5%CF%86%CE%B1%CE%BB%CE%AE%CF%82%20%CF%8C%CE%BB%CF%89%CE%BD%20%CF%84%CF%89%CE%BD%20%CF%83%CF%85%CE%BB%CE%BB%CE%BF%CE%B3%CE%B9%CE%BA%CE%BF%CF%84%CE%AE%CF%84%CF%89%CE%BD%20%CE%BC%CE%B5%20%CF%84%CE%B9%CF%82%20%CE%BF%CF%80%CE%BF%CE%AF%CE%B5%CF%82%20%CE%B7%20%CE%94%CE%97%CE%9C%CE%91%CE%A1%20%CE%B5%CE%AF%CE%BD%CE%B1%CE%B9%20%CF%83%CE%B5%20%CF%83%CF%85%CE%B6%CE%B7%CF%84%CE%AE%CF%83%CE%B5%CE%B9%CF%82%20%CE%B4%CE%B7%CE%BB%CF%8E%CE%BD%CE%B5%CE%B9%20%CE%BF%20%CE%A6%CF%8E%CF%84%CE%
Dec  9 13:14:45 server1 varnishd[4347]: Child cleanup complete
Dec  9 13:14:45 server1 varnishd[4347]: child (13526) Started
Dec  9 13:14:45 server1 varnishd[4347]: Child (13526) said Child starts

We are running
varnishd (varnish-3.0.6 revision 1899836)


Varnish is started with the following settings:

#!/bin/sh

echo starting varnish daemon

ulimit -n 131072

ulimit -l unlimited

/path/to/varnishd \

    -s malloc,48G \

    -a 0.0.0.0:80 \

    -T 127.0.0.1:7777 \

    -p thread_pools=8 \

    -p thread_pool_min=100 \

    -p thread_pool_max=4000 \

    -f /path/to/server.vcl

echo done





Any ideas and or thoughts would be appreciated.
Regards
Yiannis
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20141209/88e2532e/attachment.html>