varnish core dump

[Mahesh Ollalwar]

Can anyone help me on below error. looks like core dump is happening and child is getting restarted. This is happening recently without any change to the VCL and any system configuration.

During this time the requests are getting terminated abruptly.

Thanks in advance.


Varnish version - varnish-4.0.2


Jun 24 23:49:30 Prod-CACHE2 abrt[32341]: Saved core dump of pid 26526 (/product/varnish/sbin/varnishd) to /var/spool/abrt/ccpp-2016-06-24-23:48:19-26526 (9034817536 bytes)
Jun 24 23:49:30 Prod-CACHE2 abrtd: Directory 'ccpp-2016-06-24-23:48:19-26526' creation detected
Jun 24 23:49:31 Prod-CACHE2 abrtd: Executable '/product/varnish/sbin/varnishd' doesn't belong to any package and ProcessUnpackaged is set to 'no'
Jun 24 23:49:31 Prod-CACHE2 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2016-06-24-23:48:19-26526' exited with 1
Jun 24 23:49:31 Prod-CACHE2 abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2016-06-24-23:48:19-26526'
Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) not responding to CLI, killing it.
Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) not responding to CLI, killing it.
Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) died signal=6 (core dumped)
Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) Panic message:#012Assert error in cnt_lookup(), cache/cache_req_fsm.c line 430:#012  Condition((oc->exp_flags & (1<<7)) == 0) not true.#012thread = (cache-worker)#012ident = Linux,2.6.32-431.11.2.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit,epoll#012Backtrace:#012  0x4310e5: pan_ic+0xc5#012  0x434c30: cnt_purge+0x0#012  0x436461: CNT_Request+0x241#012  0x42b73b: HTTP1_Session+0x53b#012  0x439f98: ses_req_pool_task+0x68#012  0x43a6d4: SES_pool_accept_task+0x2a4#012  0x433b87: Pool_Work_Thread+0x357#012  0x4467d9: wrk_thread_real+0xb9#012  0x3c25e079d1: /lib64/libpthread.so.0() [0x3c25e079d1]#012  0x3c25ae8b6d: /lib64/libc.so.6(clone+0x6d) [0x3c25ae8b6d]#012req = 0x7f83baaf2cb0 {#012  sp = 0x7f84dd71c430, vxid = 1108394165,  step = R_STP_LOOKUP,#012  req_body = R_BODY_NONE,#012  restarts = 0, esi_level = 0#012  sp = 0x7f84dd71c430 {#012    fd = 64, vxid = 34652201,#012    client = 10.254.9.78 61217,#012    step = S_STP_WORKING,#012  },#012  worker = 0x7f8574c0cc40 {#012    ws = 0x7f8574c0ce58 {#012      id = "wrk",#012      {s,f,r,e} = {0x7f8574c0ac30,0x7f8574c0ac30,(nil),+8192},#012    },#012  VCL::method = 0x0,#012  VCL::return = deliver,#012  },#012  ws = 0x7f83baaf2e48 {#012    id = "req",#012    {s,f,r,e} = {0x7f83baaf4ca0,+1320,(nil),+57360},#012  },#012  http[req] = {#012    ws = 0x7f83baaf2e48[req]#012      "GET",#012      "/search-engine-web/SearchContents?action=SearchContents&moreFilters=series%3A3197&appVersion=5.2.2&query=*&startIndex=0&searchOrder=counter_week%20desc&maxResult=8&channel=ANDROID&type=CLIPS",#012      "HTTP/1.1",#012      "host: xxx.xxx.com",#012      "Accept-Encoding: gzip",#012      "Akamai-Origin-Hop: 4",#012      "True-Client-IP: 49.15.204.238",#012      "User-Agent: Dalvik/2.1.0 (Linux; U; Android 5.1; 1201 Build/LMY47I)",#012      "Via: 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 akamai.net(ghost) (AkamaiGHost)",#012      "X-Aka
Jun 24 23:49:31 Prod-SEARCH-CACHE2 varnishd[26524]: child (1277) Started
Jun 24 23:49:31 Prod-SEARCH-CACHE2 varnishd[26524]: Child (1277) said Child starts


Thanks,

Mahesh.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20160624/866191e3/attachment.html>