<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p><br>
</p>
<p>Hi,</p>
<p><br>
</p>
<p>Thanks for the response.</p>
<p>We have upgraded to latest stable version 4.1.3 but still see panics. below are the logs.</p>
<p><br>
</p>
<p></p>
<div>Sep 12 12:45:18 Prod-SHARED-CACHE6 abrt[32688]: Saved core dump of pid 24551 (/product/varnish/sbin/varnishd) to /var/spool/abrt/ccpp-2016-09-12-12:45:15-24551 (585650176 bytes)<br>
Sep 12 12:45:18 Prod-SHARED-CACHE6 abrtd: Directory 'ccpp-2016-09-12-12:45:15-24551' creation detected<br>
Sep 12 12:45:19 Prod-SHARED-CACHE6 abrtd: Executable '/product/varnish/sbin/varnishd' doesn't belong to any package and ProcessUnpackaged is set to 'no'<br>
Sep 12 12:45:19 Prod-SHARED-CACHE6 abrtd: 'post-create' on '/var/spool/abrt/ccpp-2016-09-12-12:45:15-24551' exited with 1<br>
Sep 12 12:45:19 Prod-SHARED-CACHE6 abrtd: Deleting problem directory '/var/spool/abrt/ccpp-2016-09-12-12:45:15-24551'<br>
Sep 12 12:45:19 Prod-SHARED-CACHE6 varnishd[11105]: Child (24551) not responding to CLI, killing it.<br>
Sep 12 12:45:20 Prod-SHARED-CACHE6 varnishd[11105]: Child (24551) died signal=6 (core dumped)<br>
Sep 12 12:45:20 Prod-SHARED-CACHE6 varnishd[11105]: Child (24551) Last panic at: Mon, 12 Sep 2016 07:15:20 GMT#012"Assert error in http_EstimateWS(), cache/cache_http.c line 793:#012 Condition((fm->hd[u].b) != 0) not true.#012thread = (cache-worker)#012version
= varnish-4.1.3 revision 5e3b6d2#012ident = Linux,2.6.32-431.11.2.el6.x86_64,x86_64,-junix,-smalloc,-smalloc,-hcritbit,epoll#012Backtrace:#012 0x432f13: pan_ic+0x163#012 0x42dec5: http_EstimateWS+0x435#012 0x421874: vbf_beresp2obj+0x84#012 0x422cb5: vbf_stp_error+0x235#012
0x42366b: vbf_fetch_thread+0x1db#012 0x449825: WRK_Thread+0x4a5#012 0x449c5b: pool_thread+0x2b#012 0x3c25e079d1: libpthread.so.0() [0x3c25e079d1]#012 0x3c25ae8b6d: libc.so.6(clone+0x6d) [0x3c25ae8b6d]#012busyobj = 0x7f7f505208a0 {#012 ws = 0x7f7f50520960
{#012 OVERFLOWED id = \"Bo\",#012 {s,f,r,e} = {0x7f7f50522818,+57440,(nil),+57440},#012 },#012 refcnt = 2,#012 retries = 0, failed = 1, state = 1,#012 flags = {do_gzip, do_stream, is_gunzip},#012 director_req = 0x1a16130 {#012 vcl_name = farm_anon,#012
type = round-robin {#012 },#012 },#012 director_resp = 0x1a08e98 {#012 vcl_name = PROD_LB_BE_ANON_1,#012 type = backend {#012 display_name = boot.PROD_LB_BE_ANON_1,#012 ipv4 = 10.254.4.254,#012 port = 8080,#012 hosthdr = 10.254.4.254,#012
health=healthy, admin_health=probe, changed=1473664247.5,#012 n_conn = 2,#012 },#012 },#012 http[bereq] = 0x7f7f50520f28 {#012 ws[Bo] = 0x7f7f50520960,#012 hdrs {#012 \"GET\",#012 \"/AVS/besc?action=GetArrayContentList&channel=ANDROID&categoryId=7022\",#012
\"HTTP/1.1\",#012 \"host: <a id="LPlnk528770" href="http://www.example.com" class="OWAAutoLink">
www.example.com</a> \",#012 \"Accept-Encoding: gzip\",#012 \"Akamai-Origin-Hop: 4\",#012 \"True-Client-IP: 183.83.249.187\",#012 \"User-Agent: Dalvik/2.1.0 (Linux; U; Android 6.0.1; Le X526 Build/IIXOSOP5801607291S)\",#012 \"Via: 1.1
v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 aka<br>
Sep 12 12:45:20 Prod-SHARED-CACHE6 varnishd[11105]: Child (386) said Child starts<br>
</div>
<p></p>
<p><br>
</p>
<p>Thanks in advance,</p>
<p>Mahesh.<br>
</p>
<br>
<div style="color: rgb(0, 0, 0);">
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" color="#000000" face="Calibri, sans-serif"><b>From:</b> Dag Haavi Finstad <daghf@varnish-software.com><br>
<b>Sent:</b> Monday, June 27, 2016 2:57 PM<br>
<b>To:</b> Mahesh Ollalwar<br>
<b>Cc:</b> varnish-misc@varnish-cache.org<br>
<b>Subject:</b> Re: varnish core dump</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">Hi<br>
<br>
I believe this might be #1539 [1], which is fixed in 4.0.3 and newer.<br>
<br>
Please upgrade to the latest release and file a bug if you still run into this.<br>
<br>
[1]: <a id="LPlnk497370" href="https://www.varnish-cache.org/trac/ticket/1539">https://www.varnish-cache.org/trac/ticket/1539</a><br>
<br>
On Fri, Jun 24, 2016 at 9:06 PM, Mahesh Ollalwar <mahesho@outlook.com> wrote:<br>
> Can anyone help me on below error. looks like core dump is happening and<br>
> child is getting restarted. This is happening recently without any change to<br>
> the VCL and any system configuration.<br>
><br>
> During this time the requests are getting terminated abruptly.<br>
><br>
> Thanks in advance.<br>
><br>
><br>
> Varnish version - varnish-4.0.2<br>
><br>
><br>
> Jun 24 23:49:30 Prod-CACHE2 abrt[32341]: Saved core dump of pid 26526<br>
> (/product/varnish/sbin/varnishd) to<br>
> /var/spool/abrt/ccpp-2016-06-24-23:48:19-26526 (9034817536 bytes)<br>
> Jun 24 23:49:30 Prod-CACHE2 abrtd: Directory<br>
> 'ccpp-2016-06-24-23:48:19-26526' creation detected<br>
> Jun 24 23:49:31 Prod-CACHE2 abrtd: Executable<br>
> '/product/varnish/sbin/varnishd' doesn't belong to any package and<br>
> ProcessUnpackaged is set to 'no'<br>
> Jun 24 23:49:31 Prod-CACHE2 abrtd: 'post-create' on<br>
> '/var/spool/abrt/ccpp-2016-06-24-23:48:19-26526' exited with 1<br>
> Jun 24 23:49:31 Prod-CACHE2 abrtd: Deleting problem directory<br>
> '/var/spool/abrt/ccpp-2016-06-24-23:48:19-26526'<br>
> Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) not responding to<br>
> CLI, killing it.<br>
> Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) not responding to<br>
> CLI, killing it.<br>
> Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) died signal=6<br>
> (core dumped)<br>
> Jun 24 23:49:31 Prod-CACHE2 varnishd[26524]: Child (26526) Panic<br>
> message:#012Assert error in cnt_lookup(), cache/cache_req_fsm.c line<br>
> 430:#012 Condition((oc->exp_flags & (1<<7)) == 0) not true.#012thread =<br>
> (cache-worker)#012ident =<br>
> Linux,2.6.32-431.11.2.el6.x86_64,x86_64,-smalloc,-smalloc,-hcritbit,epoll#012Backtrace:#012<br>
> 0x4310e5: pan_ic+0xc5#012 0x434c30: cnt_purge+0x0#012 0x436461:<br>
> CNT_Request+0x241#012 0x42b73b: HTTP1_Session+0x53b#012 0x439f98:<br>
> ses_req_pool_task+0x68#012 0x43a6d4: SES_pool_accept_task+0x2a4#012<br>
> 0x433b87: Pool_Work_Thread+0x357#012 0x4467d9: wrk_thread_real+0xb9#012<br>
> 0x3c25e079d1: /lib64/libpthread.so.0() [0x3c25e079d1]#012 0x3c25ae8b6d:<br>
> /lib64/libc.so.6(clone+0x6d) [0x3c25ae8b6d]#012req = 0x7f83baaf2cb0 {#012<br>
> sp = 0x7f84dd71c430, vxid = 1108394165, step = R_STP_LOOKUP,#012 req_body<br>
> = R_BODY_NONE,#012 restarts = 0, esi_level = 0#012 sp = 0x7f84dd71c430<br>
> {#012 fd = 64, vxid = 34652201,#012 client = 10.254.9.78 61217,#012<br>
> step = S_STP_WORKING,#012 },#012 worker = 0x7f8574c0cc40 {#012 ws =<br>
> 0x7f8574c0ce58 {#012 id = "wrk",#012 {s,f,r,e} =<br>
> {0x7f8574c0ac30,0x7f8574c0ac30,(nil),+8192},#012 },#012 VCL::method =<br>
> 0x0,#012 VCL::return = deliver,#012 },#012 ws = 0x7f83baaf2e48 {#012<br>
> id = "req",#012 {s,f,r,e} = {0x7f83baaf4ca0,+1320,(nil),+57360},#012<br>
> },#012 http[req] = {#012 ws = 0x7f83baaf2e48[req]#012 "GET",#012<br>
> "/search-engine-web/SearchContents?action=SearchContents&moreFilters=series%3A3197&appVersion=5.2.2&query=*&startIndex=0&searchOrder=counter_week%20desc&maxResult=8&channel=ANDROID&type=CLIPS",#012<br>
> "HTTP/1.1",#012 "host: xxx.xxx.com",#012 "Accept-Encoding:<br>
> gzip",#012 "Akamai-Origin-Hop: 4",#012 "True-Client-IP:<br>
> 49.15.204.238",#012 "User-Agent: Dalvik/2.1.0 (Linux; U; Android 5.1;<br>
> 1201 Build/LMY47I)",#012 "Via: 1.1 v1-akamaitech.net(ghost)<br>
> (AkamaiGHost), 1.1 v1-akamaitech.net(ghost) (AkamaiGHost), 1.1<br>
> v1-akamaitech.net(ghost) (AkamaiGHost), 1.1 akamai.net(ghost)<br>
> (AkamaiGHost)",#012 "X-Aka<br>
> Jun 24 23:49:31 Prod-SEARCH-CACHE2 varnishd[26524]: child (1277) Started<br>
> Jun 24 23:49:31 Prod-SEARCH-CACHE2 varnishd[26524]: Child (1277) said Child<br>
> starts<br>
><br>
><br>
> Thanks,<br>
><br>
> Mahesh.<br>
><br>
><br>
> _______________________________________________<br>
> varnish-misc mailing list<br>
> varnish-misc@varnish-cache.org<br>
> <a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
<br>
<br>
<br>
-- <br>
Dag Haavi Finstad<br>
Software Developer | Varnish Software<br>
Mobile: +47 476 64 134<br>
We Make Websites Fly!<br>
</div>
</span></font></div>
</div>
</body>
</html>