<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">Hi Guys</span><div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class=""></div><div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);">I am really struggling to debug this panic and wondered if you could give any hints were I might start looking for answers.  I have been trawling the GitHub repo and have seen several people recommend upping the thread_pool_stack value.  Here is the panic and a few other bits of data that I think are useful based on the issues I have been reading:</div><div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class=""></div><div class="" style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"> <span class="" style="color: rgb(74, 92, 110); font-family: "JetBrains Mono", monospace; font-size: 9.8pt; background-color: rgb(255, 255, 255);">varnishd[15482]: Child (15492) Panic at: Tue, 22 Nov 2022 16:10:26 GMT</span><pre class="" style="background-color: rgb(255, 255, 255); color: rgb(74, 92, 110); font-family: "JetBrains Mono", monospace; font-size: 9.8pt;">Wrong turn at cache/cache_main.c:284:<br class="">Signal 11 (Segmentation fault) received at 0xffff887fe000 si_code 2<br class="">version = varnish-6.0.11 revision a3bc025c2df28e4a76e10c2c41217c9864e9963b, vrt api = 7.1<br class="">ident = Linux,4.14.290-217.505.amzn2.aarch64,aarch64,-junix,-smalloc,-sfile,-sdefault,-hcritbit,epoll<br class="">now = 3120063.050288 (mono), 1669133426.954528 (real)<br class="">Backtrace:<br class="">  0x43f5b4: /usr/sbin/varnishd() [0x43f5b4]<br class="">  0x4a968c: /usr/sbin/varnishd(VAS_Fail+0x54) [0x4a968c]<br class="">  0x43a350: /usr/sbin/varnishd() [0x43a350]<br class="">  0xffff90fd7668: linux-vdso.so.1(__kernel_rt_sigreturn+0) [0xffff90fd7668]<br class="">  0xffff90c8f540: /lib64/libc.so.6(memset+0x100) [0xffff90c8f540]<br class="">  0x4b88f4: /usr/sbin/varnishd(deflateReset+0x48) [0x4b88f4]<br class="">  0x42e924: /usr/sbin/varnishd(VGZ_NewGzip+0x88) [0x42e924]<br class="">  0x42ebc0: /usr/sbin/varnishd() [0x42ebc0]<br class="">  0x42df28: /usr/sbin/varnishd(VFP_Open+0x98) [0x42df28]<br class="">  0x42b950: /usr/sbin/varnishd() [0x42b950]<br class="">thread = (cache-worker)<br class="">thr.req = (nil) {<br class="">},<br class="">thr.busyobj = 0xffff3d040020 {<br class="">  end = 0xffff3d050000,<br class="">  retries = 0,<br class="">  sp = 0xffff3c241a20 {<br class="">    fd = 28, vxid = 32945,<br class="">    t_open = 1669133426.953970,<br class="">    t_idle = 1669133426.953970,<br class="">    ws = 0xffff3c241a60 {<br class="">      id = \"ses\",<br class="">      {s, f, r, e} = {0xffff3c241aa0, +96, (nil), +344},<br class="">    },<br class="">    transport = HTTP/1 {<br class="">      state = HTTP1::Proc<br class="">    }<br class="">    client = 172.31.47.149 50812 :80,<br class="">  },<br class="">  worker = 0xffff832326c8 {<br class="">    ws = 0xffff83232770 {<br class="">      id = \"wrk\",<br class="">      {s, f, r, e} = {0xffff83231e00, +0, (nil), +2040},<br class="">    },<br class="">    VCL::method = BACKEND_RESPONSE,<br class="">    VCL::return = deliver,<br class="">    VCL::methods = {BACKEND_FETCH, BACKEND_RESPONSE},<br class="">  },<br class="">  vfc = 0xffff3d041f30 {<br class="">    failed = 0,<br class="">    req = 0xffff3d040640,<br class="">    resp = 0xffff3d040ab8,<br class="">    wrk = 0xffff832326c8,<br class="">    oc = 0xffff3b250640,<br class="">    filters = {<br class="">      gzip = 0xffff3d04a740 {<br class="">        priv1 = (nil),<br class="">        priv2 = 0,<br class="">        closed = 0<br class="">      },<br class="">      V1F_STRAIGHT = 0xffff3d04a660 {<br class="">        priv1 = 0xffff3d042600,<br class="">        priv2 = 674132,<br class="">        closed = 0<br class="">      },<br class="">    },<br class="">    obj_flags = 0x0,<br class="">  },<br class="">  ws = 0xffff3d040058 {<br class="">    id = \"bo\",<br class="">    {s, f, r, e} = {0xffff3d041f78, +34832, (nil), +57472},<br class="">  },<br class="">  ws_bo = 0xffff3d0425e8,<br class="">  http[bereq] = 0xffff3d040640 {<br class="">    ws = 0xffff3d040058 {<br class="">      [Already dumped, see above]<br class="">    },<br class="">    hdrs {<br class="">      \"GET\",<br class="">      \"/build/admin/css/oro.css?v=7c08a284\",<br class="">      \"HTTP/1.1\",<br class="">      \"X-Forwarded-Proto: https\",<br class="">      \"X-Forwarded-Port: 443\",<br class="">      \"Host: <a href="http://london.paperstage.doverstreetmarket.com" class="">london.paperstage.doverstreetmarket.com</a>\",<br class="">      \"X-Amzn-Trace-Id: Root=1-637cf472-4faa5da7246eda4f0a477811\",<br class="">      \"User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/107.0.0.0 Safari/537.36\",<br class="">      \"X-Amz-Cf-Id: WVvRZ7m_mF9MaQ88b0rJd_vEKVJo5YqQr6Povp9ODasHVw2FQop36w==\",<br class="">      \"Via: 3.0 <a href="http://c1685d59e35fdb859ab8a1f97feb5652.cloudfront.net" class="">c1685d59e35fdb859ab8a1f97feb5652.cloudfront.net</a> (CloudFront)\",<br class="">      \"Cookie: BAPID=mdmh5umu6v6l1qejvo0lbhse54; __utma=58316727.170235450.1647004079.1669126271.1669131591.29; __utmb=58316727.22.10.1669131591; __utmc=58316727; __utmt=1; __utmt_b=1; __utmz=58316727.1665659005.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); _csrf=5Inqz08yS1ACLDnJDFG8xoavis88G4BgHx4WpD6RuQM; _ga=GA1.2.170235450.1647004079; _ga_QNDMHZ1SBD=GS1.1.1667387720.14.1.1667387720.60.0.0; _gcl_au=1.1.255577917.1662482672; _landing_page=%2F; _orig_referrer=; _shopify_y=1531218a-8c0c-4153-86e1-37e06cc3a103; _y=1531218a-8c0c-4153-86e1-37e06cc3a103; df_preview=%7B%22preview_id%22%3A%221755%22%2C%22preview_type%22%3A%22version%22%2C%22preview_code%22%3A%22d82194c2b3c4f22164ca71d225673ff6%22%7D\",<br class="">      \"Accept-Language: en-US,en;q=0.9,fr;q=0.8\",<br class="">      \"Accept: text/css,*/*;q=0.1\",<br class="">      \"Referer: <a href="https://london.paperstage.doverstreetmarket.com/doverstreetadmin/pages/settings/1755/en\" class="">https://london.paperstage.doverstreetmarket.com/doverstreetadmin/pages/settings/1755/en\</a>",<br class="">      \"Accept-Encoding: gzip, deflate, br\",<br class="">      \"pragma: no-cache\",<br class="">      \"cache-control: no-cache\",<br class="">      \"sec-gpc: 1\",<br class="">      \"sec-fetch-site: same-origin\",<br class="">      \"sec-fetch-mode: no-cors\",<br class="">      \"sec-fetch-dest: style\",<br class="">      \"X-Forwarded-For: 32.217.249.1, 15.158.35.113, 172.31.47.149\",<br class="">      \"X-Varnish: 32947\",<br class="">    },<br class="">  },<br class="">  http[beresp] = 0xffff3d040ab8 {<br class="">    ws = 0xffff3d040058 {<br class="">      [Already dumped, see above]<br class="">    },<br class="">    hdrs {<br class="">      \"HTTP/1.1\",<br class="">      \"200\",<br class="">      \"OK\",<br class="">      \"Server: nginx/1.20.0\",<br class="">      \"Date: Tue, 22 Nov 2022 16:10:26 GMT\",<br class="">      \"Content-Type: text/css\",<br class="">      \"Content-Length: 674132\",<br class="">      \"Last-Modified: Tue, 22 Nov 2022 15:28:13 GMT\",<br class="">      \"Connection: keep-alive\",<br class="">      \"Expires: Wed, 22 Nov 2023 16:10:26 GMT\",<br class="">      \"Cache-Control: max-age=31536000, public, no-transform\",<br class="">      \"Accept-Ranges: bytes\",<br class="">      \"X-Url: /build/admin/css/oro.css?v=7c08a284\",<br class="">      \"X-Host: <a href="http://london.paperstage.doverstreetmarket.com" class="">london.paperstage.doverstreetmarket.com</a>\",<br class="">    },<br class="">  },<br class="">  objcore[fetch] = 0xffff3b250640 {<br class="">    refcnt = 2,<br class="">    flags = {busy, hfm, private},<br class="">    exp_flags = {},<br class="">    boc = 0xffff3b260160 {<br class="">      refcnt = 2,<br class="">      state = req_done,<br class="">      vary = (nil),<br class="">      stevedore_priv = (nil),<br class="">    },<br class="">    exp = {1669133426.954405, -1.000000, 300.000000, 0.000000},<br class="">    objhead = 0xffff907d01d0,<br class="">    stevedore = (nil),<br class="">  },<br class="">  http_conn = 0xffff3d042600 {<br class="">    fd = 30 (@0xffff3be704e4),<br class="">    doclose = NULL,<br class="">    ws = 0xffff3d040058 {<br class="">      [Already dumped, see above]<br class="">    },<br class="">    {rxbuf_b, rxbuf_e} = {0xffff3d042660, 0xffff3d0427a8},<br class="">    {pipeline_b, pipeline_e} = {0xffff3d0427a8, 0xffff3d04a660},<br class="">    content_length = 674132,<br class="">    body_status = length,<br class="">    first_byte_timeout = 180.000000,<br class="">    between_bytes_timeout = 180.000000,<br class="">  },<br class="">  flags = {do_gzip, do_stream, do_pass, uncacheable},<br class="">  director_req = 0xffff8ffc0190 {<br class="">    vcl_name = default,<br class="">    health = healthy,<br class="">    admin_health = probe, changed = 1669131098.893551,<br class="">    type = backend {<br class="">      display_name = boot.default,<br class="">      ipv4 = 127.0.0.1,<br class="">      port = 8080,<br class="">      hosthdr = 127.0.0.1,<br class="">      n_conn = 1,<br class="">    },<br class="">  },<br class="">  director_resp = director_req,<br class="">  vcl = {<br class="">    name = \"boot\",<br class="">    busy = 5,<br class="">    discard = 0,<br class="">    state = auto,<br class="">    temp = warm,<br class="">    conf = {<br class="">      syntax = \"40\",<br class="">      srcname = {<br class="">        \"/etc/varnish/default.vcl\",<br class="">        \"Builtin\",<br class="">      },<br class="">    },<br class="">  },<br class="">},<br class="">vmods = {<br class="">  std = {Varnish 6.0.11 a3bc025c2df28e4a76e10c2c41217c9864e9963b, 0.0},<br class="">},</pre><div class=""><div class="" style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;"><span class="" style="font-variant-ligatures: no-common-ligatures;">param.show thread_pool_stack</span></div><div class="" style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;"><span class="" style="font-variant-ligatures: no-common-ligatures;">200        </span></div><div class="" style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;"><span class="" style="font-variant-ligatures: no-common-ligatures;">thread_pool_stack</span></div><div class="" style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;"><span class="" style="font-variant-ligatures: no-common-ligatures;">        Value is: 128k [bytes] (default)</span></div><div class="" style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;"><span class="" style="font-variant-ligatures: no-common-ligatures;">        Minimum is: 128k</span></div></div><div class="" style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;"><span class="" style="font-variant-ligatures: no-common-ligatures;"><br class=""></span></div><div class="" style="margin: 0px; font-stretch: normal; font-size: 11px; line-height: normal; font-family: Menlo;"><span class="" style="font-variant-ligatures: no-common-ligatures;"><br class=""></span></div><div class="" style="margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;"><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">yum info jemalloc</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Loaded plugins: extras_suggestions, langpacks, priorities, update-motd</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">223 packages excluded due to repository priority protections</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Installed Packages</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Name        : jemalloc</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Arch        : aarch64</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Version     : 3.6.0</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Release     : 1.el7</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Size        : 374 k</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Repo        : installed</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">From repo   : epel</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Summary     : General-purpose scalable concurrent malloc implementation</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">URL         : <a href="http://www.canonware.com/jemalloc/" class="">http://www.canonware.com/jemalloc/</a></span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">License     : BSD</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Description : General-purpose scalable concurrent malloc(3) implementation.</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">            : This distribution is the stand-alone "portable" implementation of jemalloc.</span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;"><br class=""></span></div><div class="" style="font-size: 11px; font-family: Menlo; margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;"><br class=""></span></div><div class="" style="margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Any insight you can offer would be greatly appreciated.  I am going to try setting the thread_pool_stack to 1024k today, however this error is pretty intermittent so I may not know if it has resolved anything for a day or so.</span></div><div class="" style="margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;"><br class=""></span></div><div class="" style="margin: 0px; font-stretch: normal; line-height: normal;"><span class="" style="font-variant-ligatures: no-common-ligatures;">Kind Regards</span></div></span></div></div><div class="">

<div class="">
    <table cellspacing="0" width="500" cellpadding="0" border="0" class="">
        <tbody class=""><tr class="">
            <td valign="top" style="
                    border-left: 0;
                    margin: 0.1px;
                    vertical-align: top;
                    padding: 0px 0px 0px 0px;
                " class="">
                <table cellspacing="0" cellpadding="0" border="0" style="line-height: 1.4; font: 12px Verdana, sans-serif; font-size: 90%;" class="">
                    <tbody class=""><tr class="">
                        <td style="margin: 0.1px; padding: 15px 0px 0px 0px;" class="">
                            <span style="font: 12px Verdana, sans-serif; color: rgb(0, 0, 1); font-weight:bold" class="">Luke Rotherfield</span>
                        </td>
                    </tr>
                    <tr class="">
                        <td style="margin: 0.1px; " class="">
                            <span style="font: 12px Verdana, sans-serif; color: rgb(0, 0, 1);" class="">Senior Developer</span>
                        </td>
                    </tr>
                    <tr class="">
                        <td style="margin: 0.1px; padding: 10px 0px;>
                            <a href=" https:="" function.london"="" class=""><img width="130" height="27" src="https://img.mysignature.io/p/f/2/4/f24945a8-f205-5db9-ab21-ee94910f0872.png?time=1639354440" class="">
                        </td>
                    </tr>
                    <tr class="">
                        <td style="margin: 0.1px; padding: 5px 0px;" class="">
                            <div style="font: 12px Verdana, sans-serif; color: rgb(0, 0, 1);" class=""><span class=""> </span></div>
                        </td>
                    </tr>
                    <tr class="">
                        <td style="margin: 0.1px;" class="">
                            <span style="font: 12px Verdana, sans-serif;" class=""><a href="https://function.london/" style="text-decoration: none;color:#000001;undefined" target="_blank" class="">https://function.london</a></span>
                        </td>
                    </tr>
                    <tr class="">
                        <td style="margin: 0.1px; color: rgb(0, 0, 1); padding: 8px 0px 3px; display: block;" class=""><div style="font: 12px Verdana, sans-serif;" class=""></div></td>
                    </tr>
                </tbody></table>
            </td>
        </tr>
    </tbody></table>
</div>
<table fragment="b8808b9d9b" width="500" cellspacing="0" cellpadding="0" border="0" class="">
    <tbody class=""><tr class="">
        <td style="margin: 0.1px; line-height: 1px; font-size: 1px; height: 1px;" class=""> </td>
    </tr>
</tbody></table>



</div>
<br class=""></body></html>