<div dir="ltr">please keep the mailing-list CC'd<div><br></div><div>you backend is telling Varnish to not cache:</div><div>-- BerespHeader Cache-Control: no-cache<br class="gmail-Apple-interchange-newline"></div><div><br></div><div>which is acted upon in the built-in.vcl: <a href="https://github.com/varnishcache/varnish-cache/blob/6.0/bin/varnishd/builtin.vcl#L161">https://github.com/varnishcache/varnish-cache/blob/6.0/bin/varnishd/builtin.vcl#L161</a></div><div>more info here; <a href="https://varnish-cache.org/docs/trunk/users-guide/vcl-built-in-code.html#vcl-built-in-code">https://varnish-cache.org/docs/trunk/users-guide/vcl-built-in-code.html#vcl-built-in-code</a></div><div>and maybe this can help too: <a href="https://info.varnish-software.com/blog/finally-understanding-built-in-vcl">https://info.varnish-software.com/blog/finally-understanding-built-in-vcl</a></div><div><br></div><div><br clear="all"><div><div dir="ltr" data-smartmail="gmail_signature"><div dir="ltr"><div>-- <br></div>Guillaume Quintard<br></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Jun 13, 2021 at 11:16 PM Hamidreza Hosseini <<a href="mailto:hrhosseini@hotmail.com" target="_blank">hrhosseini@hotmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
This is one of hls fragments that I want to be cached:<br>
<br>
> wget <a href="http://stream.test.local/hls/mystream/1623650629260.ts" target="_blank">http://stream.test.local/hls/mystream/1623650629260.ts</a>
<div><br>
</div>
<div>```<br>
</div>
<div>* << Request >> 32770 </div>
<div>- Begin req 32769 rxreq</div>
<div>- Timestamp Start: 1623650670.552461 0.000000 0.000000</div>
<div>- Timestamp Req: 1623650670.552461 0.000000 0.000000</div>
<div>- VCL_use boot</div>
<div>- ReqStart 192.168.200.10 58016 a0</div>
<div>- ReqMethod GET</div>
<div>- ReqURL /hls/mystream/1623650629260.ts</div>
<div>- ReqProtocol HTTP/1.1</div>
<div>- ReqHeader User-Agent: Wget/1.20.3 (linux-gnu)</div>
<div>- ReqHeader Accept: */*</div>
<div>- ReqHeader Accept-Encoding: identity</div>
<div>- ReqHeader Host: stream.test.local</div>
<div>- ReqHeader Connection: Keep-Alive</div>
<div>- ReqHeader X-Forwarded-For: 192.168.200.10</div>
<div>- VCL_call RECV</div>
<div>- VCL_return hash</div>
<div>- ReqUnset Accept-Encoding: identity</div>
<div>- VCL_call HASH</div>
<div>- VCL_return lookup</div>
<div>- VCL_call MISS</div>
<div>- VCL_return fetch</div>
<div>- Link bereq 32771 fetch</div>
<div>- Timestamp Fetch: 1623650670.557642 0.005181 0.005181</div>
<div>- RespProtocol HTTP/1.1</div>
<div>- RespStatus 200</div>
<div>- RespReason OK</div>
<div>- RespHeader Server: nginx/1.20.1</div>
<div>- RespHeader Date: Mon, 14 Jun 2021 06:04:30 GMT</div>
<div>- RespHeader Content-Type: video/mp2t</div>
<div>- RespHeader Content-Length: 161868</div>
<div>- RespHeader Last-Modified: Mon, 14 Jun 2021 06:03:51 GMT</div>
<div>- RespHeader ETag: "60c6f147-2784c"</div>
<div>- RespHeader Cache-Control: no-cache</div>
<div>- RespHeader Access-Control-Allow-Origin: *</div>
<div>- RespHeader Access-Control-Expose-Headers: Content-Length</div>
<div>- RespHeader Accept-Ranges: bytes</div>
<div>- RespHeader X-Varnish: 32770</div>
<div>- RespHeader Age: 0</div>
<div>- RespHeader Via: 1.1 varnish (Varnish/6.2)</div>
<div>- VCL_call DELIVER</div>
<div>- VCL_return deliver</div>
<div>- Timestamp Process: 1623650670.557660 0.005199 0.000018</div>
<div>- Filters </div>
<div>- RespHeader Connection: keep-alive</div>
<div>- Timestamp Resp: 1623650670.558417 0.005956 0.000757</div>
<div>- ReqAcct 179 0 179 406 161868 162274</div>
<div>- End </div>
<div>** << BeReq >> 32771 </div>
<div>-- Begin bereq 32770 fetch</div>
<div>-- VCL_use boot</div>
<div>-- Timestamp Start: 1623650670.552655 0.000000 0.000000</div>
<div>-- BereqMethod GET</div>
<div>-- BereqURL /hls/mystream/1623650629260.ts</div>
<div>-- BereqProtocol HTTP/1.1</div>
<div>-- BereqHeader User-Agent: Wget/1.20.3 (linux-gnu)</div>
<div>-- BereqHeader Accept: */*</div>
<div>-- BereqHeader Host: stream.test.local</div>
<div>-- BereqHeader X-Forwarded-For: 192.168.200.10</div>
<div>-- BereqHeader Accept-Encoding: gzip</div>
<div>-- BereqHeader X-Varnish: 32771</div>
<div>-- VCL_call BACKEND_FETCH</div>
<div>-- VCL_return fetch</div>
<div>-- BackendOpen 25 b1 {Backend_ip} 80 {Varnish_ip} 49734</div>
<div>-- BackendStart {Backend_ip} 80</div>
<div>-- Timestamp Bereq: 1623650670.552739 0.000084 0.000084</div>
<div>-- Timestamp Beresp: 1623650670.557325 0.004669 0.004586</div>
<div>-- BerespProtocol HTTP/1.1</div>
<div>-- BerespStatus 200</div>
<div>-- BerespReason OK</div>
<div>-- BerespHeader Server: nginx/1.20.1</div>
<div>-- BerespHeader Date: Mon, 14 Jun 2021 06:04:30 GMT</div>
<div>-- BerespHeader Content-Type: video/mp2t</div>
<div>-- BerespHeader Content-Length: 161868</div>
<div>-- BerespHeader Last-Modified: Mon, 14 Jun 2021 06:03:51 GMT</div>
<div>-- BerespHeader Connection: keep-alive</div>
<div>-- BerespHeader ETag: "60c6f147-2784c"</div>
<div>-- BerespHeader Cache-Control: no-cache</div>
<div>-- BerespHeader Access-Control-Allow-Origin: *</div>
<div>-- BerespHeader Access-Control-Expose-Headers: Content-Length</div>
<div>-- BerespHeader Accept-Ranges: bytes</div>
<div>-- TTL RFC 120 10 0 1623650671 1623650671 1623650670 0 0 cacheable</div>
<div>-- VCL_call BACKEND_RESPONSE</div>
<div>-- TTL VCL 300 10 0 1623650671 cacheable</div>
<div>-- TTL VCL 30 10 0 1623650671 cacheable</div>
<div>-- TTL VCL 120 10 0 1623650671 cacheable</div>
<div>-- TTL VCL 120 10 0 1623650671 uncacheable</div>
<div>-- VCL_return deliver</div>
<div>-- Filters </div>
<div>-- Storage malloc Transient</div>
<div>-- Fetch_Body 3 length stream</div>
<div>-- BackendReuse 25 b1</div>
<div>-- Timestamp BerespBody: 1623650670.558352 0.005697 0.001028</div>
<div>-- Length 161868</div>
<div>-- BereqAcct 202 0 202 348 161868 162216</div>
<div>-- End </div>
<div><br>
</div>
<div>```<br>
</div>
<span></span><br>
</div>
<div id="gmail-m_-2068176009274619958gmail-m_7977940281204846255appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-2068176009274619958gmail-m_7977940281204846255divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Guillaume Quintard <<a href="mailto:guillaume@varnish-software.com" target="_blank">guillaume@varnish-software.com</a>><br>
<b>Sent:</b> Sunday, June 13, 2021 8:45 AM<br>
<b>To:</b> Hamidreza Hosseini <<a href="mailto:hrhosseini@hotmail.com" target="_blank">hrhosseini@hotmail.com</a>><br>
<b>Cc:</b> varnish-misc <<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a>><br>
<b>Subject:</b> Re: Varnish wouldn't cache HLS fragments</font>
<div> </div>
</div>
<div>
<div dir="auto">
<div>Hi,
<div dir="auto"><br>
</div>
<div dir="auto">Can you share the output of "varnishlog -g request" for one of those requests that should be cached please?</div>
<div dir="auto"><br>
</div>
<div dir="auto">Cheers,<br>
<br>
<div dir="auto">-- <br>
Guillaume Quintard</div>
</div>
<br>
<div>
<div dir="ltr">On Sun, Jun 13, 2021, 00:17 Hamidreza Hosseini <<a href="mailto:hrhosseini@hotmail.com" target="_blank">hrhosseini@hotmail.com</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hi,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I put varnish in front of my http servers to serve Hls streaming, I want varnish cache the fragments but not .m3u8 manifest file,</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I configure it but it cache nothing!</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
My configuration file:</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
```</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
vcl 4.1;
<div><br>
</div>
<div>import directors;</div>
<div><br>
</div>
<div><br>
</div>
<div>backend b1 {</div>
<div> .host = "playback-02";</div>
<div> .probe = {</div>
<div> .url = "/";</div>
<div> .timeout = 150 ms;</div>
<div> .interval = 10s;</div>
<div> .window = 6;</div>
<div> .threshold = 5;</div>
<div> }</div>
<div>}</div>
<div> </div>
<div><br>
</div>
<div><br>
</div>
<div>sub vcl_init {</div>
<div> # we use round robin director for our backend swift proxies</div>
<div><br>
</div>
<div> new hls_cluster = directors.round_robin();</div>
<div> hls_cluster.add_backend(b1);</div>
<div><br>
</div>
<div>}</div>
<div><br>
</div>
<div>acl purge {</div>
<div> "localhost";</div>
<div>}</div>
<div><br>
</div>
<div><br>
</div>
<div>sub vcl_recv {</div>
<div><br>
</div>
<div> set req.backend_hint = hls_cluster.backend();</div>
<div> if (req.method == "PURGE") {</div>
<div> if (!client.ip ~ purge) {</div>
<div> return(synth(405,"Not allowed."));</div>
<div> }</div>
<div> return (purge);</div>
<div> }</div>
<div><br>
</div>
<div> if (req.url ~ "\.m3u8$") {</div>
<div> return (pass);</div>
<div> }</div>
<div>}</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div>sub vcl_backend_response {</div>
<div> # cache for half of a day</div>
<div> set beresp.ttl=5m;</div>
<div> # Don't cache 404 responses</div>
<div><br>
</div>
<div> if (bereq.url ~ "\.(aac|dash|m4s|mp4|ts)$") {</div>
<div> set beresp.ttl = 30s;</div>
<div> }</div>
<div><br>
</div>
<div> if ( beresp.status == 404 ) {</div>
<div> set beresp.ttl = 120s;</div>
<div> set beresp.uncacheable = true;</div>
<div> return (deliver);</div>
<div> }</div>
<div> if (beresp.status == 500 || beresp.status == 502 || beresp.status == 503 || beresp.status == 504)</div>
<div> {</div>
<div> set beresp.uncacheable = true;</div>
<div> }</div>
<span>}</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
```</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Varnish version:<br>
<div>varnishd (varnish-6.0.7 revision 525d371e3ea0e0c38edd7baf0f80dc226560f26e)</div>
<div>Copyright (c) 2006 Verdens Gang AS</div>
<span>Copyright (c) 2006-2020 Varnish Software AS</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span><br>
</span></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span>Distribution: Ubuntu 20.04 LTS<br>
<br>
</span></div>
</div>
_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org" rel="noreferrer" target="_blank">varnish-misc@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" rel="noreferrer noreferrer" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote></div>