<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hi,</p>
<p><br>
</p>
<p><span class="HwtZe" lang="en">S<span class="jCAhz ChMk0b"><span class="ryNqvb">orry for my late response, but I found another video extension that must have been a problem.<span class="HwtZe" lang="en"><span class="jCAhz ChMk0b"><span class="ryNqvb"> It
seems ok for 1 month, fingers crossed ;)</span></span></span></span></span></span></p>
<p><span class="HwtZe" lang="en"><span class="jCAhz ChMk0b"><span class="ryNqvb"><span class="HwtZe" lang="en"><span class="jCAhz ChMk0b"><span class="ryNqvb">thank you Guillaume.<br>
</span></span></span></span></span></span></p>
<p><br>
</p>
<div id="Signature">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, "EmojiFont", "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p></p>
<div>Karim Ayari<br>
</div>
<p></p>
</div>
</div>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>De :</b> Guillaume Quintard <guillaume.quintard@gmail.com><br>
<b>Envoyé :</b> vendredi 27 octobre 2023 18:24<br>
<b>À :</b> AYARI KARIM<br>
<b>Cc :</b> varnish-misc@varnish-cache.org<br>
<b>Objet :</b> Re: varnish and mp4 files</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi,
<div><br>
</div>
<div>Check the varnishncsa logs to see if something fishy stands out, or even just the backend logs.
</div>
<div>Notably, try </div>
<div><font face="monospace"> varnishncsa -q 'not VCL_call eq PIPE'</font> </div>
<div>That will give you all the transactions that go through vcl_pipe{} (more log tips here:
<a href="https://MailInBlack.univ-lyon1.fr/securelink/?url=https://docs.varnish-software.com&key=eyJsYW5nIjoiRlIiLCJ1cmwiOiJodHRwczovL2RvY3MudmFybmlzaC1zb2Z0d2FyZS5jb20vdHV0b3JpYWxzL3ZzbC1xdWVyeS8iLCJ0b2tlbiI6ImdBQUFBQUJsTy1SYnh6RkpodnhKUTRfVC1JQW90YUNyRGg4MW8taTZIdFlPbTdJTDE4UG9PYVVDOFFkUmhLNWk1aXBvbE1USUFKUjVqV090VFBEOHA0RTBwMUU1eUdGY2tMeVFuWkdHaHN2dEhOTFhCdThvLXJYSEdGS0RxMmE5QjdXV1lHcklsR0RSZy00SUpSNER2c2hFMy00YnJxbHo1TE4yaUs4Nkx0SWp3WWhpRURKX2NfY3laSW85WE9aNVZNNS1TOHRrWFVCZFFWOFotZ0x1WXFNUVN2Wm8wQUZqcTYtdVJpb3pYOGR1RklwNEF5OWU4UV9ETWoxQmlrdlNwRVczQkR0ZEJQVTB1QnVEWU52VTg4dmc0V0tUckJvdmpqUkJrUTZ4MTBYTzBla2RpLWtFQmNtdGVlcmdLTTNfbDN0Qlk2QmcxN25fIn0=">
https://docs.varnish-software.com/tutorials/vsl-query/</a>) </div>
<div><br>
</div>
<div>If that doesn't work we can try to filter big files and whatnot. <br clear="all">
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>-- <br>
</div>
<div>Guillaume Quintard <br>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Oct 26, 2023 at 11:47 PM AYARI KARIM <<a href="mailto:karim.ayari@univ-lyon1.fr">karim.ayari@univ-lyon1.fr</a>> wrote:
<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div class="msg-5302022001912611345">
<div dir="ltr">
<div id="m_-5041810620345097330divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif">
<p><span lang="en"><span><span>Hello,</span></span></span></p>
<p><span lang="en"><br>
</span></p>
<p><span lang="en"><span><span>I'm coming back to this subject because after a moment of calm my problem of memory leak and OOM killer with videos is back :(</span></span></span></p>
<p><span lang="en"><span><span><span lang="en"><span><span>we have videos hosted on Moodle behind varnish which are themselves behind haproxy</span></span></span>.<br>
</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>version <span>varnish 6.2.1-2ubuntu0.2 amd64</span><br>
</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>this expression seems to work<br>
</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span><i>if (req.url ~ "(?i)^/[^?]+\.mp4($|\?)") {</i><i><br>
</i><i> std.log("ispiped:true");</i><i><br>
</i><i> return (pipe);</i><i><br>
</i><i>}</i><br>
</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>mp4 files are piped :</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span></span></span></span></p>
<div><i>- VCL_call RECV<br>
- VCL_Log ispiped:true<br>
- VCL_return pipe<br>
- VCL_call HASH<br>
- VCL_return lookup</i> </div>
<br>
<p></p>
<p><span lang="en"><span><span>first graph is memory usage (total of 16GB), the second one requests fetched, we can see 2 restarts after OOM killing at 16h25 and 18h20<br>
</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span><img naturalheight="562" naturalwidth="296" size="0" id="m_-5041810620345097330img367884" style="max-width:99.9%" src="cid:18b71ec5db3f456b1e53"><br>
</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>I don't see any other file extension to add in the expression</span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>thank you for your help<br>
</span></span></span></p>
<p><br>
</p>
<div id="m_-5041810620345097330Signature">
<div id="m_-5041810620345097330divtagdefaultwrapper" dir="ltr" style="font-size:12pt; color:rgb(0,0,0); font-family:Calibri,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols">
<p></p>
<div>--- <br>
</div>
<div>Karim Ayari <br>
</div>
</div>
</div>
<br>
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block; width:98%">
<div id="m_-5041810620345097330divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>De :</b> varnish-misc <varnish-misc-bounces+karim.ayari=<a href="mailto:univ-lyon1.fr@varnish-cache.org" target="_blank">univ-lyon1.fr@varnish-cache.org</a>>
de la part de Karim Ayari <<a href="mailto:karim.ayari@univ-lyon1.fr" target="_blank">karim.ayari@univ-lyon1.fr</a>><br>
<b>Envoyé :</b> mardi 21 février 2023 09:05<br>
<b>À :</b> Guillaume Quintard<br>
<b>Cc :</b> <a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
<b>Objet :</b> Re: varnish and mp4 files</font>
<div> </div>
</div>
<div>
<p><span lang="en">sorry I thought I noted it in my previous email.</span></p>
<div id="m_-5041810620345097330tw-target-text-container"><span lang="en">here are the good lines for using pipe :<br>
</span></div>
<p><br>
<i>if (req.url ~ "(?i)^/[^?]+\.mp4($|\?)") {</i><i><br>
</i><i> std.log("ispiped:true");</i><i><br>
</i><i> return (pipe);</i><i><br>
</i><i>}</i><br>
</p>
<p>thank you!</p>
<p>Karim.<br>
</p>
<p><br>
</p>
<p><br>
</p>
<pre cols="72"></pre>
<div>Le 20/02/2023 à 21:50, Guillaume Quintard a écrit : <br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Looks like Rainer replied directly to you and not to the list, would you mind sharing/highlighting the fix for people having the same issue?
</div>
<div><br>
</div>
<div>Cheers, </div>
<div><br>
</div>
<div>
<div>
<div dir="ltr" class="gmail_signature">
<div dir="ltr">
<div>-- <br>
</div>
<div>Guillaume Quintard <br>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Feb 20, 2023 at 12:36 PM Karim Ayari <<a href="mailto:karim.ayari@univ-lyon1.fr" target="_blank">karim.ayari@univ-lyon1.fr</a>> wrote:
<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div bgcolor="#FFFFFF">
<p><span lang="en"><span><span>thank you both for your replies.</span></span><span><span>
</span></span><span><span>I forgot the most important thing:</span></span><span><span>
</span></span><span><span>the varnish server has 16 GB of ram and the cache is 1 GB.</span></span><span><span>
</span></span><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>the cache never seems full</span></span></span></p>
<p><span lang="en"><span><span><img naturalheight="486" naturalwidth="1320" alt="" width="466" height="171" src="cid:18b71ec5db3af62603d1"><img naturalheight="478" naturalwidth="1316" alt="" width="478" height="172" src="cid:18b71ec5db3c3ba5d52"></span></span></span></p>
<p><span lang="en"><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>the command used : <b>/usr/sbin/varnishd -j unix,user=vcache -F -a:8080 -T localhost:6082 -f /etc/varnish/moodle.vcl -S /etc/varnish/secret -s malloc,1g</b></span></span><b>
</b><b><span><span>-p http_max_hdr=96</span></span></b><b><span><span> </span></span></b><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>I had tried using pipe it didn't work, but Rainer's lines works fine.</span></span></span></p>
<p><span lang="en"><span><span>this solution should suffice because the videos will soon have to be hosted on our video platform.</span></span></span><span lang="en"><span><span></span></span></span></p>
<p><span lang="en"><i><br>
</i></span></p>
<p><span lang="en">my vcl (file found on github) :</span></p>
<p><span lang="en"><i><br>
</i></span></p>
<p><span lang="en"><i>....<br>
</i></span></p>
<p><span lang="en"><i>sub vcl_recv {</i><i><br>
</i><i><br>
</i><i> # Keep client IP</i><i><br>
</i><i> if (req.restarts == 0) {</i><i><br>
</i><i> if (req.http.x-forwarded-for) {</i><i><br>
</i><i> set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;</i><i><br>
</i><i> } else {</i><i><br>
</i><i> unset req.http.X-Forwarded-For;</i><i><br>
</i><i> set req.http.X-Forwarded-For = client.ip;</i><i><br>
</i><i> }</i><i><br>
</i><i> }</i><i><br>
</i><i><br>
</i><i> if (req.http.X-Real-IP) {</i><i><br>
</i><i> set req.http.X-Forwarded-For = req.http.X-Real-IP;</i><i><br>
</i><i> } else {</i><i><br>
</i><i> set req.http.X-Forwarded-For = client.ip;</i><i><br>
</i><i> }</i><i><br>
</i><i> </i><i><br>
</i><i> # Only deal with "normal" types</i><i><br>
</i><i> if (req.method != "GET" &&</i><i><br>
</i><i> req.method != "HEAD" &&</i><i><br>
</i><i> req.method != "PUT" &&</i><i><br>
</i><i> req.method != "POST" &&</i><i><br>
</i><i> req.method != "TRACE" &&</i><i><br>
</i><i> req.method != "OPTIONS" &&</i><i><br>
</i><i> req.method != "PATCH" &&</i><i><br>
</i><i> req.method != "DELETE") {</i><i><br>
</i><i> /* Non-RFC2616 or CONNECT which is weird. */</i><i><br>
</i><i> /*Why send the packet upstream, while the visitor is using a non-valid HTTP method? */</i><i><br>
</i><i> return (synth(404, "Non-valid HTTP method!"));</i><i><br>
</i><i> }</i><i><br>
</i><i> </i><i><br>
</i><i> # Varnish don't mess with healthchecks</i><i><br>
</i><i> if (req.url ~ "^/admin/tool/heartbeat" || req.url ~ "^/healthcheck.php") {</i><i><br>
</i><i> return (pass);</i><i><br>
</i><i> }</i><i><br>
</i><i> # Pipe requests to backup.php straight to backend - prevents problem with progress bar long polling 503 problem</i><i><br>
</i><i> # This is here because backup.php is POSTing to itself - Filter before !GET&&!HEAD</i><i><br>
</i><i> if (req.url ~ "^/backup/backup.php")</i><i><br>
</i><i> {</i><i><br>
</i><i> return (pipe);</i><i><br>
</i><i> }</i><i><br>
</i><i><br>
</i><i> # Varnish only deals with GET and HEAD by default. If request method is not GET or HEAD, pass request to backend</i><i><br>
</i><i> if (req.method != "GET" && req.method != "HEAD") {</i><i><br>
</i><i> return (pass);</i><i><br>
</i><i> }</i><i><br>
</i><i><br>
</i><i> if (req.http.Cookie) {</i><i><br>
</i><i> # Remove any Google Analytics based cookies</i><i><br>
</i><i> set req.http.Cookie = regsuball(req.http.Cookie, "^_ga$", "");</i><i><br>
</i><i> set req.http.Cookie = regsuball(req.http.Cookie, "^_gid$", "");</i><i><br>
</i><i> set req.http.Cookie = regsuball(req.http.Cookie, "__gads=[^;]+(; )?", "");</i><i><br>
</i><i> set req.http.Cookie = regsuball(req.http.Cookie, "__qc.=[^;]+(; )?", "");</i><i><br>
</i><i> set req.http.Cookie = regsuball(req.http.Cookie, "__atuv.=[^;]+(; )?", "");</i><i><br>
</i><i> set req.http.Cookie = regsuball(req.http.Cookie, "^;\s*", "");</i><i><br>
</i><i> if (req.http.Cookie ~ "^\s*$") {</i><i><br>
</i><i> unset req.http.Cookie;</i><i><br>
</i><i> }</i><i><br>
</i><i> }</i><i><br>
</i><i><br>
</i><i> ### Rules for Moodle ###</i><i><br>
</i><i><br>
</i><i> # Perform lookup for selected assets that we know are static but Moodle still needs a Cookie</i><i><br>
</i><i> if( req.url ~ "^/theme/.+\.?" ||</i><i><br>
</i><i> req.url ~ "^/webservice/pluginfile.php/.+\.(png|jpg)$" ||</i><i><br>
</i><i> req.url ~ "^/lib/.+\.(png|jpg|jpeg|gif|css|js|webp)$" ||</i><i><br>
</i><i> req.url ~ "^/pluginfile.php/[0-9]+/course/.+\.(?i)(png|jpg)$" ||</i><i><br>
</i><i> req.url ~ "^/pluginfile.php/[0-9]+/theme_moove/.+\.(?i)(png|jpg)$"</i><i><br>
</i><i> )</i><i><br>
</i><i> {</i><i><br>
</i><i> # Set internal temporary header, based on which we will do things in vcl_backend_response</i><i><br>
</i><i> set req.http.X-Long-TTL = "86400";</i><i><br>
</i><i> return (hash);</i><i><br>
</i><i> }</i><i><br>
</i><i> # Requests containing "Cookie" or "Authorization" headers will not be cached</i><i><br>
</i><i> if (req.http.Authorization || req.http.Cookie) {</i><i><br>
</i><i> return (pass);</i><i><br>
</i><i> }</i><i><br>
</i><i> # Almost everything in Moodle correctly serves Cache-Control headers, if</i><i><br>
</i><i> # needed, which varnish will honor, but there are some which don't. Rather</i><i><br>
</i><i> # than explicitly finding them all and listing them here we just fail safe</i><i><br>
</i><i> # and don't cache unknown urls that get this far.</i><i><br>
</i><i> return (pass);</i><i><br>
</i><i><br>
</i><i>}</i><i><br>
</i><i><br>
</i><i>sub vcl_backend_response {</i><i><br>
</i><i> # Set backend name</i><i><br>
</i><i> set beresp.http.X-Backend = <a href="https://MailInBlack.univ-lyon1.fr/securelink/?url=http://beresp.backend.name&key=eyJsYW5nIjoiRlIiLCJ1cmwiOiJodHRwOi8vYmVyZXNwLmJhY2tlbmQubmFtZSIsInRva2VuIjoiZ0FBQUFBQmxPLVJiZUxXNHRyQWRqVVI3RDNPNEFJUXkwTVZkWnBvRk9MREQwYWlyOENmYjlUVUVrczBkck5VU05ibTBxZlZVdXpJaEUtenFBT0VLN3FETHJuV2VwZUpyVUlMMFZFN1Ayb2YwbkpVdjh4bWo2N3FTdjFUTUxZeElEdG5jVnZHcVNGVjM2U00tOTRLTF92bzR2eFhrVkdRYkRxSmxQWE5remM5OFB3VjZzSHNPMHZoTDhKekh5ZUxPLWpQdFNXTTZLLVVoelNDVzMycVFKOEFUM256RmswaEkzQVNUQjZONHF3UnBJMV9QMDM3MkU3SU9CRkF2OWtRR18wQ1BTZmtuLVMtNy16UnpTeFZJRHBqWXllUF9oS2VRYXUyalIzbGdTMlBBSU55cGZfbnV1VzJrRjFYUWNKTEpfZURzUkFzQllEeGkifQ==" target="_blank">
beresp.backend.name</a>;</i><i><br>
</i><i><br>
</i><i> if (beresp.http.Cache-Control && bereq.http.X-Long-TTL && beresp.ttl < std.duration(bereq.http.X-Long-TTL + "s", 1s) && !beresp.http.WWW-Authenticate ) { # If max-age < defined in X-Long-TTL header</i><i><br>
</i><i> set beresp.http.X-Orig-Pragma = beresp.http.Pragma; unset beresp.http.Pragma;</i><i><br>
</i><i> set beresp.http.X-Orig-Cache-Control = beresp.http.Cache-Control;</i><i><br>
</i><i> set beresp.http.Cache-Control = "public, max-age="+bereq.http.X-Long-TTL+", no-transform";</i><i><br>
</i><i> set beresp.ttl = std.duration(bereq.http.X-Long-TTL + "s", 1s);</i><i><br>
</i><i> unset bereq.http.X-Long-TTL;</i><i><br>
</i><i> }</i><i><br>
</i><i> else if (!beresp.http.Cache-Control && bereq.http.X-Long-TTL && !beresp.http.WWW-Authenticate ) {</i><i><br>
</i><i> set beresp.http.X-Orig-Pragma = beresp.http.Pragma; unset beresp.http.Pragma;</i><i><br>
</i><i> set beresp.http.Cache-Control = "public, max-age="+bereq.http.X-Long-TTL+", no-transform";</i><i><br>
</i><i> set beresp.ttl = std.duration(bereq.http.X-Long-TTL + "s", 1s);</i><i><br>
</i><i> unset bereq.http.X-Long-TTL;</i><i><br>
</i><i> }</i><i><br>
</i><i> else { # Don't touch headers if max-age > defined in X-Long-TTL header</i><i><br>
</i><i> unset bereq.http.X-Long-TTL;</i><i><br>
</i><i> }</i><i><br>
</i><i> # Here we set X-Trace header, prepending it to X-Trace header received from backend. Useful for troubleshooting</i><i><br>
</i><i> if (beresp.http.x-trace && !beresp.was_304) {</i><i><br>
</i><i> set beresp.http.X-Trace = regsub(server.identity, "^([^.]+),?.*$", "\1")+"->"+regsub(<a href="https://MailInBlack.univ-lyon1.fr/securelink/?url=http://beresp.backend.name&key=eyJsYW5nIjoiRlIiLCJ1cmwiOiJodHRwOi8vYmVyZXNwLmJhY2tlbmQubmFtZSIsInRva2VuIjoiZ0FBQUFBQmxPLVJiM0x6SzkyRVdNU0xJVzd6SGJ5bVE5UTdUemZ0UnJoemI4R1lIektnSndMY2RLUGw5V3Z2MURHODZuSzlLdW9JNmQ2X0NKb1hfNlpUWi1OZ09URDlfQ2doT1k2U2FVakNaWktwRThBRUF6NFFQR2dsV1JnY29IRml4X1NXZXpXdkZpZDN2QTROQ2RKN0k1V0lGZ0N3UVV3SkQ2c2NxYk1hRnQtUFN2a1d4V01nQ2oyWEp1dE91Q1FDbVAtbXlnWmg1ckp0bl9YcGx5VDE4X210Uk9IV1V5LWF3XzFvNEhONTJtZmNTZ2RJQWlvb0lEUU52NzRpZ2ZrZFFsSGlSTVdTdW01NWVkMl92ZVpEcUFxN3RZbHBGV3lCVjV6ZFVnZjNVZW4xZG43V2JWTjFsc3cxRVIyUHJRQWNLbU5FMTdiZmkifQ==" target="_blank">beresp.backend.name</a>,
"^(.+)\((?:[0-9]{1,3}\.){3}([0-9]{1,3})\)","\1(\2)")+"->"+beresp.http.X-Trace;</i><i><br>
</i><i> }</i><i><br>
</i><i> else {</i><i><br>
</i><i> set beresp.http.X-Trace = regsub(server.identity, "^([^.]+),?.*$", "\1")+"->"+regsub(<a href="https://MailInBlack.univ-lyon1.fr/securelink/?url=http://beresp.backend.name&key=eyJsYW5nIjoiRlIiLCJ1cmwiOiJodHRwOi8vYmVyZXNwLmJhY2tlbmQubmFtZSIsInRva2VuIjoiZ0FBQUFBQmxPLVJiM2doQzZXV2NOd0xYdkdtRk9sZXdtRHI2Q2hwWWwyT0dpaWRNQTJmMVlhdWpQYjRDdG5EMHJEQl9fTUNLRUJFSUd0U3NmckZJczJzcG5WVkowY3d6N213alZ6Z2NPc191UDAzU2R5b1FjUzRIZEJya09SejJMR2xiYklJdWREb0RmX05ZbUI3U25XMzJrbHhkOFVsQ1lVcnhYYlNpMzNvUml4TUNrc2luZzNYV0hzRzRrYm53SmpxbVJrbjdhX0U0Ui02eGlkdWhKd09WY0p4Y1Z5dnEtcmx0T29YYWw2ME1YNG55eWdMNGxFb0hyMmI4ZlpYZHBYc3M0WTFRTGgxZ1JRbkFjYVU0bDNuYlVjZDR3Mmt3YUI3TTA5cFVFWUgxdUNuSWhHb01Wa3BZODhDSFptcU1BWW9uWjZ6aVAtVUQifQ==" target="_blank">beresp.backend.name</a>,
"^(.+)\((?:[0-9]{1,3}\.){3}([0-9]{1,3})\)","\1(\2)");</i><i><br>
</i><i> }</i><i><br>
</i><i>}</i><i><br>
</i><i><br>
</i><i>sub vcl_deliver {</i><i><br>
</i><i><br>
</i><i># Revert back to original Cache-Control header before delivery to client</i><i><br>
</i><i> if (resp.http.X-Orig-Cache-Control)</i><i><br>
</i><i> {</i><i><br>
</i><i> set resp.http.Cache-Control = resp.http.X-Orig-Cache-Control;</i><i><br>
</i><i> unset resp.http.X-Orig-Cache-Control;</i><i><br>
</i><i> }</i><i><br>
</i><i> # Revert back to original Pragma header before delivery to client</i><i><br>
</i><i> if (resp.http.X-Orig-Pragma)</i><i><br>
</i><i> {</i><i><br>
</i><i> set resp.http.Pragma = resp.http.X-Orig-Pragma;</i><i><br>
</i><i> unset resp.http.X-Orig-Pragma;</i><i><br>
</i><i> }</i><i><br>
</i><i><br>
</i><i> if (obj.hits > 0) { # Add debug header to see if it's a HIT/MISS and the number of hits, disable when not needed</i><i><br>
</i><i> set resp.http.X-Cache = "HIT";</i><i><br>
</i><i> } else {</i><i><br>
</i><i> set resp.http.X-Cache = "MISS";</i><i><br>
</i><i> }</i><i><br>
</i><i> set resp.http.X-Cache-Hits = obj.hits;</i><i><br>
</i><i><br>
</i><i># If desired "Via: 1.1 Varnish-v4" response header can be removed from response</i><i><br>
</i><i> unset resp.http.Via;</i><i><br>
</i><i> unset resp.http.Server;</i><i><br>
</i><i><br>
</i><i> return (deliver);</i><i><br>
</i><i>}</i><i><br>
</i><i><br>
</i><i>sub vcl_backend_error {</i><i><br>
</i><i> # More comprehensive varnish error page. Display time, instance hostname, host header, url for easier troubleshooting.</i><i><br>
</i><i> set beresp.http.Content-Type = "text/html; charset=utf-8";</i><i><br>
</i><i> set beresp.http.Retry-After = "5";</i><i><br>
</i><i> synthetic( {"</i><i><br>
</i><i> <!DOCTYPE html></i><i><br>
</i><i> <html></i><i><br>
</i><i> <head></i><i><br>
</i><i> <title>"} + beresp.status + " " + beresp.reason + {"</title></i><i><br>
</i><i> </head></i><i><br>
</i><i> <body></i><i><br>
</i><i> <h1>Error "} + beresp.status + " " + beresp.reason + {"</h1></i><i><br>
</i><i> <p>"} + beresp.reason + {"</p></i><i><br>
</i><i> <h3>Guru Meditation:</h3></i><i><br>
</i><i> <p>Time: "} + now + {"</p></i><i><br>
</i><i> <p>Node: "} + server.hostname + {"</p></i><i><br>
</i><i> <p>Host: "} + bereq.http.host + {"</p></i><i><br>
</i><i> <p>URL: "} + bereq.url + {"</p></i><i><br>
</i><i> <p>XID: "} + bereq.xid + {"</p></i><i><br>
</i><i> <hr></i><i><br>
</i><i> <p>Varnish cache server</i><i><br>
</i><i> </body></i><i><br>
</i><i> </html></i><i><br>
</i><i> "} );</i><i><br>
</i><i> return (deliver);</i><i><br>
</i><i>}</i><i><br>
</i><i><br>
</i><i>sub vcl_synth {</i><i><br>
</i><i> #Redirect using '301 - Permanent Redirect', permanent redirect</i><i><br>
</i><i> if (resp.status == 851) { </i><i><br>
</i><i> set resp.http.Location = req.http.x-redir;</i><i><br>
</i><i> set resp.http.X-Varnish-Redirect = true;</i><i><br>
</i><i> set resp.status = 301;</i><i><br>
</i><i> return (deliver);</i><i><br>
</i><i> }</i><i><br>
</i><i> #Redirect using '302 - Found', temporary redirect</i><i><br>
</i><i> if (resp.status == 852) { </i><i><br>
</i><i> set resp.http.Location = req.http.x-redir;</i><i><br>
</i><i> set resp.http.X-Varnish-Redirect = true;</i><i><br>
</i><i> set resp.status = 302;</i><i><br>
</i><i> return (deliver);</i><i><br>
</i><i> }</i><i><br>
</i><i> #Redirect using '307 - Temporary Redirect', !GET&&!HEAD requests, dont change method on redirected requests</i><i><br>
</i><i> if (resp.status == 857) { </i><i><br>
</i><i> set resp.http.Location = req.http.x-redir;</i><i><br>
</i><i> set resp.http.X-Varnish-Redirect = true;</i><i><br>
</i><i> set resp.status = 307;</i><i><br>
</i><i> return (deliver);</i><i><br>
</i><i> }</i><i><br>
</i><i> #Respond with 403 - Forbidden</i><i><br>
</i><i> if (resp.status == 863) {</i><i><br>
</i><i> set resp.http.X-Varnish-Error = true;</i><i><br>
</i><i> set resp.status = 403;</i><i><br>
</i><i> return (deliver);</i><i><br>
</i><i> }</i><i><br>
</i><i>}</i><i><br>
</i><i><br>
</i><i>sub vcl_purge {</i><i><br>
</i><i> if (req.method != "PURGE") {</i><i><br>
</i><i> set req.http.X-Purge = "Yes";</i><i><br>
</i><i> return (restart);</i><i><br>
</i><i> }</i><i><br>
</i><i>}</i><span><span><br>
</span></span></span></p>
<p>...<br>
</p>
<p><br>
</p>
<pre cols="72"></pre>
<div>Le 20/02/2023 à 18:55, Guillaume Quintard a écrit : <br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hello Karim, </div>
<div><br>
</div>
<div>You VCL would be useful to debug this (as well as the command line you are running Varnish with), but it sounds like Varnish is using the Transient storage (<a href="https://MailInBlack.univ-lyon1.fr/securelink/?url=https://varnish-cache.org&key=eyJsYW5nIjoiRlIiLCJ1cmwiOiJodHRwczovL3Zhcm5pc2gtY2FjaGUub3JnL2RvY3MvdHJ1bmsvdXNlcnMtZ3VpZGUvc3RvcmFnZS1iYWNrZW5kcy5odG1sI3RyYW5zaWVudC1zdG9yYWdlIiwidG9rZW4iOiJnQUFBQUFCbE8tUmIxZnVmSHBWbmpscFhrWDdiMDZDaUNXVGxlVFUyc0Y0YkxMTTBPU3JWYlQzT1kwNEo0cnBBV2tWaEcxdjlaenhTcnBHZWo3cVJ1a1hSbmZYUzFBbWlZb0piRFJBNTdfcGtNY0c4Qjl5c0s2cld2SGZjX3pLVlRtYkY0ckhoVVNIMTBGbXZrOFZORVZHQVkzTEtoTm5WQXl0NHh2ZElPZEItbU9nS2tQQjF5dExCQUlkS05yRDJoZjhGMHQ2Y2p6YW5WTFYwU29qZXJsYU9lZTZxMUJ5Y3RoYzU1aWdhVzJmUkJoUFRKRHVJRGxPT0JoWHR0di1JWnNqdzhvNkw3X0ZfNEtJM2ZVemFnZzJIRkpkRFZZSWpscVRfNUhzRkRhOWlwNjN6NzcyWHloZV9CYTJPQWVaRnJwZnpmbWR5azBVeiJ9" target="_blank">https://varnish-cache.org/docs/trunk/users-guide/storage-backends.html#transient-storage</a>)
to store the file, and as the storage isn't bounded, it explodes. </div>
<div>We can fix this in a couple of ways, from storing the file in the regular cache storage, to using pipe, to waiting a few days for
<a href="https://github.com/varnishcache/varnish-cache/pull/3572#issuecomment-1305736643" target="_blank">
https://github.com/varnishcache/varnish-cache/pull/3572#issuecomment-1305736643</a> to be released.
</div>
<div><br>
</div>
<div>Question is: should that file be cached? </div>
<div><br>
</div>
<div>Cheers, </div>
<div><br>
</div>
<div>
<div>
<div dir="ltr">
<div dir="ltr">
<div>-- <br>
</div>
<div>Guillaume Quintard <br>
</div>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Feb 20, 2023 at 7:14 AM Karim Ayari <<a href="mailto:karim.ayari@univ-lyon1.fr" target="_blank">karim.ayari@univ-lyon1.fr</a>> wrote:
<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div bgcolor="#FFFFFF">
<p><span lang="en"><span><span>Hi!</span></span><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>I am currently experiencing a memory load problem with video playback.</span></span></span></p>
<p><span lang="en"><span><span></span></span><span><span>here is the infrastructure :<br>
</span></span></span></p>
<p><span lang="en"><span><span>client --> haproxy --> varnish --> moodle workers (x5)</span></span><span><span>
</span></span><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>a teacher uploaded a 400MB video to Moodle, when we start playing the video with browser player,
</span></span></span><span lang="en"><span><span><span lang="en"><span><span>Varnish</span></span></span> consumes all the memory until it runs out and oom killer to kill varnishd.</span></span><span><span>
</span></span><span><span>i have no configuration for mp4 files in my vcl file, so by default they are not hidden (?).
</span></span></span><span lang="en"><span><span>I can't find a solution :(</span></span></span><span lang="en"><span><span></span></span></span></p>
<p><span lang="en"><span><span>I can give my vcl file if necessary.</span></span><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>(I am a beginner on varnish :))</span></span></span><span lang="en"><span><span></span></span></span></p>
<p><span lang="en"><span><span>thank you for your support.</span></span><span><span><br>
</span></span></span></p>
<p><span lang="en"><span><span>Karim<br>
</span></span></span></p>
</div>
_______________________________________________ <br>
varnish-misc mailing list <br>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a>
<br>
<a href="https://MailInBlack.univ-lyon1.fr/securelink/?url=https://www.varnish-cache.org&key=eyJsYW5nIjoiRlIiLCJ1cmwiOiJodHRwczovL3d3dy52YXJuaXNoLWNhY2hlLm9yZy9saXN0cy9tYWlsbWFuL2xpc3RpbmZvL3Zhcm5pc2gtbWlzYyIsInRva2VuIjoiZ0FBQUFBQmxPLVJiZUZqNEFLdzBYZ0NGUFowNm9Fa3IwQzVOR1A5cWRPYlB1U0t2S2NWaG1VSGRSRjlYaGt4enNxV095NXVGM1ZwLWRhR0JmZmFRNkc5Qk95c1ZHUktzTVFJY1dPQ0ZqYzlwRlZlNERBUTVFdEtzUWh6YS1vX3hJOUVfYTEwSklhbVkzQ3BhMnZfMnRPbjFkQWF4cWpPY0YxVkR4ZHFFSWl1NWQ1dXZJbW00Z2hLdERoYnlSZk9jVjdHMDJDcTBKWFB1TmRWX3NwZW9vaGMxV3B2a3ZnZlpCb2dTSkEtaU5iaUtRd0Izci1qOUxQQ00zckVfajRUTDdVTUNqczQtS1otUHQtWGt3Tmx5WGNZelJvOTd4ZUR5cEtuUTFUdno4cHVMUjk2NlFVODY3LVVoc1BxcVJseTlxeDA4QVdEcGJlZVkifQ==" rel="noreferrer" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a>
<br>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</div>
_______________________________________________ <br>
varnish-misc mailing list <br>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a>
<br>
<a href="https://MailInBlack.univ-lyon1.fr/securelink/?url=https://www.varnish-cache.org&key=eyJsYW5nIjoiRlIiLCJ1cmwiOiJodHRwczovL3d3dy52YXJuaXNoLWNhY2hlLm9yZy9saXN0cy9tYWlsbWFuL2xpc3RpbmZvL3Zhcm5pc2gtbWlzYyIsInRva2VuIjoiZ0FBQUFBQmxPLVJiTW4tMVYzSGlGZ3JqSEI3QllsV01jcnp2NG1RSk9Dcy02T2w3eExrcGxwRUduYWlqb0pXdE9aT29FOWxmT2thY01zWDh3U0RONzVxVjFSUWN2NXEzS1pLbWYxY0N5bzhKbURRSV9oSlZQYnBTNEpIb2ZZV2hpYmVDTW5uWGh5QVNncWRTRERmY1FfNHU0S21FMEVRcHlMRVZMcVNHVUFqcWJDaV9TM3dDWEtpT2JRN0hXUjNaMWprVjBlUTFwSGVua3ZJcE5HNEtMelB4eHhvQzgzTUxLOUJJeFkyeXFDSmRENVZ5b3Z2b2w1ck1BVUJyUGM3X2hBeEhDT3BBOUMxaEFOMnNPaW5RYzY2Q0Ewb1ZCOEhSUGh3bnBQV0o3bFRJX0ZUcTRBcmRESVR4YlQySkpHNE9hbDBoX0VPeFhRbnYifQ==" rel="noreferrer" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a>
<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</body>
</html>