Don't cache large files

[Dridi Boukelmoune]

Hi Jason,

The pipe mode is designed for non-HTTP traffic. For example, you
switch to the WebSocket protocol, Varnish can no longer proxy anything
and simply let data go directly back and forth between the client and
the backend. But to use WebSocket, you first need to make an HTTP
request.

https://en.wikipedia.org/wiki/WebSocket#WebSocket_protocol_handshake

Cheers,
Dridi

On Fri, Apr 17, 2015 at 2:38 PM, Jason Heffner <jdh132 at psu.edu> wrote:
> Thanks Dridi. I had switched to just marking the content uncacheable and using return(deliver), per the new v4 method, temporarily. I wasn’t sure if there was any advantage to using pipe anymore for files ranging from ~10MB - 500MB in size on the server?
>
> Jason
>
>> On Apr 17, 2015, at 2:58 AM, Dridi Boukelmoune <dridi at varni.sh> wrote:
>>
>> Why pipe when you could simply have a hit-for-pass object?
>>
>> In vcl_backend_response you could simply set beresp.uncacheable to
>> true (return(pass) in vcl_fetch for v3). With such a config you'd hit
>> your backend twice for all responses of more than 10MB.
>>
>> Dridi
>>
>> On Thu, Apr 16, 2015 at 6:05 PM, Jason Heffner <jdh132 at psu.edu> wrote:
>>> I’m working on updating our Varnish 3 config to Varnish 4 and I was able to figure out how to have the same functionality for almost all of the config. I wanted to find opinions on how to handle large files. We don’t want to cache any large files and had this in place for varnish 3. I’ve read streaming instead of piping was a solution but found nothing to determine this based solely on file size from the backend.
>>>
>>> # Varnish 3 config
>>> added to vcl_recv:
>>>  /* Bypass cache for large files.  The x-pipe header is
>>>     set in vcl_fetch when a too large file is detected. */
>>>  if (req.http.x-pipe && req.restarts > 0) {
>>>    remove req.http.x-pipe;
>>>    return (pipe);
>>>  }
>>>
>>> added to vcl_fetch:
>>>  # don't cache files larger than 10MB
>>>  /* Don't try to cache too large files.  It appears
>>>     Varnish just crashes if we don't filter them. */
>>>  if (beresp.http.Content-Length ~ "[0-9]{8,}" ) {
>>>    set req.http.x-pipe = "1";
>>>    return (restart);
>>>  }
>>>
>>> Since req.* is no longer available in vcl_recv this code doesn’t function anymore.
>>>
>>> Thanks,
>>> Jason
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>