<font size="2"><font face="georgia,serif">If you have control over the backend (Apache) it should be made to emit a Cache-Control or Expires header to Varnish to make the object non-cacheable <i>if the file is too large</i>. Apache will know the file's size before a request occurs. I was talking about logic within Apache, not Varnish. This is how it's "supposed" to happen.</font></font><div>
<font size="2"><font face="georgia,serif"><br></font></font></div><div><font size="2"><font face="georgia,serif">With Varnish, I see no way to avoid downloading the entire file every time. You can control whether the file <i>stays</i> in cache, but that's it. If there were a URL pattern (e.g., magic subdirectory), you could conceivably switch to pipe in those cases.</font></font></div>
<div><font class="Apple-style-span" face="georgia, serif"><br></font></div><div><font class="Apple-style-span" face="georgia, serif">Thinking out loud... HTTP servers will send a response to a HEAD request with a Content-Length header that represents the length of the full object had a GET been performed.</font></div>
<div><font class="Apple-style-span" face="georgia, serif"><br></font></div><div><font class="Apple-style-span" face="georgia, serif">If your Apache does this (some configurations will disable this), one hack would be to have Varnish send a HEAD request to Apache for every object, set a req flag if the returned content length is too large, then restart, and then have logic that will force pipe if it's too large, otherwise pass. This will double the hits to the back-end, however, so some conditionals would help (only .mov, or only a certain subdirectory, etc.) And I've never tried changing a GET to a HEAD with VCL or inline-C.</font></div>
<div><font class="Apple-style-span" face="georgia, serif"><br></font></div><div><font class="Apple-style-span" face="georgia, serif">But usually when something is that difficult, it's a square peg and a round hole. :-)</font></div>
<div><font class="Apple-style-span" face="georgia, serif"><br></font></div><div><font class="Apple-style-span" face="georgia, serif">FWIW,</font></div><div><font face="georgia, serif">-- </font><div><font face="georgia, serif">kb</font></div>
<br>
<br><br><div class="gmail_quote">On Tue, Mar 15, 2011 at 16:56, Chris Hecker <span dir="ltr"><<a href="mailto:checker@d6.com">checker@d6.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<br>
I'm not sure I understand. I have control over the back end, the front end, the middle end, all the ends. However, I thought the problem was there was no way to get varnish to read the header without loading the file into the cache? If that's not true, then shouldn't Content-Length be enough?<br>
<br>
Chris<br>
<br>
On 2011/03/15 13:16, Ken Brownfield wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
I'm assuming that in this case it's not possible for you to have the<br>
backend server emit an appropriate Cache-Control or Expires header based<br>
on the size of the file? The server itself will know the file size<br>
before transmission, and the reindeer caching games would not be<br>
necessary. ;-)<br>
<br>
That's definitely the Right Way, but it would require control over the<br>
backend, which is often not possible. Apache unfortunately doesn't have<br>
a built-in mechanism/module to emit a header based on file size, at<br>
least that I can find. :-(<br>
--<br>
kb<br>
<br>
<br>
<br>
On Tue, Mar 15, 2011 at 00:42, Chris Hecker <<a href="mailto:checker@d6.com" target="_blank">checker@d6.com</a><br>
<mailto:<a href="mailto:checker@d6.com" target="_blank">checker@d6.com</a>>> wrote:<br>
<br>
<br>
Yeah, I think if I can't do it Right (which I define as checking the<br>
file size in the vcl), then I'm just going to make<br>
<a href="http://blah.com/uncached/*" target="_blank">blah.com/uncached/*</a> <<a href="http://blah.com/uncached/*" target="_blank">http://blah.com/uncached/*</a>> be uncached. I<br>
don't want to transfer it once just to throw it away.<br>
<br>
Chris<br>
<br>
<br>
<br>
On 2011/03/15 00:40, Martin Boer wrote:<br>
<br>
I've been reading this discussion and imho the most elegant way<br>
to do it<br>
is to have a upload directory X and 2 download directories Y and<br>
Z with<br>
a script in between that decides whether it's cacheable and move the<br>
file to Y or uncacheable and put it in Z.<br>
All the other solutions mentioned in between are far more<br>
intelligent<br>
and much more likely to backfire in some way or another.<br>
<br>
Just my 2 cents.<br>
Martin<br>
<br>
<br>
On 03/13/2011 05:28 AM, Chris Hecker wrote:<br>
<br>
<br>
I have a 400mb file that I just want apache to serve. What's<br>
the best<br>
way to do this? I can put it in a directory and tell varnish<br>
not to<br>
cache stuff that matches that dir, but I'd rather just make<br>
a general<br>
rule that varnish should ignore >=20mb files or whatever.<br>
<br>
Thanks,<br>
Chris<div class="im"><br>
<br>
<br>
_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br></div>
<mailto:<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a>><div class="im"><br>
<a href="http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
<br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
varnish-misc mailing list<br></div>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a> <mailto:<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a>><div class="im">
<br>
<a href="http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
<br>
<br>
<br>
<br>
_______________________________________________<br>
varnish-misc mailing list<br>
<a href="mailto:varnish-misc@varnish-cache.org" target="_blank">varnish-misc@varnish-cache.org</a><br>
<a href="http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc" target="_blank">http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc</a><br>
</div></blockquote>
</blockquote></div><br></div>