Varnish use for purely binary files

[Ken Brownfield]

On Jan 18, 2010, at 3:16 PM, Michael S. Fischer wrote:
> On Jan 18, 2010, at 3:08 PM, Ken Brownfield wrote:
> 
>> In the real world, sites run their applications through web servers, and this fact does (and should) guide the decision on the base web server to use, not static file serving.
> 
> I meant webservers that more than 50%+ of the world uses, which do not include those.

Depends on whether you mean 50% of companies, or 50% of web property traffic.  The latter?  Definitely.

>  I was assuming, perhaps incorrectly, that the implementor would have at least the wisdom/laziness to use a popular general-purpose webserver such as Apache for the purpose of serving static objects from the filesystem.   And that's not even really a stretch as it's the default for most servers.

This is true, though default Apache configurations vary the gamut from clean to bloated (>1ms variation I would say)

> 
>> (Though nginx may have an on-disk cache?  And don't get me started on Apache caching. :-)
> 
> Doctor, heal thyself before you call me inexperienced.  Using application-level caching for serving objects from the filesystem rarely works, which is the main point of Varnish.  Just because *you* can't get good performance out of Apache doesn't mean it's not worth using.

I'm not sure what your definition of application-level is, here.  Much of Apache's functionality could be considered an application.  But if you mean an embedded app running "inside" Apache, then that distinction has almost no bearing on whether file serving "works" or not -- an app can serve files just as fast as Apache, assuming C/C++.

Adding unnecessary software overhead will add latency to requests to the filesystem, and obviously should be avoided.  However, a cache in front of a general web server will 1) cause an object miss to have additional latency (though small) and 2) guarantee object hits will be as low as possible.  A cache in front of a dedicated static file server is unnecessary, but worst-case would introduce additional latency only for cache misses.

I'm not sure what your comment on Apache is about, since I never said Apache isn't worth using.  I've been using it in production for 11+ years now.  Does it perform "well" for static files in the absence of any other function?  Yes.  Would I choose it for anything other than an application server?  No.  There are much better solutions out there, and the proof is in the numbers.
-- 
Ken

> --Michael