Varnish crashing in production
barry at automattic.com
Fri May 11 10:47:20 CEST 2007
> Within reason - there is no point in using a cache file that is many
> times the size of your data set. I'm not sure how the allocator works
> in detail, but somewhere between 1x and 2x the size of your data set
> should be sufficient.
Varnish is sitting in front of about 600GB of data currently. We add
a header to files that were requested more than X times in Y seconds
and tell varnish to cache anything with that header and don't cache
rest. When we were trying to cache everything we were running into
serious IO issues (not to mention that we would run out of space
eventually). I am not sure how big that dataset is going to get.
The Expires headers on the content is 1 week. Once the crashing
issue is resolved and varnish stays running for a while I will be
able to get a better idea.
More information about the varnish-dev