Strategy for large cache sets

[Drew Raines]

Simon Lyall wrote:

> I was wondering how large a cache people are managing to use without any 
> problems? Maybe a survey:

Version:     trunk-r2646
Config:      file backend
Traffic:     90 req/s avg, 1.5-2K peak
RAM:         12GB, 1GB swap
Cache Size:  36GB reserved, 24GB allocated
OS:          Linux 2.6.18-6-amd64 (Debian 4.0r3)

This represents one of our image cache boxes, with a 99.6% hitrate,
in a pool fronted with nginx.  Our cache files reside on a SAN where
we can get around 500MB/s over iSCSI.  We went that route because we
could not get Linux to swap efficiently enough to make malloc
feasible.

Except for the occasional flood of connections which will cause
varnishd to consume more RSS than we've allotted, it's been extremely
stable and performant.

-Drew