push button lru nuking

[Michael Fischer]

On Sat, Jan 16, 2010 at 4:16 PM, Michael Fischer <michael at dynamine.net>wrote:

> On Sat, Jan 16, 2010 at 4:10 PM, David Birdsong <david.birdsong at gmail.com>wrote:
>
>> On Sat, Jan 16, 2010 at 3:55 PM, Michael Fischer <michael at dynamine.net>
>> wrote:
>> > This scheme seems very baroque.  Why not just reduce the size of your
>> caches
>> > so you don't page-thrash and let Varnish's builtin LRU algorithm handle
>> the
>> > eviction?
>>
>> Then I wont be able to cache nearly as much.  I want to originate as
>> much content as possible on the varnish servers ie. reduce backend
>> fetches.  There is no way I could fit any useful amount of my working
>> set into a storage that could handle the evictions without spending an
>> unreasonable amount of money (basically fit it in RAM.)  -I'd love to
>> be proven wrong though.  As far as random reads go, the SSD's are
>> really good; it's just the writes that kill me.
>>
>> Right now a mostly filled cache server with ~80-160GB allocated can
>> maintain between 90-92% cache hit ratio at 400-500Mb/sec.  When it
>> fills up completely eviction cause the machine to keel over, parent
>> can't ping the child, health checks fail -general badness.  I'd like
>> to let the eviction run under supervision (automated supervision) and
>> augment the eviction such that it buys back a few hours not minutes.
>
>
> What OS are you running?  This might be one of those rare cases where a
> little more "swappiness" (i.e., aggressiveness of the pageout algorithm)
> might buy you something.
>

This page may be useful if you're running on Linux:
http://www.westnet.com/~gsmith/content/linux-pdflush.htm

--Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20100116/2417d9eb/attachment-0001.html>