Static only caching

TiAMO tiamo at tfr.org
Mon Jul 2 00:36:04 CEST 2007


I just signed up to the list so sorry if my questions have been asked a 
million times already, if so i please point me to the responses.

This is what i want to do:

I want to setup anycasted proxys for static content that will have 48gig of 
RAM, 6TB (6x1TB disks in a raid0 setup) of storage and 2x Opteron 2214 CPU's. 
I want to have the 48gig most requested data in RAM and the rest on disk. The 
proxy will have backend content that's much larger then the storage size of 
each node. I have'nt had a chance to test how many mbit i can get out of each 
box, im hoping for gigabit+ speeds, maybe anyone have some numbers to share? 
I plan to uplink each box with a 2x1gbit etherchannel.

What fileystem is recomended to have the storage file on, i favor reiserfs for 
all my DB and http aplications, but then again i have never had 5.5TB+ files 
on them.

I think i got most of it figured out, but it's kinda hard to get it all right 
with the lack of docs and all =). 
What i want to do i limit the method to only HEAD and GET in HTTP/1.1 or else 
just return a service unavailable status (or just close the connection or 
whatever). Also i want to ignore all headers like cookies, no-cache, auth and 
so on, and always use the cache if it's in there. 

Any performance tips on how to tune whats in the cache when having more 
backend data then proxy storage, is fragmentation an issue? Also turnoff 
aging altogether so that data in the cache is only dropped when it's replaced 
by something more used.

Is it possible to have diffrent backends based on the http/1.1 host header?

Configuration examples would be much appreciated =)

If anyone is curious what it's for, it's for all the static content (like 
images, videos, torrents, audio files) of http://thepiratebay.org 
http://bayimg.com and the upcoming http://thevideobay.org http://playble.com/ 
and our other projects.

// Fredrik Neij




More information about the varnish-misc mailing list