Static only caching

[Denis Ahrens]

On 02.07.2007, at 00:36, TiAMO wrote:

> I just signed up to the list so sorry if my questions have been  
> asked a
> million times already, if so i please point me to the responses.
>
> This is what i want to do:
>
> I want to setup anycasted proxys for static content that will have  
> 48gig of
> RAM, 6TB (6x1TB disks in a raid0 setup) of storage and 2x Opteron  
> 2214 CPU's.
> I want to have the 48gig most requested data in RAM and the rest on  
> disk. The
> proxy will have backend content that's much larger then the storage  
> size of
> each node. I have'nt had a chance to test how many mbit i can get  
> out of each
> box, im hoping for gigabit+ speeds, maybe anyone have some numbers  
> to share?
> I plan to uplink each box with a 2x1gbit etherchannel.
>
> What fileystem is recomended to have the storage file on, i favor  
> reiserfs for
> all my DB and http aplications, but then again i have never had 5.5TB 
> + files
> on them.
>
> I think i got most of it figured out, but it's kinda hard to get it  
> all right
> with the lack of docs and all =).
> What i want to do i limit the method to only HEAD and GET in HTTP/ 
> 1.1 or else
> just return a service unavailable status (or just close the  
> connection or
> whatever). Also i want to ignore all headers like cookies, no-cache,  
> auth and
> so on, and always use the cache if it's in there.
>
> Any performance tips on how to tune whats in the cache when having  
> more
> backend data then proxy storage, is fragmentation an issue? Also  
> turnoff
> aging altogether so that data in the cache is only dropped when it's  
> replaced
> by something more used.
>
> Is it possible to have diffrent backends based on the http/1.1 host  
> header?
>
> Configuration examples would be much appreciated =)
>
> If anyone is curious what it's for, it's for all the static content  
> (like
> images, videos, torrents, audio files) of http://thepiratebay.org
> http://bayimg.com and the upcoming http://thevideobay.org http://playble.com/
> and our other projects.

In my tests I could easily saturate a 1GBit link with varnish on 10%  
load
on a dual quad core 2 xeons with 2.66GHz.

example for your backends would be:

at the start define two backends:

backend server1 {
         set backend.host = "backend1.domain.com";
         set backend.port = "80";
}

backend server2 {
         set backend.host = "backend2.domain.com";
         set backend.port = "80";
}

in sub vcl_recv the following:

     if (req.http.Host == "server1.piratebay.org")
     {
         set req.backend = server1;
     }

     if (req.http.Host == "server2.piratebay.org")
     {
         set req.backend = server2;
     }

I think you will figure out the rest easily!

Denis Ahrens