Varnish/Swap usage and network timeouts

[Anders Nordby]

Hi,

Please try:

- upgrade to FreeBSD 7.0, and use SCHED_ULE instead of SCHED_4BSD.

- use -s malloc,<n>G instead of -s file. Set up large swap areas
according to your needs.

- if you get "swap zone exhausted, increase kern.maxswzone" error
messages, try increasing kern.maxswzone. Default is 32 MB, try 256 MB or
something like that: kern.maxswzone="268435456"

- checking http://varnish.projects.linpro.no/wiki/Performance. :-)

Cheers,
Anders.

On Mon, Jun 02, 2008 at 11:54:46AM -0400, Erik Steigler wrote:
> Hello,
> I have 2 machines running varnish which handles around 1000 requests per 
> second each and sometimes the machines will just stop responding to any 
> network communication. A coworker used wireshark to check the connection 
> and saw a whole lot of tcp retransmissions. Normally I would suspect 
> some sort of network buffer issue but there is nothing in the logs at 
> all. This is on FreeBSD  6.3 with a default 64 bit kernel with bge 
> network cards. They are both running varnish-trunk revision 2635. 
> Attached is the VCL file that is currently running. Varnish is started 
> with the following options:
> file,/vol/data1/varnish.cache,90% -t 259200 -h classic,500009 -p 
> lru_interval=3600
> 
> The only suspicious things I've found are: Varnish says it is using 372G 
> of virtual memory when I told it to use a 58gb file and there is only 
> 2gb of physical memory in the machine and there is pretty constant 
> swapping going on, mostly under 500K but it seems to be pretty constant 
> no matter the number of connections and such. The amount of swap space 
> in the machine is 4gb and things are using 1.1gb.
> 
> Threads don't seem to be a problem, as the highest I've seen it is at 
> 500 and that was after an extended period of time around 250 or so. So I 
> don't think I'm running into thread issues
> 
> So I guess I'm asking if anyone has any other suggestions other then 
> reinstalling to FreeBSD 7.0 which seems to be the best solution I can 
> see. If that is the best solution, what is the optimal FreeBSD 
> configuration for Varnish? Lots of swap space? No swap space? Currently 
> the mmap-ed file is on an SSD so one alternative I thought of was to 
> make the entire SSD part of swap and use the malloc option and get rid 
> of any regular disk basked swap.
> 
> Thanks for any help,
> Erik
> 
> 

> backend default {
> 	.host = "xzy.com";
> 	.port = "80";
> }
> 
> sub vcl_recv {
>   if (req.url ~ "\.js|css$") {
> 	if (req.http.Accept-Encoding ~ "gzip") {
> 		set req.http.Accept-Encoding = "gzip";
> 	} elsif (req.http.Accept-Encoding ~ "deflate") {
> 		set req.http.Accept-Encoding = "deflate";
> 	} else {
> 		unset req.http.accept-encoding;
> 	}
>   } else {
> 	unset req.http.accept-encoding;
>   }
> 
> 
>   unset		req.http.Cookie;
>   set		req.grace = 10m;
> }
> 
> sub vcl_fetch {
>   set		obj.http.P3P = "CP='NOI CURa ADMa DEVa TAIa OUR BUS IND UNI COM NAV INT', policyref='http://www.okcupid.com/w3c/p3p.xml'";
>   set		obj.grace = 10m;
>   if (obj.ttl < 259200s) {
> 	set	obj.ttl = 259200s;
>   }
> }
> 

> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc



-- 
Anders.