Benchmarking and stress testing Varnish

[Rogerio Schneider]

Kristian, we do also test Varnish's behavior with slow clients, trying  
to find if it runs out of file descriptors, for instance.
There was a time that when we reach 65k fd an assert In shmlog.c made  
it to bailout. I think it was fixed but not ported to 2.0.5.

Regards,
Rogerio Schneider

Em 13/11/2009, às 11:38, Kristian Lyngstol <kristian at redpill- 
linpro.com> escreveu:

> As some of you might already know, I regular stress tests of  
> Varnish, most
> of the time it's a matter of testing the same thing over and over to  
> ensure
> that there aren't any huge surprises during the development of  
> Varnish (we
> run nightly builds and stress tests of trunk - they need to be as
> predictable as possible to have any value).
>
> However, I also do occasional tests which involve trying to find the
> various breaking points of Varnish. Last time I did this, I used  
> roughly 42
> cpu cores spread over about 30 computers to generate traffic against a
> single Varnish server on our quad xenon core machine. The result  
> thus far
> was that all of the clients ran far too close to 100% cpu usage, and
> Varnish was serving between 140 and 150 thousand requests per second.
>
> The reason I'm telling you this is because I'm looking for input on  
> aspects
> that should be tested next time I do this, which will most likely be  
> during
> Christmas (far easier to borrow machine power). So far on my list,  
> I've
> got:
>
> 1. Test performance over some time period when pages are evicted more
> frequently. (ie: X thousand pages requested repeatedly, but random  
> expiry
> time).
>
> 2. Test with fewer requests per session (this is somewhat hard  
> because the
> clients tend to turn into the bottleneck).
>
> 3. Test raw hot-hit cache rate (more of what I did before - get a high
> number).
>
> 4. Test raw performance with a huge data set that's bound to be  
> swapped
> out.
>
> 5. Various tests of -sfile versus -smalloc and large datasets,  
> combined
> with critbit/classic tests.
>
> 6. Find some reasonably optimal settings, then fidget around trying  
> to find
> a worst-case scenario.
>
> 7. Test the effect of session lingering with really slow clients.
>
> ....
>
> One thing that should be noted is that the test server is limited to  
> 1gbit,
> which means that for "raw req/s", we're basically forced to use tiny  
> pages,
> or we just end up starving the network.
>
> The goal is to test theories regarding performance, stability and
> predictability. Basically find the breaking points, what's good and  
> what's
> not, and what we have to care about and what we can silently ignore.
>
> As you can see, the list is getting long and this is off the top of my
> head, but any suggestions are welcome.
>
> -- 
> Kristian Lyngstøl
> Redpill Linpro AS
> Tlf: +47 21544179
> Mob: +47 99014497
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc