Varnish 3.0 slow backend fetches

Christian Hartlage c.hartlage at
Sun Aug 30 00:48:42 CEST 2015

Hi guys,

we're troubleshooting with a problem with our varnish setup - it would be great if you could help us.

Our server setup is as follows:

NGINX (as ssl-terminator) > Varnish  > NGINX/php-fpm (multiple web-server instances) 

Cacheable URLs work fine, even with around 5 ESI per site,  we can deliver 1000 req/s in around 100ms.

When it comes to non-cacheable URLs our setup fails completely. We can only serve about 5 req/s through varnish. If we leave out varnish for these urls and ask the nginx backends directly, we’re able to serve about 25 req/s. Without varnish, nginx directs these 25 req/s to the php-fpm processes and they get some nice cpu load.
With Varnish the php-fpm processes have nearyl zero cpu load, because varnish gets only 5req/s to the nginx backend.

Testing these requests with curl showed that it takes nginx/php-fpm about 0.2sec for a single response. For us it seems like varnish can do only 1 fetch at once, so that the backend request are blocking. Is that possible?
During load testing the client_conn counter jumps to 200 almost instantly and then increases with about 5 units per second. the backend_conn counter does not have the initial jump to 200 but increases from zero with 5 units per second.
After the load test is completed the client_conn counter does not increase anymore (obviously), but the backend_conn counter will continue increasing slowly until it reaches the amount of client_conn. This also made us think that varnish processes the backend requests sequentially, even when the clients are not actually waiting for the content anymore.

Best Regards, 

Christian Hartlage

More information about the varnish-misc mailing list