Varnish 3.0 slow backend fetches
c.hartlage at modenkg.de
Sun Aug 30 00:48:42 CEST 2015
we're troubleshooting with a problem with our varnish setup - it would be great if you could help us.
Our server setup is as follows:
NGINX (as ssl-terminator) > Varnish > NGINX/php-fpm (multiple web-server instances)
Cacheable URLs work fine, even with around 5 ESI per site, we can deliver 1000 req/s in around 100ms.
When it comes to non-cacheable URLs our setup fails completely. We can only serve about 5 req/s through varnish. If we leave out varnish for these urls and ask the nginx backends directly, we’re able to serve about 25 req/s. Without varnish, nginx directs these 25 req/s to the php-fpm processes and they get some nice cpu load.
With Varnish the php-fpm processes have nearyl zero cpu load, because varnish gets only 5req/s to the nginx backend.
Testing these requests with curl showed that it takes nginx/php-fpm about 0.2sec for a single response. For us it seems like varnish can do only 1 fetch at once, so that the backend request are blocking. Is that possible?
During load testing the client_conn counter jumps to 200 almost instantly and then increases with about 5 units per second. the backend_conn counter does not have the initial jump to 200 but increases from zero with 5 units per second.
After the load test is completed the client_conn counter does not increase anymore (obviously), but the backend_conn counter will continue increasing slowly until it reaches the amount of client_conn. This also made us think that varnish processes the backend requests sequentially, even when the clients are not actually waiting for the content anymore.
More information about the varnish-misc