max_connections, backend_busy, and queueing requests

[Jeff Potter]

Hi List,

I'm having a hard time finding documentation or other users who've done what I'm trying to do, although it seems pretty straight-forward and I must be missing something (or have something misconfigured?).

Let's say that I have a simple setup of 1 varnish server and 5 backend nodes defined in a single director; nothing fancy other than each backend's max_connections being set to 100. This setup would allow for 500 concurrent requests when everything is healthy.

As soon as there are 500 concurrent requests, varnish kicks any additional requests to vcl_error and we send out a 5xx error for the requests. And varnishstats increments the backend_busy/"Backend conn. too many" counter.

How do I get Varnish to queue up the requests instead, so that when a backend worker connection becomes available it gets sent to a backend? (Currently running Varnish 2; planning to migrate to Varnish 3 shortly. If the answer is "upgrade to varnish 3", that's useful to know too!)

It may help to know that, in our case, we expect most of the requests to go to the backend -- i.e. while we do use cacheing, most cacheable requests have been pushed off to other hosts. If the only way to do this is configure varnish such that it only handles 500 frontend requests at a time, that would be okay (but not ideal).

Thanks!
-Jeff

P.S. Thanks for a great piece of software -- Varnish is pretty amazing.