Low performance in Pipe-only mode

[Somsak Sriprayoonsakul]

Hello,

We were trying to use Varnish as L7 HTTP request router with absolutely NO
cache.

We created very minimal VCL rules to select correct backend and return PIPE
for all requests. Below are the snippet taken from the VCL.

sub vcl_recv {
        # if X-Forwarded-For has been set by CDN, leave it
        if (! req.http.X-Forwarded-For) {
                set req.http.X-Forwarded-For = client.ip;
        }

        if (req.http.host ~ "desired.domain.com" &&
                (req.request == "GET" &&
                ! ( (req.url ~ ".*\?.*[&]?.*") ||
                    (req.url ~ ".*/$") ||
                    (req.url ~ ".*/\?.*") ||
                    (req.url ~ ".*\?.*[&]?upload.*")
                ))
        ) {
                set req.backend = tier1;
        } elsif (req.request == "PUT") {
                set req.backend = tier2;
        } elsif (req.http.host ~ "another.domain.com") {
                set req.backend = tier_console;
        } else {
                set req.backend = default_tier;
        }
        return (pipe);
}

sub vcl_pipe {
        set bereq.http.connection = "close";

        return (pipe);
}

With this, Varnish work as expected, but however the performance is much
lower than in "lookup" mode.

We tested performance by using Tsung, had 3 servers simulated 15000
clients. Backends cosists of 2 servers running NGINX. We got the following
results.

- Have TSUNG did the load balancing to 2 servers (TSUNG can do round-robin
load balance by itself). We got around 4500-4800 request/sec
- TSUNG point to Varnish with above VCL. request/sec drop to around
900-1000 request/sec.
- If we change the last line to "return (lookup);" instead of "return
(pipe);", we got around 3300 request/sec.

I tried to play around with thread_pool, thread_pool_min, and
thread_pool_max as suggested on various web sites. Even with -p
thread_pool_min=100, -p thread_pool_max=5000, -p thread_pools=8, -p
session_linger=50, -p cli_timeout=25, -p listen_depth=8192 the performance
didn't improve much (NOTE: Varnish server has 8 core)

Is there anyway to increase performance of PIPE only mode to be, at the
least, in the same level as Lookup mode? What could we do to debug the
performance?

Best Regards,

Somsak

somsaks at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20121207/b0b68845/attachment.html>