[PATCH] backend conditional requests 5th release

[Dmitry Panov]

Hi Geoff,

You're right, the patch didn't make any difference. The problem was that 
I was running an earlier version of the unpatched trunk which contained 
the ws overflow bug, so the reason memory usage was dropping from time 
to time was because one of the threads was dying.

I've installed the latest trunk and it behaved exactly like the patched 
version: after about 80k requests the virtual memory size was about 2G 
(I have 1G storage configured). I ran varnishstat -1 after that (attached).

So it looks like there is a memory leak in the current trunk.

On 10/03/2011 19:41, Geoff Simmons wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 3/10/11 7:18 PM, Dmitry Panov wrote:
>> I have finally been able to test the patched version under some proper
>> load. I was using tsung (http://tsung.erlang-projects.org/). I recorded
>> a browser session in which I accessed a couple of pages, then tried to
>> replay it with tsung.
>>
>> It looks like there is a memory leak in the patched version because the
>> virtual memory size grew at a constant pace until eventually varnishd
>> filled up the whole available space. I tried exactly the same scenario
>> with an unpatched trunk (same revision that I applied the patch to) and
>> it worked fine. Note there were no stale objects or conditional requests
>> during the test because I set ttl to a high value so it's either a hit
>> or a miss for an uncacheable object.
> Thanks as always, Dmitry. That last part is interesting, if there are no
> stale objects or conditional requests, then Varnish should behave
> exactly as if the patch weren't there.
>
> I assume you tested the most recent version of the patch (v5, attached
> to the mail that you quoted)? I had fixed a memory leak in an earlier
> version; but that leak only could have happened if there had been stale
> objects.
>
> Did you notice whether there were more cache misses with the patched
> version? And did you by any chance run varnishstat -1 after load testing
> the two versions? If so, I'd be interested if there were any noticeable
> difference in stats about memory usage, such as n_object, s_hdrbytes,
> s_bodybytes or the stevedore stats.
>
> BTW, with default settings (default_keep = 0), the patched version won't
> do any conditional requests, so that should be enough to get it to
> behave the same as the unpatched version.
>
>> I have a rather complex setup which includes apache with mod_proxy_html
>> which I use to reverse proxy a real Internet web site
>> (www.telegraph.co.uk). If there is anything I can do to help you
>> diagnose the problem, please let me know.
> I'll see if I can get the same effect with my own load tests, if not
> I'll get in touch.
>
>
> Thanks again,
> Geoff

-- 
Dmitry Panov

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: stat.txt
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20110311/a2eb42a1/attachment-0003.txt>