Memory leak in trunk?

[Dmitry Panov]

Hi Geoff, and all,

Well, to me it looks like a memory leak, the virtual memory consumed by 
varnishd grew twice as much as the size of the configured storage and 
continued to grow until I stopped the test. It doesn't look normal, does 
it? Especially considering that there were only a few pages involved.

I tried the same scenario with the latest stable release and it showed a 
completely different behavior: Virtual Memory size grew at a much slower 
rate and then remained steady even after 300k requests were served. 
There is definitely a problem with the current trunk.

Does the output from varnishstat give any clue? What should I post as an 
evidence?


On 11/03/2011 07:30, Geoff Simmons wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA256
>
> On 3/11/11 2:30 AM, Dmitry Panov wrote:
>> You're right, the patch didn't make any difference. The problem was that
>> I was running an earlier version of the unpatched trunk which contained
>> the ws overflow bug, so the reason memory usage was dropping from time
>> to time was because one of the threads was dying.
>>
>> I've installed the latest trunk and it behaved exactly like the patched
>> version: after about 80k requests the virtual memory size was about 2G
>> (I have 1G storage configured). I ran varnishstat -1 after that (attached).
>>
>> So it looks like there is a memory leak in the current trunk.
> I tried a load test, but couldn't reproduce a memory leak. With the
> latest unpatched trunk using -s malloc,1G, and running httperf and an
> Apache backend all on my machine, I ran a load of about 12,500 reqs/s
> for over a half hour. varnishd's virtual memory size expanded to 911 MB
> after 4 minutes, but no further. The stevedore stats show 415 MB
> outstanding bytes after the run, which is about how much data I have in
> the test site. The stevedore had allocated 7 GB and freed 6.6 GB during
> the course of the run. All of that looks right to me.
>
> Since this is about the trunk itself rather than the patch, and if
> you're sure you can establish that there's a leak, maybe you should file
> a bug report with the evidence.
>
> Thanks again for your help with testing.
>
>
> Best,
> Geoff

Best regards,

-- 
Dmitry Panov