Varnish crashing when system starts to swap

Eden Li eden at hulu.com
Tue Apr 29 09:47:36 CEST 2008


On Mon, Apr 21, 2008 at 3:08 PM, Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
> In message <BCA2D4755E06EF44A0278A1D9B217E3F08171B3361 at exchange01.ex.local>, Ca
>
> lle Korjus writes:
>
>  >Varnish looks fine until it's had abour 1,5 million requests, then
>  >we can see the kswapd0 and kswapd1 start working and load average
>  >rises to about 200 and the machine gets totally unresponsive. Top
>  >shows a lot of cpu beeing spent on i/o waits and varnish child
>  >process restarts sometimes. In best
>  >case the process restarts and the server starts behaving within 5
>  >minutes but sometimes varnish dies completely. One thing we have
>  >noticed is that the reserved memory for varnish keeps rising and
>  >when it crashes it is usually around 14G.
>
>  You didn't say which varnish version you are using, but we have fixed
>  a number of memory leaks in -trunk recently.

I've just run into this exact issue varnish 1.1.2, except that varnish
was running for over a month until kswap started acting up.  Also, our
backends are not on the same machine.

I tried running different revisions of varnish from trunk on our
machines (r2634 and r2436 on Linux CentOS 5.0), but it keeps asserting
when daemonizing the process (varnishd.c:557).  Using an export of the
unstable branch kept at
http://varnish.projects.linpro.no/svn/branches/1.2 seems to be
working.

Do you know if this branch has some of the memory leak fixes you mention here?



More information about the varnish-misc mailing list