Varnish 2.0.3 consuming excessive memory
Harald Friessnegger
harald at webmeisterei.com
Wed Apr 8 13:42:59 CEST 2009
hi darryl
i had a simlar problem (varnish child process consuming lots of memory) a week
ago (see the thread "make varnish don't start a subprocess")
the solution for my problem seems to be to use a high-enough cache-size:
starting varnish with
-s file,"/tmp/storage",300M
will make the child process use *about* 800M RAM (the RES column in top) after
some hours. after that i needed to restart varnish to free some ram for my
webserver.
starting with a bigger cache
-s file,"/tmp/storage",600M
makes it using about 500M RAM (the RES column in top after 5 days of uptime)
and happily coexist with zope ;-)
maybe this helps you too.
regards, fRiSi
Am Mittwoch, 8. April 2009 05:05:36 schrieb Darryl Dixon - Winterhouse
Consulting:
> > Not that I have an answer, but I'd be curious to see the differences
> > in 'pmap -x <pid>' output for the different children.
> >
> > --Michael
>
> That's a good thought - I'd almost forgotten about pmap. Anyhow, the
> 'normal' process is set up with a 2GB storage cache and pmap shows pretty
> much what you would expect - the largest anonymous segment is 41MB, with
> the vast majority of allocated memory booked against storage.cache (2GB).
> The pmap for the 'runaway' process shows 256MB booked against
> storage.cache, and then one of the an anonymous allocations is 2.3GB. For
> example:
>
>
> Normal process:
> ------------------------------------------------------
> 3896: /u01/app/varnish/sbin/varnishd -a web-cms1.prd.internal:8080
> web-cms.prd.internal:8080 -T web-cms1.prd.internal:18080 -t 108000 -f
> /u01/data/prd/varnish/vcl.conf -s
> file,/u01/data/prd/varnish/storage.cache,2048M -n
> /u01/data/prd/varnish/state -P /u01/data/prd/varnish/varnishd.pid
> Address Kbytes RSS Anon Locked Mode Mapping
> 0000000000400000 312 - - - r-x-- varnishd
> 000000000064d000 8 - - - rw--- varnishd
> 000000000064f000 68 - - - rw--- [ anon ]
> 0000000040000000 4 - - - ----- [ anon ]
> 0000000040001000 10240 - - - rw--- [ anon ]
> [...snip...]
> 00002aaab0900000 2097152 - - - rw-s- storage.cache
> 00002aab30900000 28 - - - r-x-- vcl.1P9zoqAU.so
> 00002aab30907000 2044 - - - ----- vcl.1P9zoqAU.so
> 00002aab30b06000 8 - - - rw--- vcl.1P9zoqAU.so
> 00002aab30c00000 2056 - - - rw--- [ anon ]
> 00002aab30f00000 2056 - - - rw--- [ anon ]
> 00002aab31200000 41984 - - - rw--- [ anon ]
> 00007fffae8f2000 88 - - - rw--- [ stack ]
> ffffffffff600000 8192 - - - ----- [ anon ]
> ---------------- ------ ------ ------ ------
> total kB 2468512 - - -
> ------------------------------------------------------
>
>
>
> Runaway process:
> ------------------------------------------------------
> 3940: /u01/app/varnish/sbin/varnishd -a web-cms-edit1.prd.internal:8080
> web-cms-edit.prd.internal:8080 -T web-cms-edit1.prd.internal:18080 -t 3600
> -f /u01/data/prd-edit/varnish/vcl.conf -s
> file,/u01/data/prd-edit/varnish/storage.cache,256M -n
> /u01/data/prd-edit/varnish/state -P
> /u01/data/prd-edit/varnish/varnishd.pid
> Address Kbytes RSS Anon Locked Mode Mapping
> 0000000000400000 312 - - - r-x-- varnishd
> 000000000064d000 8 - - - rw--- varnishd
> 000000000064f000 68 - - - rw--- [ anon ]
> 0000000040000000 4 - - - ----- [ anon ]
> 0000000040001000 10240 - - - rw--- [ anon ]
> [...snip...]
> 00002aaab0900000 262144 - - - rw-s- storage.cache
> 00002aaac0900000 28 - - - r-x-- vcl.1P9zoqAU.so
> 00002aaac0907000 2048 - - - ----- vcl.1P9zoqAU.so
> 00002aaac0b07000 8 - - - rw--- vcl.1P9zoqAU.so
> 00002aaac0c00000 2056 - - - rw--- [ anon ]
> 00002aaac0f00000 2052 - - - rw--- [ anon ]
> 00002aaac1200000 233476 - - - rw--- [ anon ]
> 00002aaacf700000 2337792 - - - rw--- [ anon ]
> 00007fff4b619000 84 - - - rw--- [ stack ]
> ffffffffff600000 8192 - - - ----- [ anon ]
> ---------------- ------ ------ ------ ------
> total kB 3121808 - - -
> ------------------------------------------------------
>
>
> Both processes have been running for the same amount of time - 1day
> 17hours.
>
> regards,
> Darryl Dixon
> Winterhouse Consulting Ltd
> http://www.winterhouseconsulting.com
>
> > On Apr 7, 2009, at 6:27 PM, Darryl Dixon - Winterhouse Consulting wrote:
> >>> Hi All,
> >>>
> >>> I have an odd problem that I have only noticed happening since
> >>> moving from
> >>> 1.1.2 to 2.0.3 - excessive memory consumption of the varnish child
> >>> process. For example, I have a varnish instance with a 256MB cache
> >>> allocated, that is currently consuming 4.9GB of resident memory
> >>> (6.5GB
> >>> virtual). The instance has only been running for 4 days and has
> >>> only got
> >>> 25MB of objects in the cache.
> >>>
> >>> This is clearly excessive and is causing us some serious problems
> >>> in terms
> >>> of memory pressure on the machine. Our VCL is largely unchanged
> >>> from our
> >>> 1.1.2 setup to the 2.0.3 except for the obvious vcl.syntax changes,
> >>> and
> >>> the introduction of request restarts under certain limited
> >>> scenarios. Can
> >>> anyone shed some light?
> >>
> >> One further footnote to this. I have a second varnish instance
> >> running on
> >> the same machine which talks to different backend servers (still
> >> primarily
> >> the same sort of content though), with the VCL only fractionally
> >> different
> >> - it does not seem to suffer from the runaway memory consumption of
> >> the
> >> first instance. The only difference in the VCL between the two is
> >> that in
> >> the one with runaway memory this is present in vcl_recv():
> >>
> >> + if (req.http.Pragma ~ ".*no-cache.*" || req.http.Cache-Control ~
> >> ".*no-cache.*") {
> >> + purge_url(regsub(req.url, "[?].*$", ".*$"));
> >> + }
> >> +
> >>
> >> Is there possibly something in the regsub engine being triggered
> >> that is
> >> very expensive and would cause it to consume and hold on to large
> >> amounts
> >> of memory?
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
--
Webmeisterei GmbH - Büro für Netzfragen
Tel: +43 5572 908877, Fax: +43 5572 908877-66
Steinebach 18, A-6850 Dornbirn
http://www.webmeisterei.com
More information about the varnish-misc
mailing list