varnish 2.1.4 swap and increase in objects:heads.

[Ken Brownfield]

These sysctl values helped a lot for me in a 64-bit / 64GB / 1.5TB SSD / "-s
file" configuration, pretty much stabilizing the box.

vm.swappiness = 10
vm.min_free_kbytes = 131072

And don't forget to reduce thread_pool_stack to 256KB-512KB.

FWIW,
-- 
kb


On Fri, Oct 29, 2010 at 14:00, David Birdsong <david.birdsong at gmail.com>wrote:

> I attempted to run an -s file instance with 5 80GB SSD's raided
> together.  I could never get past filling it half full before the IO
> load would consume the box.
>
> On Fri, Oct 29, 2010 at 9:19 AM, Taylan Develioglu
> <tdevelioglu at ebuddy.com> wrote:
> > More of a general question:
> >
> > Is there anyone that uses varnish with a storage size of 800GB+ (single
> > instance, file or malloc) ?
> >
> > I'm just wondering because we've had so much problems with varnish
> > (2.1.2 to 2.1.4) that I'm starting to think that we're an edge case.
> >
> > On Fri, 2010-10-29 at 14:26 +0200, Per Buer wrote:
> >> Hi.
> >>
> >> The issues are actually a bit different. One uses "-s file" which
> >> really makes the kernel handle the memory management implicitly in
> >> comparison to you Rodrigo, which uses -s malloc, where you clearly
> >> state that Varnish should allocate this much memory.
> >>
> >> Regarding -s malloc we recommend that on Linux one should not use more
> >> then 75% of the physical memory for -s malloc. In addition to the
> >> actual object store Varnish uses quite some memory for it's internal
> >> data structures. If you go above 75% chances are that Linux will start
> >> swapping a bit which might have a negative effect on overall
> >> performance. Tollef is getting back to you outside the list regarding
> >> the issue you've raised with us.
> >>
> >> Per.
> >>
> >> On Fri, Oct 29, 2010 at 1:59 PM, Rodrigo Benzaquen
> >> <rodrigo at mercadolibre.com> wrote:
> >> > Hi Taylan, exactly the same is happening to us, we already open a
> ticket
> >> > with Varnish software.
> >> >
> >> > Please let me know if you find a solution.
> >> >
> >> > Thanks
> >> > Rodrigo
> >> >
> >> >
> >> >
> >> >
> >> > ------------------
> >> >
> >> > Rodrigo Benzaquen
> >> > @rbenzaquen
> >> >
> >> >
> >> >
> >> > ________________________________
> >> > From: "Taylan Develioglu" <tdevelioglu at ebuddy.com>
> >> > To: varnish-misc at varnish-cache.org
> >> > Sent: Viernes, 29 de Octubre 2010 8:37:21
> >> > Subject: varnish 2.1.4 swap and increase in objects:heads.
> >> >
> >> > Debian Lenny, Linux varnish0 2.6.33.7 #1 SMP Thu Sep 2 15:27:53 CEST
> >> > 2010 x86_64 GNU/Linux
> >> >
> >> > I have a strange situation on one of our varnish servers.
> >> >
> >> > We recently raised the expiry on previously uncached content a
> >> > (Cache-Control: max-age=0, to Cache-Control: max-age=5).
> >> >
> >> > The number of objects vs. heads increased and the system started
> >> > swapping (vm.swappiness=0).
> >> >
> >> > http://lcontent.ebuddy.com/varnish/memory-week.png
> >> > http://lcontent.ebuddy.com/varnish/varnish_objects-week.png
> >> > http://lcontent.ebuddy.com/varnish/varnish_expunge-week.png
> >> >
> >> >
> >> > I don't understand why this is happening, bug maybe ?
> >> >
> >> > DAEMON_OPTS="-a :27535 \
> >> >              -T localhost:6082 \
> >> >              -f /etc/varnish/default.vcl \
> >> >              -s file,/var/cache/varnish/sda/varnish_storage.bin,135G \
> >> >              -s file,/var/cache/varnish/sdb/varnish_storage.bin,135G \
> >> >              -s file,/var/cache/varnish/sdc/varnish_storage.bin,135G \
> >> >              -s file,/var/cache/varnish/sdd/varnish_storage.bin,135G \
> >> >              -s file,/var/cache/varnish/sde/varnish_storage.bin,135G \
> >> >              -s file,/var/cache/varnish/sdf/varnish_storage.bin,135G \
> >> >              -p thread_pools=4 \
> >> >              -p thread_pool_min=200 \
> >> >              -p thread_pool_max=2400 \
> >> >              -p thread_pool_add_delay=2 \
> >> >              -p lru_interval=20"
> >> >
> >> >
> >> >
> >> > ---vcl---
> >> > sub vcl_recv {
> >> >     unset req.http.cookie;
> >> >     unset req.http.Authorization;
> >> >
> >> >     if (req.http.host ~ "^.*host.domain.com.*") {
> >> >         set req.http.host = "host.domain.com";
> >> >     }
> >> >
> >> >     if (req.request == "HEAD") {
> >> >         return(pass);
> >> >     }
> >> >
> >> >     if (req.request == "PURGE") {
> >> >         if (!client.ip ~ purge) {
> >> >             error 405 "Not allowed.";
> >> >         }
> >> >         return(lookup);
> >> >     }
> >> >
> >> >     if (req.request == "GET" && req.url == "/monitor") {
> >> >         error 200 "OK.";
> >> >     }
> >> >
> >> >     if (req.backend.healthy) {
> >> >         set req.grace = 30s;
> >> >   } else {
> >> >         set req.grace = 48h;
> >> >     }
> >> >
> >> > }
> >> >
> >> >  sub vcl_fetch {
> >> >     if (beresp.status == 404 || beresp.status >= 300) {
> >> >          return (pass);
> >> >     }
> >> >     set beresp.grace = 48h;
> >> >
> >> >  }
> >> >
> >> > sub vcl_hit {
> >> >     if (req.request == "PURGE") {
> >> >         set obj.ttl = 0s;
> >> >         error 200 "Purged.";
> >> >     }
> >> > }
> >> >
> >> > sub vcl_miss {
> >> >     if (req.request == "PURGE") {
> >> >         error 404 "Not in cache.";
> >> >     }
> >> > }
> >> > ---EOF---
> >> >
> >> >
> >> > _______________________________________________
> >> > varnish-misc mailing list
> >> > varnish-misc at varnish-cache.org
> >> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> >> >
> >> > _______________________________________________
> >> > varnish-misc mailing list
> >> > varnish-misc at varnish-cache.org
> >> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> >> >
> >>
> >>
> >>
> >
> >
> >
> > _______________________________________________
> > varnish-misc mailing list
> > varnish-misc at varnish-cache.org
> > http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
> >
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20101029/c536ccc5/attachment-0003.html>