Varnish over memory allocation
AlexWakefield at fastmail.com.au
Mon May 25 12:38:10 UTC 2020
Thanks for the reply. I originally thought that it might've been transient storage as well however checking that shows that only 287MB of storage has been allocated from it. Its allocated and released 1.06TB though, is that perhaps the cause of the issue?
SMA.s0.g_bytes shows the correct size of 23.99GB allocated which is what I expect however a pmap of the varnish process shows 29977920K used, 6GB over what was set.
On Mon, 25 May 2020, at 8:02 PM, Dridi Boukelmoune wrote:
> On Mon, May 25, 2020 at 8:07 AM Alex Wakefield
> <AlexWakefield at fastmail.com.au> wrote:
> > Whoops, knew I forgot to specify something!
> > We're using malloc. Command line switch is specifically `-s malloc,24GB`
> The -s option only specifies the storage size (HTTP responses with
> some metadata). The rest of Varnish's memory footprint goes on top,
> things like loaded VCLs, ongoing VCL transactions, all kinds of data
> structures. VMODs like XKey may add their own footprint on top, the
> list goes on.
> Even on the storage side, if you only declare a malloc storage like
> you did, you will get an unlimited Transient storage by default for
> short-lived or uncacheable responses.
> The only way today to tell a Varnish instance to limit itself to 24GB
> (and still on a best-effort basis) is with Varnish Enterprise's memory
More information about the varnish-misc