Transient storage killing memory

[Mark Staudinger]

On Tue, 15 Nov 2016 03:42:29 -0500, Niall Murphy  
<niall.murphy at sparwelt.de> wrote:

> I'm wondering if someone could put me in the right direction in how
> to further investigate the content of transient storage, and why it's
> larger than the regular cache.

Hi Niall,

This is a longstanding Varnish behavior, if one is not specified, a  
Transient storage pool is created as type "malloc" with no limit.

https://www.varnish-cache.org/docs/5.0/users-guide/storage-backends.html#transient-storage

The short-term solution to put a limit on this is to create a storage pool  
named "Transient" when starting varnishd.  For example, this will create  
an 128M pool - add your existing "-s pool=settings" along with this one on  
the command line or startup script.

"-s Transient=malloc,128m"

It might also be useful to capture some varnishlog output and determine  
what objects are being stored in the Transient pool, and whether or not  
your "shortlived" parameter, or default grace value needs to be adjusted.   
Even if you do determine you need to do some things differently here that  
will prevent the Transient pool from growing beyond your ideal limit, IMO  
it's a good idea to keep this limited anyway.

Cheers,
-=Mark