Transient storage killing memory

[Mark Staudinger]

On Tue, 29 Nov 2016 11:09:04 -0500, Niall Murphy  
<niall.murphy at sparwelt.de> wrote:

> On Mon, 21 Nov 2016 15:09:08 -0500
> Mark Staudinger <mark.staudinger at nyi.net> wrote:
>
>> It might also be useful to capture some varnishlog output and
>> determine what objects are being stored in the Transient pool, and
>> whether or not your "shortlived" parameter, or default grace value
>> needs to be adjusted. Even if you do determine you need to do some
>> things differently here that will prevent the Transient pool from
>> growing beyond your ideal limit, IMO it's a good idea to keep this
>> limited anyway.
>>
> Hi Mark,
>
> I took a look into ttl < 10s objects and saw that they are requests we
> intentionally apply either "max-age=0, private" or "no-cache, private"
> to. However their storage field is still "malloc Transient", and
> transient storage usage only appears to be going up.
>
> Any ideas how to investigate further?
>
> It's varnish 5.0.0-1, and this didn't happen with 4.1, thought there
> may well have been configuration changes since then.
>
> Regards,
>
> --
> Niall

Hi Niall,

It's not clear if you actually wish to cache these requests.

The best way to proceed would be to look at the output of varnishlog for a  
few sample requests, and see what the values are for the "TTL" log entry,  
and make sure they match the desired settings/behavior.  Not that if you  
change TTL/grace/keep settings during the request, there will be multiple  
entries in the log. Here's a sample entry for an object that was not  
cached:

--  TTL            VCL 0 0 0 1480436502

Best,
-=Mark