Varnish Memory usage increase

[Guillaume Quintard]

So, it's probably not related to your issue, but it seems your cached data
easily fits in memory, why use the file storage?

-- 
Guillaume Quintard

On Feb 7, 2018 14:35, "Pinakee BIswas" <pinakee at waltzz.com> wrote:

> Following is the data from varnishstat:
>
> SMA.Transient.g_bytes
> 15.76K        -2.19K          .           18.53K        15.50K        15.24K
>
> SMF.s0.g_bytes
> 673.26M        47.95K          .          672.87M       669.89M
> 668.90M
>
>
> On 07/02/18 6:51 pm, Guillaume Quintard wrote:
>
> Amount of storage used is just the sum of all the g_bytes fields
>
> --
> Guillaume Quintard
>
> On Feb 7, 2018 14:12, "Pinakee BIswas" <pinakee at waltzz.com> wrote:
>
>> Hi Guillaume,
>>
>> Thanks for your response and details.
>>
>> Please find my comments inline:
>>
>> On 07/02/18 6:21 pm, Guillaume Quintard wrote:
>>
>> Hi there,
>>
>> So, varnish memory usage will mostly come from three directions:
>> - storage, include Transient, so check the g_bytes fields in varnishstat.
>> Passes and shortlived objects will use Transient, so you can either reduce
>> those, or limit the Transient storage (unbounded by defaukt).
>>
>> Checked the g_bytes for transient in varnishstat. It's in Kbs.
>>
>> - thread workspaces, one thread typically uses one workspace, so you can
>> limit the number of threads, or reduce the workspace size
>> - memory fragmentation: jemalloc will fragment up to 25%, not much you
>> can do here.
>>
>> And no, no way to modify the mmap without restarting varnish.
>>
>> May I ask how much data you are caching?
>>
>> Could you please let me know how to figure out the amount of data cached?
>>
>>
>> --
>> Guillaume Quintard
>>
>> On Wed, Feb 7, 2018 at 1:34 PM, Pinakee BIswas <pinakee at waltzz.com>
>> wrote:
>>
>>> Hi,
>>>
>>> We have been using varnish since more than a year for our ecommerce
>>> site.
>>>
>>> Current version is 4.1.8.
>>>
>>> OS is Linux. RAM is 8GB.
>>>
>>> I am observing that the varnish memory usage is increasing (right now
>>> 7%) and so is resident memory usage increasing.
>>>
>>> The storage being used is file:
>>>
>>> -s file,/tmp/varnish/,${storage}
>>>
>>> I am not sure about the reason for the increase in varnish memory usage:
>>>
>>>    - Is there a way to limit the varnish memory usage?
>>>    - How can I diagnose what is consuming memory?
>>>    - Most of our web pages have maximum 2 days of cache. Also, some of
>>>    the pages might be least visited.
>>>    - Is there a way to manipulate varnish mmap (when file storage is
>>>    used)?
>>>
>>> Would appreciate any help on the above for efficient use of varnish.
>>>
>>> Thanks,
>>>
>>> Pinakee
>>>
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org
>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180207/30606fd0/attachment.html>