Varnish Memory usage increase

[Guillaume Quintard]

There's no optimal configuration, only compromises, but I can point you
there:
https://info.varnish-software.com/blog/understanding-varnish-cache-memory-usage

-- 
Guillaume Quintard

On Thu, Feb 8, 2018 at 7:26 AM, Pinakee BIswas <pinakee at waltzz.com> wrote:

> Hi,
>
> Could you please suggest what would be the optimal configuration for
> varnishd in my scenario - in terms of memory usage and performance?
>
> Thanks,
>
> Pinakee
>
> On 07/02/18 7:19 pm, Pinakee BIswas wrote:
>
> I agree but the reason being we have other applications like DB servers,
> backend server, cache server etc running on the same machine (as our scale
> is not that big). Hence, memory is a sought after resource and needs to be
> optimized as much as possible.
>
> So, if you suggest malloc would be a better storage still, I can change
> the storage to the same.
> On 07/02/18 7:09 pm, Guillaume Quintard wrote:
>
> So, it's probably not related to your issue, but it seems your cached data
> easily fits in memory, why use the file storage?
>
> --
> Guillaume Quintard
>
> On Feb 7, 2018 14:35, "Pinakee BIswas" <pinakee at waltzz.com> wrote:
>
>> Following is the data from varnishstat:
>>
>> SMA.Transient.g_bytes
>> 15.76K        -2.19K          .           18.53K        15.50K        15.24K
>>
>> SMF.s0.g_bytes
>> 673.26M        47.95K          .          672.87M       669.89M
>> 668.90M
>>
>>
>> On 07/02/18 6:51 pm, Guillaume Quintard wrote:
>>
>> Amount of storage used is just the sum of all the g_bytes fields
>>
>> --
>> Guillaume Quintard
>>
>> On Feb 7, 2018 14:12, "Pinakee BIswas" <pinakee at waltzz.com> wrote:
>>
>>> Hi Guillaume,
>>>
>>> Thanks for your response and details.
>>>
>>> Please find my comments inline:
>>>
>>> On 07/02/18 6:21 pm, Guillaume Quintard wrote:
>>>
>>> Hi there,
>>>
>>> So, varnish memory usage will mostly come from three directions:
>>> - storage, include Transient, so check the g_bytes fields in
>>> varnishstat. Passes and shortlived objects will use Transient, so you can
>>> either reduce those, or limit the Transient storage (unbounded by defaukt).
>>>
>>> Checked the g_bytes for transient in varnishstat. It's in Kbs.
>>>
>>> - thread workspaces, one thread typically uses one workspace, so you can
>>> limit the number of threads, or reduce the workspace size
>>> - memory fragmentation: jemalloc will fragment up to 25%, not much you
>>> can do here.
>>>
>>> And no, no way to modify the mmap without restarting varnish.
>>>
>>> May I ask how much data you are caching?
>>>
>>> Could you please let me know how to figure out the amount of data cached?
>>>
>>>
>>> --
>>> Guillaume Quintard
>>>
>>> On Wed, Feb 7, 2018 at 1:34 PM, Pinakee BIswas <pinakee at waltzz.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> We have been using varnish since more than a year for our ecommerce
>>>> site.
>>>>
>>>> Current version is 4.1.8.
>>>>
>>>> OS is Linux. RAM is 8GB.
>>>>
>>>> I am observing that the varnish memory usage is increasing (right now
>>>> 7%) and so is resident memory usage increasing.
>>>>
>>>> The storage being used is file:
>>>>
>>>> -s file,/tmp/varnish/,${storage}
>>>>
>>>> I am not sure about the reason for the increase in varnish memory usage:
>>>>
>>>>    - Is there a way to limit the varnish memory usage?
>>>>    - How can I diagnose what is consuming memory?
>>>>    - Most of our web pages have maximum 2 days of cache. Also, some of
>>>>    the pages might be least visited.
>>>>    - Is there a way to manipulate varnish mmap (when file storage is
>>>>    used)?
>>>>
>>>> Would appreciate any help on the above for efficient use of varnish.
>>>>
>>>> Thanks,
>>>>
>>>> Pinakee
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>
>>>
>>>
>>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180208/6ada6693/attachment-0001.html>