Varnish Memory usage increase

[Pinakee BIswas]

Following is the data from varnishstat:

SMA.Transient.g_bytes 15.76K        -2.19K          .           18.53K 
15.50K        15.24K

SMF.s0.g_bytes 673.26M        47.95K          .          672.87M 
669.89M       668.90M


On 07/02/18 6:51 pm, Guillaume Quintard wrote:
> Amount of storage used is just the sum of all the g_bytes fields
>
> -- 
> Guillaume Quintard
>
> On Feb 7, 2018 14:12, "Pinakee BIswas" <pinakee at waltzz.com 
> <mailto:pinakee at waltzz.com>> wrote:
>
>     Hi Guillaume,
>
>     Thanks for your response and details.
>
>     Please find my comments inline:
>
>
>     On 07/02/18 6:21 pm, Guillaume Quintard wrote:
>>     Hi there,
>>
>>     So, varnish memory usage will mostly come from three directions:
>>     - storage, include Transient, so check the g_bytes fields in
>>     varnishstat. Passes and shortlived objects will use Transient, so
>>     you can either reduce those, or limit the Transient storage
>>     (unbounded by defaukt).
>     Checked the g_bytes for transient in varnishstat. It's in Kbs.
>>     - thread workspaces, one thread typically uses one workspace, so
>>     you can limit the number of threads, or reduce the workspace size
>>     - memory fragmentation: jemalloc will fragment up to 25%, not
>>     much you can do here.
>>
>>     And no, no way to modify the mmap without restarting varnish.
>>
>>     May I ask how much data you are caching?
>     Could you please let me know how to figure out the amount of data
>     cached?
>>
>>     -- 
>>     Guillaume Quintard
>>
>>     On Wed, Feb 7, 2018 at 1:34 PM, Pinakee BIswas
>>     <pinakee at waltzz.com <mailto:pinakee at waltzz.com>> wrote:
>>
>>         Hi,
>>
>>         We have been using varnish since more than a year for our
>>         ecommerce site.
>>
>>         Current version is 4.1.8.
>>
>>         OS is Linux. RAM is 8GB.
>>
>>         I am observing that the varnish memory usage is increasing
>>         (right now 7%) and so is resident memory usage increasing.
>>
>>         The storage being used is file:
>>
>>         -s file,/tmp/varnish/,${storage}
>>
>>         I am not sure about the reason for the increase in varnish
>>         memory usage:
>>
>>           * Is there a way to limit the varnish memory usage?
>>           * How can I diagnose what is consuming memory?
>>           * Most of our web pages have maximum 2 days of cache. Also,
>>             some of the pages might be least visited.
>>           * Is there a way to manipulate varnish mmap (when file
>>             storage is used)?
>>
>>         Would appreciate any help on the above for efficient use of
>>         varnish.
>>
>>         Thanks,
>>
>>         Pinakee
>>
>>
>>
>>         _______________________________________________
>>         varnish-misc mailing list
>>         varnish-misc at varnish-cache.org
>>         <mailto:varnish-misc at varnish-cache.org>
>>         https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>         <https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc>
>>
>>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20180207/e38e7f7d/attachment.html>