Varnish malloc issue

[sujith pv]

No..I'm planning to try that next. So during that time also so do we need
to cap transient storage ??

On 15-Sep-2016 01:18, "Guillaume Quintard" <guillaume at varnish-software.com>
wrote:

> Have you tried the packages from your distribution?
>
> On Sep 14, 2016 20:09, "sujith pv" <sujithnss at gmail.com> wrote:
>
>> After the full load of requests..g_space became minimal and g_bytes
>> became like 8 Gb..I had assigned malloc as 8 gb
>>
>> On 14-Sep-2016 22:42, "Guillaume Quintard" <guillaume at varnish-software.co
>> m> wrote:
>>
>>> What about the other g_space/g_bytes?
>>>
>>> On Sep 14, 2016 18:33, "sujith pv" <sujithnss at gmail.com> wrote:
>>>
>>>> It was showing as 0 for Transient_g_bytes and 1Gb for g_space as I
>>>> mentioned in my mail.
>>>>
>>>> On Wed, Sep 14, 2016 at 7:45 PM, Guillaume Quintard <
>>>> guillaume at varnish-software.com> wrote:
>>>>
>>>>> please keep using the mailing list, some other people may have ideas.
>>>>>
>>>>> what's the output of "varnishstat -1 | grep g_bytes" ?
>>>>>
>>>>> --
>>>>> Guillaume Quintard
>>>>>
>>>>> On Wed, Sep 14, 2016 at 3:50 PM, sujith pv <sujithnss at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Hi Guillaume
>>>>>>
>>>>>> I have tried capping Transient and I could see the respective values
>>>>>> under SMA.transient.g_bytes/g_space. But still on load I didnt see this
>>>>>> g_bytes being used and was showing 0 only.
>>>>>>
>>>>>> Best Regards
>>>>>> Sujith P V
>>>>>>
>>>>>> On Tue, Sep 6, 2016 at 7:39 PM, Guillaume Quintard <
>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>
>>>>>>> Maybe, not sure. Unless you have a **good** reason to do so, use the
>>>>>>> packages.
>>>>>>>
>>>>>>> --
>>>>>>> Guillaume Quintard
>>>>>>>
>>>>>>> On Tue, Sep 6, 2016 at 3:19 PM, sujith pv <sujithnss at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Thanks Guillaume. Actually in my end Varnish was compiled here and
>>>>>>>> installed (I was not part of that initially :-)). Will this have any impact
>>>>>>>> compared to as you have mentioned like try from a packaged version.
>>>>>>>>
>>>>>>>> On Tue, Sep 6, 2016 at 6:46 PM, Guillaume Quintard <
>>>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>>>
>>>>>>>>> It's just a question of storage, shortlive objects got into
>>>>>>>>> transient storage instead of the one(s) you defined in the command line.
>>>>>>>>> But it's still a cache storage, based on malloc, that is by default
>>>>>>>>> unlimited in size.
>>>>>>>>>
>>>>>>>>> The rationale is that for object with such short ttl+grace+keep, a
>>>>>>>>> simple malloc is good enough.
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Guillaume Quintard
>>>>>>>>>
>>>>>>>>> On Tue, Sep 6, 2016 at 3:04 PM, sujith pv <sujithnss at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Actually I'm confused around the short lived objects. Is these
>>>>>>>>>> objects are something like cached objects or who is creating these objects.
>>>>>>>>>>
>>>>>>>>>> On Tue, Sep 6, 2016 at 1:57 PM, Guillaume Quintard <
>>>>>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Sorry, I didn't get your question.
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Guillaume Quintard
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Sep 6, 2016 at 8:31 AM, sujith pv <sujithnss at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks Guillaume. May be Im asking a very basic question , but
>>>>>>>>>>>> still how this short lived objects are created and I'm not giving any
>>>>>>>>>>>> settings for the same as well.
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Sep 6, 2016 at 11:54 AM, Guillaume Quintard <
>>>>>>>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> https://www.varnish-cache.org/docs/4.1/reference/varnishd.ht
>>>>>>>>>>>>> ml#shortlived
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sep 6, 2016 08:07, "sujith pv" <sujithnss at gmail.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks Guillaume. I will try the transient settings first
>>>>>>>>>>>>>> and verify the same. Also could you please elaborate on the short lived
>>>>>>>>>>>>>> objects.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Best Regards
>>>>>>>>>>>>>> Sujith P V
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Fri, Sep 2, 2016 at 11:24 AM, Guillaume Quintard <
>>>>>>>>>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi Viktor,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Have a look here https://www.varnish-cache
>>>>>>>>>>>>>>> .org/docs/trunk/users-guide/storage-backends.html#transient-
>>>>>>>>>>>>>>> storage
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Sujith, please try to use a packaged version from your
>>>>>>>>>>>>>>> distribution.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> Guillaume Quintard
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Fri, Sep 2, 2016 at 1:18 AM, Viktor Villafuerte <
>>>>>>>>>>>>>>> viktor.villafuerte at optusnet.com.au> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi Guillaume,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Thu 01 Sep 2016 09:53:22, Guillaume Quintard wrote:
>>>>>>>>>>>>>>>> > Is that virtual or real memory?
>>>>>>>>>>>>>>>> > Try to cap your Transient storage to 1GB, you may be
>>>>>>>>>>>>>>>> overloaded with short
>>>>>>>>>>>>>>>> > lived objects.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Could you elaborate on this bit more? I've got very similar
>>>>>>>>>>>>>>>> problem
>>>>>>>>>>>>>>>> here. Eg: Total memory 384g
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> VIRT: 487g
>>>>>>>>>>>>>>>> RES:  311g
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> free says: 46g free
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> but SWAP is 91.3% used :(
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How can this be?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thanks
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> v
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > On Sep 1, 2016 9:40 AM, "sujith pv" <sujithnss at gmail.com>
>>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>> >
>>>>>>>>>>>>>>>> > > Hi All
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>> > > I had already mailed this query long back but this time
>>>>>>>>>>>>>>>> putting in a
>>>>>>>>>>>>>>>> > > different manner.
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>> > >    - We are using Varnish 4.0 in our end. We have a
>>>>>>>>>>>>>>>> machine with memory
>>>>>>>>>>>>>>>> > >    of 16G with 8G being malloc for Varnish. We have a
>>>>>>>>>>>>>>>> TTL for 2 hrs as well.
>>>>>>>>>>>>>>>> > >    - During our peak traffic, when we see the total
>>>>>>>>>>>>>>>> memory of the machine
>>>>>>>>>>>>>>>> > >    reaching 90% and like varnishd process is taking
>>>>>>>>>>>>>>>> some 89% .
>>>>>>>>>>>>>>>> > >    - So I'm just confused even though we had allocated
>>>>>>>>>>>>>>>> just 8G malloc
>>>>>>>>>>>>>>>> > >    which is like 50% of the total memory, how the
>>>>>>>>>>>>>>>> process is eating up 89% of
>>>>>>>>>>>>>>>> > >    memory and the memory is not releasing even after
>>>>>>>>>>>>>>>> TTL.
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>> > > Any help please...
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>> > > Best Regards
>>>>>>>>>>>>>>>> > > Sujith P V
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>> > > _______________________________________________
>>>>>>>>>>>>>>>> > > varnish-misc mailing list
>>>>>>>>>>>>>>>> > > varnish-misc at varnish-cache.org
>>>>>>>>>>>>>>>> > > https://www.varnish-cache.org/
>>>>>>>>>>>>>>>> lists/mailman/listinfo/varnish-misc
>>>>>>>>>>>>>>>> > >
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> > _______________________________________________
>>>>>>>>>>>>>>>> > varnish-misc mailing list
>>>>>>>>>>>>>>>> > varnish-misc at varnish-cache.org
>>>>>>>>>>>>>>>> > https://www.varnish-cache.org/
>>>>>>>>>>>>>>>> lists/mailman/listinfo/varnish-misc
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Viktor Villafuerte
>>>>>>>>>>>>>>>> Optus Internet Engineering
>>>>>>>>>>>>>>>> t: +61 2 80825265
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20160915/c1984966/attachment-0001.html>