Patch: Use calloc instead of malloc when running out of VSM space (common_vsm.c)

[Federico Schwindt]

In the case this would be an issue my concerns are not directly related to
performance nor latency.

If the configuration, either the default or modified by the user, would be
oversized there could be lot of waste, which might have other side effects
(as swapping).

I doubt this would the case for Fastly since you are likely on the high
side but it could be an issue for small to medium setups.
On 8 Mar 2016 3:53 p.m., "Devon H. O'Dell" <dho at fastly.com> wrote:

> Depending on the implementation of the allocator used, calloc may not have
> any additional overhead. When it does, there are really only a couple cases
> it shouldn't happen on modern systems:
>
> * If an object is allocated and freed in a tight loop. This shouldn't be
> happening anyway -- reuse / pool objects with this sort of access pattern.
> * If the object is large. Malloced memory should not be immediately
> visible to multiple concurrent processes, and objects that consist of only
> a few cache lines cost very little to zero on modern processors.
>
> It may be worth auditing for these situations, but I've done extensive
> profiling of extremely memory heavy workloads (hundreds of gb) in Varnish
> over the past few years, and I promise that calloc is not a current
> limiting factor in terms of latency or throughput.
>
> Of course, if you're concerned about swapping, I'd also argue that your
> cache is not properly sized.
>
> On Tue, Mar 8, 2016, 04:28 Poul-Henning Kamp <phk at phk.freebsd.dk> wrote:
>
>> --------
>> In message <CAJV_h0YVCRfTOFk=
>> 6N3H9jNxnrXM05ht2pQkyJ-FY-LGfu1H_g at mail.gmail.com>
>> , Federico Schwindt writes:
>>
>> >We use calloc in many places, I do wonder how many of them do really need
>> >it. The downside of using calloc when is not really needed is that by
>> >zeroing the memory you end up with resident memory and not virtual, which
>> >in turn might lead to swapping.
>>
>> This is almost always intentional, as we generally do not over-allocate.
>>
>> The exception is the malloc stevedore where we do.
>>
>>
>> --
>> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
>> phk at FreeBSD.ORG         | TCP/IP since RFC 956
>> FreeBSD committer       | BSD since 4.3-tahoe
>> Never attribute to malice what can adequately be explained by
>> incompetence.
>>
>> _______________________________________________
>> varnish-dev mailing list
>> varnish-dev at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20160308/a48f3412/attachment.html>