malloc storage memory leak?
Rogier R. Mulhuijzen
drwilco at drwilco.net
Mon Jul 23 17:20:16 CEST 2012
I would go out and drop 700 bucks on 4x16GB for my server if I were you.
600336949&IsNodeId=1&name=64GB (4 x 16GB)
22TB of tiles is quite a bit, so maybe go through the logfiles a bit and
see exactly how much having 50G of cache would gain you. And if the
zoomlevel is in the URL of the tiles, I would maybe choose to just cache
the outer N levels, so that inner levels don't blow away the cache of the
outer. Or run two varnishes, one caching outer layers, like I just
described and have a set amount of memory for that, and then run the
second one with a smaller set, just to catch those requests for inner
tiles that somehow are very popular.
Which brings me back to the fact that this is a -dev list (so your
question really doesn't belong here) and I just had an idea.
This separating storage deal, I'm a bit behind on 3/master, so do we have
stevedores exposed to VCL in any way? I recall storage hints, but not sure
if there's anything using that beyond transient?
Anyhoo, it might be worth it to segregate objects by various properties
and not have to run separate varnish instances. :)
On Mon, 23 Jul 2012, Cal Heldenbrand wrote:
> Thanks for the clarification Andreas. I'm running around 600k objects in
> memory, which should be around 586MB of overhead.
> If I could ask you guys your opinion -- is there a better way to configure
> Varnish for my environment? My backend is 22TB of mapping tiles, each file
> being anywhere from 100bytes to 3KB. So a small 4GB cache results in just
> being an LRU, caching the most popular tiles. Which makes outer zoom
> levels very fast, but misses on almost all of the low parcel levels.
> Thanks for any advice!
> On Mon, Jul 23, 2012 at 2:39 AM, Andreas Plesner Jacobsen <apj at mutt.dk>wrote:
>> On Thu, Jul 19, 2012 at 04:11:20PM -0500, Cal Heldenbrand wrote:
>>> Here's a screenshot of top. This machine was set to malloc max at
>> And it hasn't passed that. Remember that there's additional overhead of
>> 1KB/object outside the actual storage backend.
>>> SMA.s0.g_bytes 5595155188 . Bytes outstanding
>> It has allocated 5.5G
>>> SMA.Transient.g_bytes 0 . Bytes outstanding
>> And isn't gobbling up transient space at the moment.
>> I don't see a problem (in varnish at least).
More information about the varnish-dev