beresp.do_stream and memory usage

[Poul-Henning Kamp]

--------
In message <543EA1BB.8080208 at gmail.com>, Daniel writes:
>Hi,
>
>I have a question about beresp.do_stream and memory usage in Varnish 4.0.2.
>
>Example:
>
>The response (JSON) from backend is 73 kB in size uncompressed.
>
>Looking at the SMA.s0.g_bytes after one request I can see that Varnish 
>allocated 132 kB memory for this request.

I guess your backend doesn't send a Content-Length: header, so Varnish
will allocate storage in "fetch_chunksize" lumps.

When streaming is enabled, we cannot (at present) trim surplus storage
back, so you end up with 128K of storage by default.

When you disable streaming, we can trim the storage.

Depending on your average object size, you could try to reduce
fetch_chunksize to something like 16K maybe.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.