"expire superseded" in master

[Jingyi Wei]

Hello,

Thank you for your explanation. In the commit you mentioned, I see that
bo->ims_obj is set in VBF_Fetch() only if Last-Modified or ETag header
exists in the previous backend response. Otherwise |ims_obj| is NULL and
the memory will not be freed, so the actual problem is that there's no
Last-Modified header in our backend response. I commented out the check
for the existence of two headers in VBF_Fetch() and reverted my patch.
The old object is purged correctly in fetch step. Could you please tell
me why Last-Modified or ETag must present in order to free the previous
object? I looked at the places that used |ims_obj| in varnish 4.0.2, and
I found no issues to remove the check.

Thanks,
Jingyi Wei

On 12/18/14 07:51, Nils Goroll wrote:
> Hi
>
> On 17/12/14 00:43, Jingyi Wei wrote:
>> I see that varnish 4.0.2 does not free duplicate objects when ttl is
>> expired but grace is not expired. Every time when ttl expired it will
>> fetch a new object and increase memory usage. I set ttl to 1ms and grace
>> to 1 day and did stress test. Varnish memory usage continued to increase
>> and soon exceeded the max memory limit, but remained stable at a point.
> This is an old topic which we had termed "expire superseded" in the past. We had
> discussed the approach you are suggesting already, but later agreed that the
> better solution is to expire the superseded object at fetch time.
>
> This is exactly what the fix for #1530 commit
> b61c2305462c9428225a9531e664907db408fd7f should do. This is in 4.0.2, so could
> you please make sure that the effect you are seeing is really root caused by
> superseded objects?
>
> Nils
>
> P.S.: IIUC your patch would throw out old objects before they got refreshed and
> probably would break IMS (have not checked in detail)
>
>