"expire superseded" in master - Re: [PATCH] Purge all expired matching objects except the most recent one during hash lookup

[Nils Goroll]

Hi

On 17/12/14 00:43, Jingyi Wei wrote:
> I see that varnish 4.0.2 does not free duplicate objects when ttl is
> expired but grace is not expired. Every time when ttl expired it will
> fetch a new object and increase memory usage. I set ttl to 1ms and grace
> to 1 day and did stress test. Varnish memory usage continued to increase
> and soon exceeded the max memory limit, but remained stable at a point.

This is an old topic which we had termed "expire superseded" in the past. We had
discussed the approach you are suggesting already, but later agreed that the
better solution is to expire the superseded object at fetch time.

This is exactly what the fix for #1530 commit
b61c2305462c9428225a9531e664907db408fd7f should do. This is in 4.0.2, so could
you please make sure that the effect you are seeing is really root caused by
superseded objects?

Nils

P.S.: IIUC your patch would throw out old objects before they got refreshed and
probably would break IMS (have not checked in detail)