[PATCH] backend conditional requests 5th release

[Dmitry Panov]

Hi Geoff,

I have finally been able to test the patched version under some proper 
load. I was using tsung (http://tsung.erlang-projects.org/). I recorded 
a browser session in which I accessed a couple of pages, then tried to 
replay it with tsung.

It looks like there is a memory leak in the patched version because the 
virtual memory size grew at a constant pace until eventually varnishd 
filled up the whole available space. I tried exactly the same scenario 
with an unpatched trunk (same revision that I applied the patch to) and 
it worked fine. Note there were no stale objects or conditional requests 
during the test because I set ttl to a high value so it's either a hit 
or a miss for an uncacheable object.

I have a rather complex setup which includes apache with mod_proxy_html 
which I use to reverse proxy a real Internet web site 
(www.telegraph.co.uk). If there is anything I can do to help you 
diagnose the problem, please let me know.

On 09/03/2011 10:57, Geoff Simmons wrote:
> Hello again,
>
> And yet another version (v5) of the patch for conditional backend
> requests. Like the last version, this is one is to keep pace with recent
> commits to the trunk. The patch applies to:
>
> commit 13c60b522a6c7abe9cc1052c2a244f25f2ba5227
> Date:   Tue Mar 8 14:47:57 2011 +0100
>
> phk committed some changes related to this feature, mainly the addition
> of the new timer variable to VCL. There are also some changes in the way
> that previous version of the patch worked.
>
> - The name of the timer for conditional requests changed again, now to
> "keep". The management parameter is "default_keep".
>
> - In addition to beresp.keep, obj.keep and stale_obj.keep, there is now
> req.keep, which sets an upper bound for the keep time within a session.
> (Like req.ttl, which phk added yesterday.)
>
> - The keep interval now begins after ttl and grace have elapsed, so an
> object is available for conditional requests after obj.ttl + obj.grace,
> and is scheduled for eviction after obj.ttl + obj.grace + obj.keep. (In
> previous versions of the patch, the keep interval began after ttl
> elapses, independent of grace, and object were evicted after obj.ttl +
> max(obj.grace, obj.keep).)
>
> - The default value default_keep is now 0, so you need to set a non-zero
> default_keep, or set keep in VCL, to make the conditional requests happen.
>
> The Wiki page has been updated to reflect all of this:
>
> http://www.varnish-cache.org/trac/wiki/BackendConditionalRequests
>
> @phk: I made one change in the commits from yesterday. Since
> default_keep = 0 and EXP_get_*() was returning -1 for any timer = 0,
> *.keep was always -1 unless it was changed in VCL. I think VCL authors
> wouldn't expect that; I got feedback about an earlier version in which
> beresp.keep was -1 after a backend validation, which both the reporter
> and I thought was wrong, so I fixed it in the next version.
>
> Now I have this in cache_expire.c:
>
>          double                                                  \
>          EXP_Get_##fld(const struct exp *e)                      \
>          {                                                       \
> -               return (e->fld>  0. ? e->fld : -1.);            \
> +               return (e->fld>  0. ? e->fld : 0.);             \
>          }                                                       \
>
> This required a change in test case b00040.vtc, which expects req.grace
> == -1, but that doesn't seem to be a critical value there.
>
>
> Best,
> Geoff
>

Best regards,

-- 
Dmitry Panov