As we are moving more logic from varnishd into VCL, I believe some rethinking with how we deal with the default_vcl logic might be in order. This to make it easier to make the easy VCL changes and keep the default logic around still.<div>
<br></div><div>A common vcl error I have observed is where you want to match on something to perform an action, and then stop further processing on this request so further rules won't match on the url. E.g.:</div><div>
<br></div><div>sub vcl_fetch {</div><div> if (req.url ~ "\.(gif|jpg|css") {</div><div> # Cache our static resources "forever"</div><div> set beresp.ttl = 180d;</div><div> return (deliver);</div>
<div> }</div><div><br></div><div><br></div><div>}</div><div><br></div><div>This will ofc bypass any set-cookie checks performed by the default vcl_fetch logic. Also the explicit return(deliver) prevents any further application level rules that may be applied later, which is convenient. But the problem comes if for some reason one of these static resources should actually return a set-cookie header.</div>
<div><br></div><div>So the correct thing to do in this case</div><div><br></div><div>If we changed how the default vcl is structured into having all of the logic in separate subroutines, we would in cases li<br><br><div class="gmail_quote">
On Mon, Sep 17, 2012 at 9:33 AM, Poul-Henning Kamp <span dir="ltr"><<a href="mailto:phk@phk.freebsd.dk" target="_blank">phk@phk.freebsd.dk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
Right now we go directly from vcl_hash{} to either vcl_hit{}, vcl_miss{}<br>
or vcl_pass{} depending on the outcome, and that means, amongst other<br>
things, that a VCL implementation of "PURGE" as to add code to all those.<br>
<br>
Furthermore we have some semi-magic things going on behind the scenes,<br>
such as grace and saint modes.<br>
<br>
Assume:<br>
<br>
* Backend fetches will be done by a different thread than the one<br>
processing the clients request.<br>
<br>
* vcl_hash{} always is followed by vcl_lookup{} and that the above mentioned<br>
logic gets moved to VCL, rather than C code.<br>
<br>
* vcl_hit{} goes away.<br>
<br>
Then a plausible default vcl_lookup{} could then look like:<br>
<br>
sub vcl_lookup {<br>
<br>
if (!obj) {<br>
// Miss: Start a background fetch in another thread<br>
// We come back here, when the status of that fetch<br>
// is available. (ie: we go on the waiting list)<br>
return (miss);<br>
}<br>
<br>
if (obj.ttl <= 0s &&<br>
obj.ttl > -req.grace && obj.ttl > -obj.grace) {<br>
// We have a grace situation<br>
if (!obj.busy) {<br>
// Nobody is fetching yet, so start a<br>
// background fetch<br>
background_fetch();<br>
}<br>
// NB: First client does not hang on the fetch<br>
return (deliver);<br>
}<br>
<br>
if (obj.ttl <= 0s) {<br>
// object is too old, fetch new one (see miss)<br>
return (miss);<br>
}<br>
<br>
return (deliver);<br>
}<br>
<br>
Possible customizations:<br>
<br>
Purging:<br>
<br>
if (req.request == "PURGE") {<br>
// We pressume access control was in vcl_recv{}<br>
return (purge);<br>
return (purge_all);<br>
}<br>
<br>
Prefetching:<br>
<br>
if (obj.ttl < 2s && !obj.busy) {<br>
// Close to deadline, pre-fetch<br>
background_fetch();<br>
}<br>
<br>
Don't stream, only deliver complete objects:<br>
<br>
if (!obj.complete) {<br>
wait_complete(obj);<br>
}<br>
<br>
I'm seriously pondering if saintmode should become a vmod at the same<br>
time, in which case it would probably look something like:<br>
<br>
sub vcl_lookup {<br>
if (obj.ttl <= 0s && !saint.ok(req.backend, req.url)) {<br>
return (deliviver)<br>
}<br>
}<br>
<br>
sub vcl_fetch {<br>
if (beresp.status >= 400) {<br>
saint.bad(bereq.backend, bereq.url);<br>
}<br>
}<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20<br>
phk@FreeBSD.ORG | TCP/IP since RFC 956<br>
FreeBSD committer | BSD since 4.3-tahoe<br>
Never attribute to malice what can adequately be explained by incompetence.<br>
<br>
_______________________________________________<br>
varnish-dev mailing list<br>
<a href="mailto:varnish-dev@varnish-cache.org">varnish-dev@varnish-cache.org</a><br>
<a href="https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev" target="_blank">https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev</a><br>
</font></span></blockquote></div><br><br clear="all"><div><br></div>-- <br>Martin Blix Grydeland<br>Varnish Software AS<br><br>
</div>