URL rewriting / ignoring query parameters?

[Tom Pepper]

HI Poul:

Thanks for what should have probably been a glaringly obvious  
answer.  I now have a somewhat more strange problem.

My disturbingly simple vcl_recv routine is presently:

backend default {
     set backend.host = "nyp-web-3.corp.razz.com";
     set backend.port = "80";
}

sub vcl_recv {

     set req.backend = default;
     set req.http.host = "www.razz.com";

     // change all /static/ requests into /mixer/ requests
     if (req.url ~ "/static/") {
        set req.url = regsub(req.url, "/static/", "/mixer/");
     }

     // strip query parameters from all swf requests (so they cache  
as a single object)
     if (req.url ~ "\.swf?.*") {
         set req.url = regsub(req.url, "\.swf?.*", "\.swf");
     }

     if (req.request != "GET" && req.request != "HEAD") {
         pipe;
     }

     if (req.http.Expect) {
         pipe;
     }

     if (req.http.Authenticate || req.http.Cookie) {
         pass;
     }

     lookup;

}

This would (to my eyes) appear to closely mirror the default example  
in vcf's manpage.  However, in practice, when running under this  
configuration, many distinct requests seem to retrieve the same  
document from the cache.  For example:

1) client requests http://varnish:10080/  -- varnishd returns / off  
the backend correctly.
2) client requests /css/banner.css - varnishd returns correct file / 
css/banner.css off of backend.
3) client requests /css/default.css - varnishd returns banner.css
4) client requests /images/blank.gif - varnishd returns banner.css

and so on.  commenting out the entire routine seems to get things  
functioning, but has the caveat (according to the log) that there's a  
fetch for every request, probably due to the fact that the browser  
presents a cookie used site-wide, which per the above config would  
seem to force a pass on every request.  am i doing something that  
confuses the hash algorithm?  i'm invoking currently as:

varnishd -a 192.168.10.100:10080 -T 192.168.10.100:10088 -f /etc/ 
varnish/main.vcl -s file,/cache/varnish_storage.bin,1G -g nobody -u  
nobody

i issue a url.purge .* before each test run.

Thanks again,
-Tom


On Sep 10, 2007, at 11:35 AM, Poul-Henning Kamp wrote:

> In message <DB38C99E-0E5F-4FD0-8505-3ED870F758CB at razz.com>, Tom  
> Pepper writes:
>
>> 1) log the request exactly as it came from the client.  We use these
>> logs to track which distinct widget in the wild was viewed.
>
> Varnish will alway record the request exactly as received.
>
>> 2) instruct varnish to ignore the query parameters and only cache one
>> instance of the swf for all of these requests.
>
> 	sub vcl_recv {
> 		set req.url = regsub(req.url, "?.*", "");
> 	}
>
> should do it.
>
> -- 
> Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
> phk at FreeBSD.ORG         | TCP/IP since RFC 956
> FreeBSD committer       | BSD since 4.3-tahoe
> Never attribute to malice what can adequately be explained by  
> incompetence.