caching of request with cookies

[Adrian Otto]

Stewart,

I disagree. If the cached content is the same between various users with different cookie values, the cached content should be normalized so that a single cached version can shared by subsequent visitors regardless what the cookie is set to. If you use the approach you described, you will end up with numerous cached copies of the same content in this case. That is useful in the case where content differs for every individual user, but I'd argue that it would be much better to use browser cached for that situation, not a caching proxy. Simply set sensible Cache-Control: max-age=X values. The real benefit of a caching reverse proxy is when numerous users benefit from getting a common (normalized) version of the cached content.

Typically only dynamically generated content needs the cookies in it at all, so if you are only trying to cache static content that does not need any cookies, you could use an approach like the one shown below. Sometimes you want responses with certain response codes to be cached regardless of a cookie value, like this:

vcl_fetch {
        if(obj.status == 404) {
                if (obj.http.Set-Cookie) {
                        remove obj.http.Set-Cookie;
                }
                set obj.ttl = 120s;
                return(deliver);
        }
}

I use the VCL above to cache 404 requests for 2 minutes. I do the same thing for 301, 302, etc. You might also want to do it just for certain file types. This one strips cookies from, and stores a single copy of JPG files:

vcl_fetch {
        if(req.url ~ "*.\(jpg|JPG)$") {
                if (obj.http.Set-Cookie) {
                        remove obj.http.Set-Cookie;
                }
                return(deliver);
        }
}

As long as you have not added the cookie to the hash key, you will get a cache hit no matter what the cookie value is set to. This way you don't need to strip it in vcl_recv. You get a version without a cookie returned when you get a hit on the cache lookup.

Adrian

On Sep 23, 2010, at 2:22 AM, Stewart Robinson wrote:

> Whilst I don't think Varnish should change the default behaviour
> because most sites use cookies to pass around session based
> information which evolved after the http spec was written I think you
> would probably just add the cookie to your vcl_hash function and
> 
> change this in vcl_recv
>     if (req.http.Authorization || req.http.Cookie) {
>         /* Not cacheable by default */
>         return (pass);
>    }
> 
> to
>     if (req.http.Authorization) {
>         /* Not cacheable by default */
>         return (pass);
>    }
> 
> Perhaps your vcl_hash could have this appended to it
> 
>     if (req.http.Cookie) {
>         set req.hash += req.http.Cookie;
>     }
> 
> I haven't tried this but I don't see why you wouldn't have individual
> cache entries for each cookie combo. I think this would flood your
> cache if you use the cookie for session info, but if you just keep
> simple values in cookies then this should be fine.
> 
> Stewart Robinson
> fullfatthings.com/
> 
> 
> On 23 September 2010 09:59, Harm Verhagen <harm.verhagen at gmail.com> wrote:
>> Hi,
>> 
>> I noticed that varnish out of the box doesn't cache requests that carry a
>> cookie.
>> 
>> This seems a bit too conservative to me.
>> According to the http specs the mere existence of a cookie should not
>> influence cachebility.  In the content of the response depends on the
>> cookie, the server should tell in the response that the contents are not
>> cacheble.
>> The Vary: Cookie header  exists for that reason, no ?
>> 
>> In the current default configuration if you put varnish in front of a
>> 'well-behaving' (as in "adding all the required caching related http
>> headers")  website.  Most cacheble content does _not_ get cached. Unless you
>> do some non obvious (to me :) ) configuration of varnish.
>> 
>> What are the thoughts on this list about moving the default varnish
>> configuration closer to the http specification, regarding caching of request
>> where the client sends a cookie (and probably leading to problems on
>> websites that do _not_ use http headers correctly)
>> 
>> That said,
>> Does anyone has a pointer to a varnish configuration that does the thing I
>> described above.  I don't mean an example where vanish just strips off
>> cookies of  images/js/css files  request. (I found plenty examples of that
>> scenario) But leaving cookies as is, but just look at the responses to
>> decide if the content should be cached or not. So that varnish can
>> selectively cache html pages based on their response, where all requests
>> potentially have cookies).
>> 
>> 
>> Mvg,
>> Harm Verhagen
>> 
>> _______________________________________________
>> varnish-dev mailing list
>> varnish-dev at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-dev
>> 
> 
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-dev