Accept-Encoding and Vary header issue

[Igor Minar]

In an attempt to remove all the noise, I left the "if" in my snippet
and made things more confusing.

My app _always_ sends: "Vary: User-Agent", when the correct header
should be "Vary: Accept-Encoding,User-Agent".

However, since setting User-Agent as Vary is a terrible terrible
thing, I normalize the user agent and then handle the variations via
vcl_hash. I rewrite the Vary header with "Accept-Encoding" and would
like to let Varnish do the right thing, but that doesn't work.

For now I just moved to using vcl_hash for handling all the variations
and I remove Vary header in vcl_fetch. That works well.

/i

On Sat, Jul 24, 2010 at 1:46 PM, Laurence Rowe <l at lrowe.co.uk> wrote:
> Use varnishlog to see what is going on here. My bet would be on the
> backend not setting any Vary at all, so ``set beresp.http.Vary =
> "Accept-Encoding";`` never gets called. What happens when you make
> that call unconditionally?
>
> Laurence
>
> On 21 July 2010 17:40, Igor Minar <iiminar at gmail.com> wrote:
>> Hi there,
>>
>> I came across this interesting Varnish 2.1.2 behavior that I think is a bug.
>>
>> vcl:
>>
>> sub vcl_recv {
>>  call normalize_accept_encoding;
>>  ...
>> }
>>
>>
>> sub vcl_fetch {
>>  ...
>>  if (beresp.http.Vary ~ ".+") {
>>    set beresp.http.Vary = "Accept-Encoding"; #the app doesn't do this
>>  }
>>  ...
>> }
>>
>>
>> If I make an "uncompressed request" (curl -O -v http://foo.com/bar)
>> I prime the cache with uncompressed variant of bar resource
>>
>> on subsequent uncompressed request, I get cache hit and the resource
>> is returned.
>>
>> if I then (before TTL) make a "compressed request" (curl -O -v -H
>> "Accept-Encoding:gzip"  http://foo.com/bar)
>> I'd expect to get cache miss and prime the cache with compressed
>> variant of bar resource,
>> but instead I get cache hit, and the uncompressed variant is returned.
>>
>> once TTL is achieved, if I make a compressed request, varnish
>> correctly fetches the compressed variant of the resources, primes the
>> cache with it and returns it.
>>
>> if I subsequently (before TTL) make an uncompressed request, varnish
>> correctly fetches the uncompressed variant from the backend and caches
>> it.
>>
>> everything is good until the TTL for the compressed variant is
>> reached, from that moment on varnish starts serving uncompressed
>> variant for both compressed and uncompressed requests, until the TTL
>> for the uncompressed variant is reached.
>>
>> My current workaround is to control variants manually via vcl_hash:
>>
>> sub vcl_hash {
>>  # consider different Accept-Encoding requests (normalized) as new variants
>>  set req.hash += req.http.Accept-Encoding;
>> }
>>
>> but I don't think that this should be necessary. Am I doing something
>> wrong or is this a bug?
>>
>> thanks,
>> Igor
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>>
>