Varnish computes Content-Length and set it to 0 for one of my URL!

[zabrane Mikael]

Hi Ask,

Which part of the log file do you want exactly (as a company, we've
privacy stuffs to respect)?

But the problem can be easily reproduced. Here's an example:
$ cat bug.vcl
backend squid {
  .host = "127.0.0.1";
  .port = "3128";
}
backend cache {
  .host = "127.0.0.1";
  .port = "7676";
}
sub vcl_recv {
  // set default backend
  set req.backend = squid;
  // set default proto
  set req.proto = "HTTP/1.0";
  [...]
}

1) suppose squid is up and running, and used as a Varnish backend
# /etc/init.d/squid start

2) start Varnish:
# varnishd -n foo -f bug.vcl -s malloc,512M -a :6081
storage_malloc: max size 512 MB.
Using old SHMFILE

3) Test with CURL:
3.1) Getting ressource directly from live web without Varnish :

$ curl -0 -v -I --no-keepalive http://www.groupama.fr/
HTTP/1.1 200 OK
Connection: close
Date: Fri, 06 Aug 2010 12:42:24 GMT
Content-Type: text/html; charset=UTF-8
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Set-Cookie: jsessionid=McDQbrxZf2xTWGbzVGZ3G8GcG7hvMWv8Kv8Dk1PVpm3kQZty23yz!204719863;
path=/
[...]

3.2) Using CURL throught Squid (without varnish) :

HTTP/1.0 200 OK
Date: Fri, 06 Aug 2010 13:00:57 GMT
Content-Type: text/html; charset=UTF-8
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Set-Cookie: jsessionid=McHJmT7mp3hk23V4LQl1GkzQgMkS5JD7JSVhZk0hdyKFhM1shD2Z!204719863;
path=/
X-Cache: MISS from soron
X-Cache-Lookup: MISS from soron:3128
Via: 1.1 soron:3128 (squid/2.7.STABLE3)
Connection: keep-alive
Proxy-Connection: keep-alive
[...]

3.3) Now, using Varnish :
$ curl -0 -v -I --no-keepalive --proxy 127.0.0.1:6081 http://www.groupama.fr/

HTTP/1.1 200 OK
Content-Type: text/html; charset=UTF-8
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
Set-Cookie: jsessionid=McGpGDynT5phSP7QFlQ6Q8Wxqzk1vQTNpLcG210s1Sdp6QwD5wHb!204719863;
path=/
X-Cache: MISS from lyes-desktop
X-Cache-Lookup: MISS from lyes-desktop:3128
Via: 1.1 soron:3128 (squid/2.7.STABLE3)
Content-Length: 48930
Date: Fri, 06 Aug 2010 12:58:46 GMT
X-Varnish: 1996041575
Age: 7
Via: 1.1 varnish
Connection: close
[...]

Notice that for scenario 3.2, Squid replies with HTTP/1.0 (as
requested by "curl -0").
For scenario 3.3, Varnish reply with HTTP/1.1.

This is half of the problem. In my first email pointing out this bug,
our caching backend (running on port 7676) replies correctly:

$ curl -0 -v -I --no-keepalive --proxy 127.0.0.1:7676 http://www.groupama.fr/
Date	Tue, 15 Jun 2010 09:18:29 GMT
Content-Type	text/html; charset=UTF-8
X-Powered-By	ASP.NET
Set-Cookie	jsessionid=MXFJmqLZ2cwnR1gyFhVDvzG59zx2KcP2vKcg6lNP4Gnz5nVky4D1!1556069365;
path=/
X-INEODEV-CRAWL-UUID	d4807176-43f8-4547-89a9-5847bab5c326
X-INEODEV-PAGE-UUID	1a6c0abd-44dd-4c32-afc5-b8e8a7f72099
X-INEODEV-RESOURCE-LEVEL	primary

Finally, accessing our caching backend through Varnish failed (notice
the Content-Length: 0):
$ curl -0 -v -I --no-keepalive --proxy 127.0.0.1:6081 http://www.groupama.fr/

Content-Type	text/html; charset=UTF-8
X-Powered-By	ASP.NET
Set-Cookie	jsessionid=MXFJmqLZ2cwnR1gyFhVDvzG59zx2KcP2vKcg6lNP4Gnz5nVky4D1!1556069365;
path=/
X-INEODEV-CRAWL-UUID	d4807176-43f8-4547-89a9-5847bab5c326
X-INEODEV-PAGE-UUID	1a6c0abd-44dd-4c32-afc5-b8e8a7f72099
X-INEODEV-RESOURCE-LEVEL	primary
Content-Length	0        <=====
Date	Fri, 06 Aug 2010 13:09:18 GMT
X-Varnish	544081384
Age	0
Via	1.1 varnish
Connection	close


Hope this help.

--
Regards
Zabrane

2010/8/6 Ask Bjørn Hansen <ask at develooper.com>
>
> On Aug 6, 2010, at 0:30, zabrane Mikael wrote:
>
> > After days of researchs, this is a "Varnish" bug (Tiket 733: http://varnish-cache.org/ticket/733).
> > Varnish should leave the HTTP response as it is without computing any "Content-Length" in this case.
> >
> > Please, help guys? We're blocked for more than 4 weeks!
>
> You need to at least post full Varnish logs for anyone to be able to help you I suspect.   Most reasonable HTTP/1.1 implementations use either a Content-Length header or Transfer-Encoding to manage the message body length.
>
>
>  - ask
>
> --
> http://develooper.com/ - http://askask.com/