caching/hiding 302 redirects

[Stroomer, Jeff]

Varnish folks,

Apologies for the repost, but I'm still stuck, and close to running out of ideas.

I'm trying to combine Varnish with a Tomcat  servlet  to make a fancy reverse proxy.  The idea is that Varnish forwards a URL to the servlet, which uses a 302 redirect to tell Varnish what the "real" URL should be.  For this to work I need Varnish to do two things: 1) cache the 302 redirects from the servlet, and also 2) hide the redirect from the client.

Rob S pointed me to the following trac ticket, which addresses exactly this problem:

https://www.varnish-cache.org/trac/ticket/411

But when I use the solution described in the ticket, Varnish errors out. (To be precise, it gets into my vcl_deliver, and then dies with an INCOMPLETE AT error message.)   Here is the relevant part of my default.vcl:

sub vcl_hit {
  if (obj.http.X-Magic-Redirect == "1") {
    set req.url = obj.http.Location;
    restart;
  }
}
sub vcl_fetch {
  if (beresp.status == 302) {
    set beresp.http.X-Magic-Redirect = "1";
    return(deliver);
  }
}
sub vcl_deliver {
  if (resp.http.X-Magic-Redirect == "1") {
    unset resp.http.X-Magic-Redirect;
    restart;
  }
  return(deliver);
}

So far as I can tell, I have duplicated the solution from the ticket exactly (except for vcl syntax updates corresponding to the evolution of varnish since the ticket was filed).  Here's what I see in varnishlog:

    0 CLI          - Rd ping
    0 CLI          - Wr 200 PONG 1315497816 1.0
   12 SessionOpen  c ::1 57092 :6081
   12 ReqStart     c ::1 57092 611991685
   12 RxRequest    c GET
   12 RxURL        c /create
   12 RxProtocol   c HTTP/1.1
   12 RxHeader     c Host: localhost:6081
   12 RxHeader     c User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.22) Gecko/20110905 Ubuntu/10.10 (maverick) Firefox/3.6.22
   12 RxHeader     c Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
   12 RxHeader     c Accept-Language: en-us,en;q=0.5
   12 RxHeader     c Accept-Encoding: gzip,deflate
   12 RxHeader     c Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
   12 RxHeader     c Keep-Alive: 115
   12 RxHeader     c Connection: keep-alive
   12 RxHeader     c Cache-Control: max-age=0
   12 VCL_call     c recv
   12 VCL_return   c lookup
   12 VCL_call     c hash
   12 VCL_return   c hash
   12 VCL_call     c miss
   12 VCL_return   c fetch
   14 BackendOpen  b default 127.0.0.1 37812 127.0.0.1 8080
   12 Backend      c 14 default default
   14 TxRequest    b GET
   14 TxURL        b /urlredirector/#a02/create
   14 TxProtocol   b HTTP/1.1
   14 TxHeader     b Host: localhost:6081
   14 TxHeader     b User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.22) Gecko/20110905 Ubuntu/10.10 (maverick) Firefox/3.6.22
   14 TxHeader     b Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
   14 TxHeader     b Accept-Language: en-us,en;q=0.5
   14 TxHeader     b Accept-Encoding: gzip,deflate
   14 TxHeader     b Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
   14 TxHeader     b X-cip: ::1
   14 TxHeader     b X-Bucket: a02
   14 TxHeader     b X-Varnish: 611991685
   14 RxProtocol   b HTTP/1.1
   14 RxStatus     b 302
   14 RxResponse   b Moved Temporarily
   14 RxHeader     b Server: Apache-Coyote/1.1
   14 RxHeader     b Location: http://redirected-localhost:6081/production_webapp/production_env/create
   14 RxHeader     b Max-Age: 300
   14 RxHeader     b Content-Type: text/html
   14 RxHeader     b Content-Length: 86
   14 RxHeader     b Date: Thu, 08 Sep 2011 16:03:38 GMT
   12 TTL          c 611991685 RFC 120 1315497818 0 0 0 0
   12 VCL_call     c fetch
   12 VCL_return   c deliver
   12 ObjProtocol  c HTTP/1.1
   12 ObjStatus    c 302
   12 ObjResponse  c Moved Temporarily
   12 ObjHeader    c Server: Apache-Coyote/1.1
   12 ObjHeader    c Location: http://redirected-localhost:6081/production_webapp/production_env/create
   12 ObjHeader    c Max-Age: 300
   12 ObjHeader    c Content-Type: text/html
   12 ObjHeader    c Date: Thu, 08 Sep 2011 16:03:38 GMT
   12 ObjHeader    c X-Magic-Redirect: 1
   14 Length       b 86
   14 BackendReuse b default
    0 Debug        - "INCOMPLETE AT: cnt_deliver(196)"
    0 CLI          - Rd vcl.load "boot" ./vcl.1P9zoqAU.so
    0 CLI          - Wr 200 Loaded "./vcl.1P9zoqAU.so" as "boot"
    0 CLI          - Rd vcl.use "boot"
    0 CLI          - Wr 200
    0 CLI          - Rd start
    0 Debug        - "Acceptor is epoll"
    0 CLI          - Wr 200
    0 WorkThread   - 0x7f587a7f1be0 start
    0 WorkThread   - 0x7f5879ff0be0 start
    0 WorkThread   - 0x7f58797efbe0 start
    0 WorkThread   - 0x7f5878feebe0 start
    0 WorkThread   - 0x7f58787edbe0 start
    0 WorkThread   - 0x7f5877fecbe0 start
    0 WorkThread   - 0x7f58777ebbe0 start
    0 WorkThread   - 0x7f5876feabe0 start
    0 WorkThread   - 0x7f58767e9be0 start
    0 WorkThread   - 0x7f5875fe8be0 start

So far as I can tell, my servlet issues the 302 redirect, which gets me into my vcl_fetch subroutine.  vcl_fetch gets me into vcl_deliver, which is where I try to do a restart.  If, in my vcl_fetch, I do a restart instead of a return(deliver), then everything works beautifully (but, of course, then  I don't get my 302 redirect cached).

I'm running Varnish 2.1.3 under Ubuntu 10.10, but see the same problem under Varnish 2.1.4 on Ubuntu 11.04.  Thanks in advance for the help.

Jeff


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110908/a02e6b01/attachment-0003.html>