varnish incorrect back-end selection problem

[Guy Brodie]

Hi Everyone!

(sorry - re-post with subject line this time :-) )


I have a curious case of unusual backend selection.


To summarise,  I have two backend servers servicing content for two 'subsites',  eg:


www.company.com/subsite1 ->  serviced by backend 'server1'

www.company.com/subsite2 ->  serviced by backend 'server2'


My problem is with subsite 2.


It works:   when in my browser i directly enter the url "www.company.com/subsite2".

It doesn't work:   when i restart the browser and browse to www.company.com/subsite1 (handled by server1),  *then* click a link to subsite2 .   At this point,  packet captures show the request still going to server1, despite the vcl clearly defining subsite2 should be handled by server2 (and as confirmed by subsite2 working when no previous browsing history has occurred on the company site).


Furthermore,  when the request for subsite2 is incorrectly sent to server1,  the website generates a redirect. This redirect is to the correct URL that should go to server2,  but is again sent to server1, and a redirect loop between the browser and server1 occurs because varnish insists on sending the request to server1 instead of server2.


Curiously,  the varnishlog has no record of this looping behaviour at all!    I  vaguely remember reading that a malformed request does not make it into the varnishlog output?


So I'm left wondering about what is the difference between hitting the subsite2 URL directly versus navigating to it via subsite1.   Is this behaviour tied to:

* cookie based CMS being used on subsite1 - does varnish use cookies to determine back-end re-use or something?  I didn't think so...

* http tcp session re-use - the browser is re-using a single TCP session when this happens,  but I thought varnish inspects every request regardless - i.e. it doesn't blindly pass an existing TCP session from the browser through to the backend server it had previously selected…

* perhaps a malformed request or something…



Our environment is a reasonably simple one I think.  We basically use regex on URL and host to determine which backend to use. We use minimal caching functions - just for static objects , and even that is a token effort.  Varnish serves primarily as a 'URL router' for us.      We are running varnishd (varnish-3.0.0 revision 3bd5997)


So I'm kind of stuck!     I realise I haven't posted any config.  At this stage,  I'm hoping someone can help with confirming my understanding of varnish behaviour regards cookies, session re-use and malformed request handling as above.  But any ideas on how to start debugging this one will be greatly appreciated!


Thankyou!


Guy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20111110/83c75779/attachment-0003.html>