Weird backend selection policy

[Sergei Kononov]

Update:

Well, I spent almost whole day to figure out why this problem happend.
As I mentioned earlier the most strange thing is missing log entries
for 404 in varnishlog (but they are seen in tcpdump). I tried few
modifications to vcl_recv and noticed that if remove "pipe" from:
if ( req.url ~ "^/api/.*" ) {
     set req.backend = b2;
     set req.url = regsub(req.url, "^/api/", "/")
     set req.http.X-Cacheable = "false";
     pipe; # ---> remove it
  }

Everything start to working fine. It's really strange behavior and
almost unpredictable. I suppose there is some bug in parsing of VCL or
something related to vcl anyway.

On Wed, Mar 25, 2009 at 4:19 PM, Sergei Kononov
<sergei.kononov at gmail.com> wrote:
> Update: removing lookup does not fix this issue at all.
>
> b2 - is a round-robin director with 8 backends specified
> rpc - is simple backend, as media
>
> The problem really look like that all the available backends is in
> round-robin, request to rpc partially goes to b2 backends and vice
> versa. The most strange happened with logs, requests which goes to
> wrong backend can't be found neither in varnishncsa.log, nor
> varnishlog.
>
> Even running varnishlog -i RxURL,Backend (with out to terminal) does
> not show any false backends routes, but clients sees 404 and 500 for
> request.
>
> Really, really strange behavior.
>
> On Wed, Mar 25, 2009 at 3:40 PM, Sergei Kononov
> <sergei.kononov at gmail.com> wrote:
>> Hi,
>>
>> I noticed weird behavior of backend selection. Some request goes to
>> backend which is not suitable for them, as result I getting 404 or 500
>> errors, but I noticed that removing "lookup" in end of vcl_recv can
>> fix that problem, at least this problem does not occur so frequently
>> after that.
>>
>> Here is my vcl_recv:
>>
>> sub vcl_recv {
>>   set req.http.X-Cacheable = "true";
>>   set req.http.X-Forwarded-For=client.ip;
>>   set req.backend = default;
>>
>>   if ( req.url ~ "^/api/.*" ) {
>>      set req.backend = b2;
>>      set req.url = regsub(req.url, "^/api/", "/")
>>      set req.http.X-Cacheable = "false";
>>      pipe;
>>   }
>>
>>   if ( req.url ~ "^/rpc.*" ) {
>>      set req.backend = rpc;
>>      set req.http.X-Cacheable = "false";
>>      pipe;
>>   }
>>
>>   if ( req.url ~ "^/flv/.*" || req.url ~ "^/images.(128|320)/.*" ) {
>>      set req.backend = media;
>>   }
>>
>>   if ( req.url ~ "^/media.hq.flv/.*" ) {
>>      pipe;
>>   }
>>
>>   if ( req.url ~ "^/emit/.*" ) {
>>      set req.backend = default;
>>      pipe;
>>   }
>>
>>   if (req.http.Cache-Control ~ "no-cache") {
>>    pass;
>>   }
>>
>>  #if comment out lookup keyword varnish start to handle requests wierdly
>>  #lookup
>>  }
>>
>> And here is my vcl_fetch(not sure that it intended in this issue):
>> sub vcl_fetch {
>>  if (obj.http.Pragma ~ "no-cache" || obj.http.Cache-Control ~
>> "no-cache" || obj.http.Cache-Control ~ "private") {
>>   pass;
>>  }
>>
>>  if (! req.http.X-Cacheable ~ "true" ) {
>>   set obj.http.Cache-Control = "no-cache";
>>   set obj.http.Expires = "Tue, 01 Jan 2000 00:00:01 GMT";
>>   pass;
>>  }
>>
>>  set obj.ttl = 3600s;
>>
>> }
>>
>> I suppose that there is some error in vlc_ subs, but they seems ok.
>>
>> --
>> Sergei Kononov
>>
>
>
>
> --
> Sergei Kononov
>



-- 
Sergei Kononov