[Varnish] #1150: Fast expiry_sleep causing panics

[Varnish]

#1150: Fast expiry_sleep causing panics
--------------------------------+-------------------------------------------
 Reporter:  jjordan             |        Type:  defect  
   Status:  new                 |    Priority:  normal  
Milestone:                      |   Component:  varnishd
  Version:  3.0.2               |    Severity:  critical
 Keywords:  expiry_sleep panic  |  
--------------------------------+-------------------------------------------
Description changed by kristian:

Old description:

> We were having an issue with SMA.Transient.g_alloc growing unbounded
> because we were getting stuff into it faster then the expiry thread could
> remove it.
> We turned down the expiry_sleep.  With expiry_sleep at 0.02 it keeps up
> with the Transient expiring such that it doesn't grow unbounded, but
> every couple minutes after the Transient store is empty we get the
> following panic:
>
> varnish> panic.show
> 200
> Last panic at: Sun, 03 Jun 2012 08:43:09 GMT
> Assert error in oc_getobj(), cache.h line 452:
>  Condition((oc->flags & (1<<1)) == 0) not true.
> thread = (cache-timeout)
> ident = Linux,2.6.18-308.4.1.el5,x86_64,-smalloc,-smalloc,-hcritbit,epoll
> Backtrace:
>  0x42c7a6: /usr/sbin/varnishd [0x42c7a6]
>  0x420a35: /usr/sbin/varnishd [0x420a35]
>  0x42ebac: /usr/sbin/varnishd [0x42ebac]
>  0x3a9de0677d: /lib64/libpthread.so.0 [0x3a9de0677d]
>  0x3a9cad325d: /lib64/libc.so.6(clone+0x6d) [0x3a9cad325d]
>

>
> varnish> param.show expiry_sleep
> 200
> expiry_sleep               0.020000 [seconds]
>                           Default is 1
>                           How long the expiry thread sleeps when there is
>                           nothing for it to do.
>
> varnish>
>
> If I set expiry_sleep any higher, it doesn't keep up and Transient store
> acts basically as a slow memory leak.
>
> We are running 3.0.2 on Centos 5.8, Kernel 2.6.18-308.4.1.el5, SMP x86_64
> GNU/Linux

New description:

 We were having an issue with SMA.Transient.g_alloc growing unbounded
 because we were getting stuff into it faster then the expiry thread could
 remove it.
 We turned down the expiry_sleep.  With expiry_sleep at 0.02 it keeps up
 with the Transient expiring such that it doesn't grow unbounded, but every
 couple minutes after the Transient store is empty we get the following
 panic:

 {{{
 varnish> panic.show
 200
 Last panic at: Sun, 03 Jun 2012 08:43:09 GMT
 Assert error in oc_getobj(), cache.h line 452:
  Condition((oc->flags & (1<<1)) == 0) not true.
 thread = (cache-timeout)
 ident = Linux,2.6.18-308.4.1.el5,x86_64,-smalloc,-smalloc,-hcritbit,epoll
 Backtrace:
  0x42c7a6: /usr/sbin/varnishd [0x42c7a6]
  0x420a35: /usr/sbin/varnishd [0x420a35]
  0x42ebac: /usr/sbin/varnishd [0x42ebac]
  0x3a9de0677d: /lib64/libpthread.so.0 [0x3a9de0677d]
  0x3a9cad325d: /lib64/libc.so.6(clone+0x6d) [0x3a9cad325d]



 varnish> param.show expiry_sleep
 200
 expiry_sleep               0.020000 [seconds]
                           Default is 1
                           How long the expiry thread sleeps when there is
                           nothing for it to do.

 varnish>
 }}}
 If I set expiry_sleep any higher, it doesn't keep up and Transient store
 acts basically as a slow memory leak.

 We are running 3.0.2 on Centos 5.8, Kernel 2.6.18-308.4.1.el5, SMP x86_64
 GNU/Linux

--

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1150#comment:1>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator