[Varnish] #1077: "Assert error in smp_open_segs" panic with persistent storage

[Varnish]

#1077: "Assert error in smp_open_segs" panic with persistent storage
-------------------+--------------------------------------------------------
 Reporter:  acdha  |        Type:  defect  
   Status:  new    |    Priority:  normal  
Milestone:         |   Component:  varnishd
  Version:  3.0.2  |    Severity:  critical
 Keywords:         |  
-------------------+--------------------------------------------------------
 Running on Ubuntu 10.04 using the varnish-cache.org repository, I've
 noticed two repeated solid crashes using large persistent cache files.

 The first sign of a problem will be this message:
 {{{
 varnishd[27978]: Child (6953) Panic message: Assert error in
 smp_oc_getobj(), storage_persistent_silo.c line 400:
   Condition((const void*)(o) >= (const void*)((sg->sc)->base) && (const
 void*)(o) < (const void *)((sg->sc)->base + (sg->sc)->mediasize)) not
 true.
 thread = (cache-worker)
 ident = Linux,2.6.32-341-ec2,x86_64,-spersistent,-smalloc,-hcritbit,epoll
 Backtrace:
   0x42f118: /usr/sbin/varnishd() [0x42f118]
   0x44c525: /usr/sbin/varnishd() [0x44c525]
   0x428986: /usr/sbin/varnishd(HSH_Lookup+0x3a6) [0x428986]
   0x415a79: /usr/sbin/varnishd() [0x415a79]
   0x419165: /usr/sbin/varnishd(CNT_Session+0x6d5) [0x419165]
   0x430aa8: /usr/sbin/varnishd() [0x430aa8]
   0x430f31: /usr/sbin/varnishd() [0x430f31]
   0x7f4825d909ca: /lib/libpthread.so.0(+0x69ca) [0x7f4825d909ca]
   0x7f4825aed70d: /lib/libc.so.6(clone+0x6d) [0x7f4825aed70d]
 sp = 0x7f3bc659b008 {
   fd = 12, id = 12, xid = 2073042236,
   client = 10.210.214.8 34709,
   step = STP_LOOKUP,
   handling = hash,
   err_code = 200, err_reason = (null),
   restarts = 0, esi_level = 0
   flags =
   bodystatus = 4
   ws = 0x7f3bc659b080 {
     id = "sess",
     {s,f,r,e} = {0x7f3bc659bc90,+1152,+65536,+65536},
   },
 }}}

 At this point, varnishd will appear to be running but each child will
 immediately fail and no requests will actually be processed:

 {{{
 varnishd[27978]: Child cleanup complete
 varnishd[27978]: child (7670) Started
 varnishd[27978]: Pushing vcls failed:
 CLI communication error (hdr)
 varnishd[27978]: Stopping Child
 varnishd[27978]: Child (7670) died signal=6
 varnishd[27978]: Child (7670) Panic message: Assert error in
 smp_open_segs(), storage_persistent.c line 239:
   Condition(sg1->p.offset != sg->p.offset) not true.
 thread = (cache-main)
 ident =
 Linux,2.6.32-341-ec2,x86_64,-spersistent,-smalloc,-hcritbit,no_waiter
 Backtrace:
   0x42f118: /usr/sbin/varnishd() [0x42f118]
   0x44a96d: /usr/sbin/varnishd() [0x44a96d]
   0x44aacb: /usr/sbin/varnishd() [0x44aacb]
   0x447f57: /usr/sbin/varnishd(STV_open+0x27) [0x447f57]
   0x42ddfa: /usr/sbin/varnishd(child_main+0xca) [0x42ddfa]
   0x4406ee: /usr/sbin/varnishd() [0x4406ee]
   0x441037: /usr/sbin/varnishd() [0x441037]
   0x7f4826eb56a2: /usr/lib/varnish/libvarnish.so(+0x96a2) [0x7f4826eb56a2]
   0x7f4826eb5cf8: /usr/lib/varnish/libvarnish.so(vev_schedule+0x88)
 [0x7f4826eb5cf8]
   0x4408d9: /usr/sbin/varnishd(MGT_Run+0x139) [0x4408d9]
 }}}

 (see https://gist.github.com/1515327 for more logs)

 This problem is persistent - I've found two ways to restore Varnish to
 service: increasing the persistent cache size or deleting the file so it
 can be recreated.

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1077>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator