[Varnish] #1053: Persistent storage: space leakage

[Varnish]

#1053: Persistent storage: space leakage
----------------------+-----------------------------------------------------
 Reporter:  dumbbell  |        Type:  defect  
   Status:  new       |    Priority:  normal  
Milestone:            |   Component:  varnishd
  Version:  trunk     |    Severity:  major   
 Keywords:            |  
----------------------+-----------------------------------------------------
 With persistent storage, at some point in time, the silo will have the
 following layout:
 {{{
     |xxxxxxxESxxxx_|
 }}}

 where:
  * `S`: start of the segments list
  * `E`: end of the segments list
  * `x`: segments in between
  * `_`: unused space

 With such a situation, the `smp_open_segs` function
 (source:bin/varnishd/storage/storage_persistent.c) is responsible for
 finding free space. To do this, it drops the first elements of the list
 (starting from S) until it has "`free_reserve`" bytes of free space
 between `E` and the new `S`:
 {{{
     |xxxxxxxE___Sx_|
              ^^^ free_reserve
 }}}

 When the segments at the tail of the silo are all cleared and there's
 still not enough space, the function starts to reclaim space at the front
 of the silo, until it reaches the `free_reserve`:
 {{{
     |__SxxxxE______|
      ^^ free_reserve
 }}}

 It doesn't take into account the space freed at the tail.

 Unfortunately, when working on the tail of the silo, this function only
 considers the distance between `E` and the segment closest to the end of
 the silo. Therefore, it may found there's not enough space to satisfy
 `free_reserve` between those two points but there is between `E` and the
 end of the silo:
 {{{
     |xxxxxxxE____S_|
              ^^^^^^ free_reserve (but not between E and S)
 }}}

 In this special case, it wraps the list too early. And later, when the
 same situation occurs, it won't try to reclaim the space between the
 segment where the list wraps and the end of the silo:
 {{{
     |xxxxESxx______|
              ^^^^^^ leaked space
 }}}

 The bugfix (patch attached) consists of checking this situation before
 reclaiming space at the front of the silo.

-- 
Ticket URL: <https://www.varnish-cache.org/trac/ticket/1053>
Varnish <https://varnish-cache.org/>
The Varnish HTTP Accelerator