File storage problem.

[Cosimo Streppone]

In data 12 aprile 2010 alle ore 10:33:04, Sébastien FOUTREL  
<sfoutrel at bcstechno.com> ha scritto:

> We use Varnish 2.0.6 in a production site that do about 100Mb/s and 1K  
> Hits/s per cache.
>
> We found a strange phenomon when we added some 24H TTL to images.
>
> The file storage seems to have some garbage collection that deeply  
> impact performances when freeing some large amount of cache.
>
> We initialy used 8Gb file storage and encountered some GC at about 80%  
> usage that freed almost all file storage (monitored with cacti using  
> sm_balloc,sm_bfree values).
>
> We tried on one of our cache to test with 32GB and 64GB files but same  
> happened even worse because on the 64GB file it did the GC at 30GB and  
> freed until sm_balloc reached 2GB.
>
> We do not understand what is that and and to control it so if someone as  
> ideas around that problem ?

Hi Sébastien,

I'm not sure you have the same problem I experienced,
but it looks very much like the same.

Read this mail from the archives, especially the "CPU IO wait" paragraph,

   http://www.mail-archive.com/varnish-misc@projects.linpro.no/msg01571.html

it's very old as it refers to 1.1.2, but as soon as we switched to
the "malloc" storage as suggested there, all iowait/load problems
magically disappeared. See also here:

   http://varnish-cache.org/wiki/Performance

We're not using any swap, as we have servers with enough ram to keep
a useful share of the file set in memory.

I think it's worth a try.

-- 
Cosimo