2.1 plans

[Per Buer]

Jeff Anderson wrote:
> I'd like to see individual object request statistics and a method to  
> prefetch objects from the backend that are most frequently requested.   
> Perhaps also a way to prioritize objects into cache tiers based on  
> frequency of requests.  So, for example, highly requested objects are  
> maintained in RAM and less frequently requested objects are cached to  
> disk.  

Your operating system already does this today with Varnish. Squid tries
to maintain a two tier cache hierarchy without success.

> If persistent storage is on its way maybe a method to assign  
> priority to large disk cache volumes versus memory regions.

Noted.

> It might  be nice to have a distributed and/or tiered cache model where a single  
> master has a very large cache and potentially very long grace ability  
> where objects can exist even if stale.  That master in turn could host  
> frontend caches that communicate  efficiently to the master cache and  
> also have a tiered internal object priority.

I believe most of this can be achieved today. Stale objects will
hopefully reach the 2.0 series before the 2.1 revolutions - at least as
a patch, I hope.

> 
> Thanks,
> --Jeff
> 
> 
> On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:
> 
>> Hi,
>>
>> a short while before Christmas, I wrote up a small document pointing  
>> to
>> what I would like to get into 2.1 and when I'd like milestones to
>> happen.  This is a suggestion, I'm open to ideas and comments on both
>> feature set as well as if my guesstimates for dates is completely off:
>>
>> Varnish 2.1 release plan
>>
>> The theme for Varnish 2.1 is "scalability", particularly trying to
>> address the needs of sites like finn.no which has a lot of objects and
>> where priming the cache takes a long time, leading to long periods of
>> higher load on the backend servers.
>>
>> The main feature is persistent storage, see
>> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
>> for design notes. Another important scalability feature is a new
>> lockless hash algorithm which scales much better than the current
>> one.  Poul-Henning already has an implementation of this in the tree,
>> but it's still fresh.
>>
>> Minor features which would be nice to get in are:
>>
>> * Web UI, showing pretty graphs as well as allowing easy configuration
>>  of a cluster of Varnish machines.
>>
>> * Expiry randomisation.  This reduces the "lemmings" effect where you
>>  end up with a many objects with almost the same TTL (typically on
>>  startup) which then expire at the same time.  The feature will allow
>>  you to set the TTL to plus/minus X %.
>>
>> * Dynamic, user-defined counters that can be read and written from VCL
>>
>> * Forced purges, where a thread walks the list of purged objects and
>>  removes them.
>>
>> The schedule
>> ------------
>>
>> Alphas:
>> - 2009-01-15: New hash algorithm working
>> - 2009-02-15: Web UI
>> - 2009-03-15: Persistent storage
>> Beta:
>> - 2009-04-01: Feature complete
>> Release
>> - 2009-05-20: Release candidate
>> - 2009-05-01: No release critical bugs left
>> - 2009-05-10: Release
>>
>> -- 
>> Tollef Fog Heen
>> Redpill Linpro -- Changing the game!
>> t: +47 21 54 41 73
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
> 
> --Jeff
> jeff at funnyordie.com
> 
> 
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc


-- 
Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
Telefon: 21 54 41 21 - Mobil: 958 39 117
http://linpro.no/ | http://redpill.se/

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20090108/3721977f/attachment-0003.pgp>