2.1 plans

[Jeff Anderson]

I'd like to see individual object request statistics and a method to  
prefetch objects from the backend that are most frequently requested.   
Perhaps also a way to prioritize objects into cache tiers based on  
frequency of requests.  So, for example, highly requested objects are  
maintained in RAM and less frequently requested objects are cached to  
disk.  If persistent storage is on its way maybe a method to assign  
priority to large disk cache volumes versus memory regions.  It might  
be nice to have a distributed and/or tiered cache model where a single  
master has a very large cache and potentially very long grace ability  
where objects can exist even if stale.  That master in turn could host  
frontend caches that communicate  efficiently to the master cache and  
also have a tiered internal object priority.

Thanks,
--Jeff


On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:

>
> Hi,
>
> a short while before Christmas, I wrote up a small document pointing  
> to
> what I would like to get into 2.1 and when I'd like milestones to
> happen.  This is a suggestion, I'm open to ideas and comments on both
> feature set as well as if my guesstimates for dates is completely off:
>
> Varnish 2.1 release plan
>
> The theme for Varnish 2.1 is "scalability", particularly trying to
> address the needs of sites like finn.no which has a lot of objects and
> where priming the cache takes a long time, leading to long periods of
> higher load on the backend servers.
>
> The main feature is persistent storage, see
> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
> for design notes. Another important scalability feature is a new
> lockless hash algorithm which scales much better than the current
> one.  Poul-Henning already has an implementation of this in the tree,
> but it's still fresh.
>
> Minor features which would be nice to get in are:
>
> * Web UI, showing pretty graphs as well as allowing easy configuration
>  of a cluster of Varnish machines.
>
> * Expiry randomisation.  This reduces the "lemmings" effect where you
>  end up with a many objects with almost the same TTL (typically on
>  startup) which then expire at the same time.  The feature will allow
>  you to set the TTL to plus/minus X %.
>
> * Dynamic, user-defined counters that can be read and written from VCL
>
> * Forced purges, where a thread walks the list of purged objects and
>  removes them.
>
> The schedule
> ------------
>
> Alphas:
> - 2009-01-15: New hash algorithm working
> - 2009-02-15: Web UI
> - 2009-03-15: Persistent storage
> Beta:
> - 2009-04-01: Feature complete
> Release
> - 2009-05-20: Release candidate
> - 2009-05-01: No release critical bugs left
> - 2009-05-10: Release
>
> -- 
> Tollef Fog Heen
> Redpill Linpro -- Changing the game!
> t: +47 21 54 41 73
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc

--Jeff
jeff at funnyordie.com