2.1 plans

[Jeff Anderson]

Thanks for the response.

I think inline page compression would be great too.  Store gzipped  
objects in the persistent cache and unzip if uncompressed objects are  
requested.




On Jan 8, 2009, at 10:54 AM, Per Buer wrote:

> Jeff Anderson wrote:
>> I'd like to see individual object request statistics and a method to
>> prefetch objects from the backend that are most frequently requested.
>> Perhaps also a way to prioritize objects into cache tiers based on
>> frequency of requests.  So, for example, highly requested objects are
>> maintained in RAM and less frequently requested objects are cached to
>> disk.
>
> Your operating system already does this today with Varnish. Squid  
> tries
> to maintain a two tier cache hierarchy without success.
>
>> If persistent storage is on its way maybe a method to assign
>> priority to large disk cache volumes versus memory regions.
>
> Noted.
>
>> It might  be nice to have a distributed and/or tiered cache model  
>> where a single
>> master has a very large cache and potentially very long grace ability
>> where objects can exist even if stale.  That master in turn could  
>> host
>> frontend caches that communicate  efficiently to the master cache and
>> also have a tiered internal object priority.
>
> I believe most of this can be achieved today. Stale objects will
> hopefully reach the 2.0 series before the 2.1 revolutions - at least  
> as
> a patch, I hope.
>
>>
>> Thanks,
>> --Jeff
>>
>>
>> On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:
>>
>>> Hi,
>>>
>>> a short while before Christmas, I wrote up a small document pointing
>>> to
>>> what I would like to get into 2.1 and when I'd like milestones to
>>> happen.  This is a suggestion, I'm open to ideas and comments on  
>>> both
>>> feature set as well as if my guesstimates for dates is completely  
>>> off:
>>>
>>> Varnish 2.1 release plan
>>>
>>> The theme for Varnish 2.1 is "scalability", particularly trying to
>>> address the needs of sites like finn.no which has a lot of objects  
>>> and
>>> where priming the cache takes a long time, leading to long periods  
>>> of
>>> higher load on the backend servers.
>>>
>>> The main feature is persistent storage, see
>>> http://varnish.projects.linpro.no/wiki/ArchitecturePersistentStorage
>>> for design notes. Another important scalability feature is a new
>>> lockless hash algorithm which scales much better than the current
>>> one.  Poul-Henning already has an implementation of this in the  
>>> tree,
>>> but it's still fresh.
>>>
>>> Minor features which would be nice to get in are:
>>>
>>> * Web UI, showing pretty graphs as well as allowing easy  
>>> configuration
>>> of a cluster of Varnish machines.
>>>
>>> * Expiry randomisation.  This reduces the "lemmings" effect where  
>>> you
>>> end up with a many objects with almost the same TTL (typically on
>>> startup) which then expire at the same time.  The feature will allow
>>> you to set the TTL to plus/minus X %.
>>>
>>> * Dynamic, user-defined counters that can be read and written from  
>>> VCL
>>>
>>> * Forced purges, where a thread walks the list of purged objects and
>>> removes them.
>>>
>>> The schedule
>>> ------------
>>>
>>> Alphas:
>>> - 2009-01-15: New hash algorithm working
>>> - 2009-02-15: Web UI
>>> - 2009-03-15: Persistent storage
>>> Beta:
>>> - 2009-04-01: Feature complete
>>> Release
>>> - 2009-05-20: Release candidate
>>> - 2009-05-01: No release critical bugs left
>>> - 2009-05-10: Release
>>>
>>> -- 
>>> Tollef Fog Heen
>>> Redpill Linpro -- Changing the game!
>>> t: +47 21 54 41 73
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at projects.linpro.no
>>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>
>> --Jeff
>> jeff at funnyordie.com
>>
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
>
> -- 
> Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
> Telefon: 21 54 41 21 - Mobil: 958 39 117
> http://linpro.no/ | http://redpill.se/
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc

--Jeff
jeff at funnyordie.com