2.1 plans

[Michael S. Fischer]

+1.  This is a very good idea for optimizing RAM utilization.

--Michael

On Jan 8, 2009, at 11:25 AM, Jeff Anderson wrote:

> Thanks for the response.
>
> I think inline page compression would be great too.  Store gzipped
> objects in the persistent cache and unzip if uncompressed objects are
> requested.
>
>
>
>
> On Jan 8, 2009, at 10:54 AM, Per Buer wrote:
>
>> Jeff Anderson wrote:
>>> I'd like to see individual object request statistics and a method to
>>> prefetch objects from the backend that are most frequently  
>>> requested.
>>> Perhaps also a way to prioritize objects into cache tiers based on
>>> frequency of requests.  So, for example, highly requested objects  
>>> are
>>> maintained in RAM and less frequently requested objects are cached  
>>> to
>>> disk.
>>
>> Your operating system already does this today with Varnish. Squid
>> tries
>> to maintain a two tier cache hierarchy without success.
>>
>>> If persistent storage is on its way maybe a method to assign
>>> priority to large disk cache volumes versus memory regions.
>>
>> Noted.
>>
>>> It might  be nice to have a distributed and/or tiered cache model
>>> where a single
>>> master has a very large cache and potentially very long grace  
>>> ability
>>> where objects can exist even if stale.  That master in turn could
>>> host
>>> frontend caches that communicate  efficiently to the master cache  
>>> and
>>> also have a tiered internal object priority.
>>
>> I believe most of this can be achieved today. Stale objects will
>> hopefully reach the 2.0 series before the 2.1 revolutions - at least
>> as
>> a patch, I hope.
>>
>>>
>>> Thanks,
>>> --Jeff
>>>
>>>
>>> On Jan 8, 2009, at 2:29 AM, Tollef Fog Heen wrote:
>>>
>>>> Hi,
>>>>
>>>> a short while before Christmas, I wrote up a small document  
>>>> pointing
>>>> to
>>>> what I would like to get into 2.1 and when I'd like milestones to
>>>> happen.  This is a suggestion, I'm open to ideas and comments on
>>>> both
>>>> feature set as well as if my guesstimates for dates is completely
>>>> off:
>>>>
>>>> Varnish 2.1 release plan
>>>>
>>>> The theme for Varnish 2.1 is "scalability", particularly trying to
>>>> address the needs of sites like finn.no which has a lot of objects
>>>> and
>>>> where priming the cache takes a long time, leading to long periods
>>>> of
>>>> higher load on the backend servers.
>>>>
>>>> The main feature is persistent storage, see
>>>> http://varnish.projects.linpro.no/wiki/ 
>>>> ArchitecturePersistentStorage
>>>> for design notes. Another important scalability feature is a new
>>>> lockless hash algorithm which scales much better than the current
>>>> one.  Poul-Henning already has an implementation of this in the
>>>> tree,
>>>> but it's still fresh.
>>>>
>>>> Minor features which would be nice to get in are:
>>>>
>>>> * Web UI, showing pretty graphs as well as allowing easy
>>>> configuration
>>>> of a cluster of Varnish machines.
>>>>
>>>> * Expiry randomisation.  This reduces the "lemmings" effect where
>>>> you
>>>> end up with a many objects with almost the same TTL (typically on
>>>> startup) which then expire at the same time.  The feature will  
>>>> allow
>>>> you to set the TTL to plus/minus X %.
>>>>
>>>> * Dynamic, user-defined counters that can be read and written from
>>>> VCL
>>>>
>>>> * Forced purges, where a thread walks the list of purged objects  
>>>> and
>>>> removes them.
>>>>
>>>> The schedule
>>>> ------------
>>>>
>>>> Alphas:
>>>> - 2009-01-15: New hash algorithm working
>>>> - 2009-02-15: Web UI
>>>> - 2009-03-15: Persistent storage
>>>> Beta:
>>>> - 2009-04-01: Feature complete
>>>> Release
>>>> - 2009-05-20: Release candidate
>>>> - 2009-05-01: No release critical bugs left
>>>> - 2009-05-10: Release
>>>>
>>>> -- 
>>>> Tollef Fog Heen
>>>> Redpill Linpro -- Changing the game!
>>>> t: +47 21 54 41 73
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at projects.linpro.no
>>>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>>
>>> --Jeff
>>> jeff at funnyordie.com
>>>
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at projects.linpro.no
>>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>>
>>
>> -- 
>> Per Buer - Leder Infrastruktur og Drift - Redpill Linpro
>> Telefon: 21 54 41 21 - Mobil: 958 39 117
>> http://linpro.no/ | http://redpill.se/
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at projects.linpro.no
>> http://projects.linpro.no/mailman/listinfo/varnish-misc
>
> --Jeff
> jeff at funnyordie.com
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc