Conditional caching question

[David Pratt]

Hi. Sorry for not getting back sooner. The use case I have is have 
backend determine when frequency reaches a threshold and set ttl 
dynamically based on the rate. My goal is to tune cache using rates and 
ttl values to ensure only most requested makes it into the cache as 
opposed to adding 'bulk'. I am simply experimenting at this stage but 
believe it can result in a sort of 'smart caching' to maximize the 
benefit of the cache for content in demand while minimizing the overall 
cache resource to deliver the resources.

I am sure that somewhere here is an algorithm for that can provide some 
decent cache regulation for utilization and size. Its all a bit of a 
balancing act with the responsible for the backend as well I realize. 
Many thanks.

Regards
David


Barry Abrahamson wrote:
> On Jun 2, 2008, at 8:41 AM, Poul-Henning Kamp wrote:
> 
>> In message <4841723E.2020109 at eastlink.ca>, David Pratt writes:
>>
>>> Hi. In most cases, I want a request to be passed to a backend where it
>>> will be handled by server. If frequency is high, however; I want to add
>>> the object to varnish cache and have varnish handle it. I am not worried
>>> about a mechanism to keeping track of frequency of requests. Question is
>>> what is available to me to add an object/path to the varnish cache if it
>>> was originally passed?
>>
>> I wouldn't say that your way of using varnish is backwards relative
>> to the design objectives, but you do come close, since we assumed
>> caching by default, and pass as exception, rather than the other
>> way around.
> 
> We do this on WordPress.com to avoid filling our caches with 
> infrequently requested data.  The way we handle it is when an object 
> reaches a certain req/sec threshold, we send a header from the backend 
> and then have varnish configured to only insert objects into the cache 
> which contain this custom header.  Based on phk's reply, I guess we are 
> using varnish in a somewhat backwards manner as well, since we assume 
> pass as the detault, insert as the exception.
> 
> This used to work in 1.0.3.  I have started to look into upgrading to 
> trunk, and it doesn't seem to work so well anymore.  It looks like the 
> first time the URL is requested, if it is passed because it hasn't 
> reached that threshold and the header hasn't been set, all subsequent 
> requests are automatically "pass" ed.  These show up as "Cache hits for 
> pass" in varnishstat.  Any way around this?
> 
> 
> -- 
> Barry Abrahamson | Systems Wrangler | Automattic
> Blog: http://barry.wordpress.com
> 
> 
> 
> 
> 
> 
>