Using Varnish to accelerate CouchDB

[Michal Taborsky - Netretail Holding]

Hi Karel,

you will need some kind of cache invalidation. When you change the data 
in CouchDB, your application will have to notify Varnish the document 
has changed and should purge it form the cache. Next request for the 
same object will fetch the object from CouchDB and store the new version 
in Varnish. You do not need ETags for this. You only need to set correct 
caching headers that will tell Varnish to cache the object forever. But 
you definitely need the cache invalidation in your app.

Or, if you don't care Varnish will serve stale version of the document, 
you can cache it for some short period without the invalidation.

Michal

Dne 6.11.2010 11:18, Karel Minařík napsal(a):
> Hello,
>
> I've spent the last couple of days trying to figure out how to 
> accelerate CouchDB [http://couchdb.org/] with Varnish. I got kinda 
> lost and would like to ask for some help.
>
> CouchDB sends an ETag header in the request:
>
>   $ curl -I http://localhost:5984/test/abc1
>   ...
>   Etag: "3-6585b511acdd9a73040e673329369ff6"
>   Cache-Control: must-revalidate
>
> My original thought was that I could use this in Varnish to speed up 
> responses from Couch, because Varnish could "accumulate" requests to 
> the backend in certain time frame, and read cached object from memory, 
> as opposed from disk. (CouchDB view or CouchDB-Lucene queries send 
> ETag in their responses as well, so this would work very well accross 
> the whole database.)
>
> However, I can't find any definite information about if and how 
> Varnish uses ETags? Is this a bad/impossible approach in Varnish?
>
> In my tests, I got some *huge* speedup (with a default setup) when 
> querying Couch via Varnish: the mean response time dropped around 
> half, sometimes more (from ~ 2-3msec to ~0.5-1msec), and the requests 
> per second went up almost ten times (measured by simple ApacheBench 
> tests).
>
> I couldn't, though, make the cached response to expire by changing the 
> document in the database and thus changing Etag for its response. 
> (Which I consider strange given the "must-revalidate" header?)
>
> In the end, I suspect this is because Varnish has some default TTL 
> (120sec?) and that is what the acceleration is based on. Is that so? 
> (Again, I couldn't find any definite info about this in docs, or in 
> the default.vcl.)
>
> What would you consider to be the most effective Varnish configuration 
> for CouchDB? Is time-based caching (eg. setting the Expires or 
> Last-Modified header) the only option?
>
> Thanks for any help in advance!
>
> Karel
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>

-- 
Michal Táborský
chief systems architect
Netretail Holding, B.V.
http://www.nrholding.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.varnish-cache.org/pipermail/varnish-misc/attachments/20101106/0a4ea9a7/attachment-0001.html>