Using Varnish to accelerate CouchDB

[Karel Minařík]

Hello,

I've spent the last couple of days trying to figure out how to  
accelerate CouchDB [http://couchdb.org/] with Varnish. I got kinda  
lost and would like to ask for some help.

CouchDB sends an ETag header in the request:

   $ curl -I http://localhost:5984/test/abc1
   ...
   Etag: "3-6585b511acdd9a73040e673329369ff6"
   Cache-Control: must-revalidate

My original thought was that I could use this in Varnish to speed up  
responses from Couch, because Varnish could "accumulate" requests to  
the backend in certain time frame, and read cached object from memory,  
as opposed from disk. (CouchDB view or CouchDB-Lucene queries send  
ETag in their responses as well, so this would work very well accross  
the whole database.)

However, I can't find any definite information about if and how  
Varnish uses ETags? Is this a bad/impossible approach in Varnish?

In my tests, I got some *huge* speedup (with a default setup) when  
querying Couch via Varnish: the mean response time dropped around  
half, sometimes more (from ~ 2-3msec to ~0.5-1msec), and the requests  
per second went up almost ten times (measured by simple ApacheBench  
tests).

I couldn't, though, make the cached response to expire by changing the  
document in the database and thus changing Etag for its response.  
(Which I consider strange given the "must-revalidate" header?)

In the end, I suspect this is because Varnish has some default TTL  
(120sec?) and that is what the acceleration is based on. Is that so?  
(Again, I couldn't find any definite info about this in docs, or in  
the default.vcl.)

What would you consider to be the most effective Varnish configuration  
for CouchDB? Is time-based caching (eg. setting the Expires or Last- 
Modified header) the only option?

Thanks for any help in advance!

Karel