Varnish

[Poul-Henning Kamp]

In message <46DDFECE.3080004 at wheelhouse.org>, Jeff writes:

>1) The "Guru Meditation" error messages, while very Amiga-nostalgic, 
>aren't customer suitable, but appear to be hard-coded in 
>cache_synthetic.c.  If we want nice, pretty error messages, are we 
>basically on our own, or is there an imminent plan for this?

I belive its on our list somewhere.

>2) Since we have an extremely large hostname->backend map, we need to 
>choose the right one efficiently and dynamically on a per connection 
>basis; we cannot statically configure every possibility into a VCL file. 
>  How can we tie some sort of external lookup (pretty much any sort will 
>do) into VCL?

First of all, VCL is very efficient, so even very large maps in VCL
code will do well.

VCL also has an "include" facility, so you could machinegenerate
that part of your VCL program from your database.

Anyhow, what exactly is "extremely large" in this context ?

>3) The log files look like they are great for debugging obscure 
>complicated problems, but for day-to-day usage, we need something 
>similar to the squid_access format (timestamp, client IP, URL, status 
>code, fetch/cache status, bytes).  How would we approach this?

Did you miss the NCSA format writer ?

>4) We would like to limit the number of simultaneous open connections 
>from a single client IP to 10-16 or so to thwart certain types of 
>malicious crawlers that open them by the dozens, and kick back a 403 
>error to extra ones.  Is this possible with Varnish?

We have what it takes to implement this, what's missing is the VCL
access to the data.

>5) We need to If-Modified-Since: revalidate back to the origin server on 
>every request, even if 99% of the time it gets a 304 response, in order 
>to get log files on the back end that awstats can parse.  However, we 
>want to preserve Expires: and max-age values to pass along to the 
>client, so something as heavy-handed as setting the max TTL to 0 
>probably would not work.  I think this can be done in VCL, I just can't 
>seem to wrap my head around it.  What would be the best way for us to 
>handle that?

This one is tricky.

Our design assumption was that you would want to keep your backend
as much out of the loop as possible and use the varnish logfiles
for your traffic analysis.

Therefore, varnish will either request the object body unconditionally
or not bother the backend at all.

We have some room in the code where we could make this stuff more
flexible, but right now it is not on the todo list.

-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe    
Never attribute to malice what can adequately be explained by incompetence.