Possible to keep a single cache item for multiple request URLs using external logic?

[max goldberg]

Hello All,

I'm designing a sort of distributed file system used for file serving and
have been looking at multiple methods of reverse proxy caching static
content and I have to say Varnish looks like a really great product.

I have some questions as to how I can incorporate Varnish into my setup in
the most optimal way.

In the current implementation, I have multiple sites using the same content
system on the back end, using CNAME'd domains and a layer of application
logic to decide how to serve the file. In the file system, files are
completely unique in that even if a file exists across multiple sites, it
can be referred to in many ways while only being stored once.

For instance:
-----
app1.example.com is a CNAME for content.example.com
app2.example.com is a CNAME for content.example.com
etc.

A request comes in for http://app1.example.com/image.jpg.

content.example.com examines the request and responds with:

Location: http://server10.content.example.com/storage/real_image_name.jpg

Another request comes in for http://app2.example.com/test.jpg which
content.example.com identifies as being the same file as real_image_name.jpg
and responds with an identical Location header.
-----

I'm trying to keep my path translation in logic that would be external from
Varnish, as in some cases it's more extensive than a regular expression. The
whole ordeal can be an expensive process which is where caching comes in.

So is there some method of hosting the same file in multiple places while
only having to cache the file once inside Varnish using an external
application to decide what the ultimate URL is?
Does Varnish follow Location headers? Or is there some method of redirecting
Varnish to the end location of a file?

If these are features that are not currently available, can anyone point me
to any past discussions about similar setups or give me a heads up if it's
even something Varnish ever plans on doing?

This would be less of an issue if this didn't happen so often in the way I'm
using the back end file system. but as it is, it happens quite a bit and in
a straight URL to cache item configuration causes me to waste a lot of disk
space/memory/IO performance.


Thanks in advance,
-Max
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20070923/d9236049/attachment-0001.html>