Best practice for caching scenario with different backend servers but same content

[Hamidreza Hosseini]

Hi,
I read the sample config that you sent but Can I use "bereq.url"  in this way:
for example I want to shard my requests for live streams based on the url's that clients enter, for example if the following url's are different live streams (stream1 and stream2 are the name of different streams ):
```
mydomain.com/live/australia/stream1/chunk_43212123.ts
mydomain.com/live/australia/stream2/chunk_43212123.ts
mydomain.com/live/australia/stream3/chunk_43212123.ts
```
Now think I want just the url excluded with chunk file becomes hashed and sharded:
Just this part: "/live/australia/stream{1,2,3}/"
Not :"/live/australia/stream{1,2,3}/chunk_43212123.ts"

So by adjusting " p.set(by=KEY, key=bereq.url) " it would shard "bereq.url",  it means "/live/australia/stream{1,2,3}/chunk_43212123.ts"


________________________________
From: varnish-misc <varnish-misc-bounces+hrhosseini=hotmail.com at varnish-cache.org> on behalf of Hamidreza Hosseini <hrhosseini at hotmail.com>
Sent: Sunday, August 1, 2021 4:17 AM
To: varnish-misc at varnish-cache.org <varnish-misc at varnish-cache.org>
Subject: Best practice for caching scenario with different backend servers but same content

Hi,
I want to use varnish in my scenario as cache service, I have about 10 http servers that serve Hls fragments as the backend servers and about 5 varnish servers for caching purpose, the problem comes in when I use round-robin director for backend servers in varnish,
if a varnish for specific file requests to one backend server and for the same file but to another backend server it would cache that file again because of different Host headers ! so my solution is using fallback director instead of round-robin as follow:

```
In varnish-1:
    new hls_cluster = directors.fallback();
    hls_cluster.add_backend(b1());
    hls_cluster.add_backend(b2());
    hls_cluster.add_backend(b3());
    hls_cluster.add_backend(b4());
    hls_cluster.add_backend(b5());
    hls_cluster.add_backend(b6());
    hls_cluster.add_backend(b7());
    hls_cluster.add_backend(b8());
    hls_cluster.add_backend(b9());
    hls_cluster.add_backend(b10());



In varnish-2:
    new hls_cluster = directors.fallback();
    hls_cluster.add_backend(b10());
    hls_cluster.add_backend(b1());
    hls_cluster.add_backend(b2());
    hls_cluster.add_backend(b3());
    hls_cluster.add_backend(b4());
    hls_cluster.add_backend(b5());
    hls_cluster.add_backend(b6());
    hls_cluster.add_backend(b7());
    hls_cluster.add_backend(b8());
    hls_cluster.add_backend(b9());


In varnish-3:
    new hls_cluster = directors.fallback();
    hls_cluster.add_backend(b9());
    hls_cluster.add_backend(b1());
    hls_cluster.add_backend(b2());
    hls_cluster.add_backend(b3());
    hls_cluster.add_backend(b4());
    hls_cluster.add_backend(b5());
    hls_cluster.add_backend(b6());
    hls_cluster.add_backend(b7());
    hls_cluster.add_backend(b8());
    hls_cluster.add_backend(b10());

```
But I think this is not the best solution, because there is no load balancing despite, I used different backend for the first argument of fallback directive,
What is varnish recommendation for this scenario?



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20210807/f990c9fd/attachment.html>