sqlish queries for cache invalidation?

[Dridi Boukelmoune]

On Fri, Jan 26, 2018 at 4:12 PM, Alexandros Kechagias
<alexandros.kechagias at gmail.com> wrote:
> Hi there,
> thanks for the replies. I see, I didn't give you enough details for you to
> be able to help me. Sorry for that, I had a little bit of tunnel vision and
> also the title is not optimal.
>
> The reason I am using xkey, is that I am tagging content that comes from the
> Backend, with tags/names that I use afterwards to invalidate everything that
> has to do with the tags/names. The problem is that in my scenario the URLs
> can't always reliably reflect their content. So I decided to use tags, so
> the backend can tell varnish how to group the content.
>
> For Example:
> 1. User wants the pages called :
> - mysite.mars/foo
> - mysite.mars/bar
> - mysite.mars/baz
>
> 2. Varnish asks the backend for the sites and tags them with xkey
> accordingly to data that comes from the backend through beresp.http.xkey
> Let's say beresp.http.xkey gives me back the following keys/tags for each
> site (I will visualize them with brackets)
> - mysite.mars/foo [project1] [inst6]
> - mysite.mars/bar [project2] [inst6]
> - mysite.mars/baz [project2] [inst6]
>
> 3. Now I know that the purpose of xkey is that i can say:
> xkey.purge("inst6")
> That would delete all the caches.
> xkey.purge("project2")
> Would delete the last two.
>
> But I have a different problem.
> I want to be able to delete "inst6" only from "project2".
> So something like:
>
> xkey.purge("project2" && "inst6")
>
> would be nice.
>
> Has someone an idea how I can solve this problem with already existing
> varnish modules or with a VCL algorithm of a reasonable complexity?
> If I can't find anything I would have to write a module myself or maybe look
> for some different caching technology. If you could also drop me a hint
> there, that would also be nice.
>
> Alexandros

Hi,

I actually understood you well, and basically vmod-xkey sits
in-between native purge and ban. vmod-xkey invalidates based on
criteria (like bans) with purge-like behavior/performance.

So if you need complex expressions and are ready to give up the
real-time nature of xkey purges, you can reuse whatever headers (xkey
supports a couple) that contain your invalidation keys and issue bans
instead:

    ban obj.http.xkey ~ project2 && obj.http.xkey ~ inst6

Like I said, xkey gives up some of the ban flexibility and moves the
cursor to real-time processing while bans are deferred by design. What
I'm trying to say here is if you need complex invalidation schemes
combinating multiple criteria, then you have access to whatever was
present when beresp was cached. Today xkey accepts multiple keys at
once, but invalidates their union, the opposite of what you want.

Dridi