Varnish for multiple magento sites
Guillaume Quintard
guillaume.quintard at gmail.com
Wed Jul 12 15:27:34 UTC 2023
Hi Riccardo,
You are right, I assumed that the tags were uuids of some sorts and welp,
they're not.
The best way would be to ask magento to fix it by including a hash of the
host or something in each tag, but that's probably not going to happen any
time soon.
The next best way is to teach Varnish to be a bit more selective when
banning. This is done in three steps.
First, we are going to mark the backend response with the host it comes
from (maybe Magento2 already does it in some form, in which case you can
use that header instead):
# add this to the beginning of vcl_backend_response
sub vcl_backend_response {
set beresp.http.x-host = bereq.http.host;
...
Then, we change the ban() calls to only apply to the responses with the
right x-host headers:
if (req.http.X-Magento-Tags-Pattern) {
ban("obj.http.x-host == + req.http.host + " &&
obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern);
}
if (req.http.X-Pool) {
ban("obj.http.x-host == + req.http.host + "&& obj.http.X-Pool ~ "
+ req.http.X-Pool);
}
Lastly, we don't need the x-host header to be exposed to the client, so we
strip it at the beginning of vcl_deliver:
sub vcl_deliver {
unset resp.http.x-host;
...
disclaimer: This test is completely untested, and it's early in the
morning, so you probably shouldn't trust me too much and you should test
this before throwing it in prod.
Technically, you *could* use req.http.host directly, but you don't want to
because of the ban-lurker and its performance implications.
Hope this helps.
--
Guillaume Quintard
On Wed, Jul 12, 2023 at 4:15 AM Riccardo Brunetti <riccardo.brunetti at host.it>
wrote:
> Hello Guillaume.
> Thanks for your answer.
> The VCL is actually almost identical to that you mentioned in the link
> (I'm attaching it without references to names and IP anyway)
>
> What somehow worries me is that, if I understand, the ban is performed
> according to some "X-Magento-Tags".
> Now, if I look at the output of varnishlog and search for "*Tags*", what I
> get is:
>
> 1) while navigating the site:
>
> ....
> - RespHeader X-Magento-Tags: NAVIGATIONPRO_MENU_2
> - RespUnset X-Magento-Tags: NAVIGATIONPRO_MENU_2
> - RespHeader X-Magento-Tags:
> store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_bottom_strenghts,cms_b
> - RespUnset X-Magento-Tags:
> store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_bottom_strenghts,cms_b
> - BerespHeader X-Magento-Tags:
> cat_c_595,cat_c_p_595,store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_
> - BerespHeader X-Magento-Tags: NAVIGATIONPRO_MENU_2
> - RespHeader X-Magento-Tags: NAVIGATIONPRO_MENU_2
> - RespUnset X-Magento-Tags: NAVIGATIONPRO_MENU_2
> - RespHeader X-Magento-Tags:
> cat_c_595,cat_c_p_595,store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_
> - RespUnset X-Magento-Tags:
> cat_c_595,cat_c_p_595,store,cms_b,gdpr_c,theme_editor_backend_css_block,cms_b_header_cms_links,cms_b_argento_scroll_up,cms_b_footer_cms_content,cms_b_footer_payments,cms_b_header_block_custom_links,cms_b_main_bottom_newsletter,cms_b_main_
> .....
>
> 2) when performing a purge (php bin/magento c:f):
>
> ...
> - ReqHeader X-Magento-Tags-Pattern: .*
> ...
>
> In both cases I can't see any specific reference to that particular site.
>
> Thanks again.
> Riccardo
>
> 11/07/2023, 17:09 Guillaume Quintard ha scritto:
>
> Hi Ricardo,
>
> Having your VCL (even anonymized) would help here, otherwise debugging is
> pretty hard. For the moment, I'm going to assume you are using a variation
> of
> https://github.com/magento/magento2/blob/13e54e1b28a5d590ab885bd4df9f58877b549052/app/code/Magento/PageCache/etc/varnish6.vcl
> and deal in generalities.
>
> The way that vcl invalidates content is through bans:
> https://github.com/magento/magento2/blob/13e54e1b28a5d590ab885bd4df9f58877b549052/app/code/Magento/PageCache/etc/varnish6.vcl#L30-L47
> which doesn't need the host header, it's just uses unique tags pushed by
> the backend in response headers.
> If it was using the actual purge mechanism, then modifying the host should
> be sufficient because purge acts on the object found in the cache (and if
> you can get a hit, you can get purged).
>
> Here's a good primer on invalidation:
> https://docs.varnish-software.com/tutorials/cache-invalidation/
>
> Kind regards,
>
>
>
> --
> Guillaume Quintard
>
>
> On Tue, Jul 11, 2023 at 4:14 AM Riccardo Brunetti <
> riccardo.brunetti at host.it> wrote:
>
>> Hello.
>> I'm new to varnish and I have a question concerning how to manage
>> multiple sites using the same varnish cache frontend.
>>
>> More specifically, I need to setup a single varnish cache server for two
>> different Magento2 sites.
>>
>> Looking around I found that it is possible to manage different backends
>> using something like:
>>
>> if (req.http.host == "somesite") {
>> set req.backend_hint = somebackend;
>> }
>>
>> Now, I have two different Magento2 sites and, using the above expression,
>> I can handle the two different backends.
>> The problem is that I can't understand how to handle the PURGE/BAN of the
>> two independently.
>>
>> As far as I understand from the .vcl file that Magento2 itself produces
>> there is nothing inside the "purge" section that specifies which resources
>> must be purged.
>> It seems to me that is site A performs a purge, than also the cache of
>> site B resources will be cleaned.
>>
>> Can you help me with this or point me to some example or tutorials?
>>
>> Thanks a lot
>> Riccardo
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20230712/3e4c5d64/attachment.html>
More information about the varnish-misc
mailing list