Arista Networks: Caching the BODY of a POST request for subsequent GETs

[Sidhartha Agrawal]

Hi Guillaume,
Thank you for responding.

Yes, we want the POST request to also create an object in the cache i.e.
cache the body of the POST request. This way the subsequent GET can find
the object in the cache.

>From what I could gather(and I could be wrong here), the tutorial was using
the body of the POST request to compute the HASH. But was not actually
caching the BODY of the POST-request. It was caching the response to the
POST so they subsequent POST would not hit the backend.

Kind regards,
-Sid



On Sun, May 17, 2020 at 7:47 AM Guillaume Quintard <
guillaume at varnish-software.com> wrote:

> Hi Sidharta,
>
> I'm not sure why the tutorial you linked to isn't adequate. Do you need
> the POST to also create an object in cache?
>
> Kind regards,
>
> --
> Guillaume Quintard
>
>
> On Sat, May 16, 2020 at 11:44 PM Sidhartha Agrawal <sid at arista.com> wrote:
>
>> Hi Varnish Community,
>>
>> I am a software developer at Arista Network in our Tools and
>> Infrastructure Group. We use Varnish(6.3.1) to cache our build objects such
>> as RPMS and images and Varnish is an integral part of our internal build
>> system.
>>
>> Some of these objects are of the order of ~500 MB, so a GET of multiple
>> such objects can put a significant load on the backend(OpenSwift). Since
>> there is an adequate temporal locality(within a day) between the POST of
>> the large object and the subsequent GET, we would like to cache the object
>> when the POST happens, so that the subsequent GET is guaranteed to find it
>> in the cache(unless evicted or expired).
>>
>> As I understand, Varnish bypasses the cache for POST and passes it
>> straight to the backend. We would like to change this behavior of POST to
>> also add an entry for the object in the cache if the write to the backend
>> was successful.
>>
>> A cursory google search for “Caching Post requests on Varnish” lead to
>> me this article
>> <https://docs.varnish-software.com/tutorials/caching-post-requests/#step-1-prepare-the-vcl-for-caching-post-requests>.
>> Though I do not think this is exactly what we need. The article talks about
>> avoiding a write to the backend if another POST request is made with the
>> same body. That is different from what we are trying to do.
>>
>> I would appreciate any insights you can provide in to:
>>
>>    -
>>
>>    What changes do I need to make to the VCL to achieve our desired
>>    behavior
>>    -
>>
>>    Comments, on whether this is a bad idea for some reason.
>>
>>
>> I have been following the VCL state machine from this website
>> <https://varnish-cache.org/docs/6.3/reference/states.html#reference-states>.
>> I appreciate the help.
>>
>> -Sid
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20200517/7daa0274/attachment.html>