Varnish intermittently returns incomplete images

[Batanun B]

ok. Interesting. ��

99.99% of the time that this is happening (after adjusting the cache sizes), I would say that it would be only a single user requesting the image. So it would make sense it it was possible to configure Varnish so it could handle that scenario in a better way. What happens when multiple users request the same resource, and this nuke problem happens is less of a problem, and those times a broken could be acceptable (but preferably it would start serving each request separately, uncached, fetching from the backend each time).

I will increase the cache size, and look into splitting it into two storages. But, I'm guessing you mean that the small objects should be cached in-memory, and the larger ones on disk? It would make much more sense if it cached less "popular" objects on disk, and more "popular" objects in memory, and only considering the object size when the in-memory cache starts to get full. Is it possible to configure Varnish to handle that in a smart and dynamic way?
________________________________
From: Guillaume Quintard <guillaume at varnish-software.com>
Sent: Friday, May 8, 2020 8:33 PM
To: Batanun B <batanun at hotmail.com>
Cc: varnish-misc at varnish-cache.org <varnish-misc at varnish-cache.org>
Subject: Re: Varnish intermittently returns incomplete images

Good question. This is because by default varnish streams the response, so it starts sending what it has, even though it's unsure it can actually deliver. When the eviction strikes, it just aborts the transaction.

The problem with "just" passing the data to the user is that there may be more than one and things get really complicated.

Getting a bigger cache would help, and segregating the storages (smaller than 1MB, and bigger than 1MB for example) would too
--
Guillaume Quintard


On Fri, May 8, 2020 at 11:28 AM Batanun B <batanun at hotmail.com<mailto:batanun at hotmail.com>> wrote:
also... could you explain this part for me? "so it had to truncate it and throw it away"
Why does it have to truncate it? Why not avoid caching it, and returning it as is, from the backend, untouched?
________________________________
From: Guillaume Quintard <guillaume at varnish-software.com<mailto:guillaume at varnish-software.com>>
Sent: Friday, May 8, 2020 7:34 PM
To: Batanun B <batanun at hotmail.com<mailto:batanun at hotmail.com>>
Cc: varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org> <varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>>
Subject: Re: Varnish intermittently returns incomplete images

Hi,

Do you have objects that are sensibly smaller that your images in your cache?

What you are describing sounds like LRU failure (check nuke_limit in "varnishadm param.show"), basically, on a miss, varnish couldn't evict enough objects and make room for the new object, so it had to truncate it and throw it away.

If that's the issue, you can increase nuke_limit, or get a bigger cache, or segregate small and large objects into different storages.

--
Guillaume Quintard


On Fri, May 8, 2020 at 10:14 AM Batanun B <batanun at hotmail.com<mailto:batanun at hotmail.com>> wrote:
Our Varnish (test environment) intermittently returns incomplete images. So the binary content is not complete. When requesting the image from the backend directly (using curl), the complete image is returned every time (I tested 1000 times using a script).

This happens intermittently. Sometimes Varnish returns the complete image, sometimes half of it, sometimes 20% etc... The incomplete image is returned quickly, so I don't think there is a timeout involved (we have not configured any specific timeout in varnish).

I see nothing special in varnishlog when this happens. But I don't know how to troubleshoot this in a good way. Any suggestions?
_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
_______________________________________________
varnish-misc mailing list
varnish-misc at varnish-cache.org<mailto:varnish-misc at varnish-cache.org>
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20200508/de650a01/attachment-0001.html>