Compared performance of Varnish Cache on x86_64 and aarch64

[Geoff Simmons]

On 7/28/20 13:52, Martin Grigorov wrote:
> 
> I've just posted an article [1] about comparing the performance of Varnish
> Cache on two similar
> machines - the main difference is the CPU architecture - x86_64 vs aarch64.
> It uses a specific use case - the backend service just returns a static
> content. The idea is
> to compare Varnish on the different architectures but also to compare
> Varnish against the backend HTTP server.
> What is interesting is that Varnish gives the same throughput as the
> backend server on x86_64 but on aarch64 it is around 30% slower than the
> backend.

Does your test have an account of whether there were any errors in
backend fetches? Don't know if that explains anything, but with a
connect timeout of 10s and first byte timeout of 5m, any error would
have a considerable effect on the results of a 30 second test.

The test tool output doesn't say anything I can see about error rates --
whether all responses had status 200, and if not, how many had which
other status. Ideally it should be all 200, otherwise the results may
not be valid.

I agree with phk that a statistical analysis is needed for a robust
statement about differences between the two platforms. For that, you'd
need more than the summary stats shown in your blog post -- you need to
collect all of the response times. What I usually do is query Varnish
client request logs for Timestamp:Resp and save the number in the last
column.

t.test() in R runs Student's t-test (me R fanboi).


HTH,
Geoff
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20200731/3d6216e9/attachment.bin>