Compared performance of Varnish Cache on x86_64 and aarch64
phk at phk.freebsd.dk
Tue Jul 28 14:01:03 UTC 2020
Martin Grigorov writes:
> Any feedback and ideas how to tweak it (VCL or even patches) are very
First you need to tweak your benchmark setup.
Thread Stats Avg Stdev Max +/- Stdev
Latency 655.40us 798.70us 28.43ms 90.52%
Strictly speaking, you cannot rule out that the ARM machine
sends responses before it receives the request, because your
standard deviation is larger than your average.
In other words: Those numbers tell us nothing.
If you want to do this comparison, and I would love for you to do so,
you really need to take the time it takes, and get your "noise" down.
Here is how you should do it:
for machine in ARM, INTEL
For i in (at least) 1-5:
Run test for 5 minutes
If the results from the first run on each machine is very different
from the other four runs, you can disrecard it, as a startup/bootup
Report the numbers for all the runs for both machines.
Make a plot of all those numbers, where you plot the reported
average +/- stddev as a line, and the max value as a dot/cross/box.
If you want to get fancy, you can do a Student's T test to tell
you if there is any real difference. There's a program called
"ministat" which will do this for you.
Also: I can highly recommend this book:
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
More information about the varnish-dev