Getting ready for 4.0; Time-to-first-byte measurements

Lasse Karstensen lkarsten at
Mon Sep 23 16:24:07 CEST 2013

[ tl;dr: master starts sending bytes quicker than 3.0. ]

Hello all.

I've been running some benchmarks to see how master stands up to the earlier

One of the metrics are time-to-first-byte, which I've got (preliminary) results
for now.

First something to compare with; Apache (worker mpm) from ubuntu 12.04:

localhost:80 (2013-09-23 15:42:43.971109 on fryer1)
mean: 508.937μs (stdev 0.000030301) median: 505.162μs
mean: 492.875μs (stdev 0.000028883) median: 492.782μs
mean: 513.742μs (stdev 0.000026687) median: 514.409μs
mean: 512.047μs (stdev 0.000032193) median: 510.670μs
mean: 510.367μs (stdev 0.000031974) median: 510.384μs

Numbers for current 3.0 branch: (0a7e6ca)

localhost:6081 (2013-09-23 15:30:58.719143 on fryer1)
mean: 317.403μs (stdev 0.000032792) median: 320.449μs
mean: 262.953μs (stdev 0.000037567) median: 243.901μs
mean: 339.497μs (stdev 0.000031443) median: 344.593μs
mean: 344.804μs (stdev 0.000005411) median: 342.978μs
mean: 320.578μs (stdev 0.000032696) median: 326.909μs

and finally for current master: (988cd77)

localhost:6081 (2013-09-23 15:44:48.687103 on fryer1)
mean: 234.249μs (stdev 0.000013593) median: 225.852μs
mean: 272.104μs (stdev 0.000049566) median: 263.452μs
mean: 234.516μs (stdev 0.000033360) median: 226.136μs
mean: 287.847μs (stdev 0.000050505) median: 285.355μs
mean: 243.210μs (stdev 0.000048766) median: 226.050μs

Method is a single connect/GET/read(1) loop, 5 runs of 100 connections with
2s sleep between. One warmup run before measurements are done. The object
fetched was 177 bytes.

Software used is up on github, if anyone feels like replicating the
results or improving the benchmark:

With regards,
Lasse Karstensen
Varnish Software AS

More information about the varnish-dev mailing list