Getting ready for 4.0; Time-to-first-byte measurements

[Lasse Karstensen]

[ tl;dr: master starts sending bytes quicker than 3.0. ]

Hello all.

I've been running some benchmarks to see how master stands up to the earlier
version.

One of the metrics are time-to-first-byte, which I've got (preliminary) results
for now.


First something to compare with; Apache (worker mpm) from ubuntu 12.04:

localhost:80 (2013-09-23 15:42:43.971109 on fryer1)
--------------------------------------------------
mean: 508.937μs (stdev 0.000030301) median: 505.162μs
mean: 492.875μs (stdev 0.000028883) median: 492.782μs
mean: 513.742μs (stdev 0.000026687) median: 514.409μs
mean: 512.047μs (stdev 0.000032193) median: 510.670μs
mean: 510.367μs (stdev 0.000031974) median: 510.384μs


Numbers for current 3.0 branch: (0a7e6ca)

localhost:6081 (2013-09-23 15:30:58.719143 on fryer1)
--------------------------------------------------
mean: 317.403μs (stdev 0.000032792) median: 320.449μs
mean: 262.953μs (stdev 0.000037567) median: 243.901μs
mean: 339.497μs (stdev 0.000031443) median: 344.593μs
mean: 344.804μs (stdev 0.000005411) median: 342.978μs
mean: 320.578μs (stdev 0.000032696) median: 326.909μs


and finally for current master: (988cd77)

localhost:6081 (2013-09-23 15:44:48.687103 on fryer1)
--------------------------------------------------
mean: 234.249μs (stdev 0.000013593) median: 225.852μs
mean: 272.104μs (stdev 0.000049566) median: 263.452μs
mean: 234.516μs (stdev 0.000033360) median: 226.136μs
mean: 287.847μs (stdev 0.000050505) median: 285.355μs
mean: 243.210μs (stdev 0.000048766) median: 226.050μs


Method is a single connect/GET/read(1) loop, 5 runs of 100 connections with
2s sleep between. One warmup run before measurements are done. The object
fetched was 177 bytes.

Software used is up on github, if anyone feels like replicating the
results or improving the benchmark: https://github.com/lkarsten/httpttfb

-- 
With regards,
Lasse Karstensen
Varnish Software AS