phk at phk.freebsd.dk
Mon Mar 9 13:18:59 CET 2015
In message <CAOXZevBF0Y-VgfqS2E-1LfxqMyw759LJpVbfwLT+r01iu58KvA at mail.gmail.com>
, Per Buer writes:
>> But the one thing I would *really* like to get is some kind of statistics
>> of *which* test-cases we have this problem with, and I wish Jenkins could
>> somehow be persuaded to collect such info.
>That was the main objective.
>Having the testrunner have an optional interface to something like a
>database would be trivial. Having something that is able to collect a bit
>of context to a testrun in a database (load, IO response, etc) would be
>helpful in order to figure out _why_ certain tests are failing on certain
At one point I considered having varnishtest do a DNS lookup of
everytime a test failed.
>Because scheduling the tests and running the tests are different tasks.
>Scheduling and interfacing with a database is pretty trivial in Python and
>not something we should do directly in the varnishtest itself.
I disagree, varnishtest is probably a lot *better* at scheduling and
running the tests than anything python can ever do.
But it utterly lacks any support for collecting the stats, so that's where
the focus should be IMO.
>It would be really trivial to make varnishtest collect the failing
>> tests on a list and then run that list after the main-run in
>> single-threaded mode (enabled by some argv)
>Absolutely. And that is what the proposed solution does. It allows us to
>keep varnishtest a relativly simple program that deal with running a single
Uhm, varnishtest *already* has scheduling and parallelism built in...
But I wouldn't be surprised if autocrap ignores all that.
In that case, it should be obvious where I think the problem lies :-)
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
More information about the varnish-dev