Test reporting and statistics.

[Per Buer]

So, I've done a bit more work on getting data on tests. The rough idea is
something like this.

We add a new make target - "make report" which depends on testsuite.log
output from "make check". This will push the testsuite.log onto a more or
less public web service and store the file in a directory there (a
mailbox). Either a HTTP POST or a HTTP PUT. Most likely a POST since we
might want to change it a bit and add the IP address of the reporter to the
content of the database.

I've written a standalone parser for the file which extracts the various
interesting bits from it and pushes it into a local database. This seem to
work quite well and could run from cron. It's a bit hackish (it's written
in Perl with a bunch of regular expressions).

If this is OK I'll see if I can create some sort of query interface to the
database which can generate reports based on what tests are failing, what
architectures are flaky, etc.

The thing I'm missing is some way of identifying the machine which has run
the tests. This could be the IP address of the machine which ran the tests.
This would be interesting if we open this service up to the public as we'll
have all sorts of weird machines reporting to it and we might want to
ignore certain hosts or spot host specific (not arch specific) problems.

I don't think there are any privacy issues here. Some people consider IP
addresses private but I don't. :-P - the primary use case is for the
jenkins slaves.

Let me know what you think and have a good weekend.


-- 
*Per Buer*
CTO | Varnish Software AS
Cell: +47 95839117
We Make Websites Fly!
www.varnish-software.com
[image: Register now]
<http://info.varnish-software.com/varnish-summits-autumn-2014-registration>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20150313/a7e1cde2/attachment.html>