IPC semaphores in varnishtest

[Dag Haavi Finstad]

Hi guys

At the latest dev meeting in Rotterdam it was brought up that it would
be a useful addition to be able to sync a varnishtest semaphore with
Varnish itself.

The current 'sema' construct in varnishtest lives within the same
process and is implemented using regular pthread mutexes and
condvars. I've hacked together a suggestion for doing IPC semaphores
in varnishtest/Varnish, backed by SysV semaphores.

The patches introduce a new varnishtest keyword 'ipcsema' and a new
function in vmod-debug, 'debug.sema_sync()'.

'ipcsema' in varnishtest follows the syntax and semantics of the usual
'sema' keyword (with an exception described below [1]),

    ipcsema r1 sync 2

It can be used in both client/server and global scopes in varnishtest,
just like the regular 'sema' construct.

Further, to sync with this semaphore from VCL, we issue

    sub vcl_backend_fetch {
        debug.sema_sync("r1", 2);
    }

How this works under the covers is that varnishtest will on each run
generate a random number and stick that in an environment variable
($VTC_IPC_RAND) prior to launching varnishd. This string, along with
the semaphore name (e.g. "r1") goes into a hash that is used as the
key argument to semget(2). After the test case finishes, the semaphore
will be removed (however, see [2] below).

There are a couple of open issues:

[1] I have yet to find a non-racy way to reset an ipcsema back to its
  original value after the wait is over. This is something that is
  supported by the regular sema construct in varnishtest, that allows
  a sema to be reused multiple times within the same test case. (See
  for example c00013.vtc).

[2] Potential for leaking a semaphore. If the varnishtest process is
  killed and thus the cleanup code isn't called the semaphore is
  leaked. It will then have to be cleaned up via ipcrm(1). Active
  system semaphores can be shown via ipcs(1).

Please take a look - any ideas or feedback will be most welcome.

Patches attached.

Merry X-mas,
Dag

-- 
Dag Haavi Finstad
Software Developer | Varnish Software
Mobile: +47 476 64 134
We Make Websites Fly!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Prototype-varnishtest-IPC-semaphores.patch
Type: text/x-diff
Size: 10035 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20151222/8fd4c1a5/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Add-.sema_sync-to-libvmod-debug.patch
Type: text/x-diff
Size: 5576 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20151222/8fd4c1a5/attachment-0001.patch>