[master] f9a8797 Ohh, man...

Poul-Henning Kamp phk at varnish-cache.org
Mon Feb 7 22:12:57 CET 2011


commit f9a8797dedf642768a54d47248881ed2284ed590
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Mon Feb 7 21:08:09 2011 +0000

    Ohh, man...
    
    So imagine an object during fetch, where we have allocated
    the storage for the object structure, the persistent silo
    gets synced, so the data ends up in the next segment, and
    then we crash before that segment gets synched to silo.
    
    On restart the object looks good, until we try to access
    its storage... *bewm*
    
    This is a stopgap, that catches such objects and neuters
    them, using a set of paranoid sanitychecks we should
    employ in any case.
    
    There still is a relevant hole: As above, but after
    the restart we manage to write a new segment before
    the initial object is accessed, and it happens to
    have a storage structure just the same place (not
    unlikely at the beginning)
    
    We do not crash in this case, but deliver wrong content.
    
    Did I ever mention that -spersistent for all practical
    purposes is a filesytem ?

diff --git a/bin/varnishtest/tests/p00007.vtc b/bin/varnishtest/tests/p00007.vtc
new file mode 100644
index 0000000..8d0a18c
--- /dev/null
+++ b/bin/varnishtest/tests/p00007.vtc
@@ -0,0 +1,72 @@
+# $Id$
+
+test "test reload of object spanning incomplete segment"
+
+server s1 {
+	rxreq
+	expect req.url == "/1"
+	send "HTTP/1.1 200 Ok\n"
+	send "Transfer-encoding: chunked\n"
+	send "\n"
+	chunkedlen 32
+	# Tell top-level that it can sync the stevedore
+	sema r1 sync 2
+	# Top-level tells us it has synched the stevedore
+	sema r1 sync 2
+	chunkedlen 32
+	chunkedlen 0
+	accept
+
+	rxreq
+	expect req.url == "/2"
+	txresp -bodylen 100
+
+	rxreq
+	expect req.url == "/1"
+	txresp -bodylen 48
+} -start
+
+varnish v1 -storage "-spersistent,${tmpdir}/_.per,10m" \
+	-vcl+backend {} -start
+
+varnish v1 -cliok "debug.fragfetch 32"
+
+client c1 {
+	txreq -url "/1"
+	rxresp
+	expect resp.bodylen == 64
+} -start
+
+# Wait for first chunk to have been sent
+sema r1 sync 2
+delay .2
+
+# Sync the stevedore, so the next chunk ends up i segment 2
+varnish v1 -cliok "debug.persistent s0 sync"
+
+# Tell server to continue
+sema r1 sync 2
+
+# Get the result
+client c1 -wait
+
+varnish v1 -cliok "debug.persistent s0 dump"
+
+# Panic worker so second segment does not get closed
+varnish v1 -clierr 400 "debug.panic.worker"
+
+# start again
+varnish v1 -start
+
+client c1 {
+	# Make sure there is not a valid "struct storage" in second seg.
+	txreq -url "/2"
+	rxresp
+	expect resp.bodylen == 100
+
+	# Fetch the vampire object and see how that goes...
+	txreq -url "/1"
+	rxresp
+	expect resp.bodylen == 48
+} -run
+



More information about the varnish-commit mailing list