[PATCH] fileread for the std VMOD

[Sanjoy Das]

Hi!

I've attached a new patch implementing fileread with the required
corrections (std-vmod-fileread.diff) and a patch implementing a basic
test-case (fileread-test-case.diff) - I am still figuring out how to
properly write a real multi-threaded stress-test.

There are a few issues, however. Firstly, since the file is being
mmaped, files whose sizes are exact multiples of the system page size
will cause a segmentation fault. This is a direct consequence of the
fact that VCL strings are null terminated - unless there is some
(zeroed, by POSIX) space _left over_ due to a non-aligned file-size,
code that tries to read the string will end up trying to access a
non-mapped memory location.

As I see it (from within the confines of my limited experience :) ),
there are three ways by which this can be corrected:

i. Don't process files with sizes which are evenly divided by the
system page size. This is what is done now. This really does not solve
anything.
ii. Open file, copy to memory, close file. This may be too
inefficient, but this solves the second problem as well (mentioned
below).
iii. Instead of representing VCL strings as regular null-terminated C
strings, store the length somewhere; perhaps by representing them as a
struct containing a pointer to a character array and a length. When
required (for passing them to standard library functions, for
instance), they can be trivially converted to a regular C string, by
copying.

The second issue is that of the file changing after it has been
mmapped. While on Linux it seems that the changes are reflected back
to mmaped memory block, this is not a scenario well defined by POSIX.
Even in Linux, we're in trouble if a file which was not originally of
a length evenly divisible by the page size becomes so due to the
addition of text. One solution, as I see it, would be to add another
function, calling it filerefresh, which remaps (with the new length)
the file whose file-name is passed, possibly after doing a timestamp
check. This effectively leaves it to the VCL writer to decide which
files may change, and which may not.

-- 
Regards,

Sanjoy Das.

http://playingwithpointers.com

Public Key at http://playingwithpointers.com/custom/public_key.txt
-------------- next part --------------
A non-text attachment was scrubbed...
Name: fileread-test-case.diff
Type: text/x-patch
Size: 785 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20101111/a286bb92/attachment-0006.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: std-vmod-fileread.diff
Type: text/x-patch
Size: 4284 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20101111/a286bb92/attachment-0007.bin>