stack ws workspace for pcre and others #1576
slink at schokola.de
Tue Sep 2 19:12:33 CEST 2014
on the weekend I have actually stumbled over the same cause that bug #1576 has
(but unfortunately have not immediately realized the connection) and have
changed our dcs_classifier vmod (which needs some reasonably large scratch
memory in the order of ~64k) to use the client workspace rather than stack.
Using the workspace seemed more reasonable as varnish core structures have been
moved away from the stack and I thought it would be more advantagous to increase
a workspace size than increase the stack size (which has a nice tiny default of
But then Ferdericos comment in #1576 made me re-think this:
> Comment (by fgsch):
> This is actually related to the thread_pool_stack size.
> Bumping the value to 49k make the test work. Lowering pcre_match_limit
> makes the match fail without crashing.
> Should we reduce the pcre_match_limit to something more sensible? 10000
> seems like a rather large value.
If we were to need a larger default stack for pcre anyway, it wouldn't make
sense to move scratch space to a WS.
On the other hand, we might as well consider to give pcre space from workspaces:
Compiling PCRE to use heap instead of stack for pcre_exec()
In environments where stack memory is constrained, you might want to
compile PCRE to use heap memory instead of stack for remember‐
ing back-up points when pcre_exec() is running. This makes it run
a lot more slowly, however. Details of how to do this are
given in the pcrebuild documentation. When built in this way, instead of
using the stack, PCRE obtains and frees memory by calling
the functions that are pointed to by the pcre_stack_malloc and
pcre_stack_free variables. By default, these point to mal‐
loc() and free(), but you can replace the pointers to cause PCRE to use
your own functions. Since the block sizes are always the
same, and are always freed in reverse order, it may be possible to
implement customized memory handlers that are more efficient
than the standard functions.
So we could simply hand the free space of some ws to pcre via some
Whatever we do, we should either try to minimize ws usage or stack usage, but do
it consequently for all components.
More information about the varnish-dev