stack ws workspace for pcre and others #1576

[Federico Schwindt]

Only problem is that it's a build option.

Increasing the stack space will only work as long as the regex doesn't
recurse more than the stack space allows.

Not sure what's the best solution here but we should avoid crashing.



On Tue, Sep 2, 2014 at 6:12 PM, Nils Goroll <slink at schokola.de> wrote:

> Hi,
>
> on the weekend I have actually stumbled over the same cause that bug #1576
> has
> (but unfortunately have not immediately realized the connection) and have
> changed our dcs_classifier vmod (which needs some reasonably large scratch
> memory in the order of ~64k) to use the client workspace rather than stack.
>
> Using the workspace seemed more reasonable as varnish core structures have
> been
> moved away from the stack and I thought it would be more advantagous to
> increase
> a workspace size than increase the stack size (which has a nice tiny
> default of
> 48k now).
>
> But then Ferdericos comment in #1576 made me re-think this:
>
> > Comment (by fgsch):
> >
> >  This is actually related to the thread_pool_stack size.
> >  Bumping the value to 49k make the test work. Lowering pcre_match_limit
> >  makes the match fail without crashing.
> >
> >  Should we reduce the pcre_match_limit to something more sensible? 10000
> >  seems like a rather large value.
>
> If we were to need a larger default stack for pcre anyway, it wouldn't make
> sense to move scratch space to a WS.
>
> On the other hand, we might as well consider to give pcre space from
> workspaces:
>
> PCRESTACK(3):
>
>    Compiling PCRE to use heap instead of stack for pcre[16]_exec()
>
>        In environments where stack memory is constrained, you might want to
> compile PCRE to use heap memory instead of stack for remember‐
>        ing  back-up  points  when pcre[16]_exec() is running. This makes
> it run
> a lot more slowly, however.  Details of how to do this are
>        given in the pcrebuild documentation. When built in this way,
> instead of
> using the stack, PCRE obtains and frees memory by  calling
>        the  functions  that are pointed to by the pcre[16]_stack_malloc and
> pcre[16]_stack_free variables. By default, these point to mal‐
>        loc() and free(), but you can replace the pointers to cause PCRE to
> use
> your own functions. Since the block sizes  are  always  the
>        same,  and  are  always  freed in reverse order, it may be possible
> to
> implement customized memory handlers that are more efficient
>        than the standard functions.
>
> So we could simply hand the free space of some ws to pcre via some
> pcre_stack_(malloc|free).
>
>
> Whatever we do, we should either try to minimize ws usage or stack usage,
> but do
> it consequently for all components.
>
> Nils
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20140902/5edab457/attachment.html>