Lots of configs
David Helkowski
dhelkowski at sbgnet.com
Mon Mar 7 16:56:20 CET 2011
It is true that there are potentially better places to setup the hash,
but it is best to check for a null pointer
for the hash object anyway any time you use it. The setup itself is also
very fast; you just don't want to do
it every time. Note in my init function I forgot a 'hash = new hashc()'.
Also; if you are going to do this, you will likely have a preset list of
domains you are using.
In such a case, the best type of hash to use would be a 'minimal perfect
hash'.
You could use the 'gperf' library to generate a suitable algorithm to
map your domain strings
into an array.
On 3/7/2011 10:30 AM, AD wrote:
> Cool, as for the startup, i wonder if you can instead of trying to
> insert into VCL_Init, try to do just, as part of the startup process
> hit a special URL to load the hash_Table. Or another possibility
> might be to load an external module, and in there, populate the hash.
>
>
>
> On Mon, Mar 7, 2011 at 9:45 AM, David Helkowski <dhelkowski at sbgnet.com
> <mailto:dhelkowski at sbgnet.com>> wrote:
>
> vcl configuration is turned straight into C first of all.
> You can put your own C code in both the functions and globally.
> When including headers/libraries, you essentially just have to
> include the code globally.
>
> I am not sure if there is any 'init' function when varnish is
> called, so I was suggesting that
> the hash be initiated by just checking if the hash has been
> created yet.
>
> This will cause a penalty to the first vcl_recv call that goes
> through; but that shouldn't
> matter.
>
> Note that I just passed a dummy number as an example to the custom
> config, and that
> I didn't show how to do anything in the custom function. In this
> example, all custom
> stuff would be in straight C. You would need to use varnish itself
> to compile what config
> you want and look at the C code it generates to figure out how to
> tie in all your custom
> configs....
>
> eg:
>
> C{
> #include "hash.c" // a string hashing store/lookup libary;
> you'll need to write one
> // or possibly just use some freely available one.
> hashc *hash=0;
>
> void init_hash() {
> if( hash ) return;
>
> hash.store( 'test.com <http://test.com>', &test_com );
> // same for all domains
> }
>
> void test_com( int n ) {
> // custom vcl_recv stuff for domain 'test'
> }
> }
>
> sub vcl_recv {
> C{
> char *domain;
> // [ place some code to fetch domain and put it in domain here ]
> if( !hash ) init_hash();
> void (*func)(int);
> func = hash.lookup( domain );
> func(1);
>
> }
> }
>
> On 3/7/2011 9:23 AM, AD wrote:
>> but dont all the configs need to be loaded at runtime, not sure
>> the overhead here? I think what you mentioned seems like a
>> really innovative way to "call" the function but what about
>> anyimpact to "loading" all these configs?
>>
>> If i understand what you are saying, i put a "call test_func;" in
>> vcl_recv which turned into this in C
>>
>> if (VGC_function_test_func(sp))
>> return (1);
>> if
>>
>> Are you suggesting your hash_table would take over this step ?
>>
>> Adam
>>
>> On Mon, Mar 7, 2011 at 8:02 AM, David Helkowski
>> <dhelkowski at sbgnet.com <mailto:dhelkowski at sbgnet.com>> wrote:
>>
>> The best way would be to use a jump table.
>> By that, I mean to make multiple subroutines in C, and then
>> to jump to the different subroutines by looking
>> up pointers to the subroutines using a string hashing/lookup
>> system.
>>
>> You would also need a flag to indicate whether the hash has
>> been 'initialized' yet as well.
>> The initialization would consist of storing function pointers
>> at the hash locations corresponding to each
>> of the domains.
>>
>> I attempted to do this myself when I first started using
>> varnish, but I was having problems with varnish
>> crashing when attempting to use the code I wrote in C. There
>> may be limitations to the C code that can be
>> used.
>>
>>
>> On 3/6/2011 5:39 PM, AD wrote:
>>> Hello,
>>> what is the best way to run an instance of varnish that may
>>> need different vcl configurations for each hostname. This
>>> could end up being 100-500 includes to map to each hostname
>>> and then a long if/then block based on the hostname. Is
>>> there a more scalable way to deal with this? We have been
>>> toying with running one large varnish instance with tons of
>>> includes or possibly running multiple instances of varnish
>>> (with the config broken up) or spreading the load across
>>> different clusters (kind of like sharding) based on hostname
>>> to keep the configuration simple.
>>>
>>> Any best practices here? Are there any notes on the
>>> performance impact of the size of the VCL or the amount of
>>> if/then statements in vcl_recv to process a unique call
>>> function ?
>>>
>>> Thanks
>>>
>>>
>>> _______________________________________________
>>> varnish-misc mailing list
>>> varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
>>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>>
>> _______________________________________________
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
>> <mailto:varnish-misc at varnish-cache.org>
>> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>
>>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org <mailto:varnish-misc at varnish-cache.org>
> http://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20110307/9cdacff6/attachment-0003.html>
More information about the varnish-misc
mailing list