Lots of configs
dhelkowski at sbgnet.com
Tue Mar 8 14:20:01 CET 2011
First off, I would like to thank Per Buer for pointing out that I am off
by a factor of 1000 in the following
statements. I have corrected for that below so that my statements are
more clear. My mistake was in
considering modern processors as 2 megahertz instead of 2 gigahertz.
On 3/7/2011 1:52 PM, David Helkowski wrote:
> A modern CPU can run, at most, around 10 million -assembly based-
> instructions per second.
Make that 10 billion. The math I am using is 5 x 2 gigahertz.
> See http://en.wikipedia.org/wiki/Instructions_per_second
> A regular expression compare is likely at least 20 or so assembly
> That gives around 500,000 regular expression compares if you are using
> 100% of the
> CPU just for that. A reasonable amount of CPU to consume would be 30%
> ( at most ).
> So; you are left with around 150k regular expression checks per second.
The correct numbers here are 500 million. A regular expression compare
more likely takes
40 assembly instructions, so I am going to cut this to 250 million.
LIkewise, at 30%, that
leads to about 80 million.
> Lets suppose there are 500 different domains. On average, you will be
> doing 250 if/else
> checks per call. 150k / 250 = 600.
The new number is 80 million / 250 = 320k
> That means that you will get, under fair conditions, a max
> of about 600 hits per second.
320,000 hits per second.
Obviously, no server is capable of serving up such a number.
Just using regular expressions in a cascading if/then will work fine in
My apologies for the confusion in this regard.
What I can see is a server serving around 10,000 hits per second.
That would require about 30x the number of domains. You don't really
want to eat up CPU usage for just if/then though, so probably at around 10x
the number of domains you'd want to switch to a hash table.
So; correcting my conclusion; if you are altering configuration for 5000
domains, then you are going to need a hash table. Otherwise you are going
to be fine just using a cascading if/then, despite it being ugly.
> The person asking the question likely has 500 domains running.
> That gives a little over 1 hit possible per second per domain. Do you
> think that is an acceptable
> solution for this person? I think not.
> Compare it to a hash lookup. A hash lookup, using a good minimal
> perfect hashing algorithms,
> will take at most around 10 operations. Using the same math as above,
> that gives around 300k
> lookups per second. A hash would be roughly 500 times faster than
> using if/else...
Note that despite my being off by a factor of 1000, the multiplication
still holds out. If you use
a hash table, even with only 500 domains, a hash table will -still- be
500 times faster. I still think
it would be great to have a hash table solution available for use in VCL.
> On 3/7/2011 1:35 PM, Per Buer wrote:
>> On Sun, Mar 6, 2011 at 11:39 PM, AD <straightflush at gmail.com
>> <mailto:straightflush at gmail.com>> wrote:
>> what is the best way to run an instance of varnish that may need
>> different vcl configurations for each hostname. This could end
>> up being 100-500 includes to map to each hostname and then a long
>> if/then block based on the hostname. Is there a more scalable
>> way to deal with this?
>> CPU and memory bandwidth is abundant on modern servers. I'm actually
>> not sure that having a 500 entries long if/else statement will hamper
>> performance at all. Remember, there will be no system calls. I would
>> guess a modern server will execute at least a four million
>> regex-based if/else per second per CPU core if most of the code and
>> data will be in the on die cache. So executing 500 matches should
>> take about 0.5ms.
>> It might not make sense to optimize this.
>> Per Buer, Varnish Software
>> Phone: +47 21 98 92 61 / Mobile: +47 958 39 117 / Skype: per.buer
>> Varnish makes websites fly!
>> Want to learn more about Varnish?
>> varnish-misc mailing list
>> varnish-misc at varnish-cache.org
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the varnish-misc