Is anyone using ESI with a lot of traffic?

Artur Bergman sky at crucially.net
Mon Mar 2 22:42:39 CET 2009


Then the page is wrong

"Keep-alive was invented to reduce CPU usage on servers when CPUs were  
100 times slower. But what is not said is that persistent connections  
consume a lot of memory while not being usable by anybody except the  
client who openned them. Today in 2006, CPUs are very cheap and memory  
is still limited to a few gigabytes by the architecture or the price.  
If a siteneeds keep-alive, there is a real problem. Highly loaded  
sites often disable keep-alive to support the maximum number of  
simultaneous clients. The real downside of not having keep-alive is a  
slightly increased latency to fetch objects. Browsers double the  
number of concurrent connections on non-keepalive sites to compensate  
for this"

(and also widely incorrect)

:)

Artur


On Mar 2, 2009, at 1:33 PM, Cloude Porteus wrote:

> I believe TCP Keep-alive has been supported in HAProxy since version
> 1.2. We've been using 1.3.x for at least a year.
>
> -cloude
>
> On Mon, Mar 2, 2009 at 1:10 PM, Artur Bergman <sky at crucially.net>  
> wrote:
>> HAProxy doesn't do keep-alive, so it makes everything slower.
>>
>> Artur
>>
>> On Feb 27, 2009, at 9:02 PM, Cloude Porteus wrote:
>>
>>> John,
>>> Thanks so much for the info, that's a huge help for us!!!
>>>
>>> I love HAProxy and Willy has been awesome to us. We run everything
>>> through it, since it's really easy to monitor and also easy to debug
>>> where the lag is when something in the chain is not responding fast
>>> enough. It's been rock solid for us.
>>>
>>> The nice part for us is that we can use it as a content switcher to
>>> send all /xxx traffic or certain user-agent traffic to different
>>> backends.
>>>
>>> best,
>>> cloude
>>>
>>> On Fri, Feb 27, 2009 at 2:24 PM, John Adams <jna at twitter.com> wrote:
>>>>
>>>> cc'ing the varnish dev list for comments...
>>>> On Feb 27, 2009, at 1:33 PM, Cloude Porteus wrote:
>>>>
>>>> John,
>>>> Goodto hear from you. You must be slammed at Twitter. I'm happy to
>>>> hear that ESI is holding up for you. It's been in my backlog  
>>>> since you
>>>> mentioned it to me pre-Twitter.
>>>>
>>>> Any performance info would be great.
>>>>
>>>>
>>>> Any comments on our setup are welcome. You may also choose to  
>>>> call us
>>>> crazypants. Many, many thanks to Artur Bergman of Wikia for  
>>>> helping us
>>>> get
>>>> this configuration straightened out.
>>>> Right now, we're running varnish (on search) in a bit of a non- 
>>>> standard
>>>> way.
>>>> We plan to use it in the normal fashion (varnish to Internet,  
>>>> nothing
>>>> inbetween) on our API at some point. We're running version 2.0.2,  
>>>> no
>>>> patches. Cache hit rates range from 10% to 30%, or higher when a
>>>> real-time
>>>> event is flooding search.
>>>> 2.0.2 is quite stable for us, with the occasional child death  
>>>> here and
>>>> there
>>>> when we get massive headers coming in that flood sess_workspace.  
>>>> I hear
>>>> this
>>>> is fixed in 2.0.3, but haven't had time to try it yet.
>>>> We have a number of search boxes, and each search box has an apache
>>>> instance
>>>> on it, and varnish instance. We plan to merge the varnish  
>>>> instances at
>>>> some
>>>> point, but we use very low TTLs (Twitter is the real time web!)  
>>>> and don't
>>>> see much of a savings by running less of them.
>>>> We do:
>>>> Apache --> Varnish --> Apache -> Mongrels
>>>> Apaches are using mod_proxy_balancer. The front end apache is there
>>>> because
>>>> we've long had a fear that Varnish would crash on us, which it  
>>>> did many
>>>> times prior to our figuring out the proper parameters for  
>>>> startup. We
>>>> have
>>>> two entries in that balancer. Either the request goes to varnish,  
>>>> or, if
>>>> varnish bombs out, it goes directly to the mongrel.
>>>> We do this, because we need a load balancing algorithm that varnish
>>>> doesn't
>>>> support, called bybusiness. Without bybusiness, varnish tries to  
>>>> direct
>>>> requests to Mongrels that are busy, and requests end up in the  
>>>> listen
>>>> queue.
>>>> that adds ~100-150mS to load times, and that's no good for our  
>>>> desired
>>>> service times of 200-250mS (or less.)
>>>> We'd be so happy if someone put bybusiness into Varnish's backend  
>>>> load
>>>> balancing, but it's not there yet.
>>>> We also know that taking the extra hop through localhost costs us  
>>>> next to
>>>> nothing in service time, so it's good to have Apache there incase  
>>>> we need
>>>> to
>>>> yank out Varnish. In the future, we might get rid of Apache and use
>>>> HAProxy
>>>> (it's load balancing and backend monitoring is much richer than  
>>>> Apache,
>>>> and,
>>>> it has a beautiful HTTP interface to look at.)
>>>> Some variables and our decisions:
>>>>             -p obj_workspace=4096 \
>>>>     -p sess_workspace=262144 \
>>>> Absolutely vital!  Varnish does not allocate enough space by  
>>>> default for
>>>> headers, regexps on cookies, and otherwise. It was increased in  
>>>> 2.0.3,
>>>> but
>>>> really, not increased enough. Without this we were panicing every  
>>>> 20-30
>>>> requests and overflowing the sess hash.
>>>>             -p listen_depth=8192 \
>>>> 8192 is probably excessive for now. If we're queuing 8k conns,  
>>>> something
>>>> is
>>>> really broke!
>>>>             -p log_hashstring=off \
>>>> Who cares about this - we don't need it.
>>>>     -p lru_interval=60 \
>>>> We have many small objects in the search cache. Run LRU more often.
>>>>             -p sess_timeout=10 \
>>>> If you keep session data around for too long, you waste memory.
>>>>     -p shm_workspace=32768 \
>>>> Give us a bit more room in shm
>>>>             -p ping_interval=1 \
>>>> Frequent pings in case the child dies on us.
>>>>             -p thread_pools=4 \
>>>>             -p thread_pool_min=100 \
>>>> This must match up with VARNISH_MIN_THREADS. We use four pools,  
>>>> (pools *
>>>> thread_pool_min == VARNISH_MIN_THREADS)
>>>>     -p srcaddr_ttl=0 \
>>>> Disable the (effectively unused) per source-IP statistics
>>>>     -p esi_syntax=1
>>>> Disable ESI syntax verification so we can use it to process JSON
>>>> requests.
>>>> If you have more than 2.1M objects, you should also add:
>>>> # -h classic,250007 = recommeded value for 2.1M objects
>>>> #     number should be 1/10 expected working set.
>>>>
>>>> In our VCL, we have a few fancy tricks that we use. We label the  
>>>> cache
>>>> server and cache hit/miss rate in vcl_deliver with this code:
>>>> Top of VCL:
>>>> C{
>>>> #include <stdio.h>
>>>> #include <unistd.h>
>>>> char myhostname[255] = "";
>>>>
>>>> }C
>>>> vcl_deliver:
>>>> C{
>>>>    VRT_SetHdr(sp, HDR_RESP, "\014X-Cache-Svr:", myhostname,
>>>> vrt_magic_string_end);
>>>> }C
>>>>    /* mark hit/miss on the request */
>>>>    if (obj.hits > 0) {
>>>>      set resp.http.X-Cache = "HIT";
>>>>      set resp.http.X-Cache-Hits = obj.hits;
>>>>    } else {
>>>>      set resp.http.X-Cache = "MISS";
>>>>    }
>>>>
>>>> vcl_recv:
>>>> C{
>>>>   if (myhostname[0] == '\0') {
>>>>     /* only get hostname once - restart required if hostname  
>>>> changes */
>>>>     gethostname(myhostname, 255);
>>>>   }
>>>> }C
>>>>
>>>> Portions of /etc/sysconfig/varnish follow...
>>>> # The minimum number of worker threads to start
>>>> VARNISH_MIN_THREADS=400
>>>> # The Maximum number of worker threads to start
>>>> VARNISH_MAX_THREADS=1000
>>>> # Idle timeout for worker threads
>>>> VARNISH_THREAD_TIMEOUT=60
>>>> # Cache file location
>>>> VARNISH_STORAGE_FILE=/var/lib/varnish/varnish_storage.bin
>>>> # Cache file size: in bytes, optionally using k / M / G / T suffix,
>>>> # or in percentage of available disk space using the % suffix.
>>>> VARNISH_STORAGE_SIZE="8G"
>>>> #
>>>> # Backend storage specification
>>>> VARNISH_STORAGE="malloc,${VARNISH_STORAGE_SIZE}"
>>>> # Default TTL used when the backend does not specify one
>>>> VARNISH_TTL=5
>>>> # the working directory
>>>> DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
>>>>             -f ${VARNISH_VCL_CONF} \
>>>>             -T
>>>> ${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
>>>>             -t ${VARNISH_TTL} \
>>>>    -n ${VARNISH_WORKDIR} \
>>>>             -w
>>>> ${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},$ 
>>>> {VARNISH_THREAD_TIMEOUT} \
>>>>             -u varnish -g varnish \
>>>>             -p obj_workspace=4096 \
>>>>    -p sess_workspace=262144 \
>>>>             -p listen_depth=8192 \
>>>>             -p log_hashstring=off \
>>>>    -p lru_interval=60 \
>>>>             -p sess_timeout=10 \
>>>>    -p shm_workspace=32768 \
>>>>             -p ping_interval=1 \
>>>>             -p thread_pools=4 \
>>>>             -p thread_pool_min=100 \
>>>>    -p srcaddr_ttl=0 \
>>>>    -p esi_syntax=1 \
>>>>             -s ${VARNISH_STORAGE}"
>>>>
>>>> ---
>>>> John Adams
>>>> Twitter Operations
>>>> jna at twitter.com
>>>> http://twitter.com/netik
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> VP of Product Development
>>> Instructables.com
>>>
>>> http://www.instructables.com/member/lebowski
>>> _______________________________________________
>>> varnish-dev mailing list
>>> varnish-dev at projects.linpro.no
>>> http://projects.linpro.no/mailman/listinfo/varnish-dev
>>
>>
>
>
>
> -- 
> VP of Product Development
> Instructables.com
>
> http://www.instructables.com/member/lebowski




More information about the varnish-dev mailing list