Fwd: infrastructure setup using ESI

[thebog]

Oppss, forgot to include the list.

---------- Forwarded message ----------
From: thebog <thebog at gmail.com>
Date: Fri, Oct 22, 2010 at 1:38 AM
Subject: Re: infrastructure setup using ESI
To: "Gresens, August" <AGresens at scholastic.com>


Hi,

I'll try and give some advice in general terms. You are asking if this rope
is long enuff, and that question has no good answer :)

Varnish is "most often" used close to the client (in front or behind
loadbalancer). A main reason is that it handles alot of webtraffic good and
cheap. This is where it's written to shine. But it shines in other layers as
well. depending on what you are trying to do:

How many documents do you have?
Is your backend good?
How much traffic is backend seeing?
Does backend scale good/cheap?
What's your budget?
What hardware do you have/plan on getting?

Varnish works good in front of crappy backend, because it's easy/flexible
(configurationwise) to reduce/shape traffic to backend.
If your hot dataset is huge, you will need much RAM, and there the budget
kicks in. Some people dream of 64 GB RAM, some buy it by default, and for
some 64 GB is such a little part of the dataset that it really does not help
much. While others have a dataset where 4 GB RAM is more than enuff, so it's
cheap to do it. For some it does not matter in if there are separate cached
copy's, because it's free, so why not. For some it does. Hard to answer :)
So are the secondary requests to the backend.

With Varnish you can manipulate HTTP headers that otherwise does not make
sense because you have a backend you can not tweak to get what you want (Or
this is more expensive than using Varnish). Some use Varnish in front of
many different backend systems, some use it in front of crappy closed source
backends. Many find VCL (Varnish configuration laguage) to be really good
and easy (some don't bother ever to learn it because they don't need it), I
can't think of many scenarios where you can't get VCL to do what ever you'd
like with HTTP and to some extent most of the HTTP traffic.

So I would use Varnish where I have much traffic, where I need to reduce
traffic, and where I would like to shape traffic if I had to. It's a tool,
when you learn how to use it, you know when it makes sense to use it. And it
can be used many places in the stack.

I would also not install anything other than a 64 bit OS. And focus on
getting the right amount of RAM. As much as possible is always a good start
:)

Hope this answered some of your questions :) Sorry for the rant.

YS
Anders Berg

On Thu, Oct 21, 2010 at 11:14 PM, Gresens, August
<AGresens at scholastic.com>wrote:

>  Hello
>
>
>
> Varnish is a key part of a new set of apps we are building - the ESI
> feature in particular is something we want to take advantage of extensively.
>
>
>
> Where does Varnish typically reside in the network architecture? I would
> imagine in a scenario in which we would be using ESI, we would want Varnish
> in front of the application load balancers. In this way, Varnish would be
> the primary page assembler. We would likely run an HA setup for fault
> tolerance.
>
>
>
> We were also considering if Varnish could be used much farther downstream –
> in between the load balancer and the application. We thought is might
> provide for more fault tolerance, but it seems the disadvantage would be
> that there would be a separate cached copy in each redundant varnish
> instance, and with ESI secondary requests would be made back out to the same
> web server potentially, setting up a feed back loop that could cause
> problems.
>
>
>
> Thanks for your responses in advance.
>
>
>
> August
>
>
>
>
>
> August Gresens
> Director of Technology, eScholastic
>
> Scholastic Inc.
> 557 Broadway, NY, NY 10012
>
> Email :: agresens at scholastic.com
> Phone :: 917 363 3662
>
>
>
>
>
>
>
>
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at varnish-cache.org
> http://lists.varnish-cache.org/mailman/listinfo/varnish-misc
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20101022/877f928f/attachment-0003.html>