varnishd runtime parameters
Tung Nguyen
tnguyen at bleacherreport.com
Wed Mar 25 19:49:10 CET 2009
Hi guys,
So sometimes are backend is really slow in returning a response. So
slow that it looks like it
is causing varnish to timeout before the backend does and so varnish
seems to eventually give up
and returns a 503.
Ther are 4 curl requests
* the first 2 timeout and give a 503
* the 3rd one is a hit miss but gives a 200
* the 4th one is a cache hit and gives a 200
https://gist.github.com/0452c374ee21dbba138d
Here's the varnishlog filtered for client requests
http://gist.github.com/279254e0f2452814bf46
Here's varnishstat
https://gist.github.com/219534d51a503b546070
Here's are current startup run time parameters options right now:
https://gist.github.com/219534d51a503b546070
So, Im pretty certain what is happening is that our backend takes too
long and varnish times
out. How can I set the time out higher... is it sess_timeout?
Thanks guys,
Tung
On Wed, Mar 25, 2009 at 10:37 AM, Tung Nguyen <tnguyen at bleacherreport.com>wrote:
> Kristian, thank you.
>
> Im glad to hear that most defaults are good.
>
> Yup, the default thread_pool_min = 1 seems kinda funny. We'll set it to at
> least 100.
>
> I didnt even know about the cli_timeout and will set it to a base line of
> 10 seconds to test.
>
> Im wondering how you are testing, Im using ab, apachebench, to see how
> things behave with -c 10 -n 1000, on the varnished pages.
>
>
>
>
>
> Here's more specific questions more run time parameters. The general
> question I have is what to look for during testing, should I be looking at
> your varnishstat and are the most important things to look for in that
> output.
>
>
> Our varnish stack will look like this:
>
> LB -> Varnish x 2 -> Nginx x 6 -> Mongrel x 60
>
> Some questions about how best to decide how to configure them best to
> configure the run time parameters.
>
> -p obj_workspace=4096
> Cant find obj_workspace in the man page but found it in the twitter email
> post
> http://projects.linpro.no/pipermail/varnish-dev/2009-February/000968.html
>
> Is obj_workspace how much space preallocated to be used for the obj that
> gets returned from the backend? So, if my nginx backend returns a web page
> that is over 4MB than -p obj_workspace is not enough, would that crash
> varnish, or log the error somewhere.
>
> -p sess_workspace=262144
> Same deal here with the man page and twitter post.
> What is the sess_workspace?
>
> http_workspace
> How does sess_workspace and obj_workspace relate to http_workspace?
> If we use obj_workspace=4096 and sess_workspace=262144, does the default
> http_workspace=8192 make sense?
>
>
> -p lru_interval=60
> Shows up on the twitter post again, but no man notes yet. Whats the
> default for this?
>
> -p sess_timeout=10 \
> Default for this is 5. If the requests from the backend takes longer than
> 5 seconds, what happens? Sometimes we have really slow response from the
> backend..
>
> -p shm_workspace=32768 \
> Is this the same as setting the command line flag -l shmlogsize. The
> default is 80MB. So dont know twitter did both setting it to less..
>
> -p thread_pools=4 \
> -p thread_pool_min=100 \
> thread_pool_max
> The defaults are 1,1,1000 respectively. Im wondering how best to determine
> this or just leave as default.
>
>
>
>
>
> ######################################################################
> # output of varnishstat, what is best to look at here?
> ######################################################################
> 0+07:11:24
> Hitrate ratio: 4 4
> 4
> Hitrate avg: nan nan nan
>
> 400 0.00 0.02 Client connections accepted
> 400 0.00 0.02 Client requests received
> 1 0.00 0.00 Cache hits
> 5 0.00 0.00 Cache misses
> 399 0.00 0.02 Backend connections success
> 399 0.00 0.02 Backend connections failures
> 1 0.00 0.00 Backend connections reuses
> 5 0.00 0.00 Backend connections recycles
> 6 . . N struct srcaddr
> 21 . . N struct sess_mem
> 1 . . N struct sess
> 1 . . N struct object
> 1 . . N struct objecthead
> 3 . . N struct smf
> 1 . . N small free smf
> 1 . . N large free smf
> 1 . . N struct vbe_conn
> 2 . . N struct bereq
> 10 . . N worker threads
> 23 0.00 0.00 N worker threads created
> 76 0.00 0.00 N overflowed work requests
> 2 . . N backends
> 5 . . N expired objects
> 6 0.00 0.00 Objects sent with write
> 400 0.00 0.02 Total Sessions
> 400 0.00 0.02 Total Requests
> 394 0.00 0.02 Total pipe
> 5 0.00 0.00 Total fetch
> 352131 0.00 13.60 Total header bytes
> 1745 0.00 0.07 Total body bytes
> 397 0.00 0.02 Session Closed
> 3 0.00 0.00 Session herd
> 33185 2.00 1.28 SHM records
> 19306 2.00 0.75 SHM writes
> 32 0.00 0.00 SHM MTX contention
> 11 0.00 0.00 allocator requests
> 1 . . outstanding allocations
> 8192 . . bytes allocated
> 6612459520 . . bytes free
> 5 0.00 0.00 Backend requests made
> 1 0.00 0.00 N vcl total
> 1 0.00 0.00 N vcl available
> 1 . . N total active purges
> 1 0.00 0.00 N new purges added
>
>
>
>
>
>
>
> Any response is appreciated.
>
> Thanks,
> Tung
>
>
>
>
>
>
>
>
>
>
> On Tue, Mar 24, 2009 at 12:20 AM, Kristian Lyngstol <
> kristian at redpill-linpro.com> wrote:
>
>> On Mon, Mar 23, 2009 at 05:58:58PM -0700, Tung Nguyen wrote:
>> > Hi guys,
>> > So, Im reading over an archive email thread about twitters
>> configuration.
>> >
>> >
>> http://projects.linpro.no/pipermail/varnish-dev/2009-February/000968.html
>> >
>> > It looks like they had to adjust a lot of parameters... and Im not
>> finding
>> > all the parameters definitions in the varnishd man pages. Im wondering
>> if
>> > for most cases running varnish with the defaults is fine?
>> >
>> > Any caveats here which run time parameters should I focus on.
>>
>> You can mostly run it with the defaults, yes. This depends on what sort of
>> usage you have though.
>>
>> One notable exception is that I strongly recommend that you bring
>> thread_pool_min up to a decent level (reflecting how many users you
>> actually have. Numbers in the hundreds is normal). You'll also want to
>> adjust the cache size to your system, but that goes without saying.
>>
>> If you expect extremely high load, you might have to increase cli_timeout
>> too. I've run tests where even setting it to 15 seconds is insufficient
>> and
>> causes childs to be killed off. Though for most production sites, I'd
>> guess
>> 5 seconds could work and 10 seconds would definitely work.
>>
>> --
>> Kristian Lyngstøl
>> Redpill Linpro AS
>> Tlf: +47 21544179
>> Mob: +47 99014497
>>
>
>
>
> --
> Tung Nguyen, Lead Developer
> Bleacher Report, The Open Source Sports Network
> (510) 928-0475
>
--
Tung Nguyen, Lead Developer
Bleacher Report, The Open Source Sports Network
(510) 928-0475
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20090325/2cbc6565/attachment-0001.html>
More information about the varnish-misc
mailing list