Question about threads

[Rafael Umann]

If it is a 32bits system, probably the problem is that your stack size
is 10Mb. So 238 * 10mb = ~2gb

I decreased my stack size to 512Kb. Using 1gb storage files i can now
open almost 1900 threads using all the 2gb that 32bits can alloc. So, my
Varnish makes 20000 hits/second serving clients!

##### linux

# cat /etc/security/limits.conf 
* soft nofile 131072
* hard nofile 131072
* soft stack 512
* hard stack 512


# cat /etc/sysctl.conf

# Tuning webservers kernel >= 2.6.22 - Rafael Umann 20080502
net.core.somaxconn=8192
net.ipv4.tcp_timestamps=0
net.ipv4.tcp_fin_timeout=3
net.ipv4.tcp_keepalive_time=30
net.ipv4.tcp_max_orphans=262144
net.ipv4.tcp_max_syn_backlog=262144

net.ipv4.tcp_max_tw_buckets=1440000
net.ipv4.tcp_tw_recycle=1
net.ipv4.tcp_tw_reuse=1
net.ipv4.ip_local_port_range=2048 61000
net.core.rmem_max=33554432
net.core.wmem_max=33554432

net.ipv4.tcp_syncookies=1
net.ipv4.tcp_abort_on_overflow=1

net.core.rmem_default=65536
net.core.wmem_default=65536
net.core.netdev_max_backlog=262144

net.ipv4.tcp_rmem=4096 87380 16777216
net.ipv4.tcp_wmem=4096 65536 16777216
net.ipv4.tcp_mem=16777216 16777216 16777216

sunrpc.tcp_slot_table_entries=128
sunrpc.udp_slot_table_entries=128

fs.file-max=5049800

vm.min_free_kbytes=204800
vm.lowmem_reserve_ratio=256 256
vm.page-cluster=15
vm.swappiness=100
vm.overcommit_memory=1

# DSR - DO NOT USE IT IF YOU DONT USE DIRECT SERVER RESPONSE!!!!!
net.ipv4.conf.all.arp_ignore=1
net.ipv4.conf.all.arp_announce=2

# Tests with new parameters - Rafael Umann 20080613
net.ipv4.tcp_no_metrics_save=1

net.core.somaxconn = 262144
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_synack_retries = 2
net.ipv4.tcp_syn_retries = 2


Try it out and let us now!

[]s,


On Tue, 17 Jun 2008 12:37:08 +0200
<duja at torlen.net> wrote:

> I recently made a loadtest against through varnish. 
> 
> First I received a very high response time and found out that varnish
> was maxing the maximum nr of threads. 
> 
> I updated thread_min = 5 and thread_max = 300 and recevied much
> better resp. times.
> 
> Then I increased the nr of concurrent users and made another
> loadtest. The strange thing here was that I received high resp. times
> but the threads stopped at 238.
> 
> The "N worker threads not created" increased rapidly.
> 
> I increased the threads again and changed listen_depth to 2048.
> 
> Here is all the numbers:
>          238         0.00         0.22 N worker threads created
>         1318         4.98         1.21 N worker threads not created
> 
>     0 Debug        - "Create worker thread failed 12 Cannot allocate
> memory" 0 Debug        - "Create worker thread failed 12 Cannot
> allocate memory" 0 Debug        - "Create worker thread failed 12
> Cannot allocate memory" 0 Debug        - "Create worker thread failed
> 12 Cannot allocate memory" 0 Debug        - "Create worker thread
> failed 12 Cannot allocate memory" 0 Debug        - "Create worker
> thread failed 12 Cannot allocate memory" 0 Debug        - "Create
> worker thread failed 12 Cannot allocate memory" 0 Debug        -
> "Create worker thread failed 12 Cannot allocate memory"
> 
> default_ttl                120 [seconds]
> thread_pools               2 [pools] <<<<
> thread_pool_max            400 [threads] <<<<
> thread_pool_min            10 [threads] <<<<
> thread_pool_timeout        120 [seconds]
> thread_pool_purge_delay    1000 [milliseconds]
> thread_pool_add_threshold  2 [requests]
> thread_pool_add_delay      10 [milliseconds]
> thread_pool_fail_delay     200 [milliseconds]
> overflow_max               100 [%]
> rush_exponent              3 [requests per request]
> sess_workspace             8192 [bytes]
> obj_workspace              8192 [bytes]
> sess_timeout               5 [seconds]
> pipe_timeout               60 [seconds]
> send_timeout               600 [seconds]
> auto_restart               on [bool]
> fetch_chunksize            128 [kilobytes]
> vcl_trace                  off [bool]
> listen_address             0.0.0.0:80
> listen_depth               2048 [connections] <<<<
> srcaddr_hash               1049 [buckets]
> srcaddr_ttl                30 [seconds]
> backend_http11             off [bool]
> client_http11              off [bool]
> cli_timeout                5 [seconds]
> ping_interval              3 [seconds]
> lru_interval               2 [seconds]
> cc_command                 exec cc -fpic -shared -Wl,-x -o %o %s
> max_restarts               4 [restarts]
> max_esi_includes           5 [restarts]
> cache_vbe_conns            off [bool]
> connect_timeout            400 [ms]
> cli_buffer                 8192 [bytes]
> diag_bitmap                0x0 [bitmap]
> 
> Why do I get "Create worker thread failed 12 Cannot allocate memory"
> when I had 1900MB free RAM and 65GB free Disk on the server? Any
> ideas?
> 
> If "N worker threads not created" is increasing, is that a bad sign?
> 
> Thanks
> Duja
> 
> _______________________________________________
> varnish-misc mailing list
> varnish-misc at projects.linpro.no
> http://projects.linpro.no/mailman/listinfo/varnish-misc
> 
> E-mail verificado pelo Terra Anti-Spam.
> Para classificar como spam ou não spam, visite
> http://mail.terra.com.br/cgi-bin/reportspam.cgi?+_d=SCY1ODAyNDQ3I3Blcm0hdGVycmEmMSwxMjEzNjk5MDI5LjExNzYyNy4yOTM5OS50cmlidW5lLnRlcnJhLmNvbSw1OTIz
> Verifique periodicamente a pasta Spam para garantir que apenas
> mensagens indesejadas sejam classificadas como Spam.
> 
> Esta mensagem foi verificada pelo E-mail Protegido Terra.
> Atualizado em 17/06/2008
> 


-- 
Rafael Umann <rafael.umann at corp.terra.com.br>
Suporte Engenharia 1
Terra Networks Brasil S/A 
Tel: 55 (51) 3284-4344