Varnish High CPU Usage / High Load Average

[Joseph Begumisa]

Please disregard.   After using only a basic vcl file, I noticed that
cpu usage and load average was back to normal.  So I went through the
customized default.vcl and noticed my purging syntax just might be the
cause of the problem.  After correcting it and running more tests, the
issue of high cpu usage and high load average no longer existed.

Best Regards,

Joseph



On Fri, Feb 11, 2011 at 3:12 PM, Joseph Begumisa <j.begumisa at gmail.com> wrote:
> Hi,
>
> I'm having an issue where when I start varnish, CPU usage rises in
> about 3minutes to about 95% and load average rises to about 150.  I
> then have to stop varnish and restart it before the system becomes
> unresponsive.
>
> Specs of the server i'm using are: Dual Quad-Core Intel Xeon CPU E5440
> @ 2.83GHz with 16GB RAM.
>
> I'm running version 2.1.5 and varnishd is started with the following options:
>
> DAEMON_OPTS="-a 10.0.6.6:80
>             -T localhost:6082 \
>             -f /opt/varnish/etc/varnish/default.vcl \
>             -p thread_pools=8 \
>             -p thread_pool_min=100 \
>             -p thread_pool_max=2000 \
>             -p thread_pool_add_delay=2 \
>             -p cli_timeout=25 \
>             -p session_linger=50/100/150 \
>             -h classic,30007 \
>             -p listen_depth=4096 \
>             -P /var/run/varnish/varnish.pid \
>             -u varnish -g varnish \
>             -s malloc,4G"
>
> my default.vcl file is pretty basic for now:
>
> backend web1 {
>     .host = "10.0.6.66";
>     .port = "80";
> }
>
> I was running varnishstat and obtained these stats immediately after I
> stopped the varnish service since cpu usage was approaching 100%:
>
> 0+00:03:03
> Hitrate ratio:       10      100      145
> Hitrate avg:     0.9814   0.9796   0.9794
>
>       40090         0.00       219.07 Client connections accepted
>      143712         0.00       785.31 Client requests received
>       95786         0.00       523.42 Cache hits
>       11834         0.00        64.67 Cache hits for pass
>        2321         0.00        12.68 Cache misses
>        3096         0.00        16.92 Backend conn. success
>       44665         0.00       244.07 Backend conn. reuses
>           6         0.00         0.03 Backend conn. was closed
>       45827         0.00       250.42 Backend conn. recycles
>          87         0.00         0.48 Fetch head
>       29726         0.00       162.44 Fetch with Length
>        3339         0.00        18.25 Fetch chunked
>          52         0.00         0.28 Fetch wanted close
>           4         0.00         0.02 Fetch failed
>        1583          .            .   N struct sess_mem
>        1376          .            .   N struct sess
>        2314          .            .   N struct object
>        3037          .            .   N struct objectcore
>        3037          .            .   N struct objecthead
>         267          .            .   N struct vbe_conn
>         800          .            .   N worker threads
>         800         0.00         4.37 N worker threads created
>           1          .            .   N backends
>       11612          .            .   N LRU moved objects
>       60661         0.00       331.48 Objects sent with write
>       39967         0.00       218.40 Total Sessions
>      143712         0.00       785.31 Total Requests
>        1399         0.00         7.64 Total pipe
>       44178         0.00       241.41 Total pass
>       46203         0.00       252.48 Total fetch
>    51252168         0.00    280066.49 Total header bytes
>   795295815         0.00   4345878.77 Total body bytes
>        7931         0.00        43.34 Session Closed
>
>
> Any insights into what would cause that / changes I need to make would
> be helpful.  Thanks.
>
>
> Joseph
>