not sure if Varnish is working correctly
Mark Strickland
smsmail at roadrunner.com
Fri Sep 28 01:25:44 CEST 2012
I have varnish installed on my Linode VPS, I am getting good hit rates, but
site still seems to me to be slow loading. I think I have it setup to use
128M of memory instead of using a file. I am sorry if I gave to much
information or the wrong information. Thanks in advance for any help or
guidance.
Here is my configuration file:
# Configuration file for varnish
#
# /etc/init.d/varnish expects the variables $DAEMON_OPTS, $NFILES and
$MEMLOCK
# to be set from this shell script fragment.
#
# Should we start varnishd at boot? Set to "no" to disable.
START=yes
# Maximum number of open files (for ulimit -n)
NFILES=131072
# Maximum locked memory size (for ulimit -l)
# Used for locking the shared memory log in memory. If you increase log
size,
# you need to increase this number as well
MEMLOCK=82000
# Default varnish instance name is the local nodename. Can be overridden
with
# the -n switch, to have more instances on a single server.
# INSTANCE=$(uname -n)
# This file contains 4 alternatives, please use only one.
## Alternative 1, Minimal configuration, no VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# content server on localhost:8080. Use a 1GB fixed-size cache file.
#
# DAEMON_OPTS="-a :6081 \
# -T localhost:6082 \
# -b localhost:8080 \
# -u varnish -g varnish \
# -S /etc/varnish/secret \
# -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
## Alternative 2, Configuration with VCL
#
# Listen on port 6081, administration on localhost:6082, and forward to
# one content server selected by the vcl file, based on the request. Use a
1GB
# fixed-size cache file.
#
DAEMON_OPTS="-a :80 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,128M"
# -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,128M"
## Alternative 3, Advanced configuration
#
# See varnishd(1) for more information.
#
# # Main configuration file. You probably want to change it :)
# VARNISH_VCL_CONF=/etc/varnish/default.vcl
#
# # Default address and port to bind to
# # Blank address means all IPv4 and IPv6 interfaces, otherwise specify
# # a host name, an IPv4 dotted quad, or an IPv6 address in brackets.
# VARNISH_LISTEN_ADDRESS=
# VARNISH_LISTEN_PORT=6081
#
# # Telnet admin interface listen address and port
# VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1
# VARNISH_ADMIN_LISTEN_PORT=6082
#
# # The minimum number of worker threads to start
# VARNISH_MIN_THREADS=1
#
# # The Maximum number of worker threads to start
# VARNISH_MAX_THREADS=1000
#
# # Idle timeout for worker threads
# VARNISH_THREAD_TIMEOUT=120
#
# # Cache file location
# VARNISH_STORAGE_FILE=/var/lib/varnish/$INSTANCE/varnish_storage.bin
#
# # Cache file size: in bytes, optionally using k / M / G / T suffix,
# # or in percentage of available disk space using the % suffix.
# VARNISH_STORAGE_SIZE=1G
#
# # File containing administration secret
# VARNISH_SECRET_FILE=/etc/varnish/secret
#
# # Backend storage specification
# VARNISH_STORAGE="file,${VARNISH_STORAGE_FILE},${VARNISH_STORAGE_SIZE}"
#
# # Default TTL used when the backend does not specify one
# VARNISH_TTL=120
#
# # DAEMON_OPTS is used by the init script. If you add or remove options,
make
# # sure you update this section, too.
# DAEMON_OPTS="-a ${VARNISH_LISTEN_ADDRESS}:${VARNISH_LISTEN_PORT} \
# -f ${VARNISH_VCL_CONF} \
# -T
${VARNISH_ADMIN_LISTEN_ADDRESS}:${VARNISH_ADMIN_LISTEN_PORT} \
# -t ${VARNISH_TTL} \
# -w
${VARNISH_MIN_THREADS},${VARNISH_MAX_THREADS},${VARNISH_THREAD_TIMEOUT} \
# -S ${VARNISH_SECRET_FILE} \
# -s ${VARNISH_STORAGE}"
#
## Alternative 4, Do It Yourself
#
# DAEMON_OPTS=""
here is my default vcl
# This is a basic VCL configuration file for varnish. See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# TODO: Update internal subnet ACL and security.
# Define the internal network subnet.
# These are used below to allow internal access to certain files while not
# allowing access from the public internet.
# acl internal {
# "192.10.0.0"/24;
# }
# Default backend definition. Set this to point to your content
# server.
#
backend default {
.host = "127.0.0.1";
.port = "8080";
}
# Respond to incoming requests.
sub vcl_recv {
# Use anonymous, cached pages if all backends are down.
if (!req.backend.healthy) {
unset req.http.Cookie;
}
# Allow the backend to serve up stale content if it is responding slowly.
set req.grace = 6h;
# Pipe these paths directly to Apache for streaming.
#if (req.url ~ "^/admin/content/backup_migrate/export") {
# return (pipe);
#}
# Do not cache these paths.
if (req.url ~ "^/status\.php$" ||
req.url ~ "^/update\.php$" ||
req.url ~ "^/admin$" ||
req.url ~ "^/admin/.*$" ||
req.url ~ "^/flag/.*$" ||
req.url ~ "^.*/munin/.*$" ||
req.url ~ "^.*/ajax/.*$" ||
req.url ~ "^.*/api/.*$" ||
req.url ~ "^.*/apidev/.*$" ||
req.url ~ "^.*/serviioweb/.*$" ||
req.url ~ "^.*/mediabrowser/.*$" ||
req.url ~ "^.*/phpmyadmin/.*$" ||
req.url ~ "^.*/ahah/.*$") {
return (pass);
}
# Do not allow outside access to cron.php or install.php.
#if (req.url ~ "^/(cron|install)\.php$" && !client.ip ~ internal) {
# Have Varnish throw the error directly.
# error 404 "Page not found.";
# Use a custom error page that you've defined in Drupal at the path
"404".
# set req.url = "/404";
#}
# Always cache the following file types for all users. This list of
extensions
# appears twice, once here and again in vcl_fetch so make sure you edit
both
# and keep them equal.
if (req.url ~
"(?i)\.(pdf|asc|dat|txt|doc|xls|ppt|tgz|csv|png|gif|jpeg|jpg|ico|swf|css|js)
(\?.*)?$") {
unset req.http.Cookie;
}
# Remove all cookies that Drupal doesn't need to know about. We explicitly
# list the ones that Drupal does need, the SESS and NO_CACHE. If, after
# running this code we find that either of these two cookies remains, we
# will pass as the page cannot be cached.
if (req.http.Cookie) {
# 1. Append a semi-colon to the front of the cookie string.
# 2. Remove all spaces that appear after semi-colons.
# 3. Match the cookies we want to keep, adding the space we removed
# previously back. (\1) is first matching group in the regsuball.
# 4. Remove all other cookies, identifying them by the fact that they
have
# no space after the preceding semi-colon.
# 5. Remove all spaces and semi-colons from the beginning and end of the
# cookie string.
set req.http.Cookie = ";" + req.http.Cookie;
set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
set req.http.Cookie = regsuball(req.http.Cookie,
";(SESS[a-z0-9]+|NO_CACHE)=", "; \1=");
set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");
if (req.http.Cookie == "") {
# If there are no remaining cookies, remove the cookie header. If
there
# aren't any cookie headers, Varnish's default behavior will be to
cache
# the page.
unset req.http.Cookie;
}
else {
# If there is any cookies left (a session or NO_CACHE cookie), do not
# cache the page. Pass it on to Apache directly.
return (pass);
}
}
}
# Set a header to track a cache HIT/MISS.
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Varnish-Cache = "HIT";
}
else {
set resp.http.X-Varnish-Cache = "MISS";
}
}
# Code determining what to do when serving items from the Apache servers.
# beresp == Back-end response from the web server.
sub vcl_fetch {
# We need this to cache 404s, 301s, 500s. Otherwise, depending on backend
but
# definitely in Drupal's case these responses are not cacheable by
default.
if (beresp.status == 404 || beresp.status == 301 || beresp.status == 500)
{
set beresp.ttl = 10m;
}
if(beresp.status == 302 && !beresp.http.Location){
set beresp.status = 200;
set beresp.response = "OK";
}
# Don't allow static files to set cookies.
# (?i) denotes case insensitive in PCRE (perl compatible regular
expressions).
# This list of extensions appears twice, once here and again in vcl_recv
so
# make sure you edit both and keep them equal.
if (req.url ~
"(?i)\.(pdf|asc|dat|txt|doc|xls|ppt|tgz|csv|png|gif|jpeg|jpg|ico|swf|css|js)
(\?.*)?$") {
unset beresp.http.set-cookie;
}
# Allow items to be stale if needed.
set beresp.grace = 6h;
}
# In the event of an error, show friendlier messages.
sub vcl_error {
# Redirect to some other URL in the case of a homepage failure.
#if (req.url ~ "^/?$") {
# set obj.status = 302;
# set obj.http.Location = "http://backup.example.com/";
#}
# Otherwise redirect to the homepage, which will likely be in the cache.
set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
<html>
<head>
<title>Page Unavailable</title>
<style>
body { background: #303030; text-align: center; color: white; }
#page { border: 1px solid #CCC; width: 500px; margin: 100px auto 0;
padding: 30px; background: #323232; }
a, a:link, a:visited { color: #CCC; }
.error { color: #222; }
</style>
</head>
<body onload="setTimeout(function() { window.location = '/' }, 5000)">
<div id="page">
<h1 class="title">Page Unavailable</h1>
<p>The page you requested is temporarily unavailable.</p>
<p>We're redirecting you to the <a href="/">homepage</a> in 5
seconds.</p>
<div class="error">(Error "} + obj.status + " " + obj.response +
{")</div>
</div>
</body>
</html>
"};
return (deliver);
}
#
# Below is a commented-out copy of the default VCL logic. If you
# redefine any of these subroutines, the built-in logic will be
# appended to your code.
# sub vcl_recv {
# if (req.restarts == 0) {
# if (req.http.x-forwarded-for) {
Here is a copy of the waterfall chart from webpagetest
Here is the results from varnishstat -1
client_conn 7179 0.42 Client connections accepted
client_drop 0 0.00 Connection dropped, no sess/wrk
client_req 7674 0.45 Client requests received
cache_hit 5474 0.32 Cache hits
cache_hitpass 1 0.00 Cache hits for pass
cache_miss 1682 0.10 Cache misses
backend_conn 1150 0.07 Backend conn. success
backend_unhealthy 0 0.00 Backend conn. not attempted
backend_busy 0 0.00 Backend conn. too many
backend_fail 0 0.00 Backend conn. failures
backend_reuse 1049 0.06 Backend conn. reuses
backend_toolate 922 0.05 Backend conn. was closed
backend_recycle 1973 0.12 Backend conn. recycles
backend_retry 0 0.00 Backend conn. retry
fetch_head 0 0.00 Fetch head
fetch_length 1880 0.11 Fetch with Length
fetch_chunked 85 0.00 Fetch chunked
fetch_eof 0 0.00 Fetch EOF
fetch_bad 0 0.00 Fetch had bad headers
fetch_close 227 0.01 Fetch wanted close
fetch_oldhttp 0 0.00 Fetch pre HTTP/1.1 closed
fetch_zero 0 0.00 Fetch zero len
fetch_failed 0 0.00 Fetch failed
fetch_1xx 0 0.00 Fetch no body (1xx)
fetch_204 0 0.00 Fetch no body (204)
fetch_304 8 0.00 Fetch no body (304)
n_sess_mem 26 . N struct sess_mem
n_sess 1 . N struct sess
n_object 1682 . N struct object
n_vampireobject 0 . N unresurrected objects
n_objectcore 1691 . N struct objectcore
n_objecthead 1445 . N struct objecthead
n_waitinglist 10 . N struct waitinglist
n_vbc 1 . N struct vbc
n_wrk 10 . N worker threads
n_wrk_create 10 0.00 N worker threads created
n_wrk_failed 0 0.00 N worker threads not created
n_wrk_max 0 0.00 N worker threads limited
n_wrk_lqueue 0 0.00 work request queue length
n_wrk_queued 0 0.00 N queued work requests
n_wrk_drop 0 0.00 N dropped work requests
n_backend 1 . N backends
n_expired 0 . N expired objects
n_lru_nuked 0 . N LRU nuked objects
n_lru_moved 5090 . N LRU moved objects
losthdr 0 0.00 HTTP header overflows
n_objsendfile 0 0.00 Objects sent with sendfile
n_objwrite 7411 0.43 Objects sent with write
n_objoverflow 0 0.00 Objects overflowing workspace
s_sess 7179 0.42 Total Sessions
s_req 7674 0.45 Total Requests
s_pipe 0 0.00 Total pipe
s_pass 518 0.03 Total pass
s_fetch 2200 0.13 Total fetch
s_hdrbytes 4394386 257.09 Total header bytes
s_bodybytes 369487008 21616.28 Total body bytes
sess_closed 602 0.04 Session Closed
sess_pipeline 0 0.00 Session Pipeline
sess_readahead 0 0.00 Session Read Ahead
sess_linger 7117 0.42 Session Linger
sess_herd 7562 0.44 Session herd
shm_records 488273 28.57 SHM records
shm_writes 53496 3.13 SHM writes
shm_flushes 0 0.00 SHM flushes due to overflow
shm_cont 19 0.00 SHM MTX contention
shm_cycles 0 0.00 SHM cycles through buffer
sms_nreq 0 0.00 SMS allocator requests
sms_nobj 0 . SMS outstanding allocations
sms_nbytes 0 . SMS outstanding bytes
sms_balloc 0 . SMS bytes allocated
sms_bfree 0 . SMS bytes freed
backend_req 2200 0.13 Backend requests made
n_vcl 1 0.00 N vcl total
n_vcl_avail 1 0.00 N vcl available
n_vcl_discard 0 0.00 N vcl discarded
n_ban 1 . N total active bans
n_ban_gone 1 . N total gone bans
n_ban_add 1 0.00 N new bans added
n_ban_retire 0 0.00 N old bans deleted
n_ban_obj_test 0 0.00 N objects tested
n_ban_re_test 0 0.00 N regexps tested against
n_ban_dups 0 0.00 N duplicate bans removed
hcb_nolock 7155 0.42 HCB Lookups without lock
hcb_lock 1436 0.08 HCB Lookups with lock
hcb_insert 1436 0.08 HCB Inserts
esi_errors 0 0.00 ESI parse errors (unlock)
esi_warnings 0 0.00 ESI parse warnings (unlock)
accept_fail 0 0.00 Accept failures
client_drop_late 0 0.00 Connection dropped late
uptime 17093 1.00 Client uptime
dir_dns_lookups 0 0.00 DNS director lookups
dir_dns_failed 0 0.00 DNS director failed lookups
dir_dns_hit 0 0.00 DNS director cached lookups hit
dir_dns_cache_full 0 0.00 DNS director full dnscache
vmods 0 . Loaded VMODs
n_gzip 0 0.00 Gzip operations
n_gunzip 6757 0.40 Gunzip operations
LCK.sms.creat 1 0.00 Created locks
LCK.sms.destroy 0 0.00 Destroyed locks
LCK.sms.locks 0 0.00 Lock Operations
LCK.sms.colls 0 0.00 Collisions
LCK.smp.creat 0 0.00 Created locks
LCK.smp.destroy 0 0.00 Destroyed locks
LCK.smp.locks 0 0.00 Lock Operations
LCK.smp.colls 0 0.00 Collisions
LCK.sma.creat 2 0.00 Created locks
LCK.sma.destroy 0 0.00 Destroyed locks
LCK.sma.locks 7470 0.44 Lock Operations
LCK.sma.colls 0 0.00 Collisions
LCK.smf.creat 0 0.00 Created locks
LCK.smf.destroy 0 0.00 Destroyed locks
LCK.smf.locks 0 0.00 Lock Operations
LCK.smf.colls 0 0.00 Collisions
LCK.hsl.creat 0 0.00 Created locks
LCK.hsl.destroy 0 0.00 Destroyed locks
LCK.hsl.locks 0 0.00 Lock Operations
LCK.hsl.colls 0 0.00 Collisions
LCK.hcb.creat 1 0.00 Created locks
LCK.hcb.destroy 0 0.00 Destroyed locks
LCK.hcb.locks 1531 0.09 Lock Operations
LCK.hcb.colls 0 0.00 Collisions
LCK.hcl.creat 0 0.00 Created locks
LCK.hcl.destroy 0 0.00 Destroyed locks
LCK.hcl.locks 0 0.00 Lock Operations
LCK.hcl.colls 0 0.00 Collisions
LCK.vcl.creat 1 0.00 Created locks
LCK.vcl.destroy 0 0.00 Destroyed locks
LCK.vcl.locks 12 0.00 Lock Operations
LCK.vcl.colls 0 0.00 Collisions
LCK.stat.creat 1 0.00 Created locks
LCK.stat.destroy 0 0.00 Destroyed locks
LCK.stat.locks 7204 0.42 Lock Operations
LCK.stat.colls 0 0.00 Collisions
LCK.sessmem.creat 1 0.00 Created locks
LCK.sessmem.destroy 0 0.00 Destroyed locks
LCK.sessmem.locks 7547 0.44 Lock Operations
LCK.sessmem.colls 0 0.00 Collisions
LCK.wstat.creat 1 0.00 Created locks
LCK.wstat.destroy 0 0.00 Destroyed locks
LCK.wstat.locks 34286 2.01 Lock Operations
LCK.wstat.colls 0 0.00 Collisions
LCK.herder.creat 1 0.00 Created locks
LCK.herder.destroy 0 0.00 Destroyed locks
LCK.herder.locks 1 0.00 Lock Operations
LCK.herder.colls 0 0.00 Collisions
LCK.wq.creat 2 0.00 Created locks
LCK.wq.destroy 0 0.00 Destroyed locks
LCK.wq.locks 50522 2.96 Lock Operations
LCK.wq.colls 0 0.00 Collisions
LCK.objhdr.creat 1445 0.08 Created locks
LCK.objhdr.destroy 0 0.00 Destroyed locks
LCK.objhdr.locks 28872 1.69 Lock Operations
LCK.objhdr.colls 0 0.00 Collisions
LCK.exp.creat 1 0.00 Created locks
LCK.exp.destroy 0 0.00 Destroyed locks
LCK.exp.locks 18773 1.10 Lock Operations
LCK.exp.colls 0 0.00 Collisions
LCK.lru.creat 2 0.00 Created locks
LCK.lru.destroy 0 0.00 Destroyed locks
LCK.lru.locks 1682 0.10 Lock Operations
LCK.lru.colls 0 0.00 Collisions
LCK.cli.creat 1 0.00 Created locks
LCK.cli.destroy 0 0.00 Destroyed locks
LCK.cli.locks 5707 0.33 Lock Operations
LCK.cli.colls 0 0.00 Collisions
LCK.ban.creat 1 0.00 Created locks
LCK.ban.destroy 0 0.00 Destroyed locks
LCK.ban.locks 18775 1.10 Lock Operations
LCK.ban.colls 0 0.00 Collisions
LCK.vbp.creat 1 0.00 Created locks
LCK.vbp.destroy 0 0.00 Destroyed locks
LCK.vbp.locks 0 0.00 Lock Operations
LCK.vbp.colls 0 0.00 Collisions
LCK.vbe.creat 1 0.00 Created locks
LCK.vbe.destroy 0 0.00 Destroyed locks
LCK.vbe.locks 2299 0.13 Lock Operations
LCK.vbe.colls 0 0.00 Collisions
LCK.backend.creat 1 0.00 Created locks
LCK.backend.destroy 0 0.00 Destroyed locks
LCK.backend.locks 7394 0.43 Lock Operations
LCK.backend.colls 0 0.00 Collisions
SMA.s0.c_req 3364 0.20 Allocator requests
SMA.s0.c_fail 0 0.00 Allocator failures
SMA.s0.c_bytes 215415613 12602.56 Bytes allocated
SMA.s0.c_freed 204713824 11976.47 Bytes freed
SMA.s0.g_alloc 3233 . Allocations outstanding
SMA.s0.g_bytes 10701789 . Bytes outstanding
SMA.s0.g_space 123515939 . Bytes available
SMA.Transient.c_req 1028 0.06 Allocator requests
SMA.Transient.c_fail 0 0.00 Allocator failures
SMA.Transient.c_bytes 42146510 2465.72 Bytes allocated
SMA.Transient.c_freed 42146510 2465.72 Bytes freed
SMA.Transient.g_alloc 0 . Allocations outstanding
SMA.Transient.g_bytes 0 . Bytes outstanding
SMA.Transient.g_space 0 . Bytes available
VBE.default(127.0.0.1,,8080).vcls 1 . VCL references
VBE.default(127.0.0.1,,8080).happy 0 . Happy health
probes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20120927/d14b3964/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 47022 bytes
Desc: not available
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20120927/d14b3964/attachment-0001.png>
More information about the varnish-misc
mailing list