RX > TX, not storing files in cache as long as it should
Robin
imer at imer.cc
Mon Jun 30 10:19:54 CEST 2014
Hello,
I recently set up a varnish cache to serve as an edge server to take
some traffic load off our main server.
What it's supposed to do is cache ~3GB (for now, will expand later) of
static files which never change (new files will get a new url).
This seems to work mostly ok, looking at the interface statistics after
2 days of runtime it turns out it's actually recieving more data than
its sending.
One of our nodes shows (for example): RX bytes:119388151300 (111.1 GiB)
TX bytes:60809550709 (56.6 GiB)
The static files range from a few MB to (the biggest) 1.2GB, usually are
around max. 100MB though.
They are originally served using nginx with the "expires max" directive
set, this is reflected by the varnish cache too:
http://puu.sh/9Qq3A/6c934ad71c.png
As you can see though, the Age is 0 - which means (from what I
understand) the file didn't exist yet and is just being downloaded by
varnish - which is not true.
The file has been downloaded by that node multiple times, each time with
a varying content length: http://puu.sh/9Qqds/506e73e589.png
My .vcl looks like this:
> backend default {
> .host = "backend ip";
> .port = "80";
> }
> sub vcl_fetch {
> if (beresp.ttl <= 0s ||
> beresp.http.Set-Cookie ||
> beresp.http.Vary == "*") {
> /*
> * Mark as "Hit-For-Pass" for the next 2 minutes
> */
> set beresp.ttl = 120 s;
> return (hit_for_pass);
> }
> set beresp.do_stream = true;
> return (deliver);
> }
And the options passed to varnish are:
> DAEMON_OPTS="-a :80 \
> -T localhost:6082 \
> -f /etc/varnish/default.vcl \
> -S /etc/varnish/secret \
> -p send_timeout=86400 \
> -s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,7G"
Any idea why this is happening?
> n_lru_nuked 0 . N LRU nuked objects
Means the cache isnt filled up, which it shouldnt be anyways - so why is
it downloading it over and over again and why is it aborting prematurely
and not downloading the full file to cache?
Trying to download the file always results in working files, so does it
cancel the download to varnish if the user aborts? If so this is really
disadvantageous for us, cause it may cause a large file to be downloaded
multiple times before finally ending up in the cache
Hope someone can help me out
- Robin
More information about the varnish-misc
mailing list