Varnish is a state-of-the-art, high-performance web accelerator. It uses the advanced features in Linux 2.6, FreeBSD 6/7 and Solaris 10 to achieve its high performance.
Some of the features include
Features to come (Experimental):
Varnish is free software and is licenced under a modified BSD licence. Please read the introduction to get started with Varnish.
How much RAM/Disk do I need for Varnish?
That depends on pretty much everything.
I think our best current guidance is that you go for a cost-effective RAM configuration, something like 1-16GB, and a SSD disk.
Unless you positively know that you will need it, there is little point in spendng a fortune on a hand-sewn motherboard that can fit several TB in special RAM blocks, rivetet together by leftover watch-makers in Switzerland.
On the other hand, if you plot your traffic in Gb/s, you probably need all the RAM you can afford/get.
How can I limit Varnish to use less RAM?
You can not. Varnish operates in Virtual Memory and it is up to the kernel to decide which process gets to use how much RAM to map the virtual address-space of the process.
How do I instruct varnish to ignore the query parameters and only cache one instance of an object?
This can be achieved by removing the query parameters using a regexp:
sub vcl_recv {
set req.url = regsub(req.url, "\?.*", "");
}
How can I force a refresh on a object cached by varnish?
Refreshing is often called purging a document. You can purge at least 2 different ways in Varnish:
Command line
From the command line you can write:
url.purge ^/$to purge your / document. As you might see url.purge takes an regular expression as its argument. Hence the ^ and $ at the front and end. If the ^ is ommited, all the documents ending in a / in the cache would be deleted.
So to delete all the documents in the cache, write:
url.purge .*at the command line.
HTTP PURGE
VCL code to allow HTTP PURGE is to be found here. Note that this method does not support wildcard purging.
How can I debug the requests of a single client?
The “varnishlog” utility may produce a horrendous amount of output. To be able debug our own traffic can be useful.
The ReqStart token will include the client IP address. To see log entries matching this, type:
$ varnishlog -c -o ReqStart 192.0.2.123
To see the backend requests generated by a client IP address, we can match on the TxHeader token, since the IP address of the client is included in the X-Forwarded-For header in the request sent to the backend.
At the shell command line, type:
$ varnishlog -b -o TxHeader 192.0.2.123
How can I rewrite URLS before they are sent to the backend?
You can use the regsub() function to do this. Here’s an example for zope, to rewrite URL’s for the virtualhostmonster:
if (req.http.host ~ "^(www.)?example.com") {
set req.url = regsub(req.url, "^", "/VirtualHostBase/http/example.com:80/Sites/example.com/VirtualHostRoot");
}
I have a site with many hostnames, how do I keep them from multiplying the cache?
You can do this by normalizing the Host header for all your hostnames. Here’s a VCL example:
if (req.http.host ~ "^(www.)?example.com") {
set req.http.host = "example.com";
}
How do I do to alter the request going to the backend? You can use the bereq object for altering requests going to the backend, but you can only ‘set’ values to it. Therefore use req.url to build the request:
sub vcl_miss {
set bereq.url = regsub(req.url,"stream/","/");
fetch;
}
How do I force the backend to send Vary headers?
We have anectdotal evidence of non-RFC2616 compliant backends, which support content negotiation, but which do not emit a Vary header, unless the request contains Accept headers.
It may be appropriate to send no-op Accept headers to trick the backend into sending us the Vary header.
The following should be sufficient for most cases:
Accept: */*
Accept-Language: *
Accept-Charset: *
Accept-Encoding: identity
Note that Accept-Encoding can not be set to *, as the backend might then send back a compressed response which the client would be unable to process.
This can of course be implemented in VCL.
How can I customize the error messages that Varnish returns?
A custom error page can be generated by adding a vcl_error to your configuration file. The default error page looks like this:
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>"} obj.status " " obj.response {"</title>
</head>
<body>
<h1>Error "} obj.status " " obj.response {"</h1>
<p>"} obj.response {"</p>
<h3>Guru Meditation:</h3>
<p>XID: "} req.xid {"</p>
<address><a href="http://www.varnish-cache.org/">Varnish</a></address>
</body>
</html>
"};
deliver;
}
How do I instruct varnish to ignore the query parameters and only cache one instance of an object?
This can be achieved by removing the query parameters using a regexp:
sub vcl_recv {
set req.url = regsub(req.url, "\?.*", "");
}
Can I find varnish for my operating system?
We know that Varnish has been packaged for Debian, Ubuntu, RHEL, Centos, (Open)SuSE, Gentoo and FreeBSD, possibly more. Check whatever packagemanager you use. Or read Installing Varnish on your computer.
Can I use Varnish as a client-side proxy?
No. Varnish needs all backends configured in the VCL. Look at squid instead.
Can I run Varnish on a 32bit system?
Yes, recently somebody even claimed to run Varnish on his N900 mobile phone recently, but if you have the choice, go 64 bit from the start.
Varnish is written to use Virtual Memory and on a 32bit system that really cramps your style, and you will have trouble configuring more than 2 GB of storage.
Can I run Varnish on the same system as Apache?
Yes, and many people do that with good success.
There will be competition for resources, but Apache is not particular good at using RAM effectively and Varnish is, so this synergy usually more than compensates for the competition.
Can I run multiple Varnish on the same system?
Yes, as long as you give them different TCP ports and different -n arguments, you will be fine.
Can I cache multiple vhosts with one Varnish?
Yes, that works right out of the box.
Can I see what is cached in Varnish?
That is not possible for several reasons. A command to list all the contents of a Varnish cache with millions of objects would bring your Varnish to a standstill while it traverses the index.
Besides, the output is a lot less useful than you might think.
Can I use Varnish to do HTTPS?
Not at present, and while we keep an eye on this, there are no current plans to add HTTPS support, until we can find a way where it adds significant value, relative to running a stand-alone HTTPS proxy such as ngnix or pound.
Can Varnish load balance between multiple backends?
Yes, you need VCL code like this:
director foobar round-robin {
{ .backend = { .host = "www1.example.com; .port = "http"; } }
{ .backend = { .host = "www2.example.com; .port = "http"; } }
}
sub vcl_recv {
set req.backend = foobar;
}
(XXX: reference to docs, once written)
Why does it look like Varnish sends all requests to the backend? I thought it was a cache?
The object’s ttl expired. A common situation is that the backend does not set an expiry time on the requested image/file/webpage, so Varnish uses the default TTL (normally 120s).
Why are regular expressions case-sensitive?
Some HTTP headers, such as Host: and Location: contain FQDN’s which by definition is not case-sensitive. Other HTTP headers are case-sensitive, most notably the URLs. Therefore a “one size fits all” solution is not possible.
In previous releases, we used the POSIX regular expressions supplied with the operating system, and decided, because the most common use of regexps were on `Host:` headers, that they should not be case-sensitive.
From version 2.1.0 and forward, we use PCRE regular expressions, where it is possible to control case-sensitivity in the individual regular expressions, so we decided that it would probably confuse people if we made the default case-insentive. (We promise not to change our minds about this again.)
To make a PCRE regex case insensitive, put (?i) at the start:
if (req.http.host ~ "(?i)example.com$") {
...
}
See the PCRE man pages for more information.
Are regular expressions case sensitive or not? Can I change it?
In 2.1 and newer, regular expressions are case sensitive by default. In earlier versions, they were case insensitive.
To change this for a single regex in 2.1, use (?i) at the start.
See the PCRE man pages for more information.
Why does the ``Via:`` header say 1.1 in Varnish 2.1.x?
The number in the Via: header is the HTTP protocol version supported/applied, not the softwares version number.
Why did you call it *Varnish*?
Long story, but basically the instigator of Varnish spent a long time staring at an art-poster with the word “Vernisage” and ended up checking it in a dictionary, which gives the following three meanings of the word:
r.v. var·nished, var·nish·ing, var·nish·es
- To cover with varnish.
- To give a smooth and glossy finish to.
- To give a deceptively attractive appearance to; gloss over.
The three point describes happens to your backend system when you put Varnish in front of it.
Why does Varnish require the system to have a C compiler?
The VCL compiler generates C source as output (your config file), and uses the systems C-compiler to compile that into a shared library. If there is no C compiler, Varnish will not work.
Isn’t that security problem?
The days when you could prevent people from running non-approved programs by removing the C compiler from your system ended roughly with the VAX 11/780 computer.
Why am I getting a cache hit, but a request is still going to my backend?
Varnish has a feature called hit for pass, which is used when Varnish gets a response from the backend and finds out it cannot be cached. In such cases, Varnish will create a cache object that records that fact, so that the next request goes directly to “pass”.
- Since Varnish bundles multiple requests for the same URL to the backend, a common case where a client will get a hit for pass is:
- Client 1 requests url /foo
- Client 2..N request url /foo
- Varnish tasks a worker to fetch /foo for Client 1
- Client 2..N are now queued pending response from the worker
- Worker returns object to varnish which turns out to be non-cacheable.
- Client 2..N are now given the hit for pass object instructing them to go to the backend
The hit for pass object will stay cached for the duration of it’s ttl. This means that subsequent clients requesting /foo will be sent straight to the backend as long as the hit for pass object exists. The varnishstat can tell you how many hit for pass objects varnish has served. You can lower the ttl for such an object if you are sure this is needed, using the following logic:
sub vcl_fetch {
if (!obj.cacheable) {
# Limit the lifetime of all 'hit for pass' objects to 10 seconds
obj.ttl = 10s;
pass;
}
}