| Author: | Dag-Erling Smørgrav |
|---|---|
| Author: | Poul-Henning Kamp |
| Author: | Kristian Lyngstøl |
| Author: | Per Buer |
| Date: | 2010-06-02 |
| Version: | 1.0 |
| Manual section: | 7 |
The VCL language is a small domain-specific language designed to be used to define request handling and document caching policies for the Varnish HTTP accelerator.
When a new configuration is loaded, the varnishd management process translates the VCL code to C and compiles it to a shared object which is then dynamically linked into the server process.
The VCL syntax is very simple, and deliberately similar to C and Perl. Blocks are delimited by curly braces, statements end with semicolons, and comments may be written as in C, C++ or Perl according to your own preferences.
In addition to the C-like assignment (=), comparison (==) and boolean (!, && and ||) operators, VCL supports regular expression and ACL matching using the ~ operator.
Unlike C and Perl, the backslash () character has no special meaning in strings in VCL, which use the (%xx) escape mechanism just like URLs, so it can be freely used in regular expressions without doubling.
Strings are concatenated by just putting them one after each other without any operator in between.
Assignments are introduced with the set keyword. There are no user-defined variables; values can only be assigned to variables attached to backend, request or document objects. Most of these are typed, and the values assigned to them must have a compatible unit suffix.
You can use the set keyword to arbitrary HTTP headers. You can remove headers with the remove or unset keywords, which are synonym.
The return(action) keyword terminates the subroutine. action can be, depending on context one of
Please see the list of subroutines to see what return actions are available where.
VCL has if tests, but no loops.
You may log arbitrary strings to the shared memory log with the keyword log.
The contents of another VCL file may be inserted at any point in the code by using the include keyword followed by the name of the other file as a quoted string.
A backend declaration creates and initializes a named backend object::
backend www {
.host = "www.example.com";
.port = "http";
}
The backend object can later be used to select a backend at request time::
if (req.http.host ~ "^(www.)?example.com$") {
set req.backend = www;
}
To avoid overloading backend servers, .max_connections can be set to limit the maximum number of concurrent backend connections.
The timeout parameters can be overridden in the backend declaration. The timeout parameters are .connect_timeout for the time to wait for a backend connection, .first_byte_timeout for the time to wait for the first byte from the backend and .between_bytes_timeout for time to wait between each received byte.
These can be set in the declaration like this::
backend www {
.host = "www.example.com";
.port = "http";
.connect_timeout = 1s;
.first_byte_timeout = 5s;
.between_bytes_timeout = 2s;
}
To mark a backend as unhealthy after number of items have been added to it’s saintmode list .saintmode_threshold can be set to the maximum list size. Setting a value of 0 disables saintmode checking entirely for that backend. The value in the backend declaration overrides the parameter.
A director is a logical group of backend servers clustered together for redundancy. The basic role of the director is to let Varnish choose a backend server amongst several so if one is down another can be used.
There are several types of directors. The different director types use different algorithms to choose which backend to use.
Configuring a director may look like this::
director b2 random {
.retries = 5;
{
// We can refer to named backends
.backend = b1;
.weight = 7;
}
{
// Or define them inline
.backend = {
.host = "fs2";
}
.weight = 3;
}
}
The random director takes one per director option .retries. This specifies how many tries it will use to find a working backend. The default is the same as the number of backends defined for the director.
There is also a per-backend option: weight which defines the portion of traffic to send to the particular backend.
The round-robin director does not take any options.
The client director picks a backend based on the clients identity. You can set the VCL variable client.identity to identify the client by picking up the value of a session cookie or similar.
Note: in 2.1 client.identity isn’t available and the director will use client.ip to distribute clients across backends.
The client director takes one option - retries which set the number of retries the director should take in order to find a healthy backend.
The hash director will pick a backend based on the URL hash value.
This is useful is you are using Varnish to load balance in front of other Varnish caches or other web accelerators as objects won’t be duplicated across caches.
The client director takes one option - retries which set the number of retries the director should take in order to find a healthy backend.
The DNS director can use backends in three different ways. Either like the random or round-robin director or using .list:
director directorname dns {
.list = {
.host_header = "www.example.com";
.port = "80";
.connect_timeout = 0.4;
"192.168.15.0"/24;
"192.168.16.128"/25;
}
.ttl = 5m;
.suffix = "internal.example.net";
}
This will specify 384 backends, all using port 80 and a connection timeout of 0.4s. Options must come before the list of IPs in the .list statement.
The .ttl defines the cache duration of the DNS lookups.
The above example will append “internal.example.net” to the incoming Host header supplied by the client, before looking it up. All settings are optional.
Backends can be probed to see whether they should be considered healthy or not. The return status can also be checked by using req.backend.healthy .window is how many of the latest polls we examine, while .threshold is how many of those must have succeeded for us to consider the backend healthy. .initial is how many of the probes are considered good when Varnish starts - defaults to the same amount as the threshold.
A backend with a probe can be defined like this::
backend www {
.host = "www.example.com";
.port = "http";
.probe = {
.url = "/test.jpg";
.timeout = 0.3 s;
.window = 8;
.threshold = 3;
.initial = 3;
}
}
It is also possible to specify the raw HTTP request:
backend www {
.host = "www.example.com";
.port = "http";
.probe = {
# NB: \r\n automatically inserted after each string!
.request =
"GET / HTTP/1.1"
"Host: www.foo.bar"
"Connection: close";
}
}
An ACL declaration creates and initializes a named access control list which can later be used to match client addresses::
acl local {
"localhost"; // myself
"192.0.2.0"/24; // and everyone on the local network
! "192.0.2.23"; // except for the dialin router
}
If an ACL entry specifies a host name which Varnish is unable to resolve, it will match any address it is com‐ pared to. Consequently, if it is preceded by a negation mark, it will reject any address it is compared to, which may not be what you intended. If the entry is enclosed in parentheses, however, it will simply be ignored.
To match an IP address against an ACL, simply use the match operator::
if (client.ip ~ local) {
return (pipe);
}
The following built-in functions are available:
A subroutine is used to group code for legibility or reusability::
sub pipe_if_local {
if (client.ip ~ local) {
return (pipe);
}
}
Subroutines in VCL do not take arguments, nor do they return values.
To call a subroutine, use the call keyword followed by the subroutine’s name:
call pipe_if_local;
There are a number of special subroutines which hook into the Varnish workflow. These subroutines may inspect and manipulate HTTP headers and various other aspects of each request, and to a certain extent decide how the request should be handled. Each subroutine terminates by calling one of a small number of keywords which indicates the desired outcome.
Called at the beginning of a request, after the complete request has been received and parsed. Its purpose is to decide whether or not to serve the request, how to do it, and, if applicable, which backend to use.
The vcl_recv subroutine may terminate with calling return() on one of the following keywords:
Called upon entering pipe mode. In this mode, the request is passed on to the backend, and any further data from either client or backend is passed on unaltered until either end closes the connection.
The vcl_pipe subroutine may terminate with calling return() with one of the following keywords:
Called upon entering pass mode. In this mode, the request is passed on to the backend, and the backend’s response is passed on to the client, but is not entered into the cache. Subsequent requests sub‐ mitted over the same client connection are handled normally.
The vcl_recv subroutine may terminate with calling return() with one of the following keywords:
You may call hash_data() on the data you would like to add to the hash.
The vcl_hash subroutine may terminate with calling return() with one of the following keywords:
Called after a cache lookup if the requested document was found in the cache.
The vcl_hit subroutine may terminate with calling return() with one of the following keywords:
Called after a cache lookup if the requested document was not found in the cache. Its purpose is to decide whether or not to attempt to retrieve the document from the backend, and which backend to use.
The vcl_miss subroutine may terminate with calling return() with one of the following keywords:
Called after a document has been successfully retrieved from the backend.
The vcl_fetch subroutine may terminate with calling return() with one of the following keywords:
Called before a cached object is delivered to the client.
The vcl_deliver subroutine may terminate with one of the following keywords:
Called when we hit an error, either explicitly or implicitly due to backend or internal errors.
The vcl_error subroutine may terminate by calling return with one of the following keywords:
If one of these subroutines is left undefined or terminates without reaching a handling decision, control will be handed over to the builtin default. See the EXAMPLES section for a listing of the default code.
If multiple subroutines with the same name are defined, they are concatenated in the order in which the appear in the source.
Example::
# in file "main.vcl"
include "backends.vcl";
include "purge.vcl";
# in file "backends.vcl"
sub vcl_recv {
if (req.http.host ~ "example.com") {
set req.backend = foo;
} elsif (req.http.host ~ "example.org") {
set req.backend = bar;
}
}
# in file "purge.vcl"
sub vcl_recv {
if (client.ip ~ admin_network) {
if (req.http.Cache-Control ~ "no-cache") {
purge_url(req.url);
}
}
}
The builtin default subroutines are implicitly appended in this way.
Although subroutines take no arguments, the necessary information is made available to the handler subroutines through global variables.
The following variables are always available:
The following variables are available in backend declarations:
The following variables are available while processing a request:
The following variables are available while preparing a backend request (either for a cache miss or for pass or pipe mode):
The following variables are available after the requested object has been retrieved from the backend, before it is entered into the cache. In other words, they are available in vcl_fetch:
True if the request resulted in a cacheable response. A response is considered cacheable if HTTP status code is 200, 203, 300, 301, 302, 404 or 410 and pass wasn’t called in vcl_recv. If however, both the TTL and the grace time for the response are 0 beresp.cacheable will be 0.
beresp.cacheable is writable.
After the object is entered into the cache, the following (mostly read-only) variables are available when the object has been located in cache, typically in vcl_hit and vcl_deliver.
The following variables are available while determining the hash key of an object:
The following variables are available while preparing a response to the client:
Values may be assigned to variables using the set keyword::
sub vcl_recv {
# Normalize the Host: header
if (req.http.host ~ "^(www.)?example.com$") {
set req.http.host = "www.example.com";
}
}
HTTP headers can be removed entirely using the remove keyword::
sub vcl_fetch {
# Don't cache cookies
remove beresp.http.Set-Cookie;
}
If the backend takes a long time to generate an object there is a risk of a thread pile up. In order to prevent this you can enable grace. This allows varnish to serve an expired version of the object while a fresh object is being generated by the backend.
The following vcl code will make Varnish serve expired objects. All object will be kept up to two minutes past their expiration time or a fresh object is generated.:
sub vcl_recv {
set req.grace = 2m;
}
sub vcl_fetch {
set beresp.grace = 2m;
}
Saint mode is similar to grace mode and relies on the same infrastructure but functions differently. You can add VCL code to vcl_fetch to see whether or not you like the response coming from the backend. If you find that the response is not appropriate you can set beresp.saintmode to a time limit and call restart. Varnish will then retry other backends to try to fetch the object again.
If there are no more backends or if you hit max_restarts and we have an object that is younger than what you set beresp.saintmode to be Varnish will serve the object, even if it is stale.
The following code is the equivalent of the default configuration with the backend address set to “backend.example.com” and no backend port specified::
backend default {
.host = "backend.example.com";
.port = "http";
}
/*-
* Copyright (c) 2006 Verdens Gang AS
* Copyright (c) 2006-2009 Linpro AS
* All rights reserved.
*
* Author: Poul-Henning Kamp <phk@phk.freebsd.dk>
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE
* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR
* BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
* WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE
* OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
* EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* $Id$
*
* The default VCL code.
*
* NB! You do NOT need to copy & paste all of these functions into your
* own vcl code, if you do not provide a definition of one of these
* functions, the compiler will automatically fall back to the default
* code from this file.
*
* This code will be prefixed with a backend declaration built from the
* -b argument.
*/
sub vcl_recv {
if (req.restarts == 0) {
if (req.http.x-forwarded-for) {
set req.http.X-Forwarded-For =
req.http.X-Forwarded-For ", " client.ip;
} else {
set req.http.X-Forwarded-For = client.ip;
}
}
if (req.request != "GET" &&
req.request != "HEAD" &&
req.request != "PUT" &&
req.request != "POST" &&
req.request != "TRACE" &&
req.request != "OPTIONS" &&
req.request != "DELETE") {
/* Non-RFC2616 or CONNECT which is weird. */
return (pipe);
}
if (req.request != "GET" && req.request != "HEAD") {
/* We only deal with GET and HEAD by default */
return (pass);
}
if (req.http.Authorization || req.http.Cookie) {
/* Not cacheable by default */
return (pass);
}
return (lookup);
}
sub vcl_pipe {
# Note that only the first request to the backend will have
# X-Forwarded-For set. If you use X-Forwarded-For and want to
# have it set for all requests, make sure to have:
# set bereq.http.connection = "close";
# here. It is not set by default as it might break some broken web
# applications, like IIS with NTLM authentication.
return (pipe);
}
sub vcl_pass {
return (pass);
}
sub vcl_hash {
set req.hash += req.url;
if (req.http.host) {
set req.hash += req.http.host;
} else {
set req.hash += server.ip;
}
return (hash);
}
sub vcl_hit {
if (!obj.cacheable) {
return (pass);
}
return (deliver);
}
sub vcl_miss {
return (fetch);
}
sub vcl_fetch {
if (!beresp.cacheable) {
return (pass);
}
if (beresp.http.Set-Cookie) {
return (pass);
}
return (deliver);
}
sub vcl_deliver {
return (deliver);
}
sub vcl_error {
set obj.http.Content-Type = "text/html; charset=utf-8";
synthetic {"
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html>
<head>
<title>"} obj.status " " obj.response {"</title>
</head>
<body>
<h1>Error "} obj.status " " obj.response {"</h1>
<p>"} obj.response {"</p>
<h3>Guru Meditation:</h3>
<p>XID: "} req.xid {"</p>
<hr>
<p>Varnish cache server</p>
</body>
</html>
"};
return (deliver);
}
The following example shows how to support multiple sites running on separate backends in the same Varnish instance, by selecting backends based on the request URL::
backend www {
.host = "www.example.com";
.port = "80";
}
backend images {
.host = "images.example.com";
.port = "80";
}
sub vcl_recv {
if (req.http.host ~ "^(www.)?example.com$") {
set req.http.host = "www.example.com";
set req.backend = www;
} elsif (req.http.host ~ "^images.example.com$") {
set req.backend = images;
} else {
error 404 "Unknown virtual host";
}
}
The following snippet demonstrates how to force a minimum TTL for
all documents. Note that this is not the same as setting the
default_ttl run-time parameter, as that only affects document for
which the backend did not specify a TTL:::
sub vcl_fetch {
if (beresp.ttl < 120s) {
log "Adjusting TTL";
set beresp.ttl = 120s;
}
}
The following snippet demonstrates how to force Varnish to cache documents even when cookies are present::
sub vcl_recv {
if (req.request == "GET" && req.http.cookie) {
return(lookup);
}
}
sub vcl_fetch {
if (beresp.http.Set-Cookie) {
return(deliver);
}
}
The following code implements the HTTP PURGE method as used by Squid for object invalidation::
acl purge {
"localhost";
"192.0.2.1"/24;
}
sub vcl_recv {
if (req.request == "PURGE") {
if (!client.ip ~ purge) {
error 405 "Not allowed.";
}
return(lookup);
}
}
sub vcl_hit {
if (req.request == "PURGE") {
set obj.ttl = 0s;
error 200 "Purged.";
}
}
sub vcl_miss {
if (req.request == "PURGE") {
error 404 "Not in cache.";
}
}
The VCL language was developed by Poul-Henning Kamp in cooperation with Verdens Gang AS, Linpro AS and Varnish Software. This manual page was written by Dag-Erling Smørgrav and later edited by Poul-Henning Kamp and Per Buer.
This document is licensed under the same licence as Varnish itself. See LICENCE for details.