Built in subroutines¶
Various built-in subroutines are called during processing of client-
and backend requests as well as upon vcl.load and vcl.discard.
See Varnish Processing States for a detailed graphical overview of the states and how they relate to core code functions and VCL subroutines.
Subroutines always terminate with a return() on a keyword, which
determines how processing continues in the request processing state
machine.
The behaviour for return() keywords is identical or at least
similar across subroutines, so differences are only documented where
relevant.
common return keywords for the client and backend side¶
failTransition to vcl_synth on the client side as for
return(synth(503, "VCL Failed")), but with any request state changes undone as ifstd.rollback()was called and forcing a connection close.Intended for fatal errors, for which only minimal error handling is possible.
client side¶
common return keywords for the client side¶
synth(status code, reason)Transition to vcl_synth with
resp.statusandresp.reasonbeing preset to the arguments ofsynth().
passSwitch to pass mode. Control will eventually pass to vcl_pass.
pipeSwitch to pipe mode. Control will eventually pass to vcl_pipe.
restartRestart the transaction. Increases the
req.restartscounter.If the number of restarts is higher than the max_restarts parameter, control is passed to vcl_synth as for
return(synth(503, "Too many restarts"))For a restart, all modifications to
reqattributes are preserved except forreq.restartsandreq.xid, which need to change by design.
vcl_recv¶
Called at the beginning of a request, after the complete request has been received and parsed, after a restart or as the result of an ESI include.
Its purpose is to decide whether or not to serve the request, possibly modify it and decide on how to process it further. A backend hint may be set as a default for the backend processing side.
The vcl_recv subroutine may terminate with calling return() on one
of the following keywords:
failsee fail
synth(status code, reason)see synth
restartsee restart
passsee pass
pipesee pipe
hashContinue processing the object as a potential candidate for caching. Passes the control over to vcl_hash.
purgePurge the object and it’s variants. Control passes through vcl_hash to vcl_purge.
vcl(label)Switch to vcl labelled label. This will continue vcl processing in this vcl’s vcl_recv as if it was the active vcl.
See the vcl.label <label> <configname> command.
vcl_pipe¶
Called upon entering pipe mode. In this mode, the request is passed on to the backend, and any further data from both the client and backend is passed on unaltered until either end closes the connection. Basically, Varnish will degrade into a simple TCP proxy, shuffling bytes back and forth. For a connection in pipe mode, no other VCL subroutine will ever get called after vcl_pipe.
The vcl_pipe subroutine may terminate with calling return() with one
of the following keywords:
vcl_pass¶
Called upon entering pass mode. In this mode, the request is passed on to the backend, and the backend’s response is passed on to the client, but is not entered into the cache. Subsequent requests submitted over the same client connection are handled normally.
The vcl_pass subroutine may terminate with calling return() with one
of the following keywords:
vcl_hash¶
Called after vcl_recv to create a hash value for the request. This is used as a key to look up the object in Varnish.
The vcl_hash subroutine may terminate with calling return() with one
of the following keywords:
vcl_purge¶
Called after the purge has been executed and all its variants have been evicted.
The vcl_purge subroutine may terminate with calling return() with one
of the following keywords:
vcl_miss¶
Called after a cache lookup if the requested document was not found in
the cache or if vcl_hit returned fetch.
Its purpose is to decide whether or not to attempt to retrieve the document from the backend. A backend hint may be set as a default for the backend processing side.
The vcl_miss subroutine may terminate with calling return() with one
of the following keywords:
vcl_hit¶
Called when a cache lookup is successful. The object being hit may be stale: It can have a zero or negative ttl with only grace or keep time left.
The vcl_hit subroutine may terminate with calling return()
with one of the following keywords:
vcl_deliver¶
Called before any object except a vcl_synth result is delivered to the client.
The vcl_deliver subroutine may terminate with calling return() with one
of the following keywords:
vcl_synth¶
Called to deliver a synthetic object. A synthetic object is generated
in VCL, not fetched from the backend. Its body may be constructed using
the synthetic() function.
A vcl_synth defined object never enters the cache, contrary to a vcl_backend_error defined object, which may end up in cache.
The subroutine may terminate with calling return() with one of the
following keywords:
Backend Side¶
common return keywords for the backend side¶
abandonAbandon the backend request. Unless the backend request was a background fetch, control is passed to vcl_synth on the client side with
resp.statuspreset to 503.
vcl_backend_fetch¶
Called before sending the backend request. In this subroutine you typically alter the request before it gets to the backend.
The vcl_backend_fetch subroutine may terminate with calling
return() with one of the following keywords:
failsee fail
abandonsee abandon
fetchFetch the object from the backend.
error(status code, reason)Transition to vcl_backend_error with
beresp.statusandberesp.reasonbeing preset to the arguments oferror()if arguments are provided.
Before calling vcl_backend_fetch, Varnish core prepares the bereq backend request as follows:
Unless the request is a pass,
set
bereq.methodtoGETandbereq.prototoHTTP/1.1andset
bereq.http.Accept_Encodingtogzipif http_gzip_support is enabled.
If there is an existing cache object to be revalidated, set
bereq.http.If-Modified-Sincefrom itsLast-Modifiedheader and/or setbereq.http.If-None-Matchfrom itsEtagheaderSet
bereq.http.X-Varnishto the current transaction id (vxid)
These changes can be undone or modified in vcl_backend_fetch before the backend request is issued.
In particular, to cache non-GET requests, req.method needs to be
saved to a header or variable in vcl_recv and restored to
bereq.method. Notice that caching non-GET requests typically also
requires changing the cache key in vcl_hash e.g. by also
hashing the request method and/or request body.
HEAD request can be satisfied from cached GET responses.
vcl_backend_response¶
Called after the response headers have been successfully retrieved from the backend.
The vcl_backend_response subroutine may terminate with calling
return() with one of the following keywords:
failsee fail
abandonsee abandon
deliverFor a 304 response, create an updated cache object. Otherwise, fetch the object body from the backend and initiate delivery to any waiting client requests, possibly in parallel (streaming).
retryRetry the backend transaction. Increases the retries counter. If the number of retries is higher than max_retries, control will be passed to vcl_backend_error.
pass(duration)Mark the object as a hit-for-pass for the given duration. Subsequent lookups hitting this object will be turned into passed transactions, as if
vcl_recvhad returnedpass.error(status code, reason)Transition to vcl_backend_error with
beresp.statusandberesp.reasonbeing preset to the arguments oferror()if arguments are provided.
304 handling¶
For a 304 response, Varnish core code amends beresp before calling
vcl_backend_response:
If the gzip status changed,
Content-Encodingis unset and anyEtagis weakenedAny headers not present in the 304 response are copied from the existing cache object.
Content-Lengthis copied if present in the existing cache object and discarded otherwise.The status gets set to 200.
beresp.was_304 marks that this conditional response processing has happened.
Note: Backend conditional requests are independent of client conditional requests, so clients may receive 304 responses no matter if a backend request was conditional.
beresp.ttl / beresp.grace / beresp.keep¶
Before calling vcl_backend_response, core code sets beresp.ttl
based on the response status and the response headers Age,
Cache-Control or Expires and Date as follows:
If present and valid, the value of the
Ageheader is effectively deduced from all ttl calculations.For status codes 200, 203, 204, 300, 301, 304, 404, 410 and 414:
If
Cache-Controlcontains ans-maxageormax-agefield (in that order of preference), the ttl is set to the respective non-negative value or 0 if negative.Otherwise, if no
Expiresheader exists, the default ttl is used.Otherwise, if
Expirescontains a time stamp beforeDate, the ttl is set to 0.Otherwise, if no
Dateheader is present or theDateheader timestamp differs from the local clock by no more than the clock_skew parameter, the ttl is set to0 if
Expiresdenotes a past timestamp orthe difference between the local clock and the
Expiresheader otherwise.
Otherwise, the ttl is set to the difference between
ExpiresandDate
For status codes 302 and 307, the calculation is identical except that the default ttl is not used and -1 is returned if neither
Cache-ControlnorExpiresexists.For all other status codes, ttl -1 is returned.
beresp.grace defaults to the default_grace parameter.
For a non-negative ttl, if Cache-Control contains a
stale-while-revalidate field value, beresp.grace is
set to that value if non-negative or 0 otherwise.
beresp.keep defaults to the default_keep parameter.
vcl_backend_error¶
This subroutine is called if we fail the backend fetch or if max_retries has been exceeded.
Returning with abandon does not leave a cache object.
If returning with deliver and a beresp.ttl > 0s, a synthetic
cache object is generated in VCL, whose body may be constructed using
the synthetic() function.
When there is a waiting list on the object, the default ttl will
be positive (currently one second), set before entering
vcl_backend_error. This is to avoid request serialization and
hammering on a potentially failing backend.
Since these synthetic objects are cached in these special
circumstances, be cautious with putting private information there. If
you really must, then you need to explicitly set beresp.ttl to
zero in vcl_backend_error.
The vcl_backend_error subroutine may terminate with calling return()
with one of the following keywords:
failsee fail
abandonsee abandon
deliverDeliver and possibly cache the object defined in vcl_backend_error as if it was fetched from the backend, also referred to as a “backend synth”.
retryRetry the backend transaction. Increases the retries counter. If the number of retries is higher than max_retries, vcl_synth on the client side is called with
resp.statuspreset to 503.
During vcl.load / vcl.discard¶
vcl_init¶
Called when VCL is loaded, before any requests pass through it. Typically used to initialize VMODs.
The vcl_init subroutine may terminate with calling return()
with one of the following keywords:
okNormal return, VCL continues loading.
failAbort loading of this VCL.
vcl_fini¶
Called when VCL is discarded only after all requests have exited the VCL. Typically used to clean up VMODs.
The vcl_fini subroutine may terminate with calling return()
with one of the following keywords:
okNormal return, VCL will be discarded.