[master] 560b3b6 Add 5.0
Poul-Henning Kamp
phk at FreeBSD.org
Wed Sep 14 09:41:11 CEST 2016
commit 560b3b68f1e7df2ee24c237aa7779e44587cc623
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date: Wed Sep 14 07:11:21 2016 +0000
Add 5.0
diff --git a/doc/sphinx/whats-new/changes-4.1.rst b/doc/sphinx/whats-new/changes-4.1.rst
new file mode 100644
index 0000000..bbc8177
--- /dev/null
+++ b/doc/sphinx/whats-new/changes-4.1.rst
@@ -0,0 +1,165 @@
+.. _whatsnew_changes_4_1:
+
+Changes in Varnish 4.1
+======================
+
+Varnish 4.1 is the continuation of the new streaming architecture seen
+in Varnish 4.0.
+
+
+Proactive security features
+---------------------------
+
+New in 4.1 is support for different kinds of privilege separation methods,
+collectively described as jails.
+
+On most systems, the Varnish parent process will now drop effective
+privileges to normal user mode when not doing operations needing special
+access.
+
+The Varnish worker child should now be run as a separate `vcache` user.
+
+``varnishlog``, ``varnishncsa`` and other Varnish shared log utilities
+now must be run in a context with `varnish` group membership.
+
+
+Warm and cold VCL configurations
+--------------------------------
+
+Traditionally Varnish have had the concept of active and inactive
+loaded VCLs. Any loaded VCL lead to state being kept, and a separate
+set of health checks (if configured) were being run against the backends.
+
+To avoid the extra state and backend polling, a loaded VCL is now either
+warm or cold. Runtime state (incl. backend counters) and health checks
+are not present for cold VCLs.
+
+A warm VCL will automatically be set to cold after `vcl_cooldown` seconds.
+
+Output from `vcl.list`::
+
+ varnish> vcl.list
+ 200
+ available auto/warm 0 boot
+ available auto/warm 0 62f5275f-a937-4df9-9fbb-c12336bdfdb8
+
+
+A single VCL's state can be changed with the `vcl.state` call in
+``varnishadm``::
+
+ vcl.state <configname> <state>
+ Force the state of the specified configuration.
+ State is any of auto, warm or cold values.
+
+Example::
+
+
+ varnish> vcl.state 62f5275f-a937-4df9-9fbb-c12336bdfdb8 cold
+ 200
+
+ varnish> vcl.list
+ 200
+ available auto/warm 0 boot
+ available auto/cold 0 62f5275f-a937-4df9-9fbb-c12336bdfdb8
+
+
+VMOD writers should read up on the new vcl_event system to
+release unnecessary state when a VCL is transitioned to cold (see
+:ref:`ref-vmod-event-functions`).
+
+
+PROXY protocol support
+----------------------
+
+Socket support for PROXY protocol connections has been added. PROXY
+defines a short preamble on the TCP connection where (usually) a SSL/TLS
+terminating proxy can signal the real client address.
+
+The ``-a`` startup argument syntax has been expanded to allow for this::
+
+ $ varnishd -f /etc/varnish/default.vcl -a :6081 -a 127.0.0.1:6086,PROXY
+
+Both PROXY1 and PROXY2 protocols are supported on the resulting listening
+socket.
+
+For connections coming in over a PROXY socket, ``client.ip`` and
+``server.ip`` will contain the addresses given to Varnish in the PROXY
+header/preamble (the "real" IPs).
+
+The new VCL variables ``remote.ip`` and ``local.ip`` contains the local
+TCP connection endpoints. On non-PROXY connections these will be identical
+to ``client.ip`` and ``server.ip``.
+
+An expected pattern following this is `if (std.port(local.ip) == 80) { }`
+in ``vcl_recv`` to see if traffic came in over the HTTP listening socket
+(so a client redirect to HTTPS can be served).
+
+
+VMOD backends
+-------------
+
+Before Varnish 4.1, backends could only be declared in native VCL. Varnish
+4.0 moved directors from VCL to VMODs, and VMODs can now also create
+backends. It is possible to both create the same backends than VCL but
+dynamically, or create backends that don't necessarily speak HTTP/1 over
+TCP to fetch resources. More details in the :ref:`ref-writing-a-director`
+documentation.
+
+
+Backend connection timeout
+--------------------------
+
+Backend connections will now be closed by Varnish after `backend_idle_timeout`
+seconds of inactivity.
+
+Previously they were kept around forever and the backend servers would close
+the connection without Varnish noticing it. On the next traffic spike needing
+these extra backend connections, the request would fail, perhaps multiple
+times, before a working backend connection was found/created.
+
+
+Protocol support
+----------------
+
+Support for HTTP/0.9 on the client side has been retired.
+
+
+More modules available
+----------------------
+
+Varnish has an ecosystem for third-party modules (vmods). New since
+the last release, these are worth knowing about:
+
+libvmod-saintmode: Saint mode ("inferred health probes from traffic") was taken
+out of Varnish core in 4.0, and is now back as a separate vmod. This is useful
+for detecting failing backends before the health probes pick it up.
+
+libvmod-xkey: Secondary hash keys for cache objects, based on the hashtwo vmod
+written by Varnish Software. Allows for arbitrary grouping of objects to be
+purged in one go, avoiding use of ban invalidation. Also known as Cache Keys or
+Surrogate Key support.
+
+libvmod-rtstatus: Real time statistics dashboard.
+
+
+Passing data between ESI requests
+---------------------------------
+
+A new `req_top` identifier is available in VCL, which is a reference to
+`req` in the top-level ESI request.
+
+This is useful to pass data back and forth between the main ESI request
+and any ESI sub-requests it leads to.
+
+
+Other noteworthy small changes
+------------------------------
+
+* Varnish will now use the ``stale-while-revalidate`` defined in RFC5861
+ to set object grace time.
+* -smalloc storage is now recommended over -sfile on Linux systems.
+* New VCL variable ``beresp.was_304`` has been introduced in
+ ``vcl_backend_response``. Will be set to ``true`` if the response
+ from the backend was a positive result of a conditional fetch (``304
+ Not Modified``).
+
diff --git a/doc/sphinx/whats-new/changes-5.0.rst b/doc/sphinx/whats-new/changes-5.0.rst
new file mode 100644
index 0000000..c610f0f
--- /dev/null
+++ b/doc/sphinx/whats-new/changes-5.0.rst
@@ -0,0 +1,43 @@
+.. _whatsnew_changes_5.0:
+
+Changes in Varnish 5.0
+======================
+
+Varnish 5.0 changes some (mostly) internal APIs and adds som major new
+features over Varnish 4.1.
+
+
+Separate VCL files and VCL labels
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Varnish 5.0 supports jumping from the active VCL's vcl_recv{} to
+another VCL via a VCL label.
+
+The major use of this will probably be to have a separate VCL for
+each domain/vhost, in order to untangle complex VCL files, but
+it is not limited to this criteria, it would also be possible to
+send all POSTs, all JPEG images or all traffic from a certain
+IP range to a separate VCL file.
+
+VCL labels can also be used to give symbolic names to loaded VCL
+configurations, so that operations personel only need to know
+about "normal", "weekend" and "emergency", and web developers
+can update these as usual, without having to tell ops what the
+new weekend VCL is called.
+
+
+Very Experimental HTTP/2 support
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We are in the process of adding HTTP/2 support to Varnish, but
+the code is very green still - life happened.
+
+But you can actually get a bit of traffic though it already, and
+we hope to have it production ready for the next major release
+(2017-03-15).
+
+Varnish supports HTTP/1 -> 2 upgrade. For political reasons,
+no browsers support that, but tools like curl does.
+
+For encrypted HTTP/2 traffic, put a SSL proxy in front of Varnish.
+
diff --git a/doc/sphinx/whats-new/changes.rst b/doc/sphinx/whats-new/changes.rst
deleted file mode 100644
index 22e61a0..0000000
--- a/doc/sphinx/whats-new/changes.rst
+++ /dev/null
@@ -1,165 +0,0 @@
-.. _whatsnew_changes:
-
-Changes in Varnish 4.1
-======================
-
-Varnish 4.1 is the continuation of the new streaming architecture seen
-in Varnish 4.0.
-
-
-Proactive security features
-===========================
-
-New in 4.1 is support for different kinds of privilege separation methods,
-collectively described as jails.
-
-On most systems, the Varnish parent process will now drop effective
-privileges to normal user mode when not doing operations needing special
-access.
-
-The Varnish worker child should now be run as a separate `vcache` user.
-
-``varnishlog``, ``varnishncsa`` and other Varnish shared log utilities
-now must be run in a context with `varnish` group membership.
-
-
-Warm and cold VCL configurations
-================================
-
-Traditionally Varnish have had the concept of active and inactive
-loaded VCLs. Any loaded VCL lead to state being kept, and a separate
-set of health checks (if configured) were being run against the backends.
-
-To avoid the extra state and backend polling, a loaded VCL is now either
-warm or cold. Runtime state (incl. backend counters) and health checks
-are not present for cold VCLs.
-
-A warm VCL will automatically be set to cold after `vcl_cooldown` seconds.
-
-Output from `vcl.list`::
-
- varnish> vcl.list
- 200
- available auto/warm 0 boot
- available auto/warm 0 62f5275f-a937-4df9-9fbb-c12336bdfdb8
-
-
-A single VCL's state can be changed with the `vcl.state` call in
-``varnishadm``::
-
- vcl.state <configname> <state>
- Force the state of the specified configuration.
- State is any of auto, warm or cold values.
-
-Example::
-
-
- varnish> vcl.state 62f5275f-a937-4df9-9fbb-c12336bdfdb8 cold
- 200
-
- varnish> vcl.list
- 200
- available auto/warm 0 boot
- available auto/cold 0 62f5275f-a937-4df9-9fbb-c12336bdfdb8
-
-
-VMOD writers should read up on the new vcl_event system to
-release unnecessary state when a VCL is transitioned to cold (see
-:ref:`ref-vmod-event-functions`).
-
-
-PROXY protocol support
-======================
-
-Socket support for PROXY protocol connections has been added. PROXY
-defines a short preamble on the TCP connection where (usually) a SSL/TLS
-terminating proxy can signal the real client address.
-
-The ``-a`` startup argument syntax has been expanded to allow for this::
-
- $ varnishd -f /etc/varnish/default.vcl -a :6081 -a 127.0.0.1:6086,PROXY
-
-Both PROXY1 and PROXY2 protocols are supported on the resulting listening
-socket.
-
-For connections coming in over a PROXY socket, ``client.ip`` and
-``server.ip`` will contain the addresses given to Varnish in the PROXY
-header/preamble (the "real" IPs).
-
-The new VCL variables ``remote.ip`` and ``local.ip`` contains the local
-TCP connection endpoints. On non-PROXY connections these will be identical
-to ``client.ip`` and ``server.ip``.
-
-An expected pattern following this is `if (std.port(local.ip) == 80) { }`
-in ``vcl_recv`` to see if traffic came in over the HTTP listening socket
-(so a client redirect to HTTPS can be served).
-
-
-VMOD backends
-=============
-
-Before Varnish 4.1, backends could only be declared in native VCL. Varnish
-4.0 moved directors from VCL to VMODs, and VMODs can now also create
-backends. It is possible to both create the same backends than VCL but
-dynamically, or create backends that don't necessarily speak HTTP/1 over
-TCP to fetch resources. More details in the :ref:`ref-writing-a-director`
-documentation.
-
-
-Backend connection timeout
-==========================
-
-Backend connections will now be closed by Varnish after `backend_idle_timeout`
-seconds of inactivity.
-
-Previously they were kept around forever and the backend servers would close
-the connection without Varnish noticing it. On the next traffic spike needing
-these extra backend connections, the request would fail, perhaps multiple
-times, before a working backend connection was found/created.
-
-
-Protocol support
-================
-
-Support for HTTP/0.9 on the client side has been retired.
-
-
-More modules available
-======================
-
-Varnish has an ecosystem for third-party modules (vmods). New since
-the last release, these are worth knowing about:
-
-libvmod-saintmode: Saint mode ("inferred health probes from traffic") was taken
-out of Varnish core in 4.0, and is now back as a separate vmod. This is useful
-for detecting failing backends before the health probes pick it up.
-
-libvmod-xkey: Secondary hash keys for cache objects, based on the hashtwo vmod
-written by Varnish Software. Allows for arbitrary grouping of objects to be
-purged in one go, avoiding use of ban invalidation. Also known as Cache Keys or
-Surrogate Key support.
-
-libvmod-rtstatus: Real time statistics dashboard.
-
-
-Passing data between ESI requests
-=================================
-
-A new `req_top` identifier is available in VCL, which is a reference to
-`req` in the top-level ESI request.
-
-This is useful to pass data back and forth between the main ESI request
-and any ESI sub-requests it leads to.
-
-
-Other noteworthy small changes
-==============================
-
-* Varnish will now use the ``stale-while-revalidate`` defined in RFC5861
- to set object grace time.
-* -smalloc storage is now recommended over -sfile on Linux systems.
-* New VCL variable ``beresp.was_304`` has been introduced in
- ``vcl_backend_response``. Will be set to ``true`` if the response
- from the backend was a positive result of a conditional fetch (``304
- Not Modified``).
-
diff --git a/doc/sphinx/whats-new/index.rst b/doc/sphinx/whats-new/index.rst
index 2b82bb6..68d076f 100644
--- a/doc/sphinx/whats-new/index.rst
+++ b/doc/sphinx/whats-new/index.rst
@@ -1,21 +1,35 @@
.. _whats-new-index:
-%%%%%%%%%%%%%%%%%%%%%%%%%%
-What's new in Varnish 4.1
-%%%%%%%%%%%%%%%%%%%%%%%%%%
+%%%%%%%%%%%%%%%%%%%%%%
+What's new / Upgrading
+%%%%%%%%%%%%%%%%%%%%%%
-This section describes the changes that have been made for Varnish 4.1.
+This section describes the changes and improvements between different
+versions of Varnish, and what upgrading between the different versions
+entail.
-The first subsection describes overarching changes that have gone into
-Varnish 4.0, while the second subsection describes changes you need to make to
-your current configuration if you come from Varnish 3.x.
+Varnish 5.0
+-----------
-The text also considers changes in behaviour that you need to be aware of and
-take into consideration when upgrading.
+.. toctree::
+ :maxdepth: 2
+
+ changes-5.0
+ upgrading-5.0
+
+Varnish 4.1
+-----------
+
+.. toctree::
+ :maxdepth: 2
+
+ changes-4.1
+ upgrading-4.1
+
+Varnish 4.0
+-----------
.. toctree::
:maxdepth: 2
- changes
- upgrading
- upgrade-4.0
+ upgrading-4.0
diff --git a/doc/sphinx/whats-new/upgrade-4.0.rst b/doc/sphinx/whats-new/upgrade-4.0.rst
deleted file mode 100644
index f96c173..0000000
--- a/doc/sphinx/whats-new/upgrade-4.0.rst
+++ /dev/null
@@ -1,248 +0,0 @@
-.. _whatsnew_upgrading:
-
-%%%%%%%%%%%%%%%%%%%%%%%%
-Upgrading to Varnish 4.0
-%%%%%%%%%%%%%%%%%%%%%%%%
-
-Changes to VCL
-==============
-
-The backend fetch parts of VCL have changed in Varnish 4. We've tried to
-compile a list of changes needed to upgrade here.
-
-Version statement
-~~~~~~~~~~~~~~~~~
-
-To make sure that people have upgraded their VCL to the current
-version, Varnish now requires the first line of VCL to indicate the
-VCL version number::
-
- vcl 4.0;
-
-req.request is now req.method
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To align better with RFC naming, `req.request` has been renamed to
-`req.method`.
-
-vcl_fetch is now vcl_backend_response
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Directors have been moved to the vmod_directors
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To make directors (backend selection logic) easier to extend, the
-directors are now defined in loadable VMODs.
-
-Setting a backend for future fetches in `vcl_recv` is now done as follows::
-
- sub vcl_init {
- new cluster1 = directors.round_robin();
- cluster1.add_backend(b1, 1.0);
- cluster1.add_backend(b2, 1.0);
- }
-
- sub vcl_recv {
- set req.backend_hint = cluster1.backend();
- }
-
-Note the extra `.backend()` needed after the director name.
-
-Use the hash director as a client director
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Since the client director was already a special case of the hash director, it
-has been removed, and you should use the hash director directly::
-
- sub vcl_init {
- new h = directors.hash();
- h.add_backend(b1, 1);
- h.add_backend(b2, 1);
- }
-
- sub vcl_recv {
- set req.backend_hint = h.backend(client.identity);
- }
-
-vcl_error is now vcl_backend_error
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To make a distinction between internally generated errors and
-VCL synthetic responses, `vcl_backend_error` will be called when
-varnish encounters an error when trying to fetch an object.
-
-error() is now synth()
-~~~~~~~~~~~~~~~~~~~~~~
-
-And you must explicitly return it::
-
- return (synth(999, "Response"));
-
-Synthetic responses in vcl_synth
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Setting headers on synthetic response bodies made in vcl_synth are now done on
-resp.http instead of obj.http.
-
-The synthetic keyword is now a function::
-
- if (resp.status == 799) {
- set resp.status = 200;
- set resp.http.Content-Type = "text/plain; charset=utf-8";
- synthetic("You are " + client.ip);
- return (deliver);
- }
-
-obj in vcl_error replaced by beresp in vcl_backend_error
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-To better represent a the context in which it is called, you
-should now use `beresp.*` vcl_backend_error, where you used to
-use `obj.*` in `vcl_error`.
-
-hit_for_pass objects are created using beresp.uncacheable
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Example::
-
- sub vcl_backend_response {
- if (beresp.http.X-No-Cache) {
- set beresp.uncacheable = true;
- set beresp.ttl = 120s;
- return (deliver);
- }
- }
-
-req.* not available in vcl_backend_response
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-req.* used to be available in `vcl_fetch`, but after the split of
-functionality, you only have 'bereq.*' in `vcl_backend_response`.
-
-vcl_* reserved
-~~~~~~~~~~~~~~
-
-Any custom-made subs cannot be named 'vcl_*' anymore. This namespace
-is reserved for builtin subs.
-
-req.backend.healthy replaced by std.healthy(req.backend_hint)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Remember to import the std module if you're not doing so already.
-
-client.port, and server.port replaced by respectively std.port(client.ip) and std.port(server.ip)
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-`client.ip` and `server.ip` are now proper data types, which renders
-as an IP address by default. You need to use the `std.port()`
-function to get the port number.
-
-Invalidation with purge
-~~~~~~~~~~~~~~~~~~~~~~~
-
-Cache invalidation with purges is now done via `return(purge)` from `vcl_recv`.
-The `purge;` keyword has been retired.
-
-obj is now read-only
-~~~~~~~~~~~~~~~~~~~~
-
-`obj` is now read-only. `obj.last_use` has been retired.
-
-Some return values have been replaced
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Apart from the new `synth` return value described above, the
-following has changed:
-
- - `vcl_recv` must now return `hash` instead of `lookup`
- - `vcl_hash` must now return `lookup` instead of `hash`
- - `vcl_pass` must now return `fetch` instead of `pass`
-
-
-Backend restarts are now retry
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In 3.0 it was possible to do `return(restart)` after noticing that
-the backend response was wrong, to change to a different backend.
-
-This is now called `return(retry)`, and jumps back up to `vcl_backend_fetch`.
-
-This only influences the backend fetch thread, client-side handling is not affected.
-
-
-default/builtin VCL changes
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The VCL code that is appended to user-configured VCL automatically
-is now called the builtin VCL. (previously default.vcl)
-
-The builtin VCL now honors Cache-Control: no-cache (and friends)
-to indicate uncacheable content from the backend.
-
-
-The `remove` keyword is gone
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Replaced by `unset`.
-
-
-X-Forwarded-For is now set before vcl_recv
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-In many cases, people unintentionally removed X-Forwarded-For when
-implementing their own vcl_recv. Therefore it has been moved to before
-vcl_recv, so if you don't want an IP added to it, you should remove it
-in vcl_recv.
-
-
-Changes to existing parameters
-==============================
-
-session_linger
-~~~~~~~~~~~~~~
-`session_linger` has been renamed to `timeout_linger` and it is in
-seconds now (previously was milliseconds).
-
-sess_timeout
-~~~~~~~~~~~~
-`sess_timeout` has been renamed to `timeout_idle`.
-
-sess_workspace
-~~~~~~~~~~~~~~
-
-In 3.0 it was often necessary to increase `sess_workspace` if a
-lot of VMODs, complex header operations or ESI were in use.
-
-This is no longer necessary, because ESI scratch space happens
-elsewhere in 4.0.
-
-If you are using a lot of VMODs, you may need to increase
-either `workspace_backend` and `workspace_client` based on where
-your VMOD is doing its work.
-
-thread_pool_purge_delay
-~~~~~~~~~~~~~~~~~~~~~~~
-`thread_pool_purge_delay` has been renamed to `thread_pool_destroy_delay`
-and it is in seconds now (previously was milliseconds).
-
-thread_pool_add_delay and thread_pool_fail_delay
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-They are in seconds now (previously were milliseconds).
-
-New parameters since 3.0
-========================
-
-vcc_allow_inline_c
-~~~~~~~~~~~~~~~~~~
-
-You can now completely disable inline C in your VCL, and it is
-disabled by default.
-
-Other changes
-=============
-
-New log filtering
-~~~~~~~~~~~~~~~~~
-
-The logging framework has a new filtering language, which means that
-the -m switch has been replaced with a new -q switch. See
-:ref:`vsl-query(7)` for more information about the new query language.
diff --git a/doc/sphinx/whats-new/upgrading-4.0.rst b/doc/sphinx/whats-new/upgrading-4.0.rst
new file mode 100644
index 0000000..eecd5b7
--- /dev/null
+++ b/doc/sphinx/whats-new/upgrading-4.0.rst
@@ -0,0 +1,248 @@
+.. _whatsnew_upgrading_4_0:
+
+%%%%%%%%%%%%%%%%%%%%%%%%
+Upgrading to Varnish 4.0
+%%%%%%%%%%%%%%%%%%%%%%%%
+
+Changes to VCL
+==============
+
+The backend fetch parts of VCL have changed in Varnish 4. We've tried to
+compile a list of changes needed to upgrade here.
+
+Version statement
+~~~~~~~~~~~~~~~~~
+
+To make sure that people have upgraded their VCL to the current
+version, Varnish now requires the first line of VCL to indicate the
+VCL version number::
+
+ vcl 4.0;
+
+req.request is now req.method
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To align better with RFC naming, `req.request` has been renamed to
+`req.method`.
+
+vcl_fetch is now vcl_backend_response
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Directors have been moved to the vmod_directors
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To make directors (backend selection logic) easier to extend, the
+directors are now defined in loadable VMODs.
+
+Setting a backend for future fetches in `vcl_recv` is now done as follows::
+
+ sub vcl_init {
+ new cluster1 = directors.round_robin();
+ cluster1.add_backend(b1, 1.0);
+ cluster1.add_backend(b2, 1.0);
+ }
+
+ sub vcl_recv {
+ set req.backend_hint = cluster1.backend();
+ }
+
+Note the extra `.backend()` needed after the director name.
+
+Use the hash director as a client director
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Since the client director was already a special case of the hash director, it
+has been removed, and you should use the hash director directly::
+
+ sub vcl_init {
+ new h = directors.hash();
+ h.add_backend(b1, 1);
+ h.add_backend(b2, 1);
+ }
+
+ sub vcl_recv {
+ set req.backend_hint = h.backend(client.identity);
+ }
+
+vcl_error is now vcl_backend_error
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To make a distinction between internally generated errors and
+VCL synthetic responses, `vcl_backend_error` will be called when
+varnish encounters an error when trying to fetch an object.
+
+error() is now synth()
+~~~~~~~~~~~~~~~~~~~~~~
+
+And you must explicitly return it::
+
+ return (synth(999, "Response"));
+
+Synthetic responses in vcl_synth
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Setting headers on synthetic response bodies made in vcl_synth are now done on
+resp.http instead of obj.http.
+
+The synthetic keyword is now a function::
+
+ if (resp.status == 799) {
+ set resp.status = 200;
+ set resp.http.Content-Type = "text/plain; charset=utf-8";
+ synthetic("You are " + client.ip);
+ return (deliver);
+ }
+
+obj in vcl_error replaced by beresp in vcl_backend_error
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To better represent a the context in which it is called, you
+should now use `beresp.*` vcl_backend_error, where you used to
+use `obj.*` in `vcl_error`.
+
+hit_for_pass objects are created using beresp.uncacheable
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Example::
+
+ sub vcl_backend_response {
+ if (beresp.http.X-No-Cache) {
+ set beresp.uncacheable = true;
+ set beresp.ttl = 120s;
+ return (deliver);
+ }
+ }
+
+req.* not available in vcl_backend_response
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+req.* used to be available in `vcl_fetch`, but after the split of
+functionality, you only have 'bereq.*' in `vcl_backend_response`.
+
+vcl_* reserved
+~~~~~~~~~~~~~~
+
+Any custom-made subs cannot be named 'vcl_*' anymore. This namespace
+is reserved for builtin subs.
+
+req.backend.healthy replaced by std.healthy(req.backend_hint)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Remember to import the std module if you're not doing so already.
+
+client.port, and server.port replaced by respectively std.port(client.ip) and std.port(server.ip)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+`client.ip` and `server.ip` are now proper data types, which renders
+as an IP address by default. You need to use the `std.port()`
+function to get the port number.
+
+Invalidation with purge
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Cache invalidation with purges is now done via `return(purge)` from `vcl_recv`.
+The `purge;` keyword has been retired.
+
+obj is now read-only
+~~~~~~~~~~~~~~~~~~~~
+
+`obj` is now read-only. `obj.last_use` has been retired.
+
+Some return values have been replaced
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Apart from the new `synth` return value described above, the
+following has changed:
+
+ - `vcl_recv` must now return `hash` instead of `lookup`
+ - `vcl_hash` must now return `lookup` instead of `hash`
+ - `vcl_pass` must now return `fetch` instead of `pass`
+
+
+Backend restarts are now retry
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In 3.0 it was possible to do `return(restart)` after noticing that
+the backend response was wrong, to change to a different backend.
+
+This is now called `return(retry)`, and jumps back up to `vcl_backend_fetch`.
+
+This only influences the backend fetch thread, client-side handling is not affected.
+
+
+default/builtin VCL changes
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The VCL code that is appended to user-configured VCL automatically
+is now called the builtin VCL. (previously default.vcl)
+
+The builtin VCL now honors Cache-Control: no-cache (and friends)
+to indicate uncacheable content from the backend.
+
+
+The `remove` keyword is gone
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Replaced by `unset`.
+
+
+X-Forwarded-For is now set before vcl_recv
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+In many cases, people unintentionally removed X-Forwarded-For when
+implementing their own vcl_recv. Therefore it has been moved to before
+vcl_recv, so if you don't want an IP added to it, you should remove it
+in vcl_recv.
+
+
+Changes to existing parameters
+==============================
+
+session_linger
+~~~~~~~~~~~~~~
+`session_linger` has been renamed to `timeout_linger` and it is in
+seconds now (previously was milliseconds).
+
+sess_timeout
+~~~~~~~~~~~~
+`sess_timeout` has been renamed to `timeout_idle`.
+
+sess_workspace
+~~~~~~~~~~~~~~
+
+In 3.0 it was often necessary to increase `sess_workspace` if a
+lot of VMODs, complex header operations or ESI were in use.
+
+This is no longer necessary, because ESI scratch space happens
+elsewhere in 4.0.
+
+If you are using a lot of VMODs, you may need to increase
+either `workspace_backend` and `workspace_client` based on where
+your VMOD is doing its work.
+
+thread_pool_purge_delay
+~~~~~~~~~~~~~~~~~~~~~~~
+`thread_pool_purge_delay` has been renamed to `thread_pool_destroy_delay`
+and it is in seconds now (previously was milliseconds).
+
+thread_pool_add_delay and thread_pool_fail_delay
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+They are in seconds now (previously were milliseconds).
+
+New parameters since 3.0
+========================
+
+vcc_allow_inline_c
+~~~~~~~~~~~~~~~~~~
+
+You can now completely disable inline C in your VCL, and it is
+disabled by default.
+
+Other changes
+=============
+
+New log filtering
+~~~~~~~~~~~~~~~~~
+
+The logging framework has a new filtering language, which means that
+the -m switch has been replaced with a new -q switch. See
+:ref:`vsl-query(7)` for more information about the new query language.
diff --git a/doc/sphinx/whats-new/upgrading-4.1.rst b/doc/sphinx/whats-new/upgrading-4.1.rst
new file mode 100644
index 0000000..9a78d29
--- /dev/null
+++ b/doc/sphinx/whats-new/upgrading-4.1.rst
@@ -0,0 +1,72 @@
+.. _whatsnew_upgrading_4_1:
+
+%%%%%%%%%%%%%%%%%%%%%%%%
+Upgrading to Varnish 4.1
+%%%%%%%%%%%%%%%%%%%%%%%%
+
+Changes to VCL
+==============
+
+Data type conversion functions now take a fallback
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Data type conversion functions in the std vmod now takes an additional
+argument *fallback*, which is returned if the conversion does not succeed.
+
+
+Version statement is kept
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The VCL syntax has not chanced significantly, and as such the Varnish 4.0
+version marker is kept for Varnish 4.1.
+
+One of the initial lines in a Varnish 4.1 VCL should read::
+
+ vcl 4.0;
+
+Remote address accessors
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+New in 4.1 is the `local.ip` and `remote.ip` representing the (local) TCP
+connection endpoints.
+
+With PROXY listeners the `server.ip` and `client.ip` are set from the PROXY
+preamble. On normal HTTP listeners the behaviour is unchanged.
+
+
+Management interface
+====================
+
+The management interface enabled with ``-M`` previously supported the telnet
+protocol.
+
+Support for telnet control sequences have been retired. Replacement clients
+like netcat or (preferred) ``varnishadm`` should be used instead.
+
+
+Runtime users and groups
+========================
+
+With the new jail support, an additional runtime user (`vcache`) should be used
+for the Varnish worker child process.
+
+Additionally, the ``varnishlog``, ``varnishncsa`` and other Varnish shared log
+utilities must now be run in a context with `varnish` group membership.
+
+
+Changes to parameters
+=====================
+
+`vcl_cooldown` is new, and decides how long time a VCL is kept warm after being
+replaced as the active VCL.
+
+The following parameters have been retired:
+
+* `group` (security revamp)
+* `group_cc` (security revamp)
+* `listen_address` (security revamp)
+* `pool_vbc`
+* `timeout_req` - merged with `timeout_idle`.
+* `user` (security revamp)
+
+Minor changes of default values on `workspace_session` and `vsl_mask`.
diff --git a/doc/sphinx/whats-new/upgrading-5.0.rst b/doc/sphinx/whats-new/upgrading-5.0.rst
new file mode 100644
index 0000000..7cc40a4
--- /dev/null
+++ b/doc/sphinx/whats-new/upgrading-5.0.rst
@@ -0,0 +1,17 @@
+.. _whatsnew_upgrading_5_0:
+
+%%%%%%%%%%%%%%%%%%%%%%%%
+Upgrading to Varnish 5.0
+%%%%%%%%%%%%%%%%%%%%%%%%
+
+Changes to VCL
+==============
+
+
+Management interface
+====================
+
+
+Changes to parameters
+=====================
+
diff --git a/doc/sphinx/whats-new/upgrading.rst b/doc/sphinx/whats-new/upgrading.rst
deleted file mode 100644
index 49c4f08..0000000
--- a/doc/sphinx/whats-new/upgrading.rst
+++ /dev/null
@@ -1,72 +0,0 @@
-.. _whatsnew_upgrade41:
-
-%%%%%%%%%%%%%%%%%%%%%%%%
-Upgrading to Varnish 4.1
-%%%%%%%%%%%%%%%%%%%%%%%%
-
-Changes to VCL
-==============
-
-Data type conversion functions now take a fallback
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Data type conversion functions in the std vmod now takes an additional
-argument *fallback*, which is returned if the conversion does not succeed.
-
-
-Version statement is kept
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The VCL syntax has not chanced significantly, and as such the Varnish 4.0
-version marker is kept for Varnish 4.1.
-
-One of the initial lines in a Varnish 4.1 VCL should read::
-
- vcl 4.0;
-
-Remote address accessors
-~~~~~~~~~~~~~~~~~~~~~~~~
-
-New in 4.1 is the `local.ip` and `remote.ip` representing the (local) TCP
-connection endpoints.
-
-With PROXY listeners the `server.ip` and `client.ip` are set from the PROXY
-preamble. On normal HTTP listeners the behaviour is unchanged.
-
-
-Management interface
-====================
-
-The management interface enabled with ``-M`` previously supported the telnet
-protocol.
-
-Support for telnet control sequences have been retired. Replacement clients
-like netcat or (preferred) ``varnishadm`` should be used instead.
-
-
-Runtime users and groups
-========================
-
-With the new jail support, an additional runtime user (`vcache`) should be used
-for the Varnish worker child process.
-
-Additionally, the ``varnishlog``, ``varnishncsa`` and other Varnish shared log
-utilities must now be run in a context with `varnish` group membership.
-
-
-Changes to parameters
-=====================
-
-`vcl_cooldown` is new, and decides how long time a VCL is kept warm after being
-replaced as the active VCL.
-
-The following parameters have been retired:
-
-* `group` (security revamp)
-* `group_cc` (security revamp)
-* `listen_address` (security revamp)
-* `pool_vbc`
-* `timeout_req` - merged with `timeout_idle`.
-* `user` (security revamp)
-
-Minor changes of default values on `workspace_session` and `vsl_mask`.
More information about the varnish-commit
mailing list