[master] e98e8e649 Documentation updates for changed `vcl_hash{}` / `hash_data()`

[Poul-Henning Kamp]

commit e98e8e6497b9bbcca3e709a5ea0e094f2ec4b930
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Feb 12 11:12:36 2021 +0000

    Documentation updates for changed `vcl_hash{}` / `hash_data()`

diff --git a/doc/changes.rst b/doc/changes.rst
index a21ad8580..f23a79f70 100644
--- a/doc/changes.rst
+++ b/doc/changes.rst
@@ -30,6 +30,15 @@ release process.
 Varnish Cache Next (2021-03-15)
 ================================
 
+* `hash_data()` can be called from `vcl_recv`, in which case
+  `vcl_hash` is not called.  This allows `vcl_recv` and 
+  backends to take the object identity into account, for
+  instance when choosing backend and grace periods.
+
+* `hash_data()` calculates the hash-key differently than previously.
+  This means that persistent storage will be lost, and it may break
+  very specific `*.vtc` test-scripts.
+
 * counters MAIN.s_req_bodybytes and VBE.*.tools.beresp_bodybytes
   are now always the number of bodybytes moved on the wire.
 
diff --git a/doc/sphinx/reference/vcl.rst b/doc/sphinx/reference/vcl.rst
index 9dc04d6ca..d6f468e04 100644
--- a/doc/sphinx/reference/vcl.rst
+++ b/doc/sphinx/reference/vcl.rst
@@ -478,7 +478,9 @@ hash_data(input)
 ~~~~~~~~~~~~~~~~
 
   Adds an input to the hash input. In the built-in VCL ``hash_data()``
-  is called on the host and URL of the request. Available in ``vcl_hash``.
+  is called on the host and URL of the request.
+  Available in ``vcl_hash`` or ``vcl_recv``.  If used in ``vcl_recv``
+  ``vcl_hash`` will not be called.
 
 synthetic(STRING)
 ~~~~~~~~~~~~~~~~~
diff --git a/doc/sphinx/users-guide/vcl-built-in-subs.rst b/doc/sphinx/users-guide/vcl-built-in-subs.rst
index c6755ee96..bb3673687 100644
--- a/doc/sphinx/users-guide/vcl-built-in-subs.rst
+++ b/doc/sphinx/users-guide/vcl-built-in-subs.rst
@@ -129,8 +129,9 @@ of the following keywords:
 vcl_hash
 ~~~~~~~~
 
-Called after `vcl_recv` to create a hash value for the request. This is
-used as a key to look up the object in Varnish.
+Called after `vcl_recv` to create a hash value for the request,
+unless `vcl_recv` already did that.
+This is used as the key to store and look up objects in the cache.
 
 The `vcl_hash` subroutine may terminate with calling ``return()`` with one
 of the following keywords:
diff --git a/doc/sphinx/users-guide/vcl-hashing.rst b/doc/sphinx/users-guide/vcl-hashing.rst
index bdc47b0bb..5d960a6d0 100644
--- a/doc/sphinx/users-guide/vcl-hashing.rst
+++ b/doc/sphinx/users-guide/vcl-hashing.rst
@@ -1,12 +1,10 @@
 Hashing
 -------
 
-Internally, when Varnish stores content in the cache it stores the object
-together with a hash key to find the object again. In the default setup
-this key is calculated based on the content of the *Host* header or the
-IP address of the server and the URL.
-
-Behold the `default vcl`::
+Internally, when Varnish stores content in the cache indexed by a hash
+key used to find the object again. In the default setup
+this key is calculated based on `URL`, the `Host:` header, or
+if there is none, the IP address of the server::
 
     sub vcl_hash {
         hash_data(req.url);
@@ -18,7 +16,7 @@ Behold the `default vcl`::
         return (lookup);
     }
 
-As you can see it first checks in `req.url` then `req.http.host` if
+As you can see it first hashes `req.url` and then `req.http.host` if
 it exists. It is worth pointing out that Varnish doesn't lowercase the
 hostname or the URL before hashing it so in theory having "Varnish.org/"
 and "varnish.org/" would result in different cache entries. Browsers
@@ -42,7 +40,16 @@ And then add a `vcl_hash`::
         hash_data(req.http.X-Country-Code);
     }
 
-As the default VCL will take care of adding the host and URL to the hash
-we don't have to do anything else. Be careful calling ``return (lookup)``
-as this will abort the execution of the default VCL and Varnish can end
-up returning data based on more or less random inputs.
+Because there is no `return(lookup)`, the builtin VCL will take care
+of adding the URL, `Host:` or server IP# to the hash as usual.
+
+If `vcl_hash` did return, ie::
+
+    sub vcl_hash {
+        hash_data(req.http.X-Country-Code);
+        return(lookup);
+    }
+
+then *only* the country-code would matter, and Varnish would return
+seemingly random objects, ignoring the URL, (but they would always
+have the correct `X-Country-Code`).