[master] 988cd77 Start working on the Users Guide

Poul-Henning Kamp phk at varnish-cache.org
Fri Sep 20 12:49:02 CEST 2013

commit 988cd77462865f25d76db1684df183125565e359
Author: Poul-Henning Kamp <phk at FreeBSD.org>
Date:   Fri Sep 20 10:48:28 2013 +0000

    Start working on the Users Guide

diff --git a/doc/sphinx/users-guide/configuration.rst b/doc/sphinx/users-guide/configuration.rst
deleted file mode 100644
index f49f99b..0000000
--- a/doc/sphinx/users-guide/configuration.rst
+++ /dev/null
@@ -1,17 +0,0 @@
-This chapter deals with configuration of Varnish Cache. It will not
-cover the actual configuration policy that's written in VCL. 
-This guide assumes you're running Debian, Ubuntu, Red Hat or
-Centos. If you're on any other platform you're probably used to doing
-some mental translations.
-.. toctree::
-   :maxdepth: 2
-   command-line
-   storage-backends
-   params
diff --git a/doc/sphinx/users-guide/index.rst b/doc/sphinx/users-guide/index.rst
index a28b08f..7b909c9 100644
--- a/doc/sphinx/users-guide/index.rst
+++ b/doc/sphinx/users-guide/index.rst
@@ -1,22 +1,52 @@
 .. _users-guide-index:
-Using Varnish - A Users Guide
+The Varnish Users Guide
-This guide is intended for system administrators managing Varnish
+(The plan for ...)
+The Varnish documentation consists of three main documents:
-The guide is split into short chapters, each chapter explaining a
-separate topic.
+* :ref:`tutorial-index` explains the basics and gets you started with Varnish.
+* :ref:`users-guide-index` (this document), explains how Varnish works
+  and how you can use it to improve your website. 
+* :ref:`reference-index` contains hard facts and is useful for
+  looking up specific questions.
+After :ref:`users_intro`, this Users Guide is organized in sections
+along the major interfaces to Varnish as a service:
+:ref:`users_running` is about getting Varnish configured, with
+respect to storage, sockets, security and how you can control and
+communicate with Varnish once it is running.
+:ref:`users_vcl` is about getting Varnish to handle the
+HTTP requests the way you want, what to cache, how to cache it,
+modifying HTTP headers etc. etc.
+:ref:`users_report` explains how you can see and monitor what Varnish does,
+from transaction level to aggregate statistics.
+:ref:`users_performance` is about tuning your website with Varnish.
+:ref:`users_trouble` is for locating and fixing trouble with Varnish.
+Detailed Table of Contents:
 .. toctree::
    :maxdepth: 2
-   configuration
+   intro
+   running
-   operation
+   report
+   performance
+   operation
 .. customizing (which is a non ideal title)
diff --git a/doc/sphinx/users-guide/intro.rst b/doc/sphinx/users-guide/intro.rst
new file mode 100644
index 0000000..3d2fb0d
--- /dev/null
+++ b/doc/sphinx/users-guide/intro.rst
@@ -0,0 +1,119 @@
+.. _users_intro:
+The Big Varnish Picture
+What is in this package called "Varnish", what are all the different
+bits and pieces named and will you need a hex-wrench for assembly ?
+The two main parts of Varnish are the two processes in the varnishd
+The first process is called "the manager", and its job is to talk
+to you, the administrator, and make the things you ask for happen.
+The second process is called "the worker" or just "the child" and
+this is the process which does all the actual work with your HTTP
+When you start varnishd, you start the manager process, and once
+it is done handling all the command line flags, it will start the
+child process for you.
+Should the child process die, the manager will start it again for
+you, automatically and right away.
+The main reason for this division of labor is security:  The manager
+process will typically run with "root" permissions, in order to
+open TCP socket port 80, but it starts the child process with minimal
+permissions, as a defensive measure.
+The manager process is interactive, it offers a CLI -- Command Line
+Interface, which can be used manually, from scripts or programs.
+The CLI offers almost full control of what Varnish actually does
+to your HTTP traffic, and we have gone to great lengths to ensure
+that you should not need to restart the varnish processes, unless
+you need to change something very fundamental.
+The CLI can be safely accessed remotely, using a simple and flexible
+PSK -- Pre Shared Key, access control scheme, so it is easy to
+integrate Varnish into your operations and management infrastructure
+or tie it to your CMS.
+All this is covered in :ref:`users_running`.
+How the child process should deal with the HTTP requests, what to
+cache, which headers to remove etc, is al specified in a small
+programming language called VCL -- Varnish Configuration Language.
+The manager process will compile the VCL program and check it for
+errors, but it is the child process which runs the VCL program, for
+each and every HTTP request which comes in.
+Because the VCL is compiled to C code, and the C code is compiled
+to machine instructions, even very complex VCL programs execute in
+a a few microseconds, without impacting performance at all.
+And don't fret if you are not really a programmer, VCL is very
+simple to do simpel things with::
+	sub vcl_recv {
+		# Remove the cookie header to enable caching
+		unset req.http.cookie;
+	}
+The CLI interface allows you to compile and load new VCL programs
+at any time, and you can switch betweem the loaded VCL programs
+instantly, without restarting the child process and without missing
+a single HTTP request.
+VCL code can be extended using external modules, called VMODs or
+even by inline C-code if you are brave, so in terms of what Varnish
+can do for your HTTP traffic, there really is no limit.
+:ref:`users_vcl` describes VCL and what it can do in great detail.
+Varnish uses a piece of shared memory to report its activity and
+For each HTTP request, a number of very detailed records will be
+appended to the log segment in this shared memory.
+Other processes can subscribe to log-records, filter them, and
+format them, for instance as NCSA style log records.
+Another segment in shared memory is used for statistics counters,
+this allows real-time, down to microsecond resolution monitoring
+of cache hit-rate, resource usage and specific performance indicating
+Varnish comes with a number of tools which reports from shared
+memory, varnishlog, varnishstats, varnishncsa etc, and with a API
+library so you can write your own tools, should you need that.
+:ref:`users_report` explains how all that work.
+Presumably the reason why you are interested in Varnish, is that
+you want your website to work better.
+There are many aspects of performance tuning a website, from
+relatively simple policy decisions about what to cache, to designing
+a geographically diverse multilevel CDNs using ESI and automatic
+:ref:`users_performance` will take you through the possibilities
+and facilities Varnish offers.
+Finally, Murphys Law must be contended with:  Things will go wrong,
+and more likely than not, they will do so at zero-zero-dark O'clock.
+...during a hurricane, when your phones battery is flat and your
+wife had prepared a intimate evening to celebrate your aniversary.
+Yes, we've all been there, havn't we ?
+When things go wrong :ref:`users_trouble` will hopefully be of some help.
+And now, lets put som substance on this skeleton outline...
diff --git a/doc/sphinx/users-guide/operation.rst b/doc/sphinx/users-guide/operation.rst
index 4cc85fa..dc45e7e 100644
--- a/doc/sphinx/users-guide/operation.rst
+++ b/doc/sphinx/users-guide/operation.rst
@@ -1,15 +1,13 @@
+XXX: These are chapters which need to find a new home in the other sections.
 .. toctree::
    :maxdepth: 2
-   operation-logging
-   operation-statistics
-   sizing-your-cache
-   increasing-your-hitrate
diff --git a/doc/sphinx/users-guide/performance.rst b/doc/sphinx/users-guide/performance.rst
new file mode 100644
index 0000000..c28dea0
--- /dev/null
+++ b/doc/sphinx/users-guide/performance.rst
@@ -0,0 +1,12 @@
+.. _users_performance:
+Varnish and Website Performance
+This section is about tuning the performance of your Varnish server,
+and about tuning the performance of your website using Varnish.
+.. toctree::
+   :maxdepth: 2
+   increasing-your-hitrate
diff --git a/doc/sphinx/users-guide/report.rst b/doc/sphinx/users-guide/report.rst
new file mode 100644
index 0000000..5a444ab
--- /dev/null
+++ b/doc/sphinx/users-guide/report.rst
@@ -0,0 +1,14 @@
+.. _users_report:
+Reporting and statistics
+This section is about how to find out what Varnish is doing, from
+the detailed per HTTP request blow-by-blow logrecords to the global
+summary statistics counters.
+.. toctree::
+   :maxdepth: 2
+   operation-logging
+   operation-statistics
diff --git a/doc/sphinx/users-guide/running.rst b/doc/sphinx/users-guide/running.rst
new file mode 100644
index 0000000..d5f87d5
--- /dev/null
+++ b/doc/sphinx/users-guide/running.rst
@@ -0,0 +1,18 @@
+.. _users_running:
+Starting and running Varnish
+This section is about starting, running, and stopping Varnish, about
+command line flags and options, communicating with the running
+Varnish processes, configuring storage and sockets and, and about
+securing and protecting Varnish against attacks.
+.. toctree::
+   :maxdepth: 2
+   command-line
+   storage-backends
+   params
+   sizing-your-cache
diff --git a/doc/sphinx/users-guide/troubleshooting.rst b/doc/sphinx/users-guide/troubleshooting.rst
index c399958..b6f6558 100644
--- a/doc/sphinx/users-guide/troubleshooting.rst
+++ b/doc/sphinx/users-guide/troubleshooting.rst
@@ -1,10 +1,12 @@
+.. _users_trouble:
 Troubleshooting Varnish
 Sometimes Varnish misbehaves. In order for you to understand whats
 going on there are a couple of places you can check. varnishlog,
 /var/log/syslog, /var/log/messages are all places where varnish might
-leave clues of whats going on. This chapter will guide you through
+leave clues of whats going on. This section will guide you through
 basic troubleshooting in Varnish.
diff --git a/doc/sphinx/users-guide/vcl.rst b/doc/sphinx/users-guide/vcl.rst
index 0133dee..dc3fdc5 100644
--- a/doc/sphinx/users-guide/vcl.rst
+++ b/doc/sphinx/users-guide/vcl.rst
@@ -1,6 +1,11 @@
+.. _users_vcl:
 VCL - Varnish Configuration Language
+This section is about getting Varnish to do what you want to
+your HTTP traffic, using the Varnish Configuration Language.
 Varnish has a great configuration system. Most other systems use
 configuration directives, where you basically turn on and off lots of
 switches. Varnish uses a domain specific language called Varnish
@@ -29,6 +34,7 @@ code commented out in default.vcl that ships with Varnish Cache.
 .. toctree::
    :maxdepth: 1
+   vcl-intro

More information about the varnish-commit mailing list