Writing a Director

Varnish already provides a set of general-purpose directors, and since Varnish 4, it is bundled in the built-in VMOD directors - Varnish Directors Module. Writing a director boils down to writing a VMOD, using the proper data structures and APIs. Not only can you write your own director if none of the built-ins fit your needs, but since Varnish 4.1 you can even write your own backends.

Backends can be categorized as such:

• static: native backends declared in VCL

• dynamic: native backends created by VMODs

• custom: backends created and fully managed by VMODs

Backends vs Directors

The intuitive classification for backend and director is an endpoint for the former and a loadbalancer for the latter, but the actual implementation is a bit more subtle. VMODs can accept backend arguments and return backends in VCL (see VCL and C data types), but the underlying C type is struct director aka the VCL_BACKEND typedef. Under the hood director is a generic concept, and a backend is a kind of director.

The line between the two is somewhat blurry at this point, let’s look at some code instead:

// VRT interface from vrt.h

struct vdi_methods {
    unsigned                        magic;
#define VDI_METHODS_MAGIC           0x4ec0c4bb
    const char                      *type;
    vdi_http1pipe_f                 *http1pipe;
    vdi_healthy_f                   *healthy;
    vdi_resolve_f                   *resolve;
    vdi_gethdrs_f                   *gethdrs;
    vdi_getip_f                     *getip;
    vdi_finish_f                    *finish;
    vdi_event_f                     *event;
    vdi_release_f                   *release;
    vdi_destroy_f                   *destroy;
    vdi_panic_f                     *panic;
    vdi_list_f                      *list;
};

struct director {
    unsigned                        magic;
#define DIRECTOR_MAGIC              0x3336351d
    void                            *priv;
    char                            *vcl_name;
    struct vcldir                   *vdir;
    struct lock                     *mtx;
};

A director can be summed up as:

• being of a specific type with a set of operations which is identical for all instances of that particular type

• some instance specific attributes such as a vcl_name and type-specific private data

The difference between a load balancing director and a backend director is mainly the functions they will implement.

The fundamental steps towards a director implementation are:

• implement the required functions

• fill a struct vdi_methods with the name of your director type and your function pointers

  Existence of a healthy callback signifies that the director has some means of dynamically determining its health state.

• in your constructor or other initialization routine, allocate and initialize your director-specific configuration state (aka private data) and call VRT_AddDirector() with your struct vdi_methods, the pointer to your state and a printf format for the name of your director instance

• implement methods or functions returning VCL_BACKEND

• in your destructor or other finalizer, call VRT_DelDirector()

• implement a destroy callback to destroy the actual director private state. It will be called when all references to the director are gone, until then the private state must remain intact and vdi_methods functions callable (but they may return errors).

While vmods can implement functions returning directors, Objects and methods are usually a more natural representation with vmod object instances being or referring to the director private data.

Load Balancing Directors

As in VMOD directors - Varnish Directors Module, you can write directors that will group backends sharing the same role, and pick them according to a strategy. If you need more than the built-in strategies (round-robin, hash, …), even though they can be stacked, it is always possible to write your own.

In this case you simply need to implement the resolve function for the director. Directors are walked until a leaf director is found. A leaf director doesn’t have a resolve function and is used to actually make the backend request, just like the backends you declare in VCL.

load balancing directors use VRT_Assign_Backend() to take references to other directors. They must implement a release callback which has to release all references to other directors and ensure that none are gained after it returns.

Static Directors

As opposed to dynamic backends covered below, directors which are guaranteed to have VCL lifetime (that is, they do not get destroyed before the VCL goes cold) can call VRT_StaticDirector() to avoid reference counting overhead.

Dynamic Backends

If you want to speak HTTP/1 over TCP or UDS, but for some reason VCL does not fit the bill, you can instead reuse the whole backend facility. It allows you for instance to add and remove backends on-demand without the need to reload your VCL. You can then leverage your provisioning system.

Consider the following snippet:

backend default {
    .host = "localhost";
}

The VCL compiler turns this declaration into a struct vrt_backend. When the VCL is loaded, Varnish calls VRT_new_backend (or rather VRT_new_backend_clustered for VSM efficiency) in order to create the director. Varnish doesn’t expose its data structure for actual backends, only the director abstraction and dynamic backends are built just like static backends, one struct at a time. You can get rid of the struct vrt_backend as soon as you have the struct director.

A (dynamic) backend can’t exceed its VCL’s lifespan, because native backends are owned by VCLs. Though a dynamic backend can’t outlive its VCL, it can be deleted any time with VRT_delete_backend. The VCL will delete the remaining backends once discarded, you don’t need to take care of it.

Reference counting is used to ensure that backends which are no longer referenced are destroyed.

Finally, Varnish will take care of event propagation for all native backends, but dynamic backends can only be created when the VCL is warm. If your backends are created by an independent thread (basically outside of VCL scope) you must subscribe to VCL events and watch for VCL state (see Event functions). Varnish will panic if you try to create a backend on a cold VCL, and VRT_new_backend will return NULL if the VCL is cooling. You are also encouraged to comply with the VCL Temperature in general.

Health Probes

It is possible in a VCL program to query the health of a director (see BOOL healthy(BACKEND be)). A director can report its health if it implements the healthy function, it is otherwise always considered healthy.

Unless you are making a dynamic backend, you need to take care of the health probes yourselves. For load balancing directors, being healthy typically means having at least one healthy underlying backend or director.

For dynamic backends, it is just a matter of assigning the probe field in the struct vrt_backend. Once the director is created, the probe definition too is no longer needed. It is then Varnish that will take care of the health probe and disable the feature on a cold VCL (see Event functions).

Instead of initializing your own probe definition, you can get a VCL_PROBE directly built from VCL (see VCL and C data types).

Custom Backends

If you want to implement a custom backend, have a look at how Varnish implements native backends. It is the canonical implementation, and though it provides other services like connection pooling or statistics, it is essentially a director which state is a struct backend. Varnish native backends currently speak HTTP/1 over TCP or UDS, and as such, you need to make your own custom backend if you want Varnish to do otherwise such as connect over UDP or speak a different protocol.

If you want to leverage probes declarations in VCL, which have the advantage of being reusable since they are only specifications, you can. However, you need to implement the whole probing infrastructure from scratch.

You may also consider making your custom backend compliant with regards to the VCL state (see Event functions).

If you are implementing the gethdrs method of your backend (i.e. your backend is able to generate a backend response to be manipulated in vcl_backend_response), you will want to log the response code, protocol and the various headers it’ll create for easier debugging. For this, you can look at the VSL* family of functions, listed in cache/cache.h.

Data structure considerations

When you are creating a custom backend, you may want to provide the semantics of the native backends. In this case, instead of repeating the redundant fields between data structures, you can use the macros VRT_BACKEND_FIELDS and VRT_BACKEND_PROBE_FIELDS to declare them all at once. This is the little dance Varnish uses to copy data between the struct vrt_backend and its internal data structure for example.

The copy can be automated with the macros VRT_BACKEND_HANDLE and VRT_BACKEND_PROBE_HANDLE. You can look at how they can be used in the Varnish code base.