Moved from ticket #21:

Keeping an eye on a run that has now gotten into the 13th hour I spotted a behaviour which may warrant a mention in the documentation:

Most sites have a limited number of standard TTLs they use, for instance 2m, 15m, 1h, 24h & 1w or some such.

When we start varnish, a lot of the long-TTL items will be pulled in from the backend quite rapidly and then when they all expire we do a similar "mass-pull" from the backend, and so again next time and next time...

After quite some time, we can assume that it will even out over the TTL period, but it might take many days or even weeks for that to happen.

The very simple band-aid to avoiding this periodicity in the load we inflict on the backend would be to deduct a randomized percentage of long TTLs.

This is clearly something that should happen in VCL if necessary and I guess the code would look something like this (once VCL learns to do it):

             if (obj.ttl > 1h) {
                  obj.ttl = obj.ttl * random(0.75, 1.00);
             }