HELP Wanted: Redirecting repo-traffic to packagecloud.io

[Poul-Henning Kamp]

It seems that no response codes will deter linux package-management tools
and the amount of traffic we had beating up on the project server that
way was totally swamping any other traffic.

That's not a problem of course, we run varnish.

But it is a LOT of users not getting good service.

I spent some hours trying to find out where to send them on packagecloud.io
and came up with this:

        if (req.url ~ "repomd.xml") {
                set req.http.urlx = regsub(req.url,
                    ".*varnish-([345])[.](.)/el(.*)",
                    "/varnishcache/varnish\1\2/el/\3");
                return (synth(301,"packagecloud"));
        }
        if (req.url ~ "^/(ubuntu|debian)/dists/[^/]*/I*n*Release$") {
                /*
                 * We don't seem to have files on packageio which lists all
                 * releases, so point these at release5.2
                 */
                set req.http.urlx = regsub(req.url,
                    "/(ubuntu|debian)/dists/(.*)/(I*n*Release)",
                    "/varnishcache/varnish52/\1/dists/\2/\3");
                return (synth(301,"packagecloud"));
        }

I'm still seeing a lot of "deep" urls being requested (see below) but
I'm hoping these will migrate to packagecloud now that the top-level
URLs point them there.

I would appreciate if somebody with more linux package clue than me
(ie: any non-zero amount of clue will be helpful) can sanity-check
what I've done.

Poul-Henning

    12.47 ReqURL         /ubuntu/dists/trusty/InRelease
    10.23 ReqURL         /redhat/varnish-3.0/el5/x86_64/repodata/repomd.xml
     8.45 ReqURL         /redhat/varnish-4.0/el6/repodata/repomd.xml
     6.69 ReqURL         /redhat/varnish-4.1/el6/x86_64/repodata/0db82826c0003726ea45ca2ff822454afb55e209-primary.xml.gz
     6.57 ReqURL         /redhat/varnish-3.0/el6/x86_64/repodata/repomd.xml
     6.54 ReqURL         /
     6.07 ReqURL         /redhat/varnish-3.0/el6/repodata/repomd.xml
     5.51 ReqURL         /redhat/varnish-3.0/el5/x86_64/repodata/46939bc1312d4686f78d539762f83d0d3bb5f5cf-primary.xml.gz
     5.15 ReqURL         /redhat/varnish-3.0/el6/x86_64/repodata/0e86f304b9f8c0e214d431f95100ad4668e0ad9e-primary.xml.gz
     4.63 ReqURL         /ubuntu/dists/lucid/Release
     4.60 ReqURL         /ubuntu/dists/lucid/Release.gpg
     4.45 ReqURL         /redhat/varnish-4.0/el6/x86_64/repodata/repomd.xml
     4.30 ReqURL         /debian/dists/wheezy/Release
     4.30 ReqURL         /debian/dists/wheezy/Release.gpg
     3.84 ReqURL         /redhat/varnish-4.1/el6/x86_64/repodata/repomd.xml
     3.78 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-i386/Packages.gz
     3.77 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-i386/Packages.lzma
     3.74 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-i386/Packages.bz2
     3.48 ReqURL         /redhat/varnish-4.0/el6/x86_64/repodata/b3cc34523cf82349fbdce79b0588b471b842a408-primary.xml.gz
     3.47 ReqURL         /ubuntu/dists/precise/Release
     3.46 ReqURL         /ubuntu/dists/precise/Release.gpg
     3.18 ReqURL         /ubuntu/dists/lucid/varnish-3.0/i18n/Translation-en
     3.17 ReqURL         /ubuntu/dists/lucid/varnish-3.0/i18n/Translation-en.gz
     3.17 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-i386/Packages
     3.16 ReqURL         /ubuntu/dists/lucid/varnish-3.0/i18n/Translation-en.lzma
     3.14 ReqURL         /ubuntu/dists/lucid/varnish-3.0/i18n/Translation-en.xz
     3.13 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-i386/Packages.xz
     3.12 ReqURL         /ubuntu/dists/lucid/varnish-3.0/i18n/Translation-en.bz2
     3.10 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-i386/Packages.diff/Index
     3.01 ReqURL         /debian/dists/jessie/InRelease
     2.97 ReqURL         /debian/dists/wheezy/varnish-3.0/binary-amd64/Packages
     2.96 ReqURL         /debian/dists/wheezy/varnish-3.0/i18n/Translation-en
     2.96 ReqURL         /debian/dists/wheezy/varnish-3.0/binary-amd64/Packages.gz
     2.96 ReqURL         /debian/dists/wheezy/varnish-3.0/i18n/Translation-en.gz
     2.92 ReqURL         /debian/dists/wheezy/varnish-3.0/i18n/Translation-en.lzma
     2.92 ReqURL         /debian/dists/wheezy/varnish-3.0/binary-amd64/Packages.lzma
     2.92 ReqURL         /debian/dists/wheezy/varnish-3.0/binary-amd64/Packages.xz
     2.92 ReqURL         /debian/dists/wheezy/varnish-3.0/i18n/Translation-en.xz
     2.90 ReqURL         /ubuntu/GPG-key.txt
     2.89 ReqURL         /debian/dists/wheezy/varnish-3.0/binary-amd64/Packages.bz2
     2.89 ReqURL         /debian/dists/wheezy/varnish-3.0/i18n/Translation-en.bz2
     2.87 ReqURL         /debian/GPG-key.txt
     2.78 ReqURL         /debian/dists/wheezy/varnish-3.0/binary-amd64/Packages.diff/Index
     2.53 ReqURL         /redhat/varnish-4.0/el7/repodata/repomd.xml
     2.44 ReqURL         /redhat/varnish-4.0/el7/x86_64/repodata/repomd.xml
     2.40 ReqURL         /redhat/varnish-4.0/el7/x86_64/repodata/0ffac79c8b21f896830f3e6a81b4e9f8430199d3-primary.xml.gz
     2.37 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-amd64/Packages.gz
     2.36 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-amd64/Packages.lzma
     2.36 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-amd64/Packages
     2.34 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-amd64/Packages.bz2
     2.33 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-amd64/Packages.xz
     2.31 ReqURL         /ubuntu/dists/lucid/varnish-3.0/binary-amd64/Packages.diff/Index


-- 
Poul-Henning Kamp       | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG         | TCP/IP since RFC 956
FreeBSD committer       | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.