varnish-dev Digest, Vol 115, Issue 10
Phil Stanhope
stanhope at gmail.com
Wed Oct 28 01:01:53 CET 2015
I can give you live traffic. Let's talk. Definitely in Rotterdam.
-phil @phone
On Oct 27, 2015, at 18:38, varnish-dev-request at varnish-cache.org wrote:
Send varnish-dev mailing list submissions to
varnish-dev at varnish-cache.org
To subscribe or unsubscribe via the World Wide Web, visit
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
or, via email, send a message with subject or body 'help' to
varnish-dev-request at varnish-cache.org
You can reach the person managing the list at
varnish-dev-owner at varnish-cache.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of varnish-dev digest..."
Today's Topics:
1. Re: Preventing dup backend names with dynamic backends in
VMODs (Poul-Henning Kamp)
2. Varnish project autumn cleaning (Poul-Henning Kamp)
3. Re: Preventing dup backend names with dynamic backends in
VMODs (Geoff Simmons)
4. Re: Varnish project autumn cleaning (Lasse Karstensen)
5. Re: Varnish project autumn cleaning (Kacper Wysocki)
6. Re: Varnish project autumn cleaning (Federico Schwindt)
----------------------------------------------------------------------
Message: 1
Date: Tue, 27 Oct 2015 12:25:29 +0000
From: "Poul-Henning Kamp" <phk at phk.freebsd.dk>
To: geoff at uplex.de
Cc: Varnish Development <varnish-dev at varnish-cache.org>
Subject: Re: Preventing dup backend names with dynamic backends in
VMODs
Message-ID: <8599.1445948729 at critter.freebsd.dk>
Content-Type: text/plain; charset="us-ascii"
Good catch Geoff.
Yes, uniqueness of backend names affect only CLI and VSM.
The former only if people get confused, the latter always because
the VSM counter segment is named based on the backend name, so
the two will clash.
We've never had a really good model for backend names and we're
probably about to make it even worse with dynamic backends.
To _truly_ identify a backend, you need to know:
VCL name (Soon: .. or global)
VMOD name
VMOD object name
VMOD object instance (= vcl name of instantiated object)
Backend name
IPv4
IPv6
Needless to say, that doesn't work.
(Previously we used {backend name, ipv4, ipv6} that didn't work
either, and since then we grew vmods and dynamic backends.)
My current thinking is that we'll name the backend whatever the
user/vcl/vmod writer likes (ie: Backend name), and deal with the
fall-out.
That really only leaves one question: What do we do when some code
tries to add a backend with a name which already exists.
One option is to fail the backend creation, but since it is only
CLI/VSM that can get confused, that seems too draconian to me.
So I'm tempted to simply add a ".%d" suffix for ever increasing
%d's until the name becomes unique.
Would that work for people ?
Poul-Henning
PS: I also noted that backend names can become quite unwieldy, so
it might be a good idea to give all backends a serial number
which appears in CLI output, and can be used in CLI commands
to refer to a particular backend.
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
------------------------------
Message: 2
Date: Tue, 27 Oct 2015 14:48:28 +0000
From: Poul-Henning Kamp <phk at phk.freebsd.dk>
To: varnish-dev at varnish-cache.org
Subject: Varnish project autumn cleaning
Message-ID: <28767.1445957308 at critter.freebsd.dk>
Content-Type: text/plain; charset="us-ascii"
After some private consultations, this email is being sent to the
-dev list, for notice and comment.
After we have hashed it out here, I will write it up for announce@
and the world at large.
As I have mentioned to a lot of you already, I have some changes
in mind for the Varnish Project after 4.1-R and I am busy these
days trying to get from "in mind" to "a plan".
There are a number of drivers for this, which I will just quickly
sketch out. (If you want more details, I can flesh it out further
in the subsequent discussions, this email is long enough as it is.)
The reasons
===========
1. We've gone from DevOps to Developers
---------------------------------------
This is most evident in 4.1-R being so delayed mainly because of
lack of real-life testing.
Becuase none of the central group of developers actually run a
Varnish production site, we have to persuade other people to do
live tests for us.
This means we cannot use the same "QA" strategy we did previously
and that must be reflected in our release process.
2. The workshop have become cluttered
-------------------------------------
The Trac wiki has become Hotel California.
Patchwork never really worked for us.
The Forum isn't exactly a noted social venue.
The project homepage looks a lot like an advertising front end for
Varnish Software - not from any sinister plan or hostile intent,
but simply from the lack of any other content to put there.
And to top it off we're stilling running Varnish version 3.0.something
on varnish-cache.org.
It's time to clean up the workshop, and prepare it for the next
stretch of the projects life.
3. I don't have any live Varnish instances
------------------------------------------
Well, I *do* have one, but a search-engine request every 10 seconds
is not enough for me to be well informed about choices in varnish
development.
In particular for the H2 development I need to be able to see what
happens in real life.
This has been a major annoyance for me during 4.1 but it will
turn into a major problem developing H2 support, without any
access to real-life traffic.
4. Varnish-Software and Varnish-Cache are too close
---------------------------------------------------
This is a lot more perception than reality, but various people are
concerned.
It is well established that the perception that corruption *could*
happen is as bad for confidence as actual corruption happening, so
their concerns are not unreasonable.
I want to make the distinction between V-S and V-C.O much more clear.
A major sticking point is the project homepage, which V-S has
been so kind to maintain for the project until now, but which
therefore also looks a little bit too much like a V-S subsidiary.
I don't mind giving major shoutouts to the various Varnish
companies on the V-C.O homepage, but it needs to serve the
project at least as much as the companies.
So given these reasons, here is the plan I've come up with:
The plan
========
1. Release by date
------------------
In the future we will do two *major* Varnish releases a year, and
we will put them in the calendar at least 6 months ahead of time.
Whatever is in the tree and whatever works on that date will be
in that release.
If some feature or bug-fix isn't ready, it won't be in that release.
The good news is that it will only have to wait six months for the
next major release, and there may be minor releases before that.
We make this change because we are no longer able, as developers,
to run/orchestrate a comprehensive test-campaign before releases,
and therefore we have to make it possible for our users to do this
for us.
By putting releases into the calendar rather than the wish-list,
our users can plan ahead, and we minimize the consequence in terms
of time if/when one of our releases turns out to be a "dud".
We are not going to give any similar guarantees for *minor* releases,
which we will roll out as needs arise.
2. Tool cleanup
---------------
The Trac software has become a liability, and we want to get rid
of it. In particular fighting spammers is a drain.
The repository itself will move to github.
Unless anybody has better ideas, Bugtracking will move to github also.
I'm currently disinclined to have a Wiki in the future, it seems
to never really have worked for us. If we later find out we need
one, github or Wikia are obvious solutions.
Patchwork will be discontinued. Either we fall back to patches on
the -dev mailing list or github pull-requests or possibly both.
We'll find out what works best. Personally I do like to use
-dev because it works offline for me, and because it preserves
the history in the mail-archives.
The varnish-cache.org homepage will be generated from doc/sphinx
in the repository, that saves us a CMS and a database. If we
later find out we need something stronger, we can do that.
The homepage will revert to a classic "here is our software, here
is the doc, here is what's happening" FOSS project homepage,
with due shoutouts to the rest of the Varnish community.
By generating it out of doc/sphinx, all committers get ability to
push content to the home page. I hope they do, time will show.
In that respect, I will be happy to hand out commit bits for people
who want to help maintain the home-page, (similar to FreeBSD's
src/ports/doc distinctions).
Jenkins seems to work, no need to do anything at present as
far as I can tell.
Coverity has never really struck gold for us, but it's simple
for us to use and having the dog not bark works for me.
As far as I can tell, the forum is not very productive, and it might
be a better idea to focus that activity on Stackoverflow or similar,
where the chances of more user-to-user support is higher.
The mailing lists seems to work, as well as mailing lists can be
expected to, and will be continued basically unchanged.
The IRC channels also seems to work, we'll keep those. It's been
suggested we move to a "real" IRC network, but that has both up and
downsides in my view.
3. Growing up, moving out
-------------------------
After careful consideration, I have reached the conclusion that it
is high time Varnish-Cache.org moves away from its childhood home
at Varnish-Software.com.
There are no ill feelings intended on my part, I'm very grateful
for all the time V-S has spent on the projects server, software and
tools. But the time has simply come where co-habitation no longer
is the best solution for either party.
As V-S grew their company, they have chosen a strategy for their
infrastructure which makes the project server the odd-ball box in
the corner and the necessary exceptions and special rules for the
V-C.O stuff have become a PITA for their sysadmins.
Seen from the project point of view, the perfectly sensible limitations
imposed by V-S corporate needs limits what we can do, and we are
constrained by availability of time and resources at V-S.
And there is that perception thing also.
So it's time for the project to move out.
Initially I will put up a new project server in my own lab, because
it is cheap, accessible (which is important during the transition)
and by definition neutral ground for the project.
Once we've come through the transition and things have been shaken
out and settled down, that server will be migrated to some friendly
hosting service, affordable on the VML funds.
It's important for me that this does not decrease the bus-quotient
of the project to 1.0, so I still want to have a couple of
"co-roots" for this machine.
Making the transition
=====================
Lasse has started an etherpad with notes, ideas and tasks, so there
is no reason to repeat there here, go there instead:
https://etherpad.wikimedia.org/p/vc-github-move
We are not planning a "big-bang" transition, but rather taking it
bit for bit as we go.
Moving the repo to github will be the most visible part, but (famous
last word candidate:) we don't foresee any trouble.
My guess is that the order will roughly be:
* Create github repo, slave it of existing repo.
* Flip order: New github becomes master, existing repo becomes slave
* Tell developers to push to new master repo.
* Migrate (open) Trac tickets to github manually, or if the scripts
Lasse found works: Migrate all tickets to github.
Set Trac Read-Only.
* Announce that future tickets must be opened on github.
* Settling down period.
* Close down patchwork
* I (and others) create the future project homepage under doc/sphinx.
* New homepage goes live on new project server.
* Rescue whatever content deserves rescuing from Trac's Wiki
* Close down Trac
* Move mailing lists to new project server.
... or something like that.
In total it will probably take quite some weeks before we're at the
bottom of the list, which is why I'm not sweating the details too
much yet.
Summary
=======
This is not a political issue.
There are no hard feelings anywhere (I hope).
It is simply me paying some long overdue attention to the FOSS project
around the software,
I hope you all agree and will help make this reorg as uneventful as possible.
Poul-Henning
--
Poul-Henning Kamp | UNIX since Zilog Zeus 3.20
phk at FreeBSD.ORG | TCP/IP since RFC 956
FreeBSD committer | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.
------------------------------
Message: 3
Date: Tue, 27 Oct 2015 16:01:46 +0100
From: Geoff Simmons <geoff at uplex.de>
To: Poul-Henning Kamp <phk at phk.freebsd.dk>
Cc: Varnish Development <varnish-dev at varnish-cache.org>
Subject: Re: Preventing dup backend names with dynamic backends in
VMODs
Message-ID: <562F91DA.8010104 at uplex.de>
Content-Type: text/plain; charset=windows-1252
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
> On 10/27/2015 01:25 PM, Poul-Henning Kamp wrote:
>
> Yes, uniqueness of backend names affect only CLI and VSM.
>
> The former only if people get confused, the latter always because
> the VSM counter segment is named based on the backend name, so the
> two will clash.
Something else I thought of since yesterday -- the backend names used
in VSL for entries like Backend, BackendOpen, BackendClose and so forth.
If you're using the log to diagnose a problem, doesn't help much to
have to wonder which backend it's talking about.
> My current thinking is that we'll name the backend whatever the
> user/vcl/vmod writer likes (ie: Backend name), and deal with the
> fall-out.
At the very least, VMODs should have a way to discover if a backend
name is already in use by a VCL -- the VCL_HasBackend(vcl,name) idea.
Then a VMOD can choose to raise an error for duplicate names.
Maybe add a sentence or two to the "Writing a Director" doc about
backend names, what they're used for, and why this could be a problem.
> That really only leaves one question: What do we do when some
> code tries to add a backend with a name which already exists.
>
> One option is to fail the backend creation, but since it is only
> CLI/VSM that can get confused, that seems too draconian to me.
>
> So I'm tempted to simply add a ".%d" suffix for ever increasing
> %d's until the name becomes unique.
>
> Would that work for people ?
If you thought that you called your backend "be", and then see "be.1"
and "be.2" in backend.list, VSL, VBE.* stats and so forth, is it
really any less confusing? The order would be probably result from
which backend was created first, which may be difficult or impossible
to determine after the fact.
That would have the advantage that one set of VSC stats doesn't get
lost altogether.
So I guess the choices are: be draconian, or permit a situation that
will probably create confusion, however much we try to minimize it.
Best,
Geoff
- --
** * * UPLEX - Nils Goroll Systemoptimierung
Scheffelstra?e 32
22301 Hamburg
Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753
http://uplex.de
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1
iQIcBAEBCAAGBQJWL5HaAAoJEOUwvh9pJNUR1OsQALGdQ5Dn/kDeBZkGymKX31z6
4/Z60CHK1lPMPsHJKumxLGp7D2deN00XmygecYshUBjjM26rnSg1f1JhFlwIMd9t
Nb678+Zi3HXoOPJ+5a1KLjMSHfpqb95xZ9S0GR7B8eAHj4fZqKJWTPk5L0CRdeI3
ISxls4/OAmrHqccH/7RILbcwX/vQo/Za7HnPkgiABq9sts18HoLZik4Wrys7jjrR
XtnKKI0TXchzxvOH+uzpSE+zVZWfGTWalK8RtyQw/+8nV7gWvtlkSmlYDJ6IMrGR
buwf0GJPHGD8OSk2MHyBg3Z3JO3EuqiPK/sGGDPQZcCcjw9bSSwQlD0z+r9CjVcN
FzjeH/3iRhFIYMsDoplxyPj1h9FcjyjPGTlJutnvlDsny1Mes8I0Rm2B8ilJL5GT
wDHgduDsjh/IEi3sIes5Baj5rKPxTO1aOHg6iALIuZiExmcZ6f5XD54e1I2MYszW
ypGWDDZcnwuvBVzplwQo/swUnKBu+I9gTVKkUqq5bQfZMUrCZ/wE/7SfqXJqBZfW
JI6k28mxL8WZyO0HH5iWhfDr3QxYrumBYZPIAHEzHEkXSo5JfH0e4sMF31mfVJ21
2WzcdGZfeAilAzGJHALYGX9/HK8Hf9Bimw808V3vrdyZNBbpE+XyX8L52Qlur4pL
sGt66SERJ3V0J9ULMZbO
=6WOa
-----END PGP SIGNATURE-----
------------------------------
Message: 4
Date: Tue, 27 Oct 2015 17:14:28 +0100
From: Lasse Karstensen <lkarsten at varnish-software.com>
To: Poul-Henning Kamp <phk at phk.freebsd.dk>
Cc: varnish-dev at varnish-cache.org
Subject: Re: Varnish project autumn cleaning
Message-ID: <20151027161427.GA1274 at immer.varnish-software.com>
Content-Type: text/plain; charset=us-ascii
> On Tue, Oct 27, 2015 at 02:48:28PM +0000, Poul-Henning Kamp wrote:
> [cut]
> I'm currently disinclined to have a Wiki in the future, it seems
> to never really have worked for us. If we later find out we need
> one, github or Wikia are obvious solutions.
The wiki doesn't work because we've turned off user registrations
to combat spammers.
I strongly believe we should revive it when we have the chance. (==github move)
It is a natural place to keep information that is too specific to go
into the main documentation, or too small of a detail to warrant a
documentation commit.
Just making a (living) FAQ that we can use in #varnish for the normal
questions would be a big improvement. I'm happy to take point on this,
if that is what it is needed.
--
Lasse Karstensen
------------------------------
Message: 5
Date: Tue, 27 Oct 2015 18:24:35 +0100
From: Kacper Wysocki <kacperw at gmail.com>
To: Lasse Karstensen <lkarsten at varnish-software.com>
Cc: Varnish Development <varnish-dev at varnish-cache.org>
Subject: Re: Varnish project autumn cleaning
Message-ID:
<CABaBnj7hB9ueF-wdPSUe9nQnN6S4oQmoSEV=BwzQY6v+5QZ1Bg at mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
On Tue, Oct 27, 2015 at 5:14 PM, Lasse Karstensen
<lkarsten at varnish-software.com> wrote:
> On Tue, Oct 27, 2015 at 02:48:28PM +0000, Poul-Henning Kamp wrote:
> [cut]
>> I'm currently disinclined to have a Wiki in the future, it seems
>> to never really have worked for us. If we later find out we need
>> one, github or Wikia are obvious solutions.
>
> The wiki doesn't work because we've turned off user registrations
> to combat spammers.
>
> I strongly believe we should revive it when we have the chance. (==github move)
>
> It is a natural place to keep information that is too specific to go
> into the main documentation, or too small of a detail to warrant a
> documentation commit.
>
> Just making a (living) FAQ that we can use in #varnish for the normal
> questions would be a big improvement. I'm happy to take point on this,
> if that is what it is needed.
So good to hear that revamping the site will be a priority going forward.
I'm happy to help in any way that I can.
I agree with Lasse that the wiki has an important place as a living
document, and that a whole lot of VCL snippets and varnish tricks are
ending up on Other Peoples Blogs? because the trac wiki has been
broken (by spammers).
It may be useful in the future to give someone the wiki-admin-hat so
they can mark out-of-date content, avoiding the old issue of vcl
snippets for Varnish 2.0 still floating about.
That being said, getting a nicer ticketing system would be super sweet
and the old ticket system is definitely the first thing to deserve a
bullet.
cheers,
0K
------------------------------
Message: 6
Date: Tue, 27 Oct 2015 17:37:58 +0000
From: Federico Schwindt <fgsch at lodoss.net>
To: Kacper Wysocki <kacperw at gmail.com>
Cc: Varnish Development <varnish-dev at varnish-cache.org>
Subject: Re: Varnish project autumn cleaning
Message-ID:
<CAJV_h0bg_tqEVyaCX3Gi0NgRt7cFSRKu1c4ccCP_K4FiObn4ig at mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I agree with Lasse as well, we should keep the wiki.
It might be worth exploring
https://help.github.com/articles/user-organization-and-project-pages/ for
the website and/or wiki.
I'm happy to co-admin/root any server if required but IMO if we can avoid
any maintenance by using the above it'd be better.
I'm also happy contributing towards some VPS solution, Digital Ocean being
my preferred at the moment.
One question that comes to my mind is what is going to happen with
repo.varnish-cache.org after we move out of VS.
Are we going to continue providing packages?
> On Tue, Oct 27, 2015 at 5:24 PM, Kacper Wysocki <kacperw at gmail.com> wrote:
>
> On Tue, Oct 27, 2015 at 5:14 PM, Lasse Karstensen
> <lkarsten at varnish-software.com> wrote:
>> On Tue, Oct 27, 2015 at 02:48:28PM +0000, Poul-Henning Kamp wrote:
>> [cut]
>>> I'm currently disinclined to have a Wiki in the future, it seems
>>> to never really have worked for us. If we later find out we need
>>> one, github or Wikia are obvious solutions.
>>
>> The wiki doesn't work because we've turned off user registrations
>> to combat spammers.
>>
>> I strongly believe we should revive it when we have the chance.
> (==github move)
>>
>> It is a natural place to keep information that is too specific to go
>> into the main documentation, or too small of a detail to warrant a
>> documentation commit.
>>
>> Just making a (living) FAQ that we can use in #varnish for the normal
>> questions would be a big improvement. I'm happy to take point on this,
>> if that is what it is needed.
>
> So good to hear that revamping the site will be a priority going forward.
> I'm happy to help in any way that I can.
>
> I agree with Lasse that the wiki has an important place as a living
> document, and that a whole lot of VCL snippets and varnish tricks are
> ending up on Other Peoples Blogs? because the trac wiki has been
> broken (by spammers).
>
> It may be useful in the future to give someone the wiki-admin-hat so
> they can mark out-of-date content, avoiding the old issue of vcl
> snippets for Varnish 2.0 still floating about.
>
> That being said, getting a nicer ticketing system would be super sweet
> and the old ticket system is definitely the first thing to deserve a
> bullet.
>
> cheers,
> 0K
>
> _______________________________________________
> varnish-dev mailing list
> varnish-dev at varnish-cache.org
> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20151027/a990a496/attachment.html>
------------------------------
_______________________________________________
varnish-dev mailing list
varnish-dev at varnish-cache.org
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
End of varnish-dev Digest, Vol 115, Issue 10
********************************************
More information about the varnish-dev
mailing list