Gracefully stopping varnish

[Hugues Alary]

> So, there must be a misconception on my, because to me, just refusing new
connections isn't graceful. If the LB is sending you new connections, you
should honor them.

You are right. I took some shortcuts in my explanation.

> So, from what I get, k8s stops sending new connections to Varnish, and
what you want is is for Varnish to shutdown once there's no more active
connections.

That is exactly what happens, indeed. How it happens is irrelevant to our
conversation, but, for the sake of being on the same page it's important to
assume: k8s stopped sending new connections to the old instance and started
sending them to the new instance, and I want old varnish to shutdown once
there's no more active connections.

> But, let's stop being a pompous jerk obsessed with semantics and let's
try to work on a solution.

All good, it's actually important ;)

I will try your solution right away and let you know, in theory it seems
like it should work. Thanks for trying to find a solution!

-Hugues


On Wed, Oct 18, 2017 at 12:01 PM, Guillaume Quintard <
guillaume at varnish-software.com> wrote:

> Hello,
>
> So, there must be a misconception on my, because to me, just refusing new
> connections isn't graceful. If the LB is sending you new connections, you
> should honor them.
>
> So, from what I get, k8s stops sending new connections to Varnish, and
> what you want is is for Varnish to shutdown once there's no more active
> connections, which is, I would argue, kinda different from refusing new
> connections. But, let's stop being a pompous jerk obsessed with semantics
> and let's try to work on a solution.
>
> From what I get, we can kill varnish as soon as the number of active
> connections drops to 0, so your apachectl command would be equal to this in
> Varnish language:
>
> if [ `varnishstat -1 | awk '/MEMPOOL.sess[0-9]+.live/ {a+=$2} END {print
> a}'` != 0 ]; then
>     sleep 1
> fi
> killall varnishd
>
>
> There's probably a shorter version ,but that's the gist of it.
>
> Would that do?
>
> --
> Guillaume Quintard
>
> On Wed, Oct 18, 2017 at 8:22 PM, Hugues Alary <hugues at betabrand.com>
> wrote:
>
>> Hi,
>>
>> That is indeed what I want to do. Draining connections at the LB level
>> just involves more work ;).
>>
>> For context: my entire stack runs on a kubernetes cluster.
>>
>> I sometime need to replace a running instance of a server process [be it
>> nginx apache php-fpm varnish whatever] with a new instance.
>>
>> Taking Apache as an example, I simply create the new apache instance (a
>> new pod in kubernetes speak), which immediately starts getting some traffic
>> (without changing the load balancer configuration at all, a kubernetes
>> "service" automatically detects new pods), then I gracefully shutdown the
>> old instance (kubernetes actually automatically tells the pod to shutdown),
>> by issuing a "apachectl -k graceful-stop" (kubernetes is configured to
>> issue this command for me), which instructs apache to stop accepting
>> connections, finish, then shutdown.
>>
>> It's really great because instead of having to push a new config refusing
>> probes and reload it, I (/kubernetes) simply gracefully stops apache and
>> the traffic flows to the new instance. nginx and php-fpm also handle things
>> this way.
>>
>> At any rate, thanks for the advice, I will start using probes!
>>
>> Cheers,
>> -Hugues
>>
>>
>>
>> On Tue, Oct 17, 2017 at 11:51 PM, Guillaume Quintard <
>> guillaume at varnish-software.com> wrote:
>>
>>> Hi,
>>>
>>> That's not possible. However, what you really want, I think, is not
>>> sending new requests to Varnish. That's usually done at the loa-bbalancing
>>> level. If your LB use probes, you can tell Varnish to stop honoring them,
>>> drain the connections,  then kill it.
>>>
>>> --
>>> Guillaume Quintard
>>>
>>> On Oct 18, 2017 02:28, "Hugues Alary" <hugues at betabrand.com> wrote:
>>>
>>>> Hi there,
>>>>
>>>> I've been looking around and I can't find a documented way of
>>>> gracefully shutting down varnishd, and by gracefully I mean tell varnish
>>>> "stop accepting connections, but finish what you were doing, then shutdown".
>>>>
>>>> I did find something in the "first varnish design notes" (
>>>> https://varnish-cache.org/docs/5.1/phk/firstdesign.html) which seemed
>>>> to indicate that sending SIGKILL/SIGTERM would mean "suspend/stop" but KILL
>>>> doesn't seem to work, and TERM, well... terminates but not gracefully.
>>>>
>>>> I also tried using "varnishadm stop", which also doesn't gracefully
>>>> stops connection.
>>>>
>>>> Is there anyway to achieve this?
>>>>
>>>> Thanks!
>>>> -Hugues
>>>>
>>>> _______________________________________________
>>>> varnish-misc mailing list
>>>> varnish-misc at varnish-cache.org
>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20171018/e2d91caa/attachment-0001.html>