Gracefully stopping varnish

[Hugues Alary]

Your solution works, thanks much.

Cheers,
-Hugues

On Wed, Oct 18, 2017 at 12:21 PM, Hugues Alary <hugues at betabrand.com> wrote:

> > So, there must be a misconception on my, because to me, just refusing
> new connections isn't graceful. If the LB is sending you new connections,
> you should honor them.
>
> You are right. I took some shortcuts in my explanation.
>
> > So, from what I get, k8s stops sending new connections to Varnish, and
> what you want is is for Varnish to shutdown once there's no more active
> connections.
>
> That is exactly what happens, indeed. How it happens is irrelevant to our
> conversation, but, for the sake of being on the same page it's important to
> assume: k8s stopped sending new connections to the old instance and started
> sending them to the new instance, and I want old varnish to shutdown once
> there's no more active connections.
>
> > But, let's stop being a pompous jerk obsessed with semantics and let's
> try to work on a solution.
>
> All good, it's actually important ;)
>
> I will try your solution right away and let you know, in theory it seems
> like it should work. Thanks for trying to find a solution!
>
> -Hugues
>
>
> On Wed, Oct 18, 2017 at 12:01 PM, Guillaume Quintard <
> guillaume at varnish-software.com> wrote:
>
>> Hello,
>>
>> So, there must be a misconception on my, because to me, just refusing new
>> connections isn't graceful. If the LB is sending you new connections, you
>> should honor them.
>>
>> So, from what I get, k8s stops sending new connections to Varnish, and
>> what you want is is for Varnish to shutdown once there's no more active
>> connections, which is, I would argue, kinda different from refusing new
>> connections. But, let's stop being a pompous jerk obsessed with semantics
>> and let's try to work on a solution.
>>
>> From what I get, we can kill varnish as soon as the number of active
>> connections drops to 0, so your apachectl command would be equal to this in
>> Varnish language:
>>
>> if [ `varnishstat -1 | awk '/MEMPOOL.sess[0-9]+.live/ {a+=$2} END {print
>> a}'` != 0 ]; then
>>     sleep 1
>> fi
>> killall varnishd
>>
>>
>> There's probably a shorter version ,but that's the gist of it.
>>
>> Would that do?
>>
>> --
>> Guillaume Quintard
>>
>> On Wed, Oct 18, 2017 at 8:22 PM, Hugues Alary <hugues at betabrand.com>
>> wrote:
>>
>>> Hi,
>>>
>>> That is indeed what I want to do. Draining connections at the LB level
>>> just involves more work ;).
>>>
>>> For context: my entire stack runs on a kubernetes cluster.
>>>
>>> I sometime need to replace a running instance of a server process [be it
>>> nginx apache php-fpm varnish whatever] with a new instance.
>>>
>>> Taking Apache as an example, I simply create the new apache instance (a
>>> new pod in kubernetes speak), which immediately starts getting some traffic
>>> (without changing the load balancer configuration at all, a kubernetes
>>> "service" automatically detects new pods), then I gracefully shutdown the
>>> old instance (kubernetes actually automatically tells the pod to shutdown),
>>> by issuing a "apachectl -k graceful-stop" (kubernetes is configured to
>>> issue this command for me), which instructs apache to stop accepting
>>> connections, finish, then shutdown.
>>>
>>> It's really great because instead of having to push a new config
>>> refusing probes and reload it, I (/kubernetes) simply gracefully stops
>>> apache and the traffic flows to the new instance. nginx and php-fpm also
>>> handle things this way.
>>>
>>> At any rate, thanks for the advice, I will start using probes!
>>>
>>> Cheers,
>>> -Hugues
>>>
>>>
>>>
>>> On Tue, Oct 17, 2017 at 11:51 PM, Guillaume Quintard <
>>> guillaume at varnish-software.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> That's not possible. However, what you really want, I think, is not
>>>> sending new requests to Varnish. That's usually done at the loa-bbalancing
>>>> level. If your LB use probes, you can tell Varnish to stop honoring them,
>>>> drain the connections,  then kill it.
>>>>
>>>> --
>>>> Guillaume Quintard
>>>>
>>>> On Oct 18, 2017 02:28, "Hugues Alary" <hugues at betabrand.com> wrote:
>>>>
>>>>> Hi there,
>>>>>
>>>>> I've been looking around and I can't find a documented way of
>>>>> gracefully shutting down varnishd, and by gracefully I mean tell varnish
>>>>> "stop accepting connections, but finish what you were doing, then shutdown".
>>>>>
>>>>> I did find something in the "first varnish design notes" (
>>>>> https://varnish-cache.org/docs/5.1/phk/firstdesign.html) which seemed
>>>>> to indicate that sending SIGKILL/SIGTERM would mean "suspend/stop" but KILL
>>>>> doesn't seem to work, and TERM, well... terminates but not gracefully.
>>>>>
>>>>> I also tried using "varnishadm stop", which also doesn't gracefully
>>>>> stops connection.
>>>>>
>>>>> Is there anyway to achieve this?
>>>>>
>>>>> Thanks!
>>>>> -Hugues
>>>>>
>>>>> _______________________________________________
>>>>> varnish-misc mailing list
>>>>> varnish-misc at varnish-cache.org
>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20171018/ac675e8c/attachment.html>