Gracefully stopping varnish

[Guillaume Quintard]

Glad to hear it! Out of curiosity, do you run that script manually on
container you want to kill, or is there a way to do it automatically?

-- 
Guillaume Quintard

On Wed, Oct 18, 2017 at 9:47 PM, Hugues Alary <hugues at betabrand.com> wrote:

> Your solution works, thanks much.
>
> Cheers,
> -Hugues
>
> On Wed, Oct 18, 2017 at 12:21 PM, Hugues Alary <hugues at betabrand.com>
> wrote:
>
>> > So, there must be a misconception on my, because to me, just refusing
>> new connections isn't graceful. If the LB is sending you new connections,
>> you should honor them.
>>
>> You are right. I took some shortcuts in my explanation.
>>
>> > So, from what I get, k8s stops sending new connections to Varnish, and
>> what you want is is for Varnish to shutdown once there's no more active
>> connections.
>>
>> That is exactly what happens, indeed. How it happens is irrelevant to our
>> conversation, but, for the sake of being on the same page it's important to
>> assume: k8s stopped sending new connections to the old instance and started
>> sending them to the new instance, and I want old varnish to shutdown once
>> there's no more active connections.
>>
>> > But, let's stop being a pompous jerk obsessed with semantics and let's
>> try to work on a solution.
>>
>> All good, it's actually important ;)
>>
>> I will try your solution right away and let you know, in theory it seems
>> like it should work. Thanks for trying to find a solution!
>>
>> -Hugues
>>
>>
>> On Wed, Oct 18, 2017 at 12:01 PM, Guillaume Quintard <
>> guillaume at varnish-software.com> wrote:
>>
>>> Hello,
>>>
>>> So, there must be a misconception on my, because to me, just refusing
>>> new connections isn't graceful. If the LB is sending you new connections,
>>> you should honor them.
>>>
>>> So, from what I get, k8s stops sending new connections to Varnish, and
>>> what you want is is for Varnish to shutdown once there's no more active
>>> connections, which is, I would argue, kinda different from refusing new
>>> connections. But, let's stop being a pompous jerk obsessed with semantics
>>> and let's try to work on a solution.
>>>
>>> From what I get, we can kill varnish as soon as the number of active
>>> connections drops to 0, so your apachectl command would be equal to this in
>>> Varnish language:
>>>
>>> if [ `varnishstat -1 | awk '/MEMPOOL.sess[0-9]+.live/ {a+=$2} END {print
>>> a}'` != 0 ]; then
>>>     sleep 1
>>> fi
>>> killall varnishd
>>>
>>>
>>> There's probably a shorter version ,but that's the gist of it.
>>>
>>> Would that do?
>>>
>>> --
>>> Guillaume Quintard
>>>
>>> On Wed, Oct 18, 2017 at 8:22 PM, Hugues Alary <hugues at betabrand.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> That is indeed what I want to do. Draining connections at the LB level
>>>> just involves more work ;).
>>>>
>>>> For context: my entire stack runs on a kubernetes cluster.
>>>>
>>>> I sometime need to replace a running instance of a server process [be
>>>> it nginx apache php-fpm varnish whatever] with a new instance.
>>>>
>>>> Taking Apache as an example, I simply create the new apache instance (a
>>>> new pod in kubernetes speak), which immediately starts getting some traffic
>>>> (without changing the load balancer configuration at all, a kubernetes
>>>> "service" automatically detects new pods), then I gracefully shutdown the
>>>> old instance (kubernetes actually automatically tells the pod to shutdown),
>>>> by issuing a "apachectl -k graceful-stop" (kubernetes is configured to
>>>> issue this command for me), which instructs apache to stop accepting
>>>> connections, finish, then shutdown.
>>>>
>>>> It's really great because instead of having to push a new config
>>>> refusing probes and reload it, I (/kubernetes) simply gracefully stops
>>>> apache and the traffic flows to the new instance. nginx and php-fpm also
>>>> handle things this way.
>>>>
>>>> At any rate, thanks for the advice, I will start using probes!
>>>>
>>>> Cheers,
>>>> -Hugues
>>>>
>>>>
>>>>
>>>> On Tue, Oct 17, 2017 at 11:51 PM, Guillaume Quintard <
>>>> guillaume at varnish-software.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> That's not possible. However, what you really want, I think, is not
>>>>> sending new requests to Varnish. That's usually done at the loa-bbalancing
>>>>> level. If your LB use probes, you can tell Varnish to stop honoring them,
>>>>> drain the connections,  then kill it.
>>>>>
>>>>> --
>>>>> Guillaume Quintard
>>>>>
>>>>> On Oct 18, 2017 02:28, "Hugues Alary" <hugues at betabrand.com> wrote:
>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> I've been looking around and I can't find a documented way of
>>>>>> gracefully shutting down varnishd, and by gracefully I mean tell varnish
>>>>>> "stop accepting connections, but finish what you were doing, then shutdown".
>>>>>>
>>>>>> I did find something in the "first varnish design notes" (
>>>>>> https://varnish-cache.org/docs/5.1/phk/firstdesign.html) which
>>>>>> seemed to indicate that sending SIGKILL/SIGTERM would mean "suspend/stop"
>>>>>> but KILL doesn't seem to work, and TERM, well... terminates but not
>>>>>> gracefully.
>>>>>>
>>>>>> I also tried using "varnishadm stop", which also doesn't gracefully
>>>>>> stops connection.
>>>>>>
>>>>>> Is there anyway to achieve this?
>>>>>>
>>>>>> Thanks!
>>>>>> -Hugues
>>>>>>
>>>>>> _______________________________________________
>>>>>> varnish-misc mailing list
>>>>>> varnish-misc at varnish-cache.org
>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-misc/attachments/20171018/c0f1ef32/attachment-0001.html>