Support for AARCH64

Tue Mar 24 09:00:33 UTC 2020

Hi Guillaume,

On Mon, Mar 23, 2020 at 8:01 PM Guillaume Quintard <
guillaume at varnish-software.com> wrote:

> Hi Martin,
>
> Thank you for that.
> A few remarks and questions:
> - how much time does the "docker build" step takes? We can possibly speed
> things up by push images to the dockerhub, as they don't need to change
> very often.
>

Definitely such optimization would be a good thing to do!
At the moment, with 'machine' executor it fetches the base image and then
builds all the Docker layers again and again.
Here are the timings:
1) Spinning up a VM - around 10secs
2) prepare env variables - 0secs
3) checkout code (varnish-cache) - 5secs
4) activate QEMU - 2secs
5) build packages
5.1) x86 deb - 3m 30secs
5.2) x86 rpm - 2m 50secs
5.3) aarch64 rpm - 35mins
5.4) aarch64 deb - 45mins

> - any reason why you clone pkg-varnish-cache in each job? The idea was to
> have it cloned once in tar-pkg-tools for consistency and reproducibility,
> which we lose here.
>

I will extract the common steps once I see it working. This is my first
CircleCI project and I still find my ways in it!

> - do we want to change things for the amd64 platforms for the sake of
> consistency?
>

So far there is nothing specific for amd4 or aarch64, except the base
Docker images.
For example make-deb-packages.sh is reused for both amd64 and aarch64
builds. Same for -rpm- and now for -apk- (alpine).

Once I feel the change is almost finished I will open a Pull Request for
more comments!

Martin

>
> --
> Guillaume Quintard
>
>
> On Mon, Mar 23, 2020 at 6:25 AM Martin Grigorov <martin.grigorov at gmail.com>
> wrote:
>
>> Hi,
>>
>>
>> On Wed, Mar 18, 2020 at 5:31 PM Martin Grigorov <
>> martin.grigorov at gmail.com> wrote:
>>
>>> Hi,
>>>
>>> On Thu, Mar 12, 2020 at 4:35 PM Martin Grigorov <
>>> martin.grigorov at gmail.com> wrote:
>>>
>>>> Hi Guillaume,
>>>>
>>>> On Thu, Mar 12, 2020 at 3:23 PM Guillaume Quintard <
>>>> guillaume at varnish-software.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Offering arm64 packages requires a few things:
>>>>> - arm64-compatible code (all good in
>>>>> https://github.com/varnishcache/varnish-cache)
>>>>> - arm64-compatible package framework (all good in
>>>>> https://github.com/varnishcache/pkg-varnish-cache)
>>>>> - infrastructure to build the packages (uhoh, see below)
>>>>> - infrastructure to store and deliver (
>>>>> https://packagecloud.io/varnishcache)
>>>>>
>>>>> So, everything is in place, expect for the third point. At the moment,
>>>>> there are two concurrent CI implementations:
>>>>> - travis:
>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.travis.yml It's
>>>>> the historical one, and currently only runs compilation+test for OSX
>>>>>
>>>>
>>>> Actually it tests Linux AMD64 and ARM64 too.
>>>>
>>>>
>>>>> - circleci:
>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.circleci/config.yml the
>>>>> new kid on the block, that builds all the packages and distchecks for all
>>>>> the packaged platforms
>>>>>
>>>>> The issue is that cirecleci doesn't support arm64 containers (for
>>>>> now?), so we would need to re-implement the packaging logic in Travis. It's
>>>>> not a big problem, but it's currently not a priority on my side.
>>>>>
>>>>> However, I am totally ready to provide help if someone wants to take
>>>>> that up. The added benefit it that Travis would be able to handle
>>>>> everything and we can retire the circleci experiment
>>>>>
>>>>
>>>> I will take a look in the coming days and ask you if I need help!
>>>>
>>>
>>> I've took a look at the current setup and here is what I've found as
>>> problems and possible solutions:
>>>
>>> 1) Circle CI
>>> 1.1) problem - the 'machine' and 'Docker' executors run on x86_64, so
>>> there is no way to build the packages in a "native" environment
>>> 1.2) possible solutions
>>> 1.2.1) use multiarch cross build
>>> 1.2.2) use 'machine' executor that registers QEMU via
>>> https://hub.docker.com/r/multiarch/qemu-user-static/ and then builds
>>> and runs a custom Docker image that executes a shell script with the build
>>> steps
>>> It will look something like
>>> https://github.com/yukimochi-containers/alpine-vpnserver/blob/69bb0a612c9df3e4ba78064d114751b760f0df9d/.circleci/config.yml#L19-L38 but
>>> instead of uploading the Docker image as a last step it will run it.
>>> The RPM and DEB build related code from current config.yml will be
>>> extracted into shell scripts which will be copied in the custom Docker
>>> images
>>>
>>> From these two possible ways I have better picture in my head how to do
>>> 1.2.2, but I don't mind going deep in 1.2.1 if this is what you'd prefer.
>>>
>>
>> I've decided to stay with Circle CI and use 'machine' executor with QEMU.
>>
>> The changed config.yml could be seen at
>> https://github.com/martin-g/varnish-cache/tree/feature/aarch64-packages/.circleci and
>> the build at
>> https://app.circleci.com/pipelines/github/martin-g/varnish-cache/71/workflows/3a275d79-62a9-48b4-9aef-1585de1c87c8
>> The builds on x86 arch take 3-4 mins, but for aarch64 (emulation!) ~40mins
>> For now the jobs just build the .deb & .rpm packages for CentOS 7 and
>> Ubuntu 18.04, both amd64 and aarch64.
>> TODOs:
>> - migrate Alpine
>> - store the packages as CircleCI artifacts
>> - anything else that is still missing
>>
>> Adding more architectures would be as easy as adding a new Dockerfile
>> with a base image from the respective type.
>>
>> Martin
>>
>>
>>> 2) Travis CI
>>> 2.1) problems
>>> 2.1.1) generally Travis is slower than Circle!
>>> Althought if we use CircleCI 'machine' executor it will be slower than
>>> the current 'Docker' executor!
>>> 2.1.2) Travis supports only Ubuntu
>>> Current setup at CircleCI uses CentOS 7.
>>> I guess the build steps won't have problems on Ubuntu.
>>>
>>> 3) GitHub Actions
>>> GH Actions does not support ARM64 but it supports self hosted ARM64
>>> runners
>>> 3.1) The problem is that there is no way to make a self hosted runner
>>> really private. I.e. if someone forks Varnish Cache any commit in the fork
>>> will trigger builds on the arm64 node. There is no way to reserve the
>>> runner only for commits against
>>> https://github.com/varnishcache/varnish-cache
>>>
>>> Do you see other problems or maybe different ways ?
>>> Do you have preferences which way to go ?
>>>
>>> Regards,
>>> Martin
>>>
>>>
>>>>
>>>> Regards,
>>>> Martin
>>>>
>>>>
>>>>>
>>>>> --
>>>>> Guillaume Quintard
>>>>> _______________________________________________
>>>>> varnish-dev mailing list
>>>>> varnish-dev at varnish-cache.org
>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>>>>>
>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20200324/97555bad/attachment.html>