Support for AARCH64

Wed Mar 25 00:39:06 UTC 2020

Hi,

So, you are pointing at the `dist` job, whose sole role is to provide us
with a dist tarball, so we don't need that command line to work for
everyone, just for that specific platform.

On the other hand,
https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/.circleci/config.yml#L168
is
closer to what you want, `distcheck` will be call on all platform, and you
can see that it has the `--with-unwind` argument.
-- 
Guillaume Quintard

On Tue, Mar 24, 2020 at 3:05 PM Martin Grigorov <martin.grigorov at gmail.com>
wrote:

>
>
> On Tue, Mar 24, 2020, 17:19 Guillaume Quintard <
> guillaume at varnish-software.com> wrote:
>
>> Compare your configure line with what's currently in use (or the apkbuild
>> file), there are a few options (with-unwind, without-jemalloc, etc.) That
>> need to be set
>>
>
> The configure line comes from "./autogen.des":
> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/autogen.des#L35-L42
> It is called at:
>
> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/.circleci/config.yml#L40
> In my branch at:
>
> https://github.com/martin-g/varnish-cache/blob/4b4626ee9cc366b032a45f27b54d77176125ef03/.circleci/make-apk-packages.sh#L26
>
> It fails only on aarch64 for Alpine Linux. The x86_64 build for Alpine is
> fine.
> AARCH64 for CentOS 7 and Ubuntu 18.04 are also fine.
>
> Martin
>
>
>> On Tue, Mar 24, 2020, 08:05 Martin Grigorov <martin.grigorov at gmail.com>
>> wrote:
>>
>>> Hi,
>>>
>>> On Tue, Mar 24, 2020 at 11:00 AM Martin Grigorov <
>>> martin.grigorov at gmail.com> wrote:
>>>
>>>> Hi Guillaume,
>>>>
>>>> On Mon, Mar 23, 2020 at 8:01 PM Guillaume Quintard <
>>>> guillaume at varnish-software.com> wrote:
>>>>
>>>>> Hi Martin,
>>>>>
>>>>> Thank you for that.
>>>>> A few remarks and questions:
>>>>> - how much time does the "docker build" step takes? We can possibly
>>>>> speed things up by push images to the dockerhub, as they don't need to
>>>>> change very often.
>>>>>
>>>>
>>>> Definitely such optimization would be a good thing to do!
>>>> At the moment, with 'machine' executor it fetches the base image and
>>>> then builds all the Docker layers again and again.
>>>> Here are the timings:
>>>> 1) Spinning up a VM - around 10secs
>>>> 2) prepare env variables - 0secs
>>>> 3) checkout code (varnish-cache) - 5secs
>>>> 4) activate QEMU - 2secs
>>>> 5) build packages
>>>> 5.1) x86 deb - 3m 30secs
>>>> 5.2) x86 rpm - 2m 50secs
>>>> 5.3) aarch64 rpm - 35mins
>>>> 5.4) aarch64 deb - 45mins
>>>>
>>>>
>>>>> - any reason why you clone pkg-varnish-cache in each job? The idea was
>>>>> to have it cloned once in tar-pkg-tools for consistency and
>>>>> reproducibility, which we lose here.
>>>>>
>>>>
>>>> I will extract the common steps once I see it working. This is my first
>>>> CircleCI project and I still find my ways in it!
>>>>
>>>>
>>>>> - do we want to change things for the amd64 platforms for the sake of
>>>>> consistency?
>>>>>
>>>>
>>>> So far there is nothing specific for amd4 or aarch64, except the base
>>>> Docker images.
>>>> For example make-deb-packages.sh is reused for both amd64 and aarch64
>>>> builds. Same for -rpm- and now for -apk- (alpine).
>>>>
>>>> Once I feel the change is almost finished I will open a Pull Request
>>>> for more comments!
>>>>
>>>> Martin
>>>>
>>>>
>>>>>
>>>>> --
>>>>> Guillaume Quintard
>>>>>
>>>>>
>>>>> On Mon, Mar 23, 2020 at 6:25 AM Martin Grigorov <
>>>>> martin.grigorov at gmail.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 18, 2020 at 5:31 PM Martin Grigorov <
>>>>>> martin.grigorov at gmail.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> On Thu, Mar 12, 2020 at 4:35 PM Martin Grigorov <
>>>>>>> martin.grigorov at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Guillaume,
>>>>>>>>
>>>>>>>> On Thu, Mar 12, 2020 at 3:23 PM Guillaume Quintard <
>>>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> Offering arm64 packages requires a few things:
>>>>>>>>> - arm64-compatible code (all good in
>>>>>>>>> https://github.com/varnishcache/varnish-cache)
>>>>>>>>> - arm64-compatible package framework (all good in
>>>>>>>>> https://github.com/varnishcache/pkg-varnish-cache)
>>>>>>>>> - infrastructure to build the packages (uhoh, see below)
>>>>>>>>> - infrastructure to store and deliver (
>>>>>>>>> https://packagecloud.io/varnishcache)
>>>>>>>>>
>>>>>>>>> So, everything is in place, expect for the third point. At the
>>>>>>>>> moment, there are two concurrent CI implementations:
>>>>>>>>> - travis:
>>>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.travis.yml It's
>>>>>>>>> the historical one, and currently only runs compilation+test for OSX
>>>>>>>>>
>>>>>>>>
>>>>>>>> Actually it tests Linux AMD64 and ARM64 too.
>>>>>>>>
>>>>>>>>
>>>>>>>>> - circleci:
>>>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.circleci/config.yml the
>>>>>>>>> new kid on the block, that builds all the packages and distchecks for all
>>>>>>>>> the packaged platforms
>>>>>>>>>
>>>>>>>>> The issue is that cirecleci doesn't support arm64 containers (for
>>>>>>>>> now?), so we would need to re-implement the packaging logic in Travis. It's
>>>>>>>>> not a big problem, but it's currently not a priority on my side.
>>>>>>>>>
>>>>>>>>> However, I am totally ready to provide help if someone wants to
>>>>>>>>> take that up. The added benefit it that Travis would be able to handle
>>>>>>>>> everything and we can retire the circleci experiment
>>>>>>>>>
>>>>>>>>
>>>>>>>> I will take a look in the coming days and ask you if I need help!
>>>>>>>>
>>>>>>>
>>>>>>> I've took a look at the current setup and here is what I've found as
>>>>>>> problems and possible solutions:
>>>>>>>
>>>>>>> 1) Circle CI
>>>>>>> 1.1) problem - the 'machine' and 'Docker' executors run on x86_64,
>>>>>>> so there is no way to build the packages in a "native" environment
>>>>>>> 1.2) possible solutions
>>>>>>> 1.2.1) use multiarch cross build
>>>>>>> 1.2.2) use 'machine' executor that registers QEMU via
>>>>>>> https://hub.docker.com/r/multiarch/qemu-user-static/ and then
>>>>>>> builds and runs a custom Docker image that executes a shell script with the
>>>>>>> build steps
>>>>>>> It will look something like
>>>>>>> https://github.com/yukimochi-containers/alpine-vpnserver/blob/69bb0a612c9df3e4ba78064d114751b760f0df9d/.circleci/config.yml#L19-L38 but
>>>>>>> instead of uploading the Docker image as a last step it will run it.
>>>>>>> The RPM and DEB build related code from current config.yml will be
>>>>>>> extracted into shell scripts which will be copied in the custom Docker
>>>>>>> images
>>>>>>>
>>>>>>> From these two possible ways I have better picture in my head how to
>>>>>>> do 1.2.2, but I don't mind going deep in 1.2.1 if this is what you'd prefer.
>>>>>>>
>>>>>>
>>>>>> I've decided to stay with Circle CI and use 'machine' executor with
>>>>>> QEMU.
>>>>>>
>>>>>> The changed config.yml could be seen at
>>>>>> https://github.com/martin-g/varnish-cache/tree/feature/aarch64-packages/.circleci and
>>>>>> the build at
>>>>>> https://app.circleci.com/pipelines/github/martin-g/varnish-cache/71/workflows/3a275d79-62a9-48b4-9aef-1585de1c87c8
>>>>>> The builds on x86 arch take 3-4 mins, but for aarch64 (emulation!)
>>>>>> ~40mins
>>>>>> For now the jobs just build the .deb & .rpm packages for CentOS 7 and
>>>>>> Ubuntu 18.04, both amd64 and aarch64.
>>>>>> TODOs:
>>>>>> - migrate Alpine
>>>>>>
>>>>>
>>> Build on Alpine aarch64 fails with:
>>> ...
>>> automake: this behaviour will change in future Automake versions: they
>>> will
>>> automake: unconditionally cause object files to be placed in the same
>>> subdirectory
>>> automake: of the corresponding sources.
>>> automake: project, to avoid future incompatibilities.
>>> parallel-tests: installing 'build-aux/test-driver'
>>> lib/libvmod_debug/Makefile.am:12: warning: libvmod_debug_la_LDFLAGS
>>> multiply defined in condition TRUE ...
>>> lib/libvmod_debug/automake_boilerplate.am:19: ...
>>> 'libvmod_debug_la_LDFLAGS' previously defined here
>>> lib/libvmod_debug/Makefile.am:9:   'lib/libvmod_debug/
>>> automake_boilerplate.am' included from here
>>> + autoconf
>>> + CONFIG_SHELL=/bin/sh
>>> + export CONFIG_SHELL
>>> + ./configure '--prefix=/opt/varnish' '--mandir=/opt/varnish/man'
>>> --enable-maintainer-mode --enable-developer-warnings
>>> --enable-debugging-symbols --enable-dependency-tracking
>>> --with-persistent-storage --quiet
>>> configure: WARNING: dot not found - build will fail if svg files are out
>>> of date.
>>> configure: WARNING: No system jemalloc found, using system malloc
>>> configure: error: Could not find backtrace() support
>>>
>>> Does anyone know a workaround ?
>>> I use multiarch/alpine:aarch64-edge as a base Docker image
>>>
>>> Martin
>>>
>>>
>>>
>>>> - store the packages as CircleCI artifacts
>>>>>> - anything else that is still missing
>>>>>>
>>>>>> Adding more architectures would be as easy as adding a new Dockerfile
>>>>>> with a base image from the respective type.
>>>>>>
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>>> 2) Travis CI
>>>>>>> 2.1) problems
>>>>>>> 2.1.1) generally Travis is slower than Circle!
>>>>>>> Althought if we use CircleCI 'machine' executor it will be slower
>>>>>>> than the current 'Docker' executor!
>>>>>>> 2.1.2) Travis supports only Ubuntu
>>>>>>> Current setup at CircleCI uses CentOS 7.
>>>>>>> I guess the build steps won't have problems on Ubuntu.
>>>>>>>
>>>>>>> 3) GitHub Actions
>>>>>>> GH Actions does not support ARM64 but it supports self hosted ARM64
>>>>>>> runners
>>>>>>> 3.1) The problem is that there is no way to make a self hosted
>>>>>>> runner really private. I.e. if someone forks Varnish Cache any commit in
>>>>>>> the fork will trigger builds on the arm64 node. There is no way to reserve
>>>>>>> the runner only for commits against
>>>>>>> https://github.com/varnishcache/varnish-cache
>>>>>>>
>>>>>>> Do you see other problems or maybe different ways ?
>>>>>>> Do you have preferences which way to go ?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Martin
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Martin
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Guillaume Quintard
>>>>>>>>> _______________________________________________
>>>>>>>>> varnish-dev mailing list
>>>>>>>>> varnish-dev at varnish-cache.org
>>>>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>>>>>>>>>
>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20200324/50556096/attachment-0001.html>