Support for AARCH64

Wed Mar 25 18:14:48 UTC 2020

is that script running as root?

-- 
Guillaume Quintard

On Wed, Mar 25, 2020 at 2:30 AM Martin Grigorov <martin.grigorov at gmail.com>
wrote:

> Hi,
>
> I've moved 'dist' job to be executed in parallel with 'tar_pkg_tools' and
> the results from both are shared in the workspace for the actual packing
> jobs.
> Now the new error for aarch64-apk job is:
>
> abuild: varnish >>> varnish: Updating the sha512sums in APKBUILD...
> ]0; DEBUG: 4
> ]0;abuild: varnish >>> varnish: Building /varnish 6.4.0-r1 (using abuild
> 3.5.0-r0) started Wed, 25 Mar 2020 09:22:02 +0000
> >>> varnish: Checking sanity of /package/APKBUILD...
> >>> WARNING: varnish: No maintainer
> >>> varnish: Analyzing dependencies...
>   0%                                             %
> ############################################>>> varnish: Installing for
> build: build-base gcc libc-dev libgcc pcre-dev ncurses-dev libedit-dev
> py-docutils linux-headers libunwind-dev python py3-sphinx
> Waiting for repository lock
> ERROR: Unable to lock database: Bad file descriptor
> ERROR: Failed to open apk database: Bad file descriptor
> >>> ERROR: varnish: builddeps failed
> ]0; >>> varnish: Uninstalling dependencies...
> Waiting for repository lock
> ERROR: Unable to lock database: Bad file descriptor
> ERROR: Failed to open apk database: Bad file descriptor
>
> Google suggested to do this:
>    rm -rf /var/cache/apk
>    mkdir /var/cache/apk
>
> It fails at 'abuild -r' -
> https://github.com/martin-g/varnish-cache/blob/b62c357b389c0e1e31e9c001cbffb55090c2e49f/.circleci/make-apk-packages.sh#L61
>
> Any hints ?
>
> Martin
>
> On Wed, Mar 25, 2020 at 2:39 AM Guillaume Quintard <
> guillaume at varnish-software.com> wrote:
>
>> Hi,
>>
>> So, you are pointing at the `dist` job, whose sole role is to provide us
>> with a dist tarball, so we don't need that command line to work for
>> everyone, just for that specific platform.
>>
>> On the other hand,
>> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/.circleci/config.yml#L168 is
>> closer to what you want, `distcheck` will be call on all platform, and you
>> can see that it has the `--with-unwind` argument.
>> --
>> Guillaume Quintard
>>
>>
>> On Tue, Mar 24, 2020 at 3:05 PM Martin Grigorov <
>> martin.grigorov at gmail.com> wrote:
>>
>>>
>>>
>>> On Tue, Mar 24, 2020, 17:19 Guillaume Quintard <
>>> guillaume at varnish-software.com> wrote:
>>>
>>>> Compare your configure line with what's currently in use (or the
>>>> apkbuild file), there are a few options (with-unwind, without-jemalloc,
>>>> etc.) That need to be set
>>>>
>>>
>>> The configure line comes from "./autogen.des":
>>> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/autogen.des#L35-L42
>>> It is called at:
>>>
>>> https://github.com/varnishcache/varnish-cache/blob/4f9d8bed6b24bf9ee900c754f37615fdba1c44db/.circleci/config.yml#L40
>>> In my branch at:
>>>
>>> https://github.com/martin-g/varnish-cache/blob/4b4626ee9cc366b032a45f27b54d77176125ef03/.circleci/make-apk-packages.sh#L26
>>>
>>> It fails only on aarch64 for Alpine Linux. The x86_64 build for Alpine
>>> is fine.
>>> AARCH64 for CentOS 7 and Ubuntu 18.04 are also fine.
>>>
>>> Martin
>>>
>>>
>>>> On Tue, Mar 24, 2020, 08:05 Martin Grigorov <martin.grigorov at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> On Tue, Mar 24, 2020 at 11:00 AM Martin Grigorov <
>>>>> martin.grigorov at gmail.com> wrote:
>>>>>
>>>>>> Hi Guillaume,
>>>>>>
>>>>>> On Mon, Mar 23, 2020 at 8:01 PM Guillaume Quintard <
>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>
>>>>>>> Hi Martin,
>>>>>>>
>>>>>>> Thank you for that.
>>>>>>> A few remarks and questions:
>>>>>>> - how much time does the "docker build" step takes? We can possibly
>>>>>>> speed things up by push images to the dockerhub, as they don't need to
>>>>>>> change very often.
>>>>>>>
>>>>>>
>>>>>> Definitely such optimization would be a good thing to do!
>>>>>> At the moment, with 'machine' executor it fetches the base image and
>>>>>> then builds all the Docker layers again and again.
>>>>>> Here are the timings:
>>>>>> 1) Spinning up a VM - around 10secs
>>>>>> 2) prepare env variables - 0secs
>>>>>> 3) checkout code (varnish-cache) - 5secs
>>>>>> 4) activate QEMU - 2secs
>>>>>> 5) build packages
>>>>>> 5.1) x86 deb - 3m 30secs
>>>>>> 5.2) x86 rpm - 2m 50secs
>>>>>> 5.3) aarch64 rpm - 35mins
>>>>>> 5.4) aarch64 deb - 45mins
>>>>>>
>>>>>>
>>>>>>> - any reason why you clone pkg-varnish-cache in each job? The idea
>>>>>>> was to have it cloned once in tar-pkg-tools for consistency and
>>>>>>> reproducibility, which we lose here.
>>>>>>>
>>>>>>
>>>>>> I will extract the common steps once I see it working. This is my
>>>>>> first CircleCI project and I still find my ways in it!
>>>>>>
>>>>>>
>>>>>>> - do we want to change things for the amd64 platforms for the sake
>>>>>>> of consistency?
>>>>>>>
>>>>>>
>>>>>> So far there is nothing specific for amd4 or aarch64, except the base
>>>>>> Docker images.
>>>>>> For example make-deb-packages.sh is reused for both amd64 and aarch64
>>>>>> builds. Same for -rpm- and now for -apk- (alpine).
>>>>>>
>>>>>> Once I feel the change is almost finished I will open a Pull Request
>>>>>> for more comments!
>>>>>>
>>>>>> Martin
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Guillaume Quintard
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 23, 2020 at 6:25 AM Martin Grigorov <
>>>>>>> martin.grigorov at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Mar 18, 2020 at 5:31 PM Martin Grigorov <
>>>>>>>> martin.grigorov at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> On Thu, Mar 12, 2020 at 4:35 PM Martin Grigorov <
>>>>>>>>> martin.grigorov at gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Guillaume,
>>>>>>>>>>
>>>>>>>>>> On Thu, Mar 12, 2020 at 3:23 PM Guillaume Quintard <
>>>>>>>>>> guillaume at varnish-software.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> Offering arm64 packages requires a few things:
>>>>>>>>>>> - arm64-compatible code (all good in
>>>>>>>>>>> https://github.com/varnishcache/varnish-cache)
>>>>>>>>>>> - arm64-compatible package framework (all good in
>>>>>>>>>>> https://github.com/varnishcache/pkg-varnish-cache)
>>>>>>>>>>> - infrastructure to build the packages (uhoh, see below)
>>>>>>>>>>> - infrastructure to store and deliver (
>>>>>>>>>>> https://packagecloud.io/varnishcache)
>>>>>>>>>>>
>>>>>>>>>>> So, everything is in place, expect for the third point. At the
>>>>>>>>>>> moment, there are two concurrent CI implementations:
>>>>>>>>>>> - travis:
>>>>>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.travis.yml It's
>>>>>>>>>>> the historical one, and currently only runs compilation+test for OSX
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Actually it tests Linux AMD64 and ARM64 too.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> - circleci:
>>>>>>>>>>> https://github.com/varnishcache/varnish-cache/blob/master/.circleci/config.yml the
>>>>>>>>>>> new kid on the block, that builds all the packages and distchecks for all
>>>>>>>>>>> the packaged platforms
>>>>>>>>>>>
>>>>>>>>>>> The issue is that cirecleci doesn't support arm64 containers
>>>>>>>>>>> (for now?), so we would need to re-implement the packaging logic in Travis.
>>>>>>>>>>> It's not a big problem, but it's currently not a priority on my side.
>>>>>>>>>>>
>>>>>>>>>>> However, I am totally ready to provide help if someone wants to
>>>>>>>>>>> take that up. The added benefit it that Travis would be able to handle
>>>>>>>>>>> everything and we can retire the circleci experiment
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I will take a look in the coming days and ask you if I need help!
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> I've took a look at the current setup and here is what I've found
>>>>>>>>> as problems and possible solutions:
>>>>>>>>>
>>>>>>>>> 1) Circle CI
>>>>>>>>> 1.1) problem - the 'machine' and 'Docker' executors run on x86_64,
>>>>>>>>> so there is no way to build the packages in a "native" environment
>>>>>>>>> 1.2) possible solutions
>>>>>>>>> 1.2.1) use multiarch cross build
>>>>>>>>> 1.2.2) use 'machine' executor that registers QEMU via
>>>>>>>>> https://hub.docker.com/r/multiarch/qemu-user-static/ and then
>>>>>>>>> builds and runs a custom Docker image that executes a shell script with the
>>>>>>>>> build steps
>>>>>>>>> It will look something like
>>>>>>>>> https://github.com/yukimochi-containers/alpine-vpnserver/blob/69bb0a612c9df3e4ba78064d114751b760f0df9d/.circleci/config.yml#L19-L38 but
>>>>>>>>> instead of uploading the Docker image as a last step it will run it.
>>>>>>>>> The RPM and DEB build related code from current config.yml will be
>>>>>>>>> extracted into shell scripts which will be copied in the custom Docker
>>>>>>>>> images
>>>>>>>>>
>>>>>>>>> From these two possible ways I have better picture in my head how
>>>>>>>>> to do 1.2.2, but I don't mind going deep in 1.2.1 if this is what you'd
>>>>>>>>> prefer.
>>>>>>>>>
>>>>>>>>
>>>>>>>> I've decided to stay with Circle CI and use 'machine' executor with
>>>>>>>> QEMU.
>>>>>>>>
>>>>>>>> The changed config.yml could be seen at
>>>>>>>> https://github.com/martin-g/varnish-cache/tree/feature/aarch64-packages/.circleci and
>>>>>>>> the build at
>>>>>>>> https://app.circleci.com/pipelines/github/martin-g/varnish-cache/71/workflows/3a275d79-62a9-48b4-9aef-1585de1c87c8
>>>>>>>> The builds on x86 arch take 3-4 mins, but for aarch64 (emulation!)
>>>>>>>> ~40mins
>>>>>>>> For now the jobs just build the .deb & .rpm packages for CentOS 7
>>>>>>>> and Ubuntu 18.04, both amd64 and aarch64.
>>>>>>>> TODOs:
>>>>>>>> - migrate Alpine
>>>>>>>>
>>>>>>>
>>>>> Build on Alpine aarch64 fails with:
>>>>> ...
>>>>> automake: this behaviour will change in future Automake versions: they
>>>>> will
>>>>> automake: unconditionally cause object files to be placed in the same
>>>>> subdirectory
>>>>> automake: of the corresponding sources.
>>>>> automake: project, to avoid future incompatibilities.
>>>>> parallel-tests: installing 'build-aux/test-driver'
>>>>> lib/libvmod_debug/Makefile.am:12: warning: libvmod_debug_la_LDFLAGS
>>>>> multiply defined in condition TRUE ...
>>>>> lib/libvmod_debug/automake_boilerplate.am:19: ...
>>>>> 'libvmod_debug_la_LDFLAGS' previously defined here
>>>>> lib/libvmod_debug/Makefile.am:9:   'lib/libvmod_debug/
>>>>> automake_boilerplate.am' included from here
>>>>> + autoconf
>>>>> + CONFIG_SHELL=/bin/sh
>>>>> + export CONFIG_SHELL
>>>>> + ./configure '--prefix=/opt/varnish' '--mandir=/opt/varnish/man'
>>>>> --enable-maintainer-mode --enable-developer-warnings
>>>>> --enable-debugging-symbols --enable-dependency-tracking
>>>>> --with-persistent-storage --quiet
>>>>> configure: WARNING: dot not found - build will fail if svg files are
>>>>> out of date.
>>>>> configure: WARNING: No system jemalloc found, using system malloc
>>>>> configure: error: Could not find backtrace() support
>>>>>
>>>>> Does anyone know a workaround ?
>>>>> I use multiarch/alpine:aarch64-edge as a base Docker image
>>>>>
>>>>> Martin
>>>>>
>>>>>
>>>>>
>>>>>> - store the packages as CircleCI artifacts
>>>>>>>> - anything else that is still missing
>>>>>>>>
>>>>>>>> Adding more architectures would be as easy as adding a new
>>>>>>>> Dockerfile with a base image from the respective type.
>>>>>>>>
>>>>>>>> Martin
>>>>>>>>
>>>>>>>>
>>>>>>>>> 2) Travis CI
>>>>>>>>> 2.1) problems
>>>>>>>>> 2.1.1) generally Travis is slower than Circle!
>>>>>>>>> Althought if we use CircleCI 'machine' executor it will be slower
>>>>>>>>> than the current 'Docker' executor!
>>>>>>>>> 2.1.2) Travis supports only Ubuntu
>>>>>>>>> Current setup at CircleCI uses CentOS 7.
>>>>>>>>> I guess the build steps won't have problems on Ubuntu.
>>>>>>>>>
>>>>>>>>> 3) GitHub Actions
>>>>>>>>> GH Actions does not support ARM64 but it supports self hosted
>>>>>>>>> ARM64 runners
>>>>>>>>> 3.1) The problem is that there is no way to make a self hosted
>>>>>>>>> runner really private. I.e. if someone forks Varnish Cache any commit in
>>>>>>>>> the fork will trigger builds on the arm64 node. There is no way to reserve
>>>>>>>>> the runner only for commits against
>>>>>>>>> https://github.com/varnishcache/varnish-cache
>>>>>>>>>
>>>>>>>>> Do you see other problems or maybe different ways ?
>>>>>>>>> Do you have preferences which way to go ?
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Martin
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Martin
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> Guillaume Quintard
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> varnish-dev mailing list
>>>>>>>>>>> varnish-dev at varnish-cache.org
>>>>>>>>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
>>>>>>>>>>>
>>>>>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://www.varnish-cache.org/lists/pipermail/varnish-dev/attachments/20200325/c6f47c5c/attachment-0001.html>