Docker Hub images - #405
Conversation
|
@simonlegrand, the structure between Docker and your stuff is not matching precisely, see https://github.com/FreeFem/FreeFem-sources/actions/runs/29691285093, so let me know what adjustments should be made, please. |
There was a problem hiding this comment.
Pull request overview
Adds first-class Docker image build/publish support to the release workflow, including a multi-stage Dockerfile and release asset exports, so FreeFEM can be distributed via Docker Hub and GitHub Releases.
Changes:
- Add a multi-stage
etc/docker/Dockerfilethat builds FreeFEM from source and provides a runtime image with a basic smoke test. - Extend
.github/workflows/releases.ymlto build/push per-arch images, publish a multi-platform manifest, and attach exported image tarballs to GitHub Releases. - Add a
.dockerignoreto reduce Docker build context size and avoid copying build artifacts into image builds.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
etc/docker/Dockerfile |
Defines builder/runtime stages to compile and run FreeFEM inside an Ubuntu 24.04 container. |
.github/workflows/releases.yml |
Adds Docker build + multi-arch publish jobs and includes Docker image tarballs in release assets. |
.dockerignore |
Excludes common build outputs and large 3rd-party directories from Docker build context. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
3e2c4f6 to
cde3709
Compare
simonlegrand
left a comment
There was a problem hiding this comment.
Nice! Thanks for the structure mods. I don't understand the failure, could we try to run the whole pipeline again to see if it was only a "one-off" github problem?
|
Are you done with the changes, @simonlegrand? Sorry, I lost track of this one. |
|
It seems ok to me. The CI kept failing randomly when downloading freeyams (https://github.com/FreeFem/FreeFem-sources/actions/runs/30818261818/job/91760697332), it's probably a Github problem but still it's a bit annoying not to have all the green lights, and packages :). Do you want to give it a last try? |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
Suppressed comments (4)
.github/workflows/release_docker.yml:89
- The uploaded artifact name uses
matrix.version, but the job matrix doesn't define aversionkey. This will produce confusing artifact names (or fail expression evaluation depending on runner) and makes the artifact hard to locate.
uses: actions/upload-artifact@v7
with:
name: docker-image-${{ matrix.version }}-${{ matrix.arch }}
path: ./docker-assets/*
.github/workflows/release_docker.yml:106
- The digest artifact name also uses
matrix.version, but the matrix doesn't define it. This can break the downstreamdownload-artifactstep (patterning / expected digests) and makes digests ambiguous across releases.
uses: actions/upload-artifact@v7
with:
name: docker-digest-${{ matrix.version }}-${{ matrix.arch }}
path: ${{ runner.temp }}/digests/*
.github/workflows/releases.yml:83
actions/download-artifact'spatterninput expects a glob; the extglob-style negation pattern!(docker-digest-*|install_tree*)is not a standard glob and may result in downloading no artifacts (breaking the release). Consider downloading all artifacts and relying on the release uploadfiles:globs to select what is published.
uses: actions/download-artifact@v4
with:
path: ./release-assets
pattern: "!(docker-digest-*|install_tree*)"
merge-multiple: true
.github/workflows/release_docker.yml:65
matrix.versionis referenced in the Buildx cache scope, but the matrix only definesarch, somatrix.versionwill be empty/undefined and caching will not work as intended (and may collide across builds). Use the workflow inputrelease_version(orgithub.ref_name) instead.
This issue also appears in the following locations of the same file:
- line 86
- line 103
cache-from: type=gha,scope=docker-${{ matrix.version }}-${{ matrix.arch }}
cache-to: type=gha,scope=docker-${{ matrix.version }}-${{ matrix.arch }},mode=max
provenance: mode=max
|
Well, the issue is that the code is broken, so we can’t merge it as is. |
|
Why are there now so many unneeded assets being downloaded https://github.com/FreeFem/FreeFem-sources/actions/runs/31295773341/job/93238864712? I did not go through all your previous changes, @simonlegrand, but could you please have a look and go back to the initial behavior of the PR (modulo your changes to the workflow)? |
|
Sure, I'll try to do that by the end of the week. |
|
Thanks, take your time. |
|
Those network problems are very annoying and seem to only happen with Ubuntu images. I won't push the tag to trigger the release workflow today, there might be some sort of quota or I don't know what that has already been reached. I'll be on vacation next week so feel free to push it, and when I'll be back I'll try to configure local ubuntu runners on Inria CI platform. |
|
Enjoy your week off! I've tagged the latest commit of the branch, let's see what happens... |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (2)
.github/workflows/releases.yml:66
- This called workflow publishes the version manifest—and may move
latest—before the Debian, Windows, and GitHub release jobs have succeeded. A failure in either package build orcreate_releasetherefore leaves Docker Hub advertising a release that the overall release workflow rejected. Split Docker build/export from publication and make the publishing step depend on the package jobs orcreate_release.
get_docker_images:
needs: set_versions
uses: ./.github/workflows/release_docker.yml
etc/docker/Dockerfile:100
- This batch command can hide a failed input: without
set -e, the loop continues and the container returns only the status of the lastff-mpirun. If an earlier.edpfails and a later one succeeds, CI reports the container as successful. Exit on the first failed solve so every input failure is propagated.
CMD ["bash", "-c", "shopt -s nullglob; files=(/data/$GLOB); for file in \"${files[@]}\"; do ff-mpirun -n 1 \"$file\"; done"]
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Suppressed comments (1)
.github/workflows/release_docker.yml:79
- The image's published default path is not exercised: the Dockerfile smoke test invokes
FreeFem++-nwdirectly, whileCMDruns every workload throughff-mpirun. Manifest inspection will still pass if the installed MPI launcher or its runtime libraries are broken. Run a mounted smoke.edpthrough the image's default command for each architecture before creating the multi-platform manifest, as the package release check does in.github/workflows/release_check.yml:62.
docker pull --platform "linux/${ARCH}" "${SOURCE_IMAGE}"
- Fix download filter pattern
- Remove ${{ matrix.version }}
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
.github/workflows/releases.yml:83
- Adding
pattern: v<version>*changes which artifacts are downloaded for the GitHub Release. If the.deb/.exeartifacts uploaded byget_deb_packages/get_exe_packagesare not named with the samev${version}prefix, they won’t be downloaded, and the release step will fail to attach them (or will publish an incomplete release). Either ensure all producing workflows use consistent artifact naming with that prefix, or adjust/remove the pattern to match all required artifacts.
pattern: v${{ needs.set_versions.outputs.ff_version }}*
.github/workflows/full-msmpi.yml:51
- GitHub Actions expression functions are typically written as
fromJSON(matching the official docs and usage elsewhere in this PR, e.g.fromJSONinrelease_docker.yml). Consider standardizing tofromJSONfor consistency/readability across workflows.
windows_version: ${{ fromJson(inputs.windows_versions || '[2025]') }}
build_type: ${{ fromJson(inputs.build_types || '["debug", "release"]') }}
[ci skip]
|
@simonlegrand, I hope you had a good week off. What do you think of the latest iteration of the PR https://github.com/FreeFem/FreeFem-sources/actions/runs/32513979260? Could you maybe try to simplify the dependency graph? |
|
Hi @prj- , thanks for the fixes. However I don't understand the need to put the Docker image construction into the get_deb_packages workflow. To me, .deb packages and docker images are two unrelated things, even if the docker image is based on Debian. Furthermore, it creates this additional 'Publication' job in between the others. If you really want to keep it like this, I suggest we move the docker publication into the create_release workflow to simplify the graph. What do you think? |
|
Please fix as you see fit. |
4177d7f to
ef8b6b5
Compare
|
I just rolled back to your previous commit. Weirdly, I can't trigger the releases workflow... But it should be working fine. |
|
So we go back to this workflow? https://github.com/FreeFem/FreeFem-sources/actions/runs/32029504102 |
|
Rather this one: https://github.com/FreeFem/FreeFem-sources/actions/runs/32369463268 (there is just ef8b6b5 in between)
It seems fine in this case since get_docker_images depends on get_deb_packages and get_exe_packages. |
|
I don't think this is right either.
|
No description provided.