Skip to content

Docker Hub images - #405

Open
prj- wants to merge 6 commits into
developfrom
release-docker
Open

Docker Hub images#405
prj- wants to merge 6 commits into
developfrom
release-docker

Conversation

@prj-

@prj- prj- commented Jul 19, 2026

Copy link
Copy Markdown
Member

No description provided.

@prj-

prj- commented Jul 19, 2026

Copy link
Copy Markdown
Member Author

@simonlegrand, the structure between Docker and your stuff is not matching precisely, see https://github.com/FreeFem/FreeFem-sources/actions/runs/29691285093, so let me know what adjustments should be made, please.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds first-class Docker image build/publish support to the release workflow, including a multi-stage Dockerfile and release asset exports, so FreeFEM can be distributed via Docker Hub and GitHub Releases.

Changes:

  • Add a multi-stage etc/docker/Dockerfile that builds FreeFEM from source and provides a runtime image with a basic smoke test.
  • Extend .github/workflows/releases.yml to build/push per-arch images, publish a multi-platform manifest, and attach exported image tarballs to GitHub Releases.
  • Add a .dockerignore to reduce Docker build context size and avoid copying build artifacts into image builds.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
etc/docker/Dockerfile Defines builder/runtime stages to compile and run FreeFEM inside an Ubuntu 24.04 container.
.github/workflows/releases.yml Adds Docker build + multi-arch publish jobs and includes Docker image tarballs in release assets.
.dockerignore Excludes common build outputs and large 3rd-party directories from Docker build context.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .github/workflows/releases.yml
Comment thread .github/workflows/releases.yml Outdated
Comment thread .github/workflows/releases.yml Outdated
Comment thread .github/workflows/releases.yml Outdated
Comment thread .github/workflows/releases.yml Outdated
Comment thread etc/docker/Dockerfile Outdated
Comment thread etc/docker/Dockerfile Outdated
Comment thread etc/docker/Dockerfile Outdated
@prj-
prj- force-pushed the release-docker branch 2 times, most recently from 3e2c4f6 to cde3709 Compare July 24, 2026 04:02

@simonlegrand simonlegrand left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Thanks for the structure mods. I don't understand the failure, could we try to run the whole pipeline again to see if it was only a "one-off" github problem?

Comment thread .github/workflows/release_docker.yml Outdated
Comment thread .github/workflows/releases.yml Outdated
@prj-

prj- commented Aug 6, 2026

Copy link
Copy Markdown
Member Author

Are you done with the changes, @simonlegrand? Sorry, I lost track of this one.

@simonlegrand

Copy link
Copy Markdown
Contributor

It seems ok to me. The CI kept failing randomly when downloading freeyams (https://github.com/FreeFem/FreeFem-sources/actions/runs/30818261818/job/91760697332), it's probably a Github problem but still it's a bit annoying not to have all the green lights, and packages :). Do you want to give it a last try?

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Suppressed comments (4)

.github/workflows/release_docker.yml:89

  • The uploaded artifact name uses matrix.version, but the job matrix doesn't define a version key. This will produce confusing artifact names (or fail expression evaluation depending on runner) and makes the artifact hard to locate.
        uses: actions/upload-artifact@v7
        with:
          name: docker-image-${{ matrix.version }}-${{ matrix.arch }}
          path: ./docker-assets/*

.github/workflows/release_docker.yml:106

  • The digest artifact name also uses matrix.version, but the matrix doesn't define it. This can break the downstream download-artifact step (patterning / expected digests) and makes digests ambiguous across releases.
        uses: actions/upload-artifact@v7
        with:
          name: docker-digest-${{ matrix.version }}-${{ matrix.arch }}
          path: ${{ runner.temp }}/digests/*

.github/workflows/releases.yml:83

  • actions/download-artifact's pattern input expects a glob; the extglob-style negation pattern !(docker-digest-*|install_tree*) is not a standard glob and may result in downloading no artifacts (breaking the release). Consider downloading all artifacts and relying on the release upload files: globs to select what is published.
        uses: actions/download-artifact@v4
        with:
          path: ./release-assets
          pattern: "!(docker-digest-*|install_tree*)"
          merge-multiple: true

.github/workflows/release_docker.yml:65

  • matrix.version is referenced in the Buildx cache scope, but the matrix only defines arch, so matrix.version will be empty/undefined and caching will not work as intended (and may collide across builds). Use the workflow input release_version (or github.ref_name) instead.

This issue also appears in the following locations of the same file:

  • line 86
  • line 103
          cache-from: type=gha,scope=docker-${{ matrix.version }}-${{ matrix.arch }}
          cache-to: type=gha,scope=docker-${{ matrix.version }}-${{ matrix.arch }},mode=max
          provenance: mode=max

Comment thread etc/docker/Dockerfile
Comment thread .github/workflows/release_docker.yml
@prj-

prj- commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

Well, the issue is that the code is broken, so we can’t merge it as is.

@prj-

prj- commented Aug 9, 2026

Copy link
Copy Markdown
Member Author

Why are there now so many unneeded assets being downloaded https://github.com/FreeFem/FreeFem-sources/actions/runs/31295773341/job/93238864712? I did not go through all your previous changes, @simonlegrand, but could you please have a look and go back to the initial behavior of the PR (modulo your changes to the workflow)?

@simonlegrand

Copy link
Copy Markdown
Contributor

Sure, I'll try to do that by the end of the week.

@prj-

prj- commented Aug 11, 2026

Copy link
Copy Markdown
Member Author

Thanks, take your time.

@simonlegrand

Copy link
Copy Markdown
Contributor

Those network problems are very annoying and seem to only happen with Ubuntu images. I won't push the tag to trigger the release workflow today, there might be some sort of quota or I don't know what that has already been reached. I'll be on vacation next week so feel free to push it, and when I'll be back I'll try to configure local ubuntu runners on Inria CI platform.

@prj-

prj- commented Aug 17, 2026

Copy link
Copy Markdown
Member Author

Enjoy your week off! I've tagged the latest commit of the branch, let's see what happens...

@prj-
prj- requested a balanced review from Copilot August 20, 2026 06:25

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (2)

.github/workflows/releases.yml:66

  • This called workflow publishes the version manifest—and may move latest—before the Debian, Windows, and GitHub release jobs have succeeded. A failure in either package build or create_release therefore leaves Docker Hub advertising a release that the overall release workflow rejected. Split Docker build/export from publication and make the publishing step depend on the package jobs or create_release.
  get_docker_images:
    needs: set_versions
    uses: ./.github/workflows/release_docker.yml

etc/docker/Dockerfile:100

  • This batch command can hide a failed input: without set -e, the loop continues and the container returns only the status of the last ff-mpirun. If an earlier .edp fails and a later one succeeds, CI reports the container as successful. Exit on the first failed solve so every input failure is propagated.
CMD ["bash", "-c", "shopt -s nullglob; files=(/data/$GLOB); for file in \"${files[@]}\"; do ff-mpirun -n 1 \"$file\"; done"]

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Suppressed comments (1)

.github/workflows/release_docker.yml:79

  • The image's published default path is not exercised: the Dockerfile smoke test invokes FreeFem++-nw directly, while CMD runs every workload through ff-mpirun. Manifest inspection will still pass if the installed MPI launcher or its runtime libraries are broken. Run a mounted smoke .edp through the image's default command for each architecture before creating the multi-platform manifest, as the package release check does in .github/workflows/release_check.yml:62.
          docker pull --platform "linux/${ARCH}" "${SOURCE_IMAGE}"

@prj-
prj- requested a balanced review from Copilot August 20, 2026 12:35

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Suppressed comments (2)

.github/workflows/releases.yml:83

  • Adding pattern: v<version>* changes which artifacts are downloaded for the GitHub Release. If the .deb/.exe artifacts uploaded by get_deb_packages / get_exe_packages are not named with the same v${version} prefix, they won’t be downloaded, and the release step will fail to attach them (or will publish an incomplete release). Either ensure all producing workflows use consistent artifact naming with that prefix, or adjust/remove the pattern to match all required artifacts.
          pattern: v${{ needs.set_versions.outputs.ff_version }}*

.github/workflows/full-msmpi.yml:51

  • GitHub Actions expression functions are typically written as fromJSON (matching the official docs and usage elsewhere in this PR, e.g. fromJSON in release_docker.yml). Consider standardizing to fromJSON for consistency/readability across workflows.
        windows_version: ${{ fromJson(inputs.windows_versions || '[2025]') }}
        build_type: ${{ fromJson(inputs.build_types || '["debug", "release"]') }}

Comment thread etc/docker/Dockerfile
@prj-
prj- requested a balanced review from Copilot August 22, 2026 04:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated no new comments.

@prj-

prj- commented Aug 24, 2026

Copy link
Copy Markdown
Member Author

@simonlegrand, I hope you had a good week off. What do you think of the latest iteration of the PR https://github.com/FreeFem/FreeFem-sources/actions/runs/32513979260? Could you maybe try to simplify the dependency graph?

@simonlegrand

Copy link
Copy Markdown
Contributor

Hi @prj- , thanks for the fixes.

However I don't understand the need to put the Docker image construction into the get_deb_packages workflow. To me, .deb packages and docker images are two unrelated things, even if the docker image is based on Debian. Furthermore, it creates this additional 'Publication' job in between the others.

If you really want to keep it like this, I suggest we move the docker publication into the create_release workflow to simplify the graph. What do you think?

@prj-

prj- commented Aug 24, 2026

Copy link
Copy Markdown
Member Author

Please fix as you see fit.

@simonlegrand

Copy link
Copy Markdown
Contributor

I just rolled back to your previous commit. Weirdly, I can't trigger the releases workflow... But it should be working fine.

@prj-

prj- commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

So we go back to this workflow? https://github.com/FreeFem/FreeFem-sources/actions/runs/32029504102
It isn’t right, we shouldn’t publish anything on Docker Hub until we know the .deb and .exe have been generated.

@simonlegrand

Copy link
Copy Markdown
Contributor

Rather this one: https://github.com/FreeFem/FreeFem-sources/actions/runs/32369463268 (there is just ef8b6b5 in between)

It isn’t right, we shouldn’t publish anything on Docker Hub until we know the .deb and .exe have been generated.

It seems fine in this case since get_docker_images depends on get_deb_packages and get_exe_packages.

@prj-

prj- commented Aug 25, 2026

Copy link
Copy Markdown
Member Author

I don't think this is right either.

  • Why should the Docker images be built after the .exe and .deb, and not simultaneously?
  • There is no real point, IMHO, of having 5 workflows in sequence, can't they be factored somehow?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants