Skip to content

👷 QD-16987 Build arm64 base images on native runners instead of QEMU - #1137

Merged
Ivan Efimov (ivanovec) merged 2 commits into
mainfrom
QD-16987-arm-native-base
Nov 12, 2026
Merged

Ivan Efimov (ivanovec) merged 2 commits into
mainfrom
QD-16987-arm-native-base

Conversation

@ivanovec

Copy link
Copy Markdown
Collaborator

Pull Request Details

Description

Build arm64 base images on native GitHub arm64 runners instead of QEMU emulation.

  • Each bake group is built per platform: linux/amd64 on ubuntu-latest, linux/arm64 on ubuntu-24.04-arm. setup-qemu is removed.
  • On push, every target is pushed untagged by digest (a tagged ref can't be pushed by digest), using a generated bake override: tags are cleared and output gets name=<repository of the target's first tag>. Nothing is hardcoded — tags and repositories still come from docker-bake.hcl.
  • A new merge (<group>) job reads the tags via docker buildx bake --print, the per-platform digests from the bake metadata artifacts, and creates the multi-arch tags with docker buildx imagetools create. A group whose platform build is missing fails on its own; other groups are still merged.
  • GHA cache: a separate scope per group and platform (base-<group>-<arch>) instead of the single provenance scope that all 13 jobs overwrote.

Behavior changes

  • Job names: base (jvm) → base (jvm, amd64) / base (jvm, arm64) + merge (jvm). Not part of required status checks.
  • Tags of a group are updated only if both platforms of the whole group succeeded (previously each target was pushed as soon as it finished, so a group could be partially updated).
  • The first run after the change starts with a cold cache.

Unchanged

  • Published tags and repositories, image contents (Dockerfiles/HCL untouched; arch detection uses only TARGETPLATFORM / dpkg --print-architecture, no BUILDPLATFORM).
  • Provenance attestations (preserved by imagetools create).
  • Push only on push events; PRs build both platforms without pushing.

To watch on the first runs

  • GHA cache size limit with 26 scopes in mode=max (may cause LRU eviction → slower builds, not failures).
  • Free disk space on ubuntu-24.04-arm (only one platform per runner now).

Related Issue

QD-16987

Motivation and Context

QEMU-emulated arm64 builds are slow and intermittently crash Node/V8 with SIGILL, e.g. https://github.com/JetBrains/qodana-cli/actions/runs/43056082577/job/130645796321:

qemu: uncaught target signal 4 (Illegal instruction) - core dumped
... did not complete successfully: exit code: 132

The same step passes on re-run. ci.yml already uses ubuntu-24.04-arm.

How Has This Been Tested

  • actionlint 1.7.7 (with shellcheck): clean.
  • docker buildx bake --print with the generated override for all 13 groups (26 targets): push mode → tags cleared, single platform, output to registry.jetbrains.team/p/sa/containers/qodana by digest; PR mode → tags kept, no output.
  • End-to-end locally with a local registry: a two-target chain via contexts, separate amd64/arm64 builds pushed by digest, then the merge step extracted verbatim from the workflow → each tag (incl. multiple tags per target) points to an index with linux/amd64 + linux/arm64 + attestation manifests; a missing platform fails the merge with exit 1.
  • Not yet run on GitHub runners: the merge job only runs on push, so it will be exercised after merge to main.

Types of changes

  • Docs change / refactoring / dependency upgrade
  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

QEMU-emulated arm64 builds on ubuntu-latest are slow and intermittently
crash Node/V8 with SIGILL (exit code 132).

- Build each bake group per platform: amd64 on ubuntu-latest,
  arm64 on ubuntu-24.04-arm; drop setup-qemu
- On push, push untagged images by digest (a tagged ref can't be pushed
  by digest), then the merge job creates the multi-arch tags from
  docker-bake.hcl via `docker buildx imagetools create`
- Use a separate GHA cache scope per group and platform instead of one
  shared "provenance" scope overwritten by all jobs
@ivanovec
Ivan Efimov (ivanovec) requested a review from a team as a code owner November 12, 2026 07:57
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T07:59:03.127588Z 317990c PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Qodana for Go

127 new problems were found

Inspection name Severity Problems
Vulnerable declared dependency 🔴 Failure 2
Check GO source code coverage 🔶 Warning 67
Unhandled error 🔶 Warning 13
Interprocedural potential nil dereference 🔶 Warning 10
Deprecated element 🔶 Warning 4
Imported package name as a name identifier 🔶 Warning 3
Vulnerable declared dependency 🔶 Warning 2
Reserved word used as name 🔶 Warning 1
Potential resource leak 🔶 Warning 1
Missing 'case' statements for 'iota' consts in 'switch' 🔶 Warning 1
Unsorted imports ◽️ Notice 9
Name starts with a package name ◽️ Notice 8
Empty slice declared using a literal ◽️ Notice 3
Vulnerable declared dependency ◽️ Notice 1
Redundant type conversion ◽️ Notice 1
Comment of exported element starts with the incorrect name ◽️ Notice 1
@@ Code coverage @@
+ 65% total lines covered
10631 lines analyzed, 6980 lines covered
# Calculated according to the filters of your coverage tool

☁️ View the detailed Qodana report

Contact Qodana team

Contact us at qodana-support@jetbrains.com

Comment thread .github/workflows/base.yml Resolved
Comment thread .github/workflows/base.yml Resolved
Comment thread .github/workflows/base.yml Resolved
Comment thread .github/workflows/base.yml Resolved
… in Python

- docker-bake.hcl: BASE_PLATFORMS / PUSH_BY_DIGEST variables and a tags() function
  replace the jq bake override (based on the patch from review)
- matrix platform as objects instead of include
- group list shared via YAML anchor, comments restored, groups job removed
- merge logic moved to .github/scripts/merge-base-images.py; validates the whole
  group before tagging, so tags are never partially updated
@qodana-release-bot

Copy link
Copy Markdown
Contributor

Successfully created backport PR for 263:

Ivan Efimov (ivanovec) added a commit that referenced this pull request Nov 13, 2026
…1137) (#1143)

* 👷 QD-16987 Build arm64 base images on native runners instead of QEMU

QEMU-emulated arm64 builds on ubuntu-latest are slow and intermittently
crash Node/V8 with SIGILL (exit code 132).

- Build each bake group per platform: amd64 on ubuntu-latest,
  arm64 on ubuntu-24.04-arm; drop setup-qemu
- On push, push untagged images by digest (a tagged ref can't be pushed
  by digest), then the merge job creates the multi-arch tags from
  docker-bake.hcl via `docker buildx imagetools create`
- Use a separate GHA cache scope per group and platform instead of one
  shared "provenance" scope overwritten by all jobs

* 👷 QD-16987 Address review: push mode in docker-bake.hcl, merge script in Python

- docker-bake.hcl: BASE_PLATFORMS / PUSH_BY_DIGEST variables and a tags() function
  replace the jq bake override (based on the patch from review)
- matrix platform as objects instead of include
- group list shared via YAML anchor, comments restored, groups job removed
- merge logic moved to .github/scripts/merge-base-images.py; validates the whole
  group before tagging, so tags are never partially updated

(cherry picked from commit f603f9a)

Co-authored-by: Ivan Efimov <ivan.efimov@jetbrains.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants