Skip to content

Fallback to bwrap in linux-sandbox when AppArmor denies mount in userns - #32315

Open
tripleyoung wants to merge 1 commit into
bazelbuild:masterfrom
tripleyoung:fix/linux-sandbox-ubuntu24-bwrap-fallback
Open

tripleyoung wants to merge 1 commit into
bazelbuild:masterfrom
tripleyoung:fix/linux-sandbox-ubuntu24-bwrap-fallback

Conversation

@tripleyoung

Copy link
Copy Markdown

Motivation

On Ubuntu 24.04+, kernel.apparmor_restrict_unprivileged_userns=1 is enabled by default (#24081). When linux-sandbox creates an unprivileged user namespace via clone(CLONE_NEWUSER), AppArmor transitions it into the unprivileged_userns profile, which denies CAP_SYS_ADMIN and mount(2) with EACCES (src/main/tools/linux-sandbox-pid1.cc: "mount": Permission denied). As a result, LinuxSandboxedSpawnRunner.isSupported() fails and Bazel silently falls back to processwrapper-sandbox.

Previously, #26434 explored wrapping linux-sandbox with busybox, but:

  1. Canonical subsequently removed the unconfined busybox AppArmor profile in Ubuntu 24.04 security updates (so busybox sh -c '"$@"' -- linux-sandbox now also fails with "mount": Permission denied).
  2. Wrapping linux-sandbox in an unconfined profile left the sandboxed payload itself unconfined with full capabilities inside the user namespace.

By contrast, /usr/bin/bwrap (bubblewrap, pre-installed on Ubuntu 24.04 Desktop/Server) has an official Canonical-maintained AppArmor profile (/etc/apparmor.d/bwrap-userns-restrict) designed specifically for unprivileged sandboxing:

  • profile bwrap is granted userns, mount, umount, and pivot_root to construct the sandbox mount table.
  • Upon execve of the child process, AppArmor transitions the child into the stacked &bwrap//&unpriv_bwrap profile (audit deny capability), stripping capabilities inside the user namespace and preserving Ubuntu's security model.

Changes

  1. src/main/tools/linux-sandbox-pid1.cc:
    • In SetupMountNamespace(), if the initial mount(nullptr, "/", nullptr, MS_REC | MS_PRIVATE, nullptr) fails with EACCES or EPERM (and !opt.hermetic and /usr/bin/bwrap is executable), close the readiness pipe and exit with kAppArmorBwrapFallbackExitCode before signaling the parent.
    • Implement SpawnBwrap() and RunInBwrapPid1(): translate the parsed Options (-W, -w, -e, -M/-m, -N/-n, -H, -R, -U, -P) into /usr/bin/bwrap arguments, and re-enter linux-sandbox inside the mounted namespace (BAZEL_LINUX_SANDBOX_BWRAP_PID1=1) so SpawnChild(), shebang interpreter diagnostics, ForwardSignal(SIGTERM), and WaitForChild() behave identically to native Pid1Main.
  2. src/main/tools/linux-sandbox.cc:
    • Detect kAppArmorBwrapFallbackExitCode in SpawnPid1() and transparently fall back to SpawnBwrap(), while keeping outer -T timeout, -t polite SIGTERM grace period, -l/-L output redirection, and -S wait4 rusage/cgroup statistics collection in the outer process.

Verification on Ubuntu 24.04 (kernel.apparmor_restrict_unprivileged_userns = 1)

  • Verified linux-sandbox basic execution, read-only root filesystem, writable -W/-w paths, -M/-m bind mounts, -e tmpfs mounts, -R (root) / -U (nobody) user mapping, missing shebang interpreter diagnostics, polite SIGTERM forwarding (-t), SIGKILL timeout (-T), and zero FD leaks.
  • Verified with Bazel 10.0.0rc2 on Ubuntu 24.04 that LinuxSandboxedSpawnRunner.isSupported() succeeds out-of-the-box and actions execute with # Runner: linux-sandbox instead of falling back to processwrapper-sandbox.

Fixes #24081

@tripleyoung
tripleyoung force-pushed the fix/linux-sandbox-ubuntu24-bwrap-fallback branch from d4e3c18 to 86a365c Compare November 1, 2026 09:22
@google-cla

google-cla Bot commented Nov 1, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@github-actions github-actions Bot added the awaiting-review PR is awaiting review from an assigned reviewer label Nov 1, 2026
On Ubuntu 24.04+, kernel.apparmor_restrict_unprivileged_userns=1 is
enabled by default. Non-allowlisted binaries that create an unprivileged
user namespace transition into the 'unprivileged_userns' AppArmor
profile, which drops capabilities and denies mount(2) with EACCES,
causing Bazel to silently fall back from linux-sandbox to
processwrapper-sandbox.

Unlike busybox (whose unconfined AppArmor profile was removed in recent
Ubuntu security updates and which left child processes unconfined inside
the user namespace), /usr/bin/bwrap ships with an official AppArmor
profile (/etc/apparmor.d/bwrap-userns-restrict) designed specifically
for unprivileged sandboxes: bwrap is permitted to set up user, mount,
PID, IPC, UTS, and network namespaces and bind/tmpfs mounts, and then
transitions its child process into the 'unpriv_bwrap' stacked profile
which strips capabilities inside the user namespace.

When SetupMountNamespace() in Pid1Main fails with EACCES or EPERM and
/usr/bin/bwrap is available, exit with a dedicated status code before
signaling readiness to the parent process and transparently re-spawn the
sandbox namespace and mount hierarchy via /usr/bin/bwrap, re-entering
linux-sandbox inside the mounted namespace to preserve child process
management, shebang error reporting, polite SIGTERM forwarding, and
rusage/cgroup statistics collection.

Fixes bazelbuild#24081
@tripleyoung
tripleyoung force-pushed the fix/linux-sandbox-ubuntu24-bwrap-fallback branch from 86a365c to 99b6793 Compare November 1, 2026 10:30
@fmeum
fmeum requested a review from meisterT November 1, 2026 17:54
@fmeum

fmeum commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator

Nice find, this can really help improve the situation on Ubuntu by.

Rather than falling back to bwrap from within the sandbox binary, would it be possible to detect the availability directly from the Java sandbox runner and run it without the indirection?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting-review PR is awaiting review from an assigned reviewer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Difficulties in using Linux-sandbox on Ubuntu 24

2 participants