fix(ebuild): bash interpreter hardening for v0.9.4#69
Merged
Conversation
…context tracking - Add InterpreterPanicError type for descriptive panic-to-error conversion - Add defer/recover to all interpreter entry points (Run, RunFile, Eval) - Add panic recovery to dispatchPhase() with package name in errors - Fix nested eclass inheritance: save/restore currentEclass in loadEclass() - Add panic recovery to eclass loading with retry-loop prevention - Add panicContext() helper for rich error messages (package + eclass) - Add 8 new tests covering all hardening paths
…all phase - Add runtimeEnv field to Helpers for bash-to-Go variable bridge - Update getEnvVar to check runner's environment first, enabling Go eclass handlers to read variables set in ebuild scripts (e.g., DISTUTILS_USE_PEP517, PYTHON_COMPAT) - Sync all runtime vars in InheritWithEnv before eclass loading, fixing 'Unknown DISTUTILS_USE_PEP517=' errors for Python packages - Set runtimeEnv in exec handler before dispatching to command map - Make phaseInstall/phaseCompile return success when no Makefile (correct for virtual/*, acct-*, and eclass-managed packages) - Skip .provenance files in unpack (attestation files, not archives) - Update TestIntegration_PhaseInstall_NoMakefile to match new behavior Tested on Gentoo WSL2: 10+ Python packages now build successfully, zero DISTUTILS_USE_PEP517 errors (previously ~39% of failures).
- Add preprocessScript for ${VAR@a} param expansion (fixes app-alternatives/*)
- Fix ln flag parsing: handle -snf, -n, single-target, multi-target
- Fix rm flag parsing: handle --, combined flags like -Rfv
- Add symbolic mode support in fperms (+x, u+rw, a+rx)
- Add cat stdin support (CatWithStdin) for pipe patterns
- Add newins/newman stdin support (newins - filename <<EOF)
- Add get_alternative stub for app-alternatives.eclass
- Add _python_set_impls, _python_export stubs for eclass functions
- Fix python_setup: auto-detect Python for python-any-r1 packages
- Fix ver_test: support 2-arg form (ver_test -ge 40.0 uses $PVR)
- Fix sed: handle -e flags, alternate delimiters, better error reporting
- Fix distfile SRC_URI: pass UseFlags as activeFlags to filter .sig files
- Redirect Go helpers stdout to context stdout for command substitution
- Add shell completion dir helpers (get_bashcompdir, get_zshcompdir, etc.) - Add eautoreconf, eprefixify stubs - Add distutils_enable_tests, kernel_is, linux-info_pkg_setup stubs - Add unpacker and llvm.org eclass no-op stubs - Auto-detect PYTHON_SINGLE_TARGET from PYTHON_COMPAT - Fix PythonSingleR1PkgSetup to not fail on missing target
… resolution - mv: fall back to copy+remove on cross-device link error (fixes pillow) - chmod: support symbolic modes (+x, u+rw, a+rx) via applyMode() - sed: resolve relative file paths against $S workdir (fixes cmake, maturin)
… unpack - CRITICAL: IUSE flags without + prefix now correctly default to false - Apply effective USE flags (make.conf/profiles) to Package.UseFlags - Distfile service always passes non-nil activeFlags (fixes .sig downloads) - Unpack copies non-archive files to WORKDIR instead of erroring - Touch resolves relative paths against $S, creates parent dirs - Econf includes source dir in error message for debugging
Ebuilds can override $S with custom source directories (e.g.,
S="${WORKDIR}/${MY_P}"). Previously, the Go env.S was set once at
initialization and never updated. Now a __grpm_sync_env command runs
after sourcing the ebuild to sync S and WORKDIR back to Go.
- getWorkDir/getSourceDir now check runtimeEnv for live $S value - Added resolvePath() helper for consistent relative path resolution - Updated rm, cp, mv, ln, sed, touch to resolve against $S - Added glob expansion in eapply for unexpanded patterns (*.patch) This fixes packages where Go helpers used incorrect working directory because env.S wasn't updated from bash runtime.
… AT_M4DIR - ver_cut: rewrite with verSplit matching Portage's __ver_split exactly, preserving original separators (. - _) instead of always joining with . - ver_cut/ver_rs: support 1-arg form defaulting to $PV per Portage spec - cmake: add cmakeCheckBuildDir() matching Portage's _cmake_check_build_dir, called in CmakeSrcConfigure/Compile/Install/Test to set BUILD_DIR - eautoreconf: support AT_M4DIR for additional aclocal search paths, set ACLOCAL_FLAGS accordingly
…elpers - Add cd to $S/$WORKDIR before each phase per Portage's phase-functions.sh (src_unpack uses WORKDIR, src_prepare..src_install use S with fallback) - Pass hc.Dir (bash runtime CWD) to helpers via runtimeDir field - Update resolvePath to prefer runtimeDir over static $S - Fix chmod/find to use resolvePath for relative path resolution - Fixes ~15 packages failing with 'sed: can't read' / 'chmod: stat'
…eclass loading - Apply effective USE flags (USE_EXPAND + make.conf + package.use) to packages before building, not just for display output - Set USE_EXPAND variables (PYTHON_TARGETS, etc.) as environment variables alongside USE flags, matching Portage behavior - Pass CFLAGS/CXXFLAGS/LDFLAGS from make.conf to ebuild environment (os.Getenv is empty since GRPM runs standalone) - Use DefaultOptions() for ExecutorOptions to enable dynamic eclass loading (was disabled because zero-value bool is false)
… dynamic EXPORT_FUNCTIONS - Share eclass loader and registry from main interpreter to per-phase interpreters so dynamically loaded eclasses (meson, cmake, etc.) are available when the ebuild is sourced in the combined script - Check dynamic loader's EXPORT_FUNCTIONS state in HasPhaseFunction and RunPhaseFunction, not just the legacy eclassRegistry - Use DefaultOptions() for executor to ensure EnableDynamicEclass=true
… IUSE Filter USE_EXPAND display (PYTHON_TARGETS, etc.) to only show for packages that have at least one matching flag in their original IUSE. Previously, PYTHON_TARGETS was shown on all packages even non-Python ones.
isUseEnabled now checks the bash runtime environment for USE changes made by eclasses (e.g., python_setup modifying USE). isInIuse now checks the runtime IUSE variable for flags added dynamically by eclasses (e.g., python-r1.eclass extends IUSE with python_targets_*).
…direct panic - Move eclass MarkLoaded before Inherit to prevent recursive loading (matches Portage ebuild.sh line 384-385 behavior) - Add multilib/ABI profile defaults (DEFAULT_ABI, MULTILIB_ABIS, etc.) from profiles/arch/amd64/make.defaults - Pass ARCH, KERNEL, ACCEPT_KEYWORDS and other profile vars to env - Transform >& file to &> file in preprocessor (mvdan/sh panics on >& with filename, only supports >& fd for descriptor duplication)
Per Portage phase-helpers.sh:514, econf should check ECONF_SOURCE for configure location (defaulting to current dir). Also read CHOST/CBUILD from interpreter environment instead of only os.Getenv, which is empty in standalone build mode.
…aces
- InheritWithEnv now passes ALL bash variables (not just hardcoded list)
to eclass executor. Fixes distutils-r1 'Unknown DISTUTILS_USE_PEP517='
error — eclasses need variables set before inherit to be visible.
- Add expandBraceArgs() to exec handler to work around mvdan.cc/sh
limitation where brace expansion fails when mixed with variable
expansion (e.g., ${PN}/config.{sub,guess}). Fixes gnuconfig and
similar packages using brace patterns in fperms/doins/etc.
…tted Per Portage spec, ver_cut and ver_rs default to $PV when the version argument is not provided. The eclass executor's handler was requiring all arguments explicitly, causing gnome.org and llvm.org eclasses to fail at source time.
…and display - Remove regex-based S variable parsing (parseSVariable) which could not handle conditional assignments inside if-blocks. Rely on __grpm_sync_env to read the correct S from bash after ebuild conditionals are evaluated. - Fix unpack to always extract into WORKDIR, not S, matching Portage behavior. Previously getWorkDir() returned S which caused tarballs to extract one level too deep (WORKDIR/P/P instead of WORKDIR/P). - Add --nodeps/-O flag to skip dependency resolution for targeted testing. - Add KEYWORDS filtering in --nodeps mode via loadBestAcceptableVersion() to avoid selecting live/9999 ebuilds. - Fix doubled version display (e.g. diffutils-3.12-3.12) by using p.Name instead of the map key which already contained the version.
MULTILIB_ABIS was set to 'amd64 x86' but ABI_X86='64' means only 64-bit is enabled. Building x86 (32-bit) ABI requires a cross-compiler (i686-pc-linux-gnu-gcc) which isn't available on most systems. Removed CHOST_x86 and LIBDIR_x86 entries accordingly.
Embed eclass content directly in the combined bash script instead of
relying on separate interpreter runs. This fixes the multilib build
chain (configure/compile/install) for packages like zlib and diffutils.
Key changes:
- Set EAPI before eclass content to pass EAPI guards (case 7|8)
- Replace unsupported declare -f/p with __grpm_has_func/__grpm_has_var
- Override multibuild_foreach_variant without process substitution
- Use ${S} as BUILD_DIR fallback (matching original eclass behavior)
- Initialize MULTILIB_CHOST_TOOLS/WRAPPED_HEADERS as empty arrays
- Scan eclasses for EXPORT_FUNCTIONS and register in Go registry
- Add PATH to interpreter env, filtering Windows /mnt/ paths (WSL)
- Add DefaultOpenHandler for source/dot file support
… python eclasses - Split ebuild at inherit line so pre-inherit vars (PYTHON_COMPAT, DISTUTILS_USE_PEP517) are available when eclass top-level code runs - Replace type -P with command -v (type -P returns NOT IMPLEMENTED in mvdan.cc/sh) - Set BASH_VERSINFO=(4...) so eclasses use declare -p path instead of bash 5+ var@a - Fix __grpm_has_var to write declare -p output to hc.Stdout for command substitution - Add unit tests for splitEbuildAtInherit, scanExportFunctions, preprocessing
Strip phase function bodies before metadata extraction to avoid
mvdan.cc/sh parse errors from unsupported bash features (brace
expansion in variable names, ${var@a} parameter attributes).
Add 3-layer defense for signature file filtering:
- extractRawSrcURI: regex fallback for SRC_URI/SRC_URI+= extraction
- filterSignatureFiles: safety net filtering .sig/.asc/.sign files
- Remove silent manifest fallbacks in executor (align with Portage)
Redirect eclass sourcing stdout to prevent usev/echo pollution.
Verified on real Gentoo: findutils, sed, make no longer download .sig.
Add unpackTarLz using external decompressor (xz/plzip/pdlzip/lzip), matching Portage unpacker.eclass strategy. Fixes dev-build/make build. Remove unused parseSVariable, fix errcheck on fmt.Fprintf. Add //nolint:gocyclo on 7 inherently complex functions (command dispatchers, phase orchestrators, flag parsers). golangci-lint: 0 issues.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Major hardening of the bash interpreter (mvdan.cc/sh) for reliable ebuild metadata extraction, distfile resolution, and source building.
verify-sig? ( *.sig )in SRC_URI no longer download.sig/.ascfiles when the flag is disabled (3-layer defense: eval → regex fallback → manifest filter)${var@a}).tar.lzunpack support: Uses external decompressor (xz/plzip/lzip), matching Portageunpacker.eclassstrategyVerified on Real Gentoo (WSL2)
sys-apps/findutils— no.sigdownloadsys-apps/sed— no.sigdownloaddev-build/make—.tar.lzunpacks correctly, no.sigdownloadTest plan
go test -race ./internal/...— all passgolangci-lint run— 0 issuesgofmt -l internal/— clean