fix: harden package name resolution + add pyp2spec dep tooling - #3014
Merged
Merged
Conversation
Closes #2988 (items 1 and 3; item 2 dropped as too costly on bump PRs). - find_package.py: expose --resolve-dir <name> CLI flag so callers can resolve a raw PyPI name to its on-disk spec directory suffix or exit 1 on failure, instead of silently succeeding with a wrong path. - update_packages.sh: use find_package.py --resolve-dir to build the spec path (handles separator/prefix mismatches like ruamel.yaml → python-ruamel-yaml); add explicit error guards around rpmspec so a missing or malformed spec fails loudly instead of silently exiting 0. - build_matrix.py: apply PEP-503 canonicalization (ruamel.yaml → ruamel-yaml) before emitting the matrix so branch names and spec paths are always clean. Documents the intentional bypass of full resolve_package_dir() resolution (scope is limited to automation/requirements.txt packages by design). - automation/dep_diff.py: standalone tool (for agent/manual use) that compares a spec's mandatory Requires against the PyPI JSON API for a given version, outputting a markdown diff table of added/removed deps. - automation/update_deps.py: standalone tool that rewrites the Requires: python3.12-* library entries in a spec in-place based on PyPI mandatory runtime deps, preserving toolchain BuildRequires. - scaffold_package.sh: generate a new RPM spec for a PyPI package using pyp2spec -a, set up the packages/ directory, fetch the tarball via spectool, and add via git annex. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Odilhao
force-pushed
the
fix/harden-package-resolution
branch
from
August 31, 2026 21:15
b13da8d to
ed242f4
Compare
zjhuntin
reviewed
Aug 31, 2026
zjhuntin
approved these changes
Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2988 (items 1 and 3; item 2 dropped — triggering test CI on
packages/**changes is too costly at bump-wave scale).Changes
A1 — Silent failure hardening (
find_package.py,update_packages.sh)find_package.pynow exposes--resolve-dir <name>: resolves a raw PyPI name to its on-disk spec directory suffix, or exits 1 with a clear error.update_packages.shuses this instead of the naivepackages/python-$pkg/template — so separator/prefix mismatches (e.g.ruamel.yaml→python-ruamel-yaml) are handled correctly, and a missing or malformed spec fails loudly instead of silently exiting 0.A3 —
build_matrix.pynormalization + design commentApplies PEP-503 canonicalization to matrix entries before emitting them (
ruamel.yaml==0.19.1→ruamel-yaml 0.19.1), so branch names and spec paths are always clean. Adds a comment documenting that the bypass of fullresolve_package_dir()is intentional — this workflow is scoped toautomation/requirements.txtpackages only.B — Dependency tooling (
automation/dep_diff.py,automation/update_deps.py,scaffold_package.sh)Three new tools using the PyPI JSON API (stdlib
urllibonly, no extra installs):automation/dep_diff.py— compares a spec'sRequires:against the new version's declared mandatory runtime deps from PyPI, outputs a markdown table. Runs as a CI step inupdate-pulp-packages.ymland injects the diff into the PR body.automation/update_deps.py— rewrites theRequires: python3.12-*library entries in a spec in-place based on PyPI deps, preserving toolchainBuildRequires. Runs insideupdate_packages.shas part of each bump.scaffold_package.sh— scaffolds a newpackages/python-<name>/directory for a PyPI package usingpyp2spec -a, fetches the tarball, and adds it viagit annex add.Test
Verified in CentOS Stream 10 container against PRs #3001 and #3002 (
ruamel-yaml/ruamel-yaml-clib):--resolve-dir ruamel.yaml→python-ruamel-yaml✓--resolve-diron unknown package → exit 1 ✓build_matrix.pywithruamel.yaml==0.19.1→ruamel-yaml 0.19.1✓dep_diff.pycorrectly flagspython3.12-ruamel-yaml-clibas removed (became optional-only in 0.19.1) ✓update_deps.pycleanly drops the staleRequires: python%{python3_pkgversion}-ruamel-yaml-clib >= 0.2.6✓🤖 Generated with Claude Code