Skip to content

Automate IAM template sync from upstream ingestion sources#222

Merged
gastonyelmini merged 6 commits into
mainfrom
sync-permissions-from-prod
May 8, 2026
Merged

Automate IAM template sync from upstream ingestion sources#222
gastonyelmini merged 6 commits into
mainfrom
sync-permissions-from-prod

Conversation

@gastonyelmini
Copy link
Copy Markdown
Contributor

Add a script + GitHub Actions workflow that regenerates all CloudFormation, Markdown, and Terraform files under cloudformation/ from the public ingestion-sources endpoint. The workflow runs every 8 hours and commits any changes automatically; on PRs it validates there is no drift.

  • scripts/lib/source.ts: parse per-step authorization (permissions + roles)
  • scripts/lib/policy.ts: partition actions into size-bounded policy documents
  • scripts/lib/templates.ts: build detailed, standard, and govcloud variants
  • scripts/lib/render.ts: render to JSON/Markdown/Terraform with prettier
  • scripts/lib/sync.ts: orchestrator returning FileWrite pairs
  • scripts/sync-permissions.ts: CLI (--apply / --check modes)
  • scripts/tests: 29 unit tests with a fixture payload
  • .github/workflows/sync-permissions.yml: scheduled + PR validation workflow
  • Regenerate all 9 templates from prod (179 exact actions, 70 wildcard actions)

Add a script + GitHub Actions workflow that regenerates all CloudFormation,
Markdown, and Terraform files under cloudformation/ from the public
ingestion-sources endpoint. The workflow runs every 8 hours and commits any
changes automatically; on PRs it validates there is no drift.

- scripts/lib/source.ts: parse per-step authorization (permissions + roles)
- scripts/lib/policy.ts: partition actions into size-bounded policy documents
- scripts/lib/templates.ts: build detailed, standard, and govcloud variants
- scripts/lib/render.ts: render to JSON/Markdown/Terraform with prettier
- scripts/lib/sync.ts: orchestrator returning FileWrite pairs
- scripts/sync-permissions.ts: CLI (--apply / --check modes)
- scripts/__tests__: 29 unit tests with a fixture payload
- .github/workflows/sync-permissions.yml: scheduled + PR validation workflow
- Regenerate all 9 templates from prod (179 exact actions, 70 wildcard actions)
@gastonyelmini gastonyelmini requested a review from a team as a code owner April 30, 2026 22:14
Copy link
Copy Markdown
Contributor

@VDubber VDubber left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review — Automate IAM template sync

Findings

# Severity Category File(s) Finding Suggestion
1 High Security / Supply chain .github/workflows/sync-permissions.yml:14,4 runs-on: scaleset-jupiterone-infra-arm64 (self-hosted) + pull_request: trigger on a public repo. Fork PRs run arbitrary code on the self-hosted runner (postinstall hooks, modified scripts/sync-permissions.ts, etc.). Even though secrets.AUTO_GITHUB_PAT_TOKEN is not passed to fork runs, the runner host itself is exposed. Use runs-on: ubuntu-latest for the PR-validation path; keep self-hosted only for schedule/workflow_dispatch. Or gate self-hosted with if: github.event.pull_request.head.repo.full_name == github.repository.
2 High Bug (rendered output) scripts/lib/render.ts:168-172cloudformation/iam-cloudformation-govcloud/terraform.tf:1-3 GovCloud branch emits output "aws_iam_user_jupiterone_access_user" { value = "${aws_iam_role.jupiterone.arn}" }, but no aws_iam_role.jupiterone resource exists in that file (only aws_iam_user.jupiterone_access_user). terraform validate/plan fails with "Reference to undeclared resource". Pre-existing on main but now permanently re-emitted by codegen. Change to value = "${aws_iam_user.jupiterone_access_user.arn}".
3 Medium Security / Supply chain .github/workflows/sync-permissions.yml:54 Third-party action EndBug/add-and-commit@v9 pinned to mutable tag. If hijacked, it commits arbitrary content to main under the bot identity with contents: write. Pin to a full commit SHA: EndBug/add-and-commit@<sha> # v9.x.x.
4 Medium CI reliability .github/workflows/sync-permissions.yml:42, scripts/sync-permissions.ts:91 PR validation calls pnpm sync:permissions:check which fetch()es https://api.us.jupiterone.io/... live. Endpoint outage / transient drift breaks every open PR's checks, including unrelated PRs. For PR mode, point at a checked-in fixture or last-known-good snapshot via INGESTION_SOURCES_FILE. Reserve live fetch for schedule/dispatch.
5 Medium CI scope .github/workflows/sync-permissions.yml:5 pull_request: runs on every PR — including doc/typo PRs that don't touch templates. Wastes CI and amplifies the runner-exposure risk in #1. Add paths: ['cloudformation/**', 'scripts/**', '.github/workflows/sync-permissions.yml', 'package.json', 'pnpm-lock.yaml'].
6 Medium Security / Trust boundary workflow + script Generator faithfully transcribes upstream JSON into IAM policies and auto-commits to main every 8h. Compromise/misconfig of the upstream endpoint → expanded IAM permissions deployed to consumers without human review. Add a sanity gate before applyWrites: cap on max actions / policy docs / unexpected services; require diff above a threshold to be raised as a PR rather than auto-committed.
7 Low Code quality scripts/lib/render.ts:132-135 JSON.stringify(JUPITERONE_ACCOUNT_ARNS).replace(/","/g, '","') — replacement string equals the match. Dead code. Drop the .replace(...) or implement the intended transform.
8 Low Code quality scripts/lib/render.ts:107-115 statementResourceListEqualsRoot declared, immediately void'd as "exported nowhere". Dead. Remove.
9 Low Consistency scripts/lib/render.ts:30-37 vs scripts/lib/templates.ts:46-57 Terraform output embeds 6 J1 account ARNs; CloudFormation JupiterOneAwsAccountArns.Default lists only 1 (612791702201). Possibly intentional (CF default is an example), but the lists can silently drift. Co-locate the canonical list and reference it from both renderers, or document the difference.
10 Low Scheduled commit hygiene .github/workflows/sync-permissions.yml:60 add: "['cloudformation/']" only. If the script ever generates files outside that tree, drift won't be committed but --check will fail next run. Widen the add path to mirror what syncTemplates writes, or assert that all generated paths fall under the configured add list.
11 Info package.json pnpm-lock.yaml not visible in the diff but pnpm install --frozen-lockfile is required. Confirm the lockfile is committed alongside the new tsx/prettier versions.
12 Info Test coverage scripts/__tests__/* No tests exercise the loadSource fetch path or the --check exit-code behavior. Add a fixture-driven CLI smoke test (already easy via --input).

Note on #2: Verified the same broken reference exists on main, so this PR is not regressing — but it makes the bug permanent across regenerations. Worth fixing while the renderer is being touched.

Scores

Dimension Score Rationale
Security risk 3 Self-hosted runner exposed to fork PRs on a public repo (#1) is the dominant concern; mitigated by ephemeral scaleset pods and absence of secrets on fork PRs, but still real. Mutable third-party action tag (#3) and auto-commit trust in upstream (#6) compound.
Data volume impact 1 No pipeline data effect. One JSON fetch every 8h, 9 files written.
Performance risk 1 Codegen is small and bounded; partition logic is O(n) over actions.

Verdict

Request changes#1 (self-hosted runner accepting fork-PR code on a public repo) and #2 (broken GovCloud Terraform output) should be addressed before merge. #3#6 strongly recommended. Rest are nits / follow-ups.

Security / CI:
- Use ubuntu-latest runner for PR validation so fork PRs cannot run code
  on the self-hosted runner; keep self-hosted for schedule/dispatch only
- Pin EndBug/add-and-commit to commit SHA (v9.1.4) instead of mutable tag
- PR validation now uses checked-in fixture instead of live fetch to avoid
  endpoint outage breaking unrelated PRs
- Add paths filter to pull_request trigger to skip unrelated PRs

Bug fix:
- Fix GovCloud terraform output: was referencing aws_iam_role.jupiterone.arn
  (undeclared resource) instead of aws_iam_user.jupiterone_access_user.arn

Code quality:
- Remove dead statementResourceListEqualsRoot function and its void() call
- Remove no-op .replace() in trustArns serialization
- Add sanity gate in syncTemplates: rejects payloads with 0 or >600 actions
  to catch upstream source misconfigurations before they overwrite templates
The fixture check compares committed templates (generated from prod) against
a synthetic test fixture — they will never match. PR validation needs the
live endpoint to detect actual drift.
Comment thread .github/workflows/sync-permissions.yml Outdated
The upstream payload was extended with an aggregated authorization block at
the root of the document (in addition to per-step blocks under
childIngestionSourcesMetadata). Without reading it the parser only saw a
fraction of the data: 183 actions instead of the full set.

- source.ts: include payload.authorization in collectSources output
- sync.ts: bump sanity-check ceiling from 600 to 1500 actions
- regenerate all 9 templates: detailed now has 561 exact actions
  (vs. 509 in main) split across 3 partitioned managed policies
@gastonyelmini gastonyelmini merged commit 5d8bcd7 into main May 8, 2026
7 checks passed
@gastonyelmini gastonyelmini deleted the sync-permissions-from-prod branch May 8, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants