Skip to content

fix(measurement): allow rewrite W&B records and project dirs#221

Open
lipikaramaswamy wants to merge 3 commits into
mainfrom
lramaswamy/bugfix/220-rewrite-wandb-seal
Open

fix(measurement): allow rewrite W&B records and project dirs#221
lipikaramaswamy wants to merge 3 commits into
mainfrom
lramaswamy/bugfix/220-rewrite-wandb-seal

Conversation

@lipikaramaswamy

@lipikaramaswamy lipikaramaswamy commented Jul 13, 2026

Copy link
Copy Markdown
Collaborator

Related Issue

Fixes #220.

Plan Document

No plan required: narrow bug fix for W&B measurement schema/path validation.

Summary

  • Allows W&B measurement ingress to accept rewrite-mode run records when replacement metadata is null or omitted.
  • Keeps active-mode validation strict: replace runs still require replace metadata, and rewrite runs still require rewrite metadata.
  • Allows measurement completion seals and W&B output directory validation to pass through root-owned, group-writable project-directory ancestors while still rejecting unsafe world-writable or non-root-owned writable ancestors.
  • Adds regression coverage for rewrite run metadata, active-mode metadata rejection, completion-seal path validation, and W&B output path validation.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update
  • Refactoring
  • CI, release, or contributor workflow update

Contributor Checklist

  • PR title follows Conventional Commits, for example fix: handle empty entity list
  • Related issue is linked, or a maintainer-owned no-issue reason is documented above
  • For non-trivial changes, a plan document is linked above, or the no-plan reason is documented above
  • Public API impact checked; skills/anonymizer/SKILL.md updated if needed
  • No real PII added to tests, docs, notebooks, fixtures, or artifacts
  • No API keys, service tokens, private keys, credentials, or real endpoint secrets added

Validation

  • Commands run:
    • uv run pytest tests/tools/test_measurement_ingress_completion.py -k "rewrite or active_mode or root_owned"
    • uv run pytest tests/tools/test_measurement_wandb_logging.py -k "root_owned or isolates_routing"
    • TMPDIR=/tmp uv run pytest tests/tools/test_measurement_ingress_completion.py tests/tools/test_measurement_wandb_logging.py
    • make check
  • Skipped checks or known failures:
    • make format was not run separately because make check includes the read-only format-check, which passed with 179 files already formatted.
    • make check completed successfully; its advisory typecheck step reported existing diagnostics and is ignored by the Makefile.

Documentation and Artifacts

  • Docs updated, or not needed
  • If docs changed: make docs-build passes locally
  • If tutorial sources changed: notebooks regenerated with make convert-notebooks
  • If e2e, benchmark, or model-provider behavior changed: relevant validation is listed above

Signed-off-by: lipikaramaswamy <lramaswamy@nvidia.com>
@lipikaramaswamy lipikaramaswamy marked this pull request as ready for review July 13, 2026 20:50
@lipikaramaswamy lipikaramaswamy requested review from a team as code owners July 13, 2026 20:50
@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR updates W&B measurement validation and related path handling. The main changes are:

  • Allows rewrite run records without replacement metadata.
  • Keeps active-mode metadata required for replace and rewrite runs.
  • Allows root-owned group-writable intermediate directories for W&B output and completion seals.
  • Adds tests for the schema and directory validation paths.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.

Important Files Changed

Filename Overview
tools/measurement/measurement_tools/wandb_ingress.py Makes inactive run metadata optional while still requiring metadata for the selected mode.
tools/measurement/measurement_tools/wandb_completion.py Updates completion-seal directory validation to permit root-owned group-writable intermediate directories.
tools/measurement/measurement_tools/wandb_setup.py Updates W&B output directory validation to match the completion-seal path rules.
tests/tools/test_measurement_ingress_completion.py Adds tests for rewrite metadata validation and completion-seal directory handling.
tests/tools/test_measurement_wandb_logging.py Adds a W&B output directory validation test for root-owned group-writable parents.

Reviews (3): Last reviewed commit: "fix(measurement): allow wandb output und..." | Re-trigger Greptile

@binaryaaron binaryaaron left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agent (review-pr): Approved with two non-blocking test suggestions

The mode-aware validation looks correct, and I verified both positive and negative cases locally.

Two non-blocking suggestions:

  • Add regression tests showing that replace mode still rejects missing replace metadata and rewrite mode rejects missing rewrite metadata. That validator is what prevents the nullable fields from widening the accepted schema, and the current positive tests would still pass if it were removed.
  • Use moderate or low instead of the unsupported medium risk tolerance in the rewrite fixture.

Otherwise, this looks ready.

@lipikaramaswamy lipikaramaswamy changed the title fix(measurement): allow rewrite run metadata without replace config fix(measurement): allow rewrite W&B records and project dirs Jul 14, 2026
@lipikaramaswamy

Copy link
Copy Markdown
Collaborator Author

Thanks for the review @binaryaaron. Since your approval, I pushed follow-ups to address the test coverage suggestions and another shared cluster storage / W&B path-validation issue.

Changes since your review:

  • Added the completion-seal path validation fix for root-owned, group-writable project-directory ancestors.
  • Added the matching W&B output directory validation fix for the same project-directory layout.
  • Added regression coverage for completion-seal path validation and W&B output path validation.
  • Added negative regression tests showing replace mode still rejects missing replace metadata and rewrite mode still rejects missing rewrite metadata.
  • Updated the rewrite fixture risk tolerance from medium to low.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

W&B completion seal rejects rewrite run measurements without replace metadata

2 participants