Skip to content

EffortlessMetrics/ripr

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,857 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

ripr logo

ripr

CI Codecov

ripr open gaps ripr+ test-efficiency gaps

GitHub release crates.io downloads docs.rs

VS Marketplace installs Open VSX downloads

MSRV License: MIT OR Apache-2.0

Static mutation-exposure analysis for Rust and Cargo workspaces.

ripr points reviewers and coding agents at changed Rust behavior whose tests appear to reach the code but miss the check that would catch the change.

It reads a PR diff, finds changed branches, return values, error paths, fields, side effects, and output text, then turns the useful cases into repair cards: what changed, why the current proof is weak, where to add the focused test or output golden, and which command verifies movement.

The first successful use should feel small:

one PR
-> one repairable gap
-> one focused test or output proof
-> one before/after receipt

Start with the front-door packet:

ripr first-pr --root . --base origin/main --head HEAD

Use it while a PR is still moving:

  • in generated CI, as an advisory first-screen summary and artifact packet;
  • in VS Code, as diagnostics, hovers, and repair packets for the current work;
  • from the CLI, as a before/after receipt for one improved Rust gap;
  • with coding agents, as a bounded packet that names the repair route and stop conditions.

ripr is not coverage and does not run mutants. Coverage asks whether code executed. Mutation testing asks whether tests fail against a concrete mutant. ripr asks the cheaper draft-time question:

Does this changed behavior appear exposed to a meaningful test assertion?

The mature repair loop is Rust/Cargo: find one named gap, add one focused proof outside ripr, and keep the receipt. TypeScript remains an opt-in preview surface. Python now has a scoped usable alpha repair-routing loop for selected pytest/unittest workflows: direct weak Python findings can emit repair cards, verify commands, bounded agent packets, and before/after receipts. Broader Python static facts remain preview/advisory and do not run imports, execute tests, generate tests, or claim mutation adequacy. See Support tiers for what is usable, stable, preview, advisory, blocked, or unsupported.

The Problem

Coverage can tell you that code executed.

Mutation testing can tell you whether a concrete mutated version of the code was caught by the test suite.

During everyday development, teams often need a cheaper question answered earlier:

This behavior changed. Do the nearby tests actually check the thing that changed?

That is what ripr is for.

It is designed to find weak or missing test discriminators, such as:

  • a boundary change without equality-boundary assertions
  • an error-path change checked only with is_err()
  • a return-value change covered only by a smoke assertion
  • a field construction change without field or object assertions
  • a side effect without a mock, event, state, persistence, or metric oracle

What ripr Does

ripr analyzes a diff and classifies static exposure evidence for changed behavior.

It looks for:

Reachability:
  can a related test reach the changed code?

Infection:
  could the changed expression alter local state or control?

Propagation:
  could that altered state reach a visible boundary?

Revealability:
  does a test oracle appear to observe the affected behavior?

Then it reports whether the current tests appear to expose the changed behavior to a meaningful discriminator.

What ripr Does Not Do

ripr does not run mutants.

It does not report killed or survived, prove test adequacy, replace coverage, or replace real mutation testing.

Use ripr for fast, static, draft-time oracle-gap feedback. Use a real mutation runner, such as cargo-mutants, when the change is ready for execution-backed confirmation.

Where ripr Fits

coverage:
  did this code execute?

ripr:
  does changed behavior appear exposed to a meaningful test oracle?

mutation testing:
  did tests fail when a concrete mutant was run?

ripr shifts the mutation signal left: it answers the same weak-oracle question mutation testing answers, but at draft time and without running mutants. Mutation testing stays as the slower runtime backstop when the change is ready for execution-backed confirmation; coverage stays as the execution-surface signal.

Coverage

The Codecov badge above indicates execution-surface evidence only: whether code paths executed during testing. Coverage does not prove test adequacy, correctness, safety, or completeness. See Coverage for what the coverage signal measures and does not claim.

Quick Start

Most users start from the surface they already use. The CLI is the shared engine behind each path, but it is not the required first interface for editor or CI users.

User type First action Main doc
VS Code user Install EffortlessMetrics.ripr, open a Rust workspace, then use the status bar, Problems, hover, and seam actions. Quickstart
CI user Generate or copy the advisory GitHub workflow. Quickstart
CLI user Run ripr pilot --root . and add one focused test for the top seam. Quickstart
Agent or reviewer Start from ripr agent status --root . or a selected-seam workflow packet. LLM operator guide

For the full first-hour path, including troubleshooting and known limits, read Quickstart. For a single real PR adoption proof, use the first successful PR workflow.

Before rollout, check Support tiers for the plain-language map of what is usable, usable alpha, preview, scaffold, blocked, or deferred. It also keeps the public-badge boundary separate from PR-local evidence and gate authority.

VS Code

Install EffortlessMetrics.ripr from VS Code Marketplace or Open VSX, then open a Rust/Cargo workspace. Normal editor installs do not require cargo install ripr; the extension resolves the server from configuration, bundled or cached assets, GitHub Releases, or PATH.

Use the ripr status bar item or ripr: Show Status first. Then inspect RIPR Problems diagnostics, hover evidence, Write targeted test actions, agent handoff commands, and Open Best Related Test.

CI

ripr init --ci github

The generated workflow is advisory by default. It runs ripr pilot, writes a RIPR advisory summary, uploads pilot/workflow/agent/report/review artifacts, writes badge JSON, can optionally upload SARIF, and does not run mutation tests.

CLI

Install from crates.io:

cargo install ripr

Links:

For local development from this repository:

cargo install --path crates/ripr

ripr targets Rust 2024 and requires Rust 1.95 or newer.

Run a zero-config pilot packet:

ripr pilot --root .

Read target/ripr/pilot/pilot-summary.md, add one focused test for the top missing discriminator, then compare before and after snapshots:

ripr check --root . --mode ready --format repo-exposure-json > target/ripr/pilot/after.repo-exposure.json
ripr outcome --before target/ripr/pilot/repo-exposure.json --after target/ripr/pilot/after.repo-exposure.json

For agent handoff or review automation, write the matching verification packet and focused receipt:

mkdir -p target/ripr/agent
ripr agent verify --root . --before target/ripr/pilot/repo-exposure.json --after target/ripr/pilot/after.repo-exposure.json --json > target/ripr/agent/agent-verify.json
ripr agent receipt --root . --verify-json target/ripr/agent/agent-verify.json --seam-id <seam_id> --json --out target/ripr/agent/agent-receipt.json

For local agent handoff state, start with:

ripr agent status --root .
ripr agent start --root . --seam-id <seam_id> --out target/ripr/workflow
ripr agent review-summary --root .

If the first run behaves unexpectedly:

ripr doctor

ripr.toml is optional. ripr init materializes repo-local policy when a team wants to review, version, and tune it; it does not unlock basic usefulness.

Example Finding

WARNING src/pricing.rs:88

Static exposure: weakly_exposed
Probe: predicate

Changed behavior:
  if amount >= discount_threshold {

Evidence:
  Reachability:     related tests found
  Infection:        changed predicate can alter branch behavior
  Propagation:      branch appears to influence returned total
  Revealability:    tests assert returned values, but no equality-boundary case was found

Gap:
  No detected test input for amount == discount_threshold.

Recommended next step:
  Add below, equal, and above-threshold tests with exact assertions.

The wording is intentionally conservative. Static analysis can identify evidence and gaps; it should not claim real mutation outcomes.

Classifications

Classification Meaning
exposed Static evidence suggests a complete RIPR path to a strong oracle.
weakly_exposed A path exists, but infection or discrimination appears weak.
reachable_unrevealed Related tests appear reachable, but no meaningful oracle was found.
no_static_path No static test path was found for the changed owner.
infection_unknown Reachability exists, but input or fixture evidence is opaque.
propagation_unknown The changed behavior crosses an opaque propagation boundary.
static_unknown Static analysis cannot make a credible judgment.

ripr should not use mutation-runtime outcome language such as killed or survived unless explicit real mutation data is being reported in a calibration context.

Current Scope

The current alpha line is intentionally narrow:

  • one published package: ripr
  • one CLI binary: ripr
  • one shared analysis engine
  • syntax-backed unified diff analysis
  • parser-backed Rust function, test, assertion, owner, and probe facts with lexical fallback
  • human, JSON, and GitHub outputs
  • experimental LSP sidecar

The package is not split into ripr-core, ripr-cli, or ripr-lsp. Public crate boundaries can be added later if external consumers need them.

Current Capability Snapshot

ripr is currently strongest as a fast, syntax-backed draft signal with first-class repo seam evidence.

Current capabilities:

Capability Current state Next checkpoint
Distribution 0.8.0 is the current public release line for the Rust crate, GitHub Release server assets, VS Code/Open VSX extension metadata, generated CI workflow artifacts, and the Rust 1.95 MSRV. ripr-swarm remains the development trunk; source ripr remains the release/distribution authority. Promotion and release-maintenance proof.
Diff analysis Evidence-first Voice A findings with syntax-backed changed-line probes, probe-relative oracle strength, local flow sinks, observed/missing activation values, and explicit stop reasons. Maintenance; no active analyzer-refactor lane.
Repo seam inventory First-class RepoSeam model with deterministic seam IDs, cached seam fact layers, test-grip evidence across the five RIPR stages, and 11-class SeamGripClass classification. Maintenance; no active analyzer-refactor lane.
Test discovery Parser-backed test and assertion facts with exact, broad, relational, snapshot, mock, smoke, custom-helper, side-effect observer, and unknown oracle kinds; per-test efficiency ledger with smoke/broad/disconnected/opaque/circular/likely-vacuous reasons and duplicate-discriminator groups. Maintenance; no active analyzer-refactor lane.
Output Human, JSON, context, and GitHub formats render evidence-first findings with stop reasons; ripr pilot writes a zero-config first-run packet; ripr outcome compares before/after repo exposure snapshots; repo exposure report and v2 agent seam packets render classified seam evidence; public ripr and ripr+ Shields badges count unresolved actionable static repair gaps while diff badge artifacts remain finding-exposure based. Output contract maintenance.
LSP Experimental tower-lsp-server sidecar with evidence-aware Finding diagnostics, related-test links, hovers, server-side context packets, repo seam diagnostics + hover, and seam code actions for copying packets/assertions and opening related tests. Saved-workspace diagnostics remain advisory; unsaved-buffer overlays are not default behavior. Editor contract maintenance.
Agent context Compact context packet plus per-seam write_targeted_test and inspect_static_limitation packets carrying recommended test placement, nearest tests to imitate, candidate values, missing discriminators, patterns to imitate/avoid, and assertion templates. GapRecord-backed agent packets also carry allowed files, forbidden files, conflict groups, receipt command/status, and preview-language guardrails when supplied. ripr agent start --root . --seam-id <id> --out target/ripr/workflow writes a source-edit-free workflow packet, ripr agent status --root . --json reports local LLM loop artifact state and the next command without rerunning analysis, ripr agent receipt emits provenance plus bounded next-action guidance, and ripr agent review-summary --root . joins existing loop artifacts into compact review Markdown or schema 0.1 JSON. Agent loop maintenance.
RIPR swarm repair loop cargo xtask ripr-swarm plan --top <n> ranks actionable canonical gap packets into swarm-ready, blocked, missing-context, static-limitation, and high-confidence buckets; cargo xtask ripr-swarm attempt --packet <id> --dry-run prints one bounded repair context; and RIPR swarm human workflow explains packet selection, repair, verify, receipt, and outcome handling. The 0.7 dogfood receipt records live audit timeout handling with fail-closed packet suppression, fixture-backed packet planning, dry-run attempts, verify/receipt separation, and outcome joins. The 0.7 readiness closeout keeps the loop advisory/dry-run-first and routes release proof back to source ripr. Recent Lane 1 route evidence includes analysis/assertion-target-return-value-owner-call-tracing for a non-actionable static-limitation subroute. These surfaces do not edit files, run tests, call providers, generate tests, create receipts, run mutation testing, change PR/CI rendering, change LSP/editor behavior, change gates, or change public badges. Swarm repair-loop maintenance.
First useful action ripr first-action writes advisory first-useful-action.{json,md} from explicit PR guidance, assistant proof, PR evidence ledger, baseline delta, receipt, optional gate, optional coverage/grip frontier, and editor context inputs without hidden analysis, source edits, generated tests, provider calls, mutation execution, or default CI blocking; generated CI projects the report as advisory summary/artifact content, and VS Code status/Show Status can project an existing workspace-matched report without new diagnostics. docs/first-useful-action-workflow
Gap decision ledger ripr reports gap-ledger --records ... renders explicit GapRecord input into advisory JSON and Markdown so repairability, projection eligibility, safe gate predicates, missing artifacts, preview ineligibility, and receipt movement share one decision vocabulary before downstream surfaces consume them. --check-output can also derive PR-local Python preview repair records from actionable python_repair_card findings without rerunning analysis. report/first-useful-action-gap-record
Python repair routing Scoped usable alpha loop for selected pytest/unittest findings: repair cards, verify commands, bounded agent packets, queue/ingest handling, real-repo eval receipts, and before/after outcome receipts. Broader Python static facts and static limits remain preview/advisory. dogfood/python-stability-evals-v1
Python preview static facts Opt-in/advisory Python facts cover project/test discovery, owners, probes, oracle shapes, static limits, related-test hints, and stable canonical gap IDs. Static-limit findings remain limitation evidence, not repair cards or agent packets. Maintenance; no promotion of broader preview facts.
Repository config Repo-root ripr.toml can set analysis mode, oracle policy, severity mapping, suppressions path, report related-test caps, and LSP seam-diagnostic defaults. Explicit CLI flags and LSP initialization options still win. Policy feedback after adoption.
SARIF and CI policy ripr check --format sarif emits diff-scoped Finding SARIF and --format repo-sarif emits repo seam SARIF with configured severity, suppression metadata, stable rule IDs, and stable fingerprints. ripr init --ci github generates a non-blocking GitHub Actions report workflow with pilot/report artifacts, repo badge JSON, and optional SARIF rendering/upload; cargo xtask sarif-policy compares current SARIF to a baseline only when explicitly requested. Advisory policy feedback after adoption.
Calibration Advisory ripr calibrate cargo-mutants and repo-local cargo xtask mutation-calibration join imported cargo-mutants runtime data to static seam evidence by seam_id or unambiguous file/line; ambiguous file/line candidates stay unassigned. fixtures/CALIBRATION_CORPUS.md maps current fixtures to controlled calibration scenarios, fixtures/EXAMPLE_CORPUS.md links the checked boundary-gap calibration sample into the operator loop, and fixtures/boundary_gap/calibration/runtime-fixtures-v1/ pins the main static/runtime agreement buckets. Maintenance; runtime mutation language stays inside calibration/runtime reports.

Current product surfaces:

  • Editor: diagnostics, evidence-rich hovers, and intent-titled focused-test actions in VS Code and Open VSX. Status bar projects an existing first-useful-action report without rerunning analysis.
  • CLI: ripr pilot, ripr outcome, ripr agent status / start / verify / receipt / review-summary, and ripr review-comments for local review and agent handoff.
  • CI: advisory PR summaries with PR review front panel, report packet index, and uploaded pilot/workflow/agent/report/review artifacts. The default generated workflow is non-blocking.
  • Governance: optional ripr.toml policy, reviewed baselines, suppressions, RIPR Zero status, calibration imports, and an opt-in calibrated gate for teams that want explicit policy control.

The full build history of these surfaces (Campaigns 5A through 26) lives in Implementation campaigns. Deeper capability state lives in Capability matrix and Metrics.

Editor Extension

The VS Code extension starts ripr lsp --stdio and can resolve the server from:

1. explicit ripr.server.path
2. bundled server binary, if present
3. downloaded cached server binary
4. verified first-run GitHub Release download
5. ripr on PATH
6. actionable error

Normal editor install should not require cargo install ripr. The Cargo install path remains available for offline, pinned, or controlled environments. The v0.8.0 public release line includes the server manifest, per-target server archives, checksums, and VSIX needed for this default path. Future public release claims must refresh the same asset family during release execution.

See:

For Contributors

Most contributors should use the repo automation instead of remembering the gate order manually.

cargo xtask shape
cargo xtask fix-pr
cargo xtask check-pr
cargo xtask pr-summary

Useful evidence commands:

cargo xtask fixtures
cargo xtask goldens check
cargo xtask golden-drift
cargo xtask test-oracle-report
cargo xtask evidence-health
cargo xtask dogfood
cargo xtask reports index
cargo xtask receipts
cargo xtask receipts check
cargo xtask suggested-fixes
cargo xtask check-allow-attributes
cargo xtask check-local-context
cargo xtask metrics

A good ripr PR is scoped by production risk, not line count:

small production delta
complete evidence package
clear spec-test-code-output-metric trail

Large fixture, golden, spec, docs, metrics, or traceability diffs are welcome when they make one production behavior reviewable.

See:

Supporting Docs

Start here:

Need Doc
Choose the first-hour path by user type Quickstart
Map plain language to the internal model Terminology
Understand the model Static exposure model
Understand JSON/context output Output schema
Turn seam evidence into a test Targeted test workflow
Adopt a baseline debt ledger Baseline ledger workflow
Understand easy-start defaults Defaults-first adoption spec
See current product direction Roadmap
Understand source-of-truth rails Repo tracking model and context system
See active campaigns Implementation campaigns
See implementation checkpoints Implementation plan
Run Codex Goals Codex Goals
See behavior contracts Specs
See design decisions ADRs
Add or review fixtures Testing and Test taxonomy
Understand repo automation PR automation
Roll out Droid review Roll out Factory Droid review and Droid rollout checklist
Understand architecture Architecture
Review capability state Capability matrix and Metrics
Contribute a scoped PR Contributing and Scoped PR contract
Understand CI CI strategy
Understand swarm development Swarm development
Understand dogfooding Dogfooding
Understand docs organization Documentation system
Capture repo knowledge Learnings
Release the crate or extension Release, Release copy checklist, Publishing, and Open VSX
Work on extension assets Brand assets
Find repo instructions for agents Agent instructions

EffortlessMetrics/ripr-swarm is the public same-repo development landing zone for trusted agent and maintainer PRs. The release-facing repository, package publishing, marketplace publishing, GitHub Releases, and public distribution authority remain in EffortlessMetrics/ripr. See Swarm development.

Development

Run the normal local gate:

cargo xtask check-pr

Run the full Rust checks directly:

cargo fmt --check
cargo check --workspace --all-targets
cargo test --workspace
cargo clippy --workspace --all-targets -- -D warnings
cargo doc --workspace --no-deps

Release/package checks:

cargo package -p ripr --list
cargo publish -p ripr --dry-run

Useful sample commands:

cargo run -p ripr -- check --diff crates/ripr/examples/sample/example.diff
cargo run -p ripr -- check --diff crates/ripr/examples/sample/example.diff --json

About

Find Rust test-oracle gaps before mutation testing: static RIPR evidence for CI, editors, and agents.

Topics

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors

Languages