(ROSAENG-62419) Coderabbit Review Skill - #362
Conversation
Add sandbox-compatible script to collect CodeRabbit findings for PRs
and emit them as JSON for FullSend review synthesis.
Script features:
- Two modes: comment (S3, default) and cli (S2, runner-only)
- Sandbox-compatible: uses gh + node (both allowlisted)
- Fail-soft behavior: emits empty array on unavailability
- Exact bot login matching: prevents injection attacks
- Current commit filtering: excludes stale comments
- CLI output normalization: transforms to required JSON schema
- API key authentication: for optional CLI mode
Comment mode (default):
- Reads CodeRabbit's existing PR review comments via gh API
- Filters by current PR head commit to exclude historical comments
- No API key or external network required
- Uses node for JSON processing (jq not allowlisted)
CLI mode (runner-only):
- Runs CodeRabbit CLI with --format json
- Requires CODERABBIT_API_KEY as CI secret
- Normalizes output to {source, path, line, body, url}[] format
- Fails gracefully if CLI unavailable
Security:
- API key passed via --api-key flag (not in env visible to logs)
- Exact case-insensitive login comparison (no regex injection)
- All errors emit warnings to stderr + empty array to stdout
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add comprehensive documentation for the coderabbit-review skill that explains how to integrate CodeRabbit findings into FullSend reviews. Documentation covers: - When to use the skill (during /fs-review on PRs) - Two ingestion sources with priority (S2 injected file, S3 GitHub comments) - CI setup instructions for CODERABBIT_API_KEY - Step-by-step finding gathering and mapping process - Repo path exclusion patterns from .coderabbit.yaml - Security constraints (never log API keys, treat as advisory) CI Setup (S2 mode): - Store CODERABBIT_API_KEY as CI secret - Run pre-script before review harness - Output to /tmp/workspace/coderabbit-findings.json - host_files mapping copies into sandbox - API key never enters sandbox environment GitHub Ingest (S3 mode - default): - Uses gh client already available in review agent - No extra network, binary, or secret required - Reads existing CodeRabbit PR review comments - Sandbox-safe and fail-soft Finding mapping: - Map severity to review scale (default: info) - Prefix categories with coderabbit- for deduplication - Extract file, line, description, remediation, url - Respect .coderabbit.yaml path exclusions Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Add comprehensive README for the .fullsend/skills/ directory, documenting the available skills and how to create new ones. Documented skills: - prow-ci: Access and analyze OpenShift Prow CI results - coderabbit-review: Ingest CodeRabbit findings for PR reviews Documentation includes: - Purpose and usage for each skill - Skill invocation patterns for Claude conversations - Skill components structure (frontmatter, commands, troubleshooting) - Creating new skills (directory structure, SKILL.md format) - Integration with agents, hooks, and commands - Future planned skills (dependency-update, release-prep, etc.) Directory structure note: - Skills live in .fullsend/skills/ - .claude/skills is a symlink for portability across agent runtimes - Skills extend built-in FullSend skills (don't replace them) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Reorganize skill files to follow FullSend conventions: - Move .claude/skills/ to .fullsend/skills/ - Create .claude/skills symlink to ../.fullsend/skills - Migrate prow-ci skill to new location This change improves portability across agent runtimes while maintaining backward compatibility via the symlink. Files migrated: - .claude/skills/prow-ci/SKILL.md → .fullsend/skills/prow-ci/SKILL.md - .claude/skills/README.md → deleted (already added to .fullsend/skills/) - .claude/skills → new symlink to ../.fullsend/skills Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Repository YAML (base), Central YAML (inherited) Review profile: CHILL Plan: Team Run ID: Comment |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: Sushant2504 The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #362 +/- ##
=======================================
Coverage 65.39% 65.39%
=======================================
Files 23 23
Lines 1598 1598
=======================================
Hits 1045 1045
Misses 473 473
Partials 80 80 🚀 New features to boost your workflow:
|
Add review.yaml harness configuration that extends the FullSend base review harness with CodeRabbit findings ingestion. Review harness configuration: - Composes upstream FullSend review harness (v0.38.0) - Adds coderabbit-review skill as complementary finding source - Inherits triggers from base (no custom override needed) - Allows remote resources from fullsend-ai/agents - Maps host_files for S2 CLI mode (coderabbit-findings.json) CodeRabbit integration: - Findings mapped with coderabbit-* category prefixes - Built-in review dimensions remain authoritative - Synthesizer can dedupe and attribute properly API key setup (CI only): - CODERABBIT_API_KEY stored as CI secret (Tekton/Prow) - Runner-side pre-script generates findings JSON - host_files mapping copies into sandbox - API key never injected into sandbox environment Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
9ffe194 to
acaab51
Compare
|
/test lint |
|
/test rosa-sts-e2e |
|
@Sushant2504: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
duplicate: #364 |
|
🤖 Finished Retro · ✅ Success · Started 9:39 AM UTC · Completed 9:52 AM UTC Commit: Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $3.28 |
Retro: PR #362 — (ROSAENG-62419) Coderabbit Review SkillTimelinePR #362 was one of four PRs opened for the same JIRA ticket (ROSAENG-62419 — adding CodeRabbit as a complementary finding source for FullSend reviews):
None of the four PRs were merged. All were closed on Sep 7. PR #362 was closed by samanthajayasinghe with the comment "duplicate: #364". PR #364 received Agent Performance AssessmentThis retro was triggered on PR #362, which had zero fullsend agent involvement. No triage, code, review, or fix agents ran on this PR. The only agent dispatched was the retro itself (workflow run 34107107318). However, the sibling PR #363 received a substantive FullSend review (claude-opus-4-6, high effort). The review agent produced 11 findings, including one genuinely high-value catch:
The review agent's config-gap finding demonstrates strong structural awareness — it cross-referenced the new harness file against the existing config and identified the inconsistency. This is the kind of finding that often slips through human review on config-only PRs. Key Observations
Evidence for Existing Issues
|
3 participants
Summary
Adds CodeRabbit AI review as a complementary finding source for FullSend code reviews. Findings are synthesized alongside built-in dimensions (correctness, protected paths, intent/coherence).
Changes
New Files:
Modified:
Stats: 7 files changed, 329 insertions(+), 5 deletions(-)
Key Features
Usage
Test comment mode (default, no API key required)
.fullsend/skills/coderabbit-review/scripts/run-coderabbit.sh <PR_NUMBER>
Use in FullSend review
fullsend review --pr <PR_NUMBER>
CI Setup (Optional - CLI mode only)
export CODERABBIT_API_KEY="$(cat /path/to/secret)"
export CODERABBIT_MODE="cli"
.fullsend/skills/coderabbit-review/scripts/run-coderabbit.sh "$PR_NUMBER" \
Commits