Skip to content

test(content): add regression fixtures for unsafe hook script patterns#461

Closed
Kelvinchen03 wants to merge 4 commits into
JSONbored:mainfrom
Kelvinchen03:test/hook-script-security-regression-fixtures
Closed

test(content): add regression fixtures for unsafe hook script patterns#461
Kelvinchen03 wants to merge 4 commits into
JSONbored:mainfrom
Kelvinchen03:test/hook-script-security-regression-fixtures

Conversation

@Kelvinchen03
Copy link
Copy Markdown

Pull Request

Summary

Adds security regression fixtures for hook script bodies as requested in Issue #435. Created 10 test fixtures (5 vulnerable + 5 safe) under tests/fixtures/hooks/ demonstrating common hook security patterns, plus comprehensive tests in tests/content-validation.test.ts (new), tests/registry-artifacts.test.ts, and tests/submission-workflows.test.ts that validate hook submissions against security vulnerabilities without executing untrusted code.

Submission Source

  • New content file(s) added under tests/fixtures/hooks/
  • Existing content updated
  • Submission issue resolved (link it here): test(content): add security regression fixtures for hook script bodies #435
  • Direct content submissions include submittedBy and submittedByUrl frontmatter matching the PR author.
  • I did not modify README.md, generated registry outputs, or apps/web/public/downloads/** unless this is a maintainer/internal automation branch.
  • I did not request HeyClaude-hosted /downloads/... package hosting for community-submitted ZIP/MCPB artifacts.

Schema and Quality Checks

  • pnpm validate:content passed
  • pnpm validate:packages passed
  • pnpm scan:packages passed when package artifacts changed
  • pnpm audit:content ran and I reviewed findings
  • No forbidden fields were added (viewCount, copyCount, popularityScore)
  • Install/use/copy paths are practical and complete
  • Skill submissions include capability metadata when applicable (skillType, skillLevel, verificationStatus, verifiedAt, retrievalSources, testedPlatforms)

Validation

  • Local build passed (pnpm build)
  • I spot-checked the affected detail page(s)

Notes

All 19 tests added for Issue #435 pass successfully. The 5 pre-existing test failures in the output (Windows path separators, line endings, etc.) are unrelated to this PR and predate these changes. Closes #435.

@Kelvinchen03 Kelvinchen03 requested a review from JSONbored as a code owner May 21, 2026 11:47
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 21, 2026

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (13)
  • tests/content-validation.test.ts is excluded by !**/*.test.ts, !tests/** and included by none
  • tests/fixtures/hooks/safe-no-external-downloads.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/safe-proper-quoting.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/safe-secure-tmp-usage.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/safe-with-privacy-notes.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/safe-with-safety-notes.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/vulnerable-community-download.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/vulnerable-invalid-quoting.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/vulnerable-missing-privacy-notes.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/vulnerable-missing-safety-notes.mdx is excluded by !tests/** and included by none
  • tests/fixtures/hooks/vulnerable-tmp-debug-log.mdx is excluded by !tests/** and included by none
  • tests/registry-artifacts.test.ts is excluded by !**/*.test.ts, !tests/** and included by none
  • tests/submission-workflows.test.ts is excluded by !**/*.test.ts, !tests/** and included by none

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Repository UI (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 2ed83f41-1696-4085-ad5e-ea0fe77439d3

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@superagent-security superagent-security Bot added contributor:verified Contributor passed trust analysis. pr:verified PR passed security analysis. labels May 21, 2026
Signed-off-by: Kelvinchen03 <Kelvinchen03@outlook.com>
Copy link
Copy Markdown
Owner

@JSONbored JSONbored left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this, but I don’t think this PR satisfies #435 in its current form. The issue asked for deterministic regression tests where vulnerable fixtures fail for the intended reason and safe fixtures pass. The added tests mostly check that fixture files exist and contain certain strings, so they do not exercise the real validation path or prove the policy catches unsafe hook bodies. CI is also failing in registry/web/Raycast validation, including the new Hook script assertion. I’d close this version and ask for a smaller rewrite that adds real validator-backed fixture tests without generated artifact churn.

@JSONbored JSONbored closed this May 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor:verified Contributor passed trust analysis. pr:verified PR passed security analysis.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(content): add security regression fixtures for hook script bodies

2 participants