Write an eval suite for the pf-project-scaffolder skill (react plugin). This skill scaffolds PF React projects with PF6-safe dependencies, imports, and starter layout.
Discriminating signal: PF6-safe dependency versions, correct starter imports, PF-specific project structure. Baseline Claude may scaffold a generic React project or use outdated PF5 dependencies and import patterns.
MCP dependency: None — full eval with scaffolding prompts.
Acceptance criteria:
-
eval/pf-project-scaffolder/eval.yaml exists following the pf-unit-test-generator template
-
5+ test cases covering explicit invocation, implicit/contextual prompts, and at least 1 negative control (e.g., "scaffold a Vue app" — wrong framework)
-
Inline Python judges test discriminating behavior (PF6 dependency versions and project structure, not general React scaffolding)
-
With-skill/without-skill A/B delta documented
-
All judges pass at defined thresholds
Jira Issue: PF-4347
Write an eval suite for the
pf-project-scaffolderskill (reactplugin). This skill scaffolds PF React projects with PF6-safe dependencies, imports, and starter layout.Discriminating signal: PF6-safe dependency versions, correct starter imports, PF-specific project structure. Baseline Claude may scaffold a generic React project or use outdated PF5 dependencies and import patterns.
MCP dependency: None — full eval with scaffolding prompts.
Acceptance criteria:
eval/pf-project-scaffolder/eval.yamlexists following the pf-unit-test-generator template5+ test cases covering explicit invocation, implicit/contextual prompts, and at least 1 negative control (e.g., "scaffold a Vue app" — wrong framework)
Inline Python judges test discriminating behavior (PF6 dependency versions and project structure, not general React scaffolding)
With-skill/without-skill A/B delta documented
All judges pass at defined thresholds
Jira Issue: PF-4347