Eval: pf-ai-experience-patterns

[patternfly-jira-sync]

Write an eval suite for the pf-ai-experience-patterns skill (design-guide plugin). This skill applies Red Hat's AI design language to AI-powered features — chatbots, assistants, generation UIs.

Discriminating signal: Red Hat-specific AI UX patterns that baseline Claude wouldn't know — chatbot interaction models, generation UI patterns, assistant affordances per the RH AI design language. Baseline Claude may suggest generic AI UX patterns that don't align with RH brand.

MCP dependency: None — full eval with UI scenario descriptions.

Acceptance criteria:

• eval/pf-ai-experience-patterns/eval.yaml exists following the pf-unit-test-generator template

• 5+ test cases covering explicit invocation, implicit/contextual prompts, and at least 1 negative control (e.g., generic form UI that shouldn't trigger AI patterns)

• Inline Python judges test discriminating behavior (RH AI-specific recommendations, not general chatbot/AI UX advice)

• With-skill/without-skill A/B delta documented

• All judges pass at defined thresholds

---

Jira Issue: PF-4341

[coderabbitai]

🔗 Related PRs

#6 - feat(skills): add PatternFly 6 development skills for Claude Code [merged]
#29 - Add skill for writing unit tests for PatternFly ecosystem components [merged]
#62 - feat(ai-guidelines): skills framework [merged]
#67 - feat(design-mode): New design skills [merged]
#115 - feat(PF-4273): add eval for pf-unit-test-generator skill [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!