Skip to content

Eval: pf-figma-icon-finder #119

Description

@patternfly-jira-sync

Write an eval suite for the pf-figma-icon-finder skill (design-audit plugin). This skill identifies PatternFly icons in Figma mockups and provides correct React import statements. It requires the Figma MCP.

Discriminating signal: Mapping visual icons to exact PF React import paths (e.g., @patternfly/react-icons/dist/esm/icons/check-circle-icon). Baseline Claude may suggest plausible but incorrect paths.

MCP dependency: Figma MCP required. Mock icon screenshot/component data as eval input. Tag Figma-dependent cases with requires_mcp: figma for gated CI execution.

Acceptance criteria:

  • eval/pf-figma-icon-finder/eval.yaml exists following the pf-unit-test-generator template

  • 5+ test cases covering explicit invocation, implicit/contextual prompts, and at least 1 negative control

  • Inline Python judges test discriminating behavior (correct PF icon import resolution, not general icon identification)

  • With-skill/without-skill A/B delta documented

  • All judges pass at defined thresholds


Jira Issue: PF-4338

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions