Add configurable adversarial benchmark pipeline - #2436

Open

hannahwestra25 wants to merge 13 commits into

microsoft:mainfrom

hannahwestra25:hannahwestra25-adversarial-benchmark-system-prompt

Commits on Aug 20, 2026

DEV: add adversarial benchmark pipeline prototype
hannahwestra25
committed
da7672c
DEV: make adversarial benchmark pipeline configurable
hannahwestra25
committed
6cd70c7
DEV: add balanced HarmBench benchmark dataset
hannahwestra25
committed
f2adb3b
FIX: run benchmark dataset preparation as module
hannahwestra25
committed
821733f
FIX: make benchmark artifacts reliable
hannahwestra25
committed
88e6919
FIX: pass benchmark parameters through environment
hannahwestra25
committed
7f7b98a
MAINT: use public dataset provider in benchmark prep
hannahwestra25
committed
bbff19d
FIX: export partial adversarial benchmark results
hannahwestra25
committed
b2be470
DEV: add per-technique benchmark metrics
hannahwestra25
committed
5c9fdbe
FIX: deduplicate benchmark retries in metrics
hannahwestra25
committed
ba5b9ea
MAINT: use scanner run command in benchmark pipeline
hannahwestra25
committed
41e3940
MAINT: simplify benchmark technique configuration
hannahwestra25
committed
fdbe4b4
Merge branch 'main' into hannahwestra25-adversarial-benchmark-system-prompt
hannahwestra25
authored
dc67c7f