Skip to content

Add SocialiToM generation and evaluation pipeline#3

Open
lishitian15-ops wants to merge 1 commit into
TomTraining:mainfrom
lishitian15-ops:socialitom-pipeline-eval
Open

Add SocialiToM generation and evaluation pipeline#3
lishitian15-ops wants to merge 1 commit into
TomTraining:mainfrom
lishitian15-ops:socialitom-pipeline-eval

Conversation

@lishitian15-ops

Copy link
Copy Markdown

Summary

  • Add SocialiToM generation pipeline
  • Add QC judge and Qwen pass@5 evaluation
  • Add pilot generation scripts and output adapters

本次修改概述

新增了 SocialiToM 的完整生成流水线,支持从社会场景上下文生成故事、问题、CoT、干扰项和标准答案。
补充了上下文池、故事填充、输出适配、问题生成和社会后果规则等核心模块,增强了不同任务类型的生成能力。
新增了质量判断模块,用于对生成样本做规则/LLM 质量筛选。
新增了批量生成脚本和评测脚本,支持本地 vLLM / OpenAI-compatible 接口,方便大规模合成与评测。
新增了 Qwen pass@5 评测流程,并输出 thinking、acc@1、majority_vote_acc 等更细粒度指标。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant