Skip to content

fix(ci): replace empty example tests with smoke coverage - #6460

Merged
yuxuan-lou merged 1 commit into
mainfrom
fix/empty-example-test-ci
Sep 30, 2026
Merged

yuxuan-lou merged 1 commit into
mainfrom
fix/empty-example-test-ci

Conversation

@richardoo-707

Copy link
Copy Markdown
Collaborator

Summary

  • reject missing or empty test_ci.sh files in the shared example-check runner
  • add deterministic tiny smoke tests for Llama, Mixtral, and DeepSeek V3
  • add correctness smoke coverage for benchmark RMSNorm and rotary embedding operators
  • replace the unsupported RoBERTa empty script with an explicit warning and tracked skip linked to CI: replace explicit example skips with hermetic smoke tests #6453

The smoke tests use local synthetic inputs, do not download models or tokenizers, and do not change the examples' public APIs, training entry points, default model configurations, or benchmark grids. Performance values do not determine success.

Why

An empty test_ci.sh exits successfully without testing anything, so example CI can report green while the example is broken. The runner now treats missing or empty scripts as configuration errors; runnable examples perform real work, while unsupported coverage is explicit and visible.

Validation

  • shared runner unit tests: 4 passed
  • Llama: real two-GPU tensor-parallel training step passed
  • Mixtral: real two-GPU expert-parallel + ZeRO-1 training step passed
  • DeepSeek V3: built-in tiny two-GPU expert-parallel + ZeRO-1 forward/loss/backward/optimizer step passed in offline mode with an empty HF cache
  • benchmark_ops: RMSNorm and rotary embedding correctness smoke passed on CUDA
  • Python compilation, YAML parsing, Bash syntax, targeted pre-commit checks, and git diff --check passed
  • GPU processes and memory were released after each smoke test

The remaining RoBERTa migration/deprecation decision stays tracked in #6453; its CI path now warns and skips explicitly instead of producing a silent false green.

@richardoo-707
richardoo-707 requested a review from a team as a code owner September 30, 2026 04:49
@yuxuan-lou
yuxuan-lou merged commit 99e44f1 into main Sep 30, 2026
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants