Skip to content

ci: split Responses tests across simulator and vLLM - #1195

Closed
leseb wants to merge 1 commit into
praxis-proxy:mainfrom
leseb:speed-up-vllm-ci
Closed

leseb wants to merge 1 commit into
praxis-proxy:mainfrom
leseb:speed-up-vllm-ci

Conversation

@leseb

@leseb leseb commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • run 41 deterministic Responses integration tests against pinned llm-d-inference-sim in the SQLite and PostgreSQL PR jobs
  • keep an 8-test live CPU-vLLM smoke lane on relevant changes, including OGX file resolution plus buffered and streaming file search
  • keep the remaining 12 live inference/compatibility tests available through workflow dispatch for on-demand confidence until GPU runners are available
  • adapt simulator-only Chat tool selection and hosted file-search declarations at the test backend boundary

Validation

  • cargo build -p praxis-ai-proxy
  • simulator Responses lane: 41 passed
  • OpenAI Conversations SDK suite: 26 passed
  • OGX file-search simulator subset: 4 passed
  • live critical marker selection: 8 tests
  • remaining live marker selection: 12 tests
  • actionlint .github/workflows/vllm-integration.yaml
  • Python bytecode compilation and git diff --check

Run deterministic gateway and protocol coverage against llm-d-inference-sim in the SQLite and PostgreSQL jobs.

Keep a focused CPU vLLM smoke suite on relevant changes, including critical OGX file resolution and file search scenarios. Leave the remaining live inference and compatibility cases available through workflow dispatch until GPU runners are available.

Add test-only simulator adapters for deterministic Chat tool turns and hosted file search.

Signed-off-by: Sébastien Han <seb@redhat.com>
@leseb
leseb requested review from a team and asaadbalum September 16, 2026 09:00

@franciscojavierarceo franciscojavierarceo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reviewed the workflow split, test adapters, and existing recording coverage. the simulator lanes pass for both SQLite and PostgreSQL, and the critical live-vLLM smoke lane passes. this test-infrastructure change needs no new inference recording. no blocking issues found.

leseb added a commit to leseb/praxis-ai that referenced this pull request Sep 16, 2026
Signed-off-by: Sébastien Han <seb@redhat.com>
@leseb

leseb commented Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

superseeded by #1129

@leseb leseb closed this Sep 16, 2026
@leseb
leseb deleted the speed-up-vllm-ci branch September 16, 2026 09:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants