Skip to content

feat(rerank): add TEI provider#2605

Open
evaldass wants to merge 1 commit into
volcengine:mainfrom
evaldass:feature/tei-rerank-provider
Open

feat(rerank): add TEI provider#2605
evaldass wants to merge 1 commit into
volcengine:mainfrom
evaldass:feature/tei-rerank-provider

Conversation

@evaldass

Copy link
Copy Markdown
Contributor

Summary

  • add a dedicated Hugging Face Text Embeddings Inference (TEI) rerank provider
  • support TEI base URLs and full /rerank endpoint URLs, optional Bearer auth, extra headers, and optional model names
  • document TEI rerank configuration and refresh stale rerank config tests for current auto-detect behavior

Tests

  • PYTHONPATH=. .venv/bin/python -m pytest -o addopts='' --confcutdir=tests/unit tests/unit/test_tei_rerank.py tests/unit/test_cohere_rerank.py tests/unit/models/rerank/test_openai_rerank_extra_headers.py tests/unit/config/test_rerank_extra_headers_config.py -q
  • PYTHONPATH=. .venv/bin/python -m pytest -o addopts='' --confcutdir=tests/misc tests/misc/test_rerank_openai.py -q
  • .venv/bin/python -m ruff check openviking/models/rerank/tei_rerank.py openviking/models/rerank/init.py openviking/models/rerank/volcengine_rerank.py openviking_cli/utils/config/rerank_config.py tests/unit/test_tei_rerank.py tests/misc/test_rerank_openai.py
  • .venv/bin/python -m ruff format --check openviking/models/rerank/tei_rerank.py openviking/models/rerank/init.py openviking/models/rerank/volcengine_rerank.py openviking_cli/utils/config/rerank_config.py tests/unit/test_tei_rerank.py tests/misc/test_rerank_openai.py
  • python3 -m compileall openviking/models/rerank openviking_cli/utils/config/rerank_config.py

Smoke test

  • Verified against a live TEI reranker using BAAI/bge-reranker-v2-m3: POST /rerank returns ordered scores through TEIRerankClient.

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🏅 Score: 75
🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 Multiple PR themes

Sub-PR theme: Add TEI rerank provider

Relevant files:

  • openviking/models/rerank/tei_rerank.py
  • openviking/models/rerank/init.py
  • openviking/models/rerank/volcengine_rerank.py
  • openviking_cli/utils/config/rerank_config.py
  • tests/unit/test_tei_rerank.py

Sub-PR theme: Refresh stale rerank config tests

Relevant files:

  • tests/misc/test_rerank_openai.py

Sub-PR theme: Update rerank configuration docs

Relevant files:

  • docs/en/guides/01-configuration.md

⚡ Recommended focus areas for review

Blocking I/O in Rerank Method

TEIRerankClient.rerank_batch uses blocking requests.post which may starve the event loop if called from an async context. Use httpx.AsyncClient instead for async HTTP calls.

response = requests.post(
    url=self.rerank_url,
    headers=headers,
    json=req_body,
    timeout=30,
)
TEI Auto-Detection Missing

RerankConfig._effective_provider does not auto-detect TEI provider. TEI only requires api_base, but current logic skips it, requiring explicit provider="tei" to work.

if self.api_key and self.api_base:
    return "openai"
if self.api_key:
    return "cohere"
if self.ak and self.sk:
    return "vikingdb"
return None
Overly Broad Exception Catch

Bare except Exception catches all errors including unexpected ones. Narrow to specific exceptions like requests.exceptions.RequestException and json.JSONDecodeError.

except Exception as e:
    logger.error(f"[TEIRerankClient] Rerank failed: {e}")
    return None

@github-actions

Copy link
Copy Markdown

PR Code Suggestions ✨

No code suggestions found for the PR.

@evaldass evaldass force-pushed the feature/tei-rerank-provider branch from bd9d31a to de35b78 Compare June 14, 2026 11:38
@evaldass

Copy link
Copy Markdown
Contributor Author

Follow-up on the automated review notes:

  • TEI auto-detection is addressed in the latest push: api_base without api_key now resolves to provider=tei, with a unit test and docs note. api_key + api_base still resolves to openai for backwards compatibility, so TEI with auth should set provider=tei explicitly.
  • The synchronous requests.post call is consistent with the existing rerank client interface and the OpenAI/Cohere rerank implementations in this package.
  • The broad exception fallback is also consistent with the existing rerank clients, which return None on provider failure so retrieval can fall back gracefully.

@evaldass evaldass force-pushed the feature/tei-rerank-provider branch from de35b78 to 6d9b446 Compare June 14, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant