Skip to content

Add new results of Querit/Querit#540

Merged
Samoed merged 2 commits into
embeddings-benchmark:mainfrom
moshesbeta:update-querit-results
May 14, 2026
Merged

Add new results of Querit/Querit#540
Samoed merged 2 commits into
embeddings-benchmark:mainfrom
moshesbeta:update-querit-results

Conversation

@moshesbeta
Copy link
Copy Markdown
Contributor

@moshesbeta moshesbeta commented May 14, 2026

Checklist

  • My model has a model sheet, report, or similar
  • My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
  • The results submitted are obtained using the reference implementation
  • My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
  • I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 14, 2026

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: Querit/Querit
Tasks: AlloprofReranking, AskUbuntuDupQuestions, RuBQReranking, T2Reranking, VoyageMMarcoReranking, WebLINXCandidatesReranking, WikipediaRerankingMultilingual

Results for Querit/Querit

task_name Querit/Querit google/gemini-embedding-001 intfloat/multilingual-e5-large Max result Model with max result In Training Data
AlloprofReranking 0.7939 0.8177 0.6944 0.8540 Octen/Octen-Embedding-8B False
AskUbuntuDupQuestions 0.6402 0.6424 0.5924 0.7528 IEITYuan/Yuan-embedding-2.0-en True
RuBQReranking 0.7537 0.7384 0.756 0.8051 ai-sage/Giga-Embeddings-instruct False
T2Reranking 0.6898 0.6795 0.6632 0.7315 tencent/Youtu-Embedding True
VoyageMMarcoReranking 0.6859 0.6673 0.6821 0.8366 codefuse-ai/F2LLM-v2-14B False
WebLINXCandidatesReranking 0.1163 0.1097 0.0778 0.2246 codefuse-ai/F2LLM-v2-8B False
WikipediaRerankingMultilingual 0.9105 0.9224 0.8981 0.9308 jinaai/jina-reranker-v3 False
Average 0.6558 0.6539 0.6234 0.7336 nan -

Training datasets: AskUbuntuDupQuestions, AskUbuntuDupQuestions-VN, CQADupStack, MIRACLRanking, MSMARCO, MSMARCO-Fa, MSMARCO-FaHardNegatives, MSMARCO-PL, MSMARCO-PLHardNegatives, MSMARCO-VN, MSMARCOHardNegatives, MSMARCOv2, MindSmallReranking, MrTidyRetrieval, MrTyDiJaRetrievalLite, MultiLongDocReranking, MultiLongDocRetrieval, NanoMSMARCO-VN, NanoMSMARCORetrieval, T2Reranking, ruri-v3-dataset-reranker



Note: Content truncated due to GitHub API limits. See the full report in the workflow artifacts.

@Samoed Samoed merged commit 5b386b3 into embeddings-benchmark:main May 14, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants