Update Querit results

[moshesbeta]

Checklist

• My model has a model sheet, report, or similar
• My model has a reference implementation in mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here
  • No, but there is an existing PR Fix: Querit model revision and release_data mteb#4569
• The results submitted are obtained using the reference implementation
• My model is available, either as a publicly accessible API or publicly on e.g., Huggingface
• I solemnly swear that for all results submitted I have not trained on the evaluation dataset including training splits. If I have, I have disclosed it clearly.

[github-actions]

Model Results Comparison

Reference models: intfloat/multilingual-e5-large, google/gemini-embedding-001
New models evaluated: Querit/Querit
Tasks: AlloprofReranking, RuBQReranking, T2Reranking, VoyageMMarcoReranking, WebLINXCandidatesReranking, WikipediaRerankingMultilingual

Results for Querit/Querit

task_name	Querit/Querit	google/gemini-embedding-001	intfloat/multilingual-e5-large	Max result	Model with max result	In Training Data
AlloprofReranking	0.7919	0.8177	0.6944	0.8540	Octen/Octen-Embedding-8B	False
RuBQReranking	0.7535	0.7384	0.756	0.8051	ai-sage/Giga-Embeddings-instruct	False
T2Reranking	0.6895	0.6795	0.6632	0.7315	tencent/Youtu-Embedding	True
VoyageMMarcoReranking	0.6788	0.6673	0.6821	0.8366	codefuse-ai/F2LLM-v2-14B	False
WebLINXCandidatesReranking	0.1184	0.1097	0.0778	0.2246	codefuse-ai/F2LLM-v2-8B	False
WikipediaRerankingMultilingual	0.9092	0.9224	0.8981	0.9308	jinaai/jina-reranker-v3	False
Average	0.6569	0.6558	0.6286	0.7304	nan	-

Training datasets: AskUbuntuDupQuestions, AskUbuntuDupQuestions-VN, CQADupStack, MIRACLRanking, MSMARCO, MSMARCO-Fa, MSMARCO-FaHardNegatives, MSMARCO-PL, MSMARCO-PLHardNegatives, MSMARCO-VN, MSMARCOHardNegatives, MSMARCOv2, MindSmallReranking, MrTidyRetrieval, MrTyDiJaRetrievalLite, MultiLongDocReranking, MultiLongDocRetrieval, NanoMSMARCO-VN, NanoMSMARCORetrieval, T2Reranking, ruri-v3-dataset-reranker

---

---

Note: Content truncated due to GitHub API limits. See the full report in the workflow artifacts.

[KennethEnevoldsen]

this is deleted but never added

[KennethEnevoldsen]

this is still missing

[moshesbeta]

Our model primarily focuses on multilingual reranking tasks, and it's not tested on this dataset. Is it necessary to provide results from this test set?