Add new results of Querit/Querit#540
Conversation
Model Results ComparisonReference models: Results for
|
| task_name | Querit/Querit | google/gemini-embedding-001 | intfloat/multilingual-e5-large | Max result | Model with max result | In Training Data |
|---|---|---|---|---|---|---|
| AlloprofReranking | 0.7939 | 0.8177 | 0.6944 | 0.8540 | Octen/Octen-Embedding-8B | False |
| AskUbuntuDupQuestions | 0.6402 | 0.6424 | 0.5924 | 0.7528 | IEITYuan/Yuan-embedding-2.0-en | True |
| RuBQReranking | 0.7537 | 0.7384 | 0.756 | 0.8051 | ai-sage/Giga-Embeddings-instruct | False |
| T2Reranking | 0.6898 | 0.6795 | 0.6632 | 0.7315 | tencent/Youtu-Embedding | True |
| VoyageMMarcoReranking | 0.6859 | 0.6673 | 0.6821 | 0.8366 | codefuse-ai/F2LLM-v2-14B | False |
| WebLINXCandidatesReranking | 0.1163 | 0.1097 | 0.0778 | 0.2246 | codefuse-ai/F2LLM-v2-8B | False |
| WikipediaRerankingMultilingual | 0.9105 | 0.9224 | 0.8981 | 0.9308 | jinaai/jina-reranker-v3 | False |
| Average | 0.6558 | 0.6539 | 0.6234 | 0.7336 | nan | - |
Training datasets: AskUbuntuDupQuestions, AskUbuntuDupQuestions-VN, CQADupStack, MIRACLRanking, MSMARCO, MSMARCO-Fa, MSMARCO-FaHardNegatives, MSMARCO-PL, MSMARCO-PLHardNegatives, MSMARCO-VN, MSMARCOHardNegatives, MSMARCOv2, MindSmallReranking, MrTidyRetrieval, MrTyDiJaRetrievalLite, MultiLongDocReranking, MultiLongDocRetrieval, NanoMSMARCO-VN, NanoMSMARCORetrieval, T2Reranking, ruri-v3-dataset-reranker
Note: Content truncated due to GitHub API limits. See the full report in the workflow artifacts.
Checklist
mteb/models/model_implementations/, this can be as an API. Instruction on how to add a model can be found here