Leaderboard
One pinned corpus, one document renderer, exact search — so a difference between two rows is a difference between two systems and not between two experimental setups. Every row carries the number of queries it is actually computed over, because a row scored on 894 queries and one scored on 946 are not the same measurement.
| R@k curve | ||||||
|---|---|---|---|---|---|---|
| 0.571 | 0.744[0.72–0.77] | 0.519 | 894/946 | 2068.6 ms * | ||
Nemotron-3-Embed-8Bnvidia/Nemotron-3-Embed-8B-BF16 · 8.0B · 4096d · OpenMDW-1.1 | 0.526 | 0.683[0.65–0.71] | 0.482 | 894/946 | 64.6 ms * | |
Qwen3-Embedding-8BQwen/Qwen3-Embedding-8B · 8B · 4096d · Apache-2.0 | 0.456 | 0.601[0.57–0.63] | 0.417 | 894/946 | 55.4 ms * | |
Qwen3-Embedding-4BQwen/Qwen3-Embedding-4B · 4B · 2560d · Apache-2.0 | 0.427 | 0.573[0.54–0.61] | 0.389 | 894/946 | 29.9 ms * | |
| 0.385 | 0.565[0.53–0.60] | 0.340 | 894/946 | 1457.2 ms * | ||
Nemotron-3-Embed-1Bnvidia/Nemotron-3-Embed-1B-BF16 · 1.1B · 2048d · OpenMDW-1.1 | 0.354 | 0.510[0.48–0.54] | 0.315 | 894/946 | 33.2 ms * | |
Qwen3-Embedding-0.6BQwen/Qwen3-Embedding-0.6B · 0.6B · 1024d · Apache-2.0 | 0.331 | 0.464[0.44–0.50] | 0.298 | 894/946 | 1136.5 ms | |
BGE-M3BAAI/bge-m3 · 568M · 1024d · MIT | 0.266 | 0.379[0.35–0.41] | 0.238 | 894/946 | 272.1 ms | |
EmbeddingGemma-300Mgoogle/embeddinggemma-300m · 308M · 768d · Gemma Terms of Use | 0.243 | 0.361[0.33–0.39] | 0.215 | 894/946 | 211.2 ms | |
BM25lexical | 0.227 | 0.326[0.29–0.36] | 0.205 | 894/946 | 15.7 ms |
Latencies marked * are not our end-to-end in-container measurement: on a published engine the figure is an internet round trip including the service's own queueing, and on a row whose query encoder is too large for the CPU searcher it is the vector search alone, with the GPU encode and any cold start excluded. Unmarked rows are measured inside the serving container, encode included, with no network in the path.
This task has exactly one relevant declaration per query, so precision@k is a mechanical R@k⁄k decay and carries no information beyond recall. It is suppressed here and on Compare rather than plotted as a misleading curve. Bracketed figures on R@10 are bootstrap 95% intervals over queries.
The curves show the R@k shape and the query-style breakdown behind these cut-offs, and Explore datasets shows the individual pairs each system got right or wrong.