SauerkrautLM-MoEdernColBERT-27M

A compact multilingual ColBERT retriever: 27M parameters, 1/22 the size of Qwen3-Embedding-0.6B, 6,708 documents per second on one GPU. Built for German and six other European languages, with documents of up to 32k tokens.

The SauerkrautLM-MoEdernColBERT family introduces MoEdernBERT, a new ModernBERT-style encoder architecture with a mixture of experts and hash embeddings. MoEdernColBERT-27M has 27.0M parameters, of which only 12.3M are active per token.

Highlights

• Ahead of 3.6× larger granite-97m-multilingual-r2 on all 11 benchmarks in the overview.
• 4.3× the throughput of granite-97m-multilingual-r2 and 47× that of Qwen3-Embedding-0.6B: 6,708 documents per second on one GPU.
• German retrieval ahead of models up to 22× larger. MKQA de→en 0.690 vs. 0.671 for Qwen3-Embedding-0.6B.
• Spanish, French, Italian, Dutch and Portuguese retrieval ahead of models up to 22× larger. MKQA Spanish → English 0.679 vs. 0.677 for Qwen3-Embedding-0.6B.
• German legal texts: ahead of 8B models and commercial APIs. LegalQuAD 0.796 vs. 0.772† for Nemotron-3-Embed-8B.
• Cross-lingual retrieval ahead of 22× larger Qwen3-Embedding-0.6B. MKQA 0.684 vs. 0.681.
• Long documents up to 32k tokens. LongEmbed 86.9, ahead of the 7B–14B models on the public LongEmbed leaderboard.

† public MTEB result, not measured by us.

Architecture: MoEdernBERT

2 transformer layers, hidden size 640.

• Hash embedding. A factorized embedding: every token id owns a short row of 80 values, which a shared layer projects to the hidden size. This needs 2.45M parameters instead of 19.2M.
• Mixture-of-experts MLP. In every transformer layer, a router sends each token to 2 of 8 GeGLU experts, and one shared expert always runs.
• Attention and long context. Global attention with RoPE, trained on 512 tokens. Above 513 tokens, a ±256-token sliding window keeps every token within the trained range, so documents of up to 32k tokens work.
• ColBERT head. Every token is mapped to 128 dimensions. Queries are padded to 32 tokens with [MASK], and relevance is scored with MaxSim.

Training: Stella-style embedding distillation, then knowledge-distillation pre-training and English fine-tuning.

Speed

Sparse activation makes MoEdernColBERT-27M fast on a GPU and on a laptop.

One NVIDIA RTX 6000 Ada, bf16, 512-token documents, bundled Triton MoE kernel.

Results

German

Other languages

German legal texts

Long documents

The MoEdernColBERT family

	MoEdernColBERT-1.5M	MoEdernColBERT-27M	MoEdernColBERT-115M
Parameters total / active per token	1.5M / 0.88M	27.0M / 12.3M	114.6M / 67.4M
NanoBEIR English	0.591	0.619	0.643
NanoBEIR, 6 languages	0.497	0.544	0.585
MKQA, cross-lingual (Recall@20)	0.588	0.684	0.724
German retrieval, 4 tasks	0.858	0.870	0.897
LegalQuAD (German legal texts)	0.794	0.796	0.851
LongEmbed, up to 32k tokens (×100)	85.2	86.9	90.1
GPU throughput (docs/s)	43,255	6,708	1,132
Size as 4-bit CoreML	0.87 MB	13.8 MB	57.9 MB
Query latency, Apple M5 Max	0.13 ms	0.41 ms	1.19 ms

NDCG@10 unless noted, higher is better. Throughput measured as under Speed.

Usage

The model ships its own modeling code, so trust_remote_code=True is required. Tested with sentence-transformers 6.1 and transformers 5.

pip install -U "sentence-transformers>=6.1"
pip install -U triton   # Linux + NVIDIA GPU: enables the bundled fused MoE kernel

Fast GPU path. On NVIDIA GPUs, the bundled Triton MoE kernel switches on automatically during inference; without a CUDA GPU or without Triton, the model falls back to plain PyTorch. Set MOEDERN_TRITON=0 to turn the kernel off.

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder("VAGOsolutions/SauerkrautLM-MoEdernColBERT-27M", trust_remote_code=True)

queries = ["Wie hoch ist die Zugspitze?", "What is the capital of France?"]
documents = [
    "Die Zugspitze ist mit 2962 Metern der höchste Berg Deutschlands.",
    "Paris is the capital and largest city of France.",
    "Berlin ist die Hauptstadt Deutschlands.",
]

query_embeddings = model.encode_query(queries)          # list of (32, 128) tensors
document_embeddings = model.encode_document(documents)  # list of (num_tokens, 128) tensors

scores = model.similarity(query_embeddings, document_embeddings)  # MaxSim, shape (2, 3)
print(scores.argmax(dim=1))  # tensor([0, 1])

Long documents. Documents are cut at 512 tokens by default. Raise the limit, and the ±256 window switches on automatically:

model[0].document_length = 32768

Precision. Use fp32 or bf16. Do not call .half(): fp16 overflows and produces NaN embeddings.

GGUF and CoreML. Quantized versions run on CPUs with our ggml-based engine (GGUF) and on Apple GPUs and the Neural Engine (CoreML). The 4-bit CoreML file is 13.8 MB and scores 0.621 on NanoBEIR English (fp32: 0.619).

Evaluation

All numbers without † come from our own runs with sentence-transformers 6, each model with its own configuration. On the German tasks, our models use the better of 512 or 8,192 document tokens per task. The paper has the full protocol and all results.

Languages and limitations. German, English, Spanish, Dutch, French, Italian and Portuguese. Arabic, Japanese and Korean are not supported.

License

Apache 2.0, see LICENSE.

Citation

@misc{sauerkrautlm_moederncolbert_2026,
  title={MoEdernBERT: Hash-Embedded Mixture-of-Experts Encoders for Tiny Multilingual Late-Interaction Retrieval},
  author={David Golchinfar},
  organization={VAGO Solutions},
  url={https://huggingface.co/VAGOsolutions/SauerkrautLM-MoEdernColBERT-27M},
  year={2026}
}

Contact

VAGO solutions