kirillbilchenko/Kolibri-1-MXFP4-MxWave

Kolibri-1 MXFP4 — MxWave softer RMS

Model parameters: 78.10B total (78,103,074,560), about 3.46B active per token. This matches the original Kolibri architecture. The Hub's approximately 40.35B count reflects packed tensor elements: each uint8 expert element contains two four-bit MXFP4 weights. Doubling the 37,748,736,000 packed expert elements and adding 2,605,602,560 backbone parameters recovers the full 78,103,074,560 parameters.

Weight precision: 4-bit MXFP4 routed experts with an FP8 backbone. Hugging Face currently adds an automatic 8-bit precision badge for this mixed checkpoint. Its config.json declares separate 4-bit expert and 8-bit backbone groups.

Experimental release. The unchanged strict KL target was missed: 0.055140 nats versus 0.030. The likelihood bounds and all six behavior checks passed. The original failed screen is included in validation/; its threshold was not changed.

The 32 safetensors weight files total 40.45 GiB (43,433,901,512 bytes), 44.9% smaller than the official FP8 files. All 57,600 routed expert matrices use OCP MXFP4 E2M1 with one E8M0 scale per 32 weights. The official block-FP8 attention and shared-expert backbone, scales, routers, embeddings, norms, and head retain their original payload bytes. Expert inference uses Marlin W4A16; backbone activation quantization remains dynamic FP8.

Recipe and provenance

Expert source: Aleph-Alpha/Kolibri-1-BF16 at 7a8f290e7858825c3cf5e4c447ba68345de9f1d3. Backbone and behavioral reference: Aleph-Alpha/Kolibri-1 at e52eb4627d11516b0c01de49210ab5a4e4061444.

Scale selection blends uniform MSE with routed activation variance: alpha = 0.25 * n / (n + 128); weight = (1 - alpha) + alpha * clip(RMS² / mean(RMS²), 0.25, 4). Target-wide gamma normalization preserves this objective. MSE clip depth is 4, percentile 99.5, with round-to-nearest-even mxfp4-rne-v2. No rotation is used. The teacher collected 32,768 tokens in 64 training-split windows from English WikiText-2, GermanQuAD, and MBPP. Evaluation passages were excluded. 53,646 projections were observed; 3,954 unobserved projections retain their unweighted payloads. 53,591 projections changed, affecting 0.3033% of expert blocks.

Every one of the 116,303 exported payloads matches the measured compact trial. This standalone checkpoint was then measured with the stock mixed-format vLLM loader. Serving requires no private trial worker or patch files. The converter manifest is immutable export-time provenance; subsequent qualification is recorded in validation/qualification-result.json. Soft-RMS SQNR was not independently measured.

Run on DGX Spark

Tested on NVIDIA GB10 / SM121 with vLLM 0.29.0 and aleph-alpha-inference==1.0.0. The plugin implements Kolibri's routing, sandwich norms, and mixed sliding/full attention; install it in every runtime process. It declares vllm>=0.29.0,<0.30.0. vLLM 0.30 was not qualified for this release.

python -m pip install 'vllm==0.29.0' 'aleph-alpha-inference==1.0.0'
VLLM_USE_DEEP_GEMM=0 VLLM_USE_DEEP_GEMM_E8M0=0 \
vllm serve kirillbilchenko/Kolibri-1-MXFP4-MxWave \
  --load-format safetensors --dtype bfloat16 --moe-backend marlin \
  --max-model-len 9216 --max-num-batched-tokens 1024 --max-num-seqs 4 \
  --kv-cache-dtype bfloat16 --kv-cache-memory-bytes 2147483648 \
  --gpu-memory-utilization 0.85 --no-enable-prefix-caching \
  --compilation-config '{"mode":0,"cudagraph_mode":"FULL_DECODE_ONLY","cudagraph_capture_sizes":[1,2,4]}' \
  --enable-auto-tool-choice --tool-call-parser kolibri1 --reasoning-parser kolibri1

Startup must select MarlinExperts for routed experts and automatic block-FP8 dense kernels for the backbone. Inference memory includes weights, KV cache, and kernel workspaces in addition to the 40.45 GiB weight files.

Measured quality and speed

The frozen screen scores 48 passages: 16 English WikiText-2 validation windows, 16 unique GermanQuAD test passages, and 16 HumanEval prompts with canonical solutions. Code PPL is a likelihood screen, not HumanEval pass@1.

Domain	Scored tokens	Official FP8 PPL	Softer RMS PPL	Relative PPL
en	16,214	32.470410	32.759410	+0.89%
de	6,963	21.668688	22.146479	+2.20%
code	3,311	7.240783	7.475029	+3.24%
all	26,488	24.201922	24.571060	+1.53%

Mean full-vocabulary forward KL uses one fixed prefix per passage. The six behavior checks cover English/German arithmetic, German retrieval, JSON, a tool call, and retrieval near 8k tokens, with reasoning disabled. The screen does not validate the advertised full context or broad task accuracy. The original compact trial measured PPL 24.482698 and mean KL 0.053467, 8.7% lower mean KL than the unweighted mixed candidate's 0.058545. Its paired passage-bootstrap interval includes zero, and some prefixes worsen. This bounded result does not establish a broad quality improvement.

Model / decode graphs	Output tok/s, c1	Aggregate output tok/s, c4
Official FP8	43.61	113.18
Softer RMS MXFP4	49.62	136.37

Throughput uses 512 input tokens, 128 forced output tokens, BF16 KV, no prefix caching, and the median of three repetitions. Both use graph captures [1,2,4] with Torch compilation disabled. The standalone run overlapped a CPU/network weight upload; its timing describes that run and may include host I/O contention. The earlier compact trial measured 48.86 / 135.26 tok/s at c1/c4.

The frozen protocol, raw full-vocabulary distributions, hashes, export proof, and original failed qualification are included in validation/ and mxwave-manifest.json. Exact routed statistics, training tokens, source pins, and dataset licenses are included in calibration/. WikiText uses CC BY-SA 3.0, GermanQuAD CC BY 4.0, HumanEval MIT, and MBPP CC BY 4.0 as recorded in the protocols. The licensed converter and tests are in mxwave-source.tar.gz; archive and Git provenance are in validation/release-source.json. Extract and run pip install -e '.[dev]'. Vendor plugin source is not bundled. MxWave source.

GGUF

These files use compressed-tensors for vLLM. A separately measured 42.30 GiB MXFP4 GGUF release is available. It preserves the expert MXFP4 codes and scales and dequantizes the FP8 backbone to BF16. See its card for the recommended eight-bit activation setting and pinned, licensed Kolibri runtime. Stock llama.cpp and Ollama compatibility has not been established.