BeardedMonster/afro-xlmr-base-masakhanews-multilingual

afro-xlmr-base fine-tuned on masakhanews (multilingual)

Fine-tuned from Davlan/afro-xlmr-base on masakhanews, multilingual protocol, over 16 language(s): amh, eng, fra, hau, ibo, lin, lug, orm, pcm, run, sna, som, swa, tir, xho, yor

setting	value
epochs	25
learning rate	5e-05
batch size	32
max length	256
metric	weighted_f1
mean score	0.8900

Per-language scores are in metrics.json.

Loading

This is a mean-pooling classifier, not a stock AutoModelForSequenceClassification: the base encoder's <cls> token was never trained, so pooling over the attention mask is what the head expects. The encoder is saved in the usual format and the head is a single linear layer in head.pt.

import json, torch, torch.nn as nn
from transformers import AutoTokenizer, AutoModel

tok = AutoTokenizer.from_pretrained("BeardedMonster/afro-xlmr-base-masakhanews-multilingual")
enc = AutoModel.from_pretrained("BeardedMonster/afro-xlmr-base-masakhanews-multilingual")
labels = json.load(open("labels.json"))            # hf_hub_download(repo, "labels.json")
head = nn.Linear(enc.config.hidden_size, len(labels))
head.load_state_dict(torch.load("head.pt"))        # hf_hub_download(repo, "head.pt")

def predict(texts):
    b = tok(texts, return_tensors="pt", padding=True, truncation=True, max_length=256)
    h = enc(**b).last_hidden_state
    m = b["attention_mask"].unsqueeze(-1).float()
    pooled = (h * m).sum(1) / m.sum(1).clamp(min=1e-9)   # mean pooling
    return [labels[i] for i in head(pooled).argmax(-1).tolist()]

Provenance and limits

Produced by this project's benchmark harness, which reproduces the published MasakhaNER 2.0 baselines to within ~0.25 F1, so these numbers are calibrated rather than ad hoc. Scores come from a single run; the baseline papers average over 5, so treat small differences as noise.

The base model's own licence and terms govern any derivative use — check Davlan/afro-xlmr-base before relying on this checkpoint.