Ugiat/spanish-variety-fasttext

Spanish variety classifier: Spain (es-ES) vs Latin America (fastText)

A fast document classifier that estimates whether a Spanish text was written in Spain (es_ES) or in Latin America (es_LATAM). It was trained on MADLAD-400 web documents labelled by their website's domain and was used to annotate the whole Spanish split of MADLAD-400 (250.9M documents, Ugiat/madlad-400-es-annotations) and to select Castilian text (Ugiat/madlad-400-es_ES-sentences).

• fastText supervised, 64 dimensions, word bigrams; about 70,000 documents per second on 24 CPU cores.
• For documents, not single sentences (see Limitations).
• Needs its own normalisation (normalize.py, included): lowercase, capitalized words, digits and URLs removed.

This model was prepared by Ugiat Technologies. It is an independent work and is not produced, reviewed or endorsed by Google, AllenAI or the MADLAD-400 authors.

Usage

import fasttext
from huggingface_hub import hf_hub_download
from normalize import normalize          # from this repository

model = fasttext.load_model(hf_hub_download("Ugiat/spanish-variety-fasttext", "variety.bin"))
labels, probs = model.predict(normalize(document_text), k=2)
p_es_ES = dict(zip(labels, probs))["__label__es_ES"]

normalize(text, max_chars=3000) keeps the first 3,000 characters, removes URLs and e-mail addresses, keeps only letter tokens, drops those starting with a capital letter (mostly proper nouns and sentence starts) and lowercases the rest. Documents with fewer than 20 tokens after normalisation are unreliable. With numpy >= 2, the fasttext 0.9.3 package fails in predict; use numpy < 2.

Training

• Data: 200 random shards (4 %) of the MADLAD-400 Spanish clean split (revision 9d886a76bd8fa69b294f2dd3843dacb8388ee5a5), 10M documents.
• Weak labels from the domain: documents on .es sites → es_ES; documents on Latin American country domains (.mx, .ar, .cl, .pe, .ve, .uy, .ec, .cu, .do, .bo, .py, .gt, .hn, .sv, .ni, .cr, .pa, .pr) → es_LATAM. .co was left out (widely used as a generic domain); all other domains were unlabelled.
• At most 200 documents per website, so large sites do not dominate; 5 % of websites held out for validation, so the validation score is not inflated by site-specific boilerplate.
• Balanced: 912,477 training documents per class; 47,505 validation documents per class.
• fastText: 5 epochs, learning rate 0.5, word bigrams, minimum count 5, 2M hash buckets.

Evaluation

On the held-out websites, against the domain labels (which are themselves noisy, so the real precision is likely somewhat higher):

Threshold on p(es_ES)	Precision (es_ES)	Recall (es_ES)
0.5	0.917	0.928
0.7	0.941	0.876
0.8	0.951	0.831
0.9	0.963	0.743
0.95	0.971	0.646
0.98	0.978	0.512

Independent check with dialect markers that the classifier does not use in any special way: among validation documents predicted es_ES, 94 per 1,000 contain vosotros forms and 1 per 1,000 voseo; among those predicted es_LATAM, 7 and 13. On the whole MADLAD-400 Spanish split, documents scored ≥ 0.9 have 175 vosotros-marker and 5 voseo-marker documents per 1,000, against 12 and 32 for documents scored < 0.1: the model tracks how the text is written, not only its topic.

Averaged per website (≥ 5 documents), scores are very stable: only 0.2–0.3 % of .es or Latin American websites land on the opposite side.

Limitations

• Documents only. On single sentences the scores are overconfident and close to random: clearly Spanish sentences get scores near 0. Aggregate over a document or a website.
• Two classes only. Latin America is one class; the model does not tell Mexico from Argentina.
• Weak labels. Trained on domains, not on human judgements: Latin American text on .es sites (and the reverse) is part of the training noise.
• Topic leakage is reduced, not removed. Removing capitalized words hides most place names and brands, but lowercase topical words (euros, ayuntamiento, pesos) still carry signal.
• Neutral text has no dialect evidence; its score reflects weak cues.
• Web domain. Trained on MADLAD-400 web text; other genres (speech transcripts, literature, social media) may behave differently.

License and attribution

• License: Apache-2.0 for the model and normalize.py.
• Training data: MADLAD-400 (ODC-BY 1.0 on Hugging Face; its README also states CC BY 4.0), built from Common Crawl. Please credit MADLAD-400:

@misc{kudugunta2023madlad400,
      title={MADLAD-400: A Multilingual And Document-Level Large Audited Dataset},
      author={Sneha Kudugunta and Isaac Caswell and Biao Zhang and Xavier Garcia and Christopher A. Choquette-Choo and Katherine Lee and Derrick Xin and Aditya Kusupati and Romi Stella and Ankur Bapna and Orhan Firat},
      year={2023},
      eprint={2309.04662},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Suggested reference for this model:

@misc{spanish_variety_fasttext,
      title={Spanish variety classifier: Spain (es-ES) vs Latin America (fastText)},
      author={Ugiat},
      organization={Ugiat Technologies},
      year={2026},
      howpublished={\url{https://huggingface.co/Ugiat/spanish-variety-fasttext}}
}

Contact

Ugiat Technologies, ugiat@ugiat.com.