A new study shows that mixing high‑resource language data into low‑resource pre‑training beats extensive hyperparameter tuning. The authors compare aggressive weight‑decay regularization with a tunable mixing ratio, finding the latter expands the training distribution and improves generalization in data‑constrained settings. This offers a practical shortcut for multilingual model builders.
Opening Kapyn…