2.8 trillion tokens in 22 Indian languages: what NVIDIA and AI4Bharat are releasing
A translated pretraining corpus covering every scheduled Indian language, an open pipeline to generate more, and a first model trained on it. What is in it, and what translation cannot fix.
On 16 October, NVIDIA and AI4Bharat at IIT Madras published details of Nemotron-pretraining-Indic-v1: about 2.8 trillion tokens of pretraining data across all 22 scheduled Indian languages. The dataset itself is listed as coming soon. The pipeline used to build it is already open.
What is being released
- The corpus. About 2.8 trillion tokens, counted with the Gemma-4 tokenizer: 2,601 billion in native scripts and 224.6 billion in romanized form. It spans 34 language-script configurations, such as
hi-Deva-IN(Hindi in Devanagari) andhi-Latn-IN(romanized Hindi). - The pipeline. A Nemotron Indic translation plugin for NVIDIA's NeMo Data Designer, so other teams can generate similar data.
- A report and an eval suite. A companion report on Hugging Face (nvidia/nemotron-indic-pretraining-report), and Indic-LLM-Eval-Suite, a benchmark collection AI4Bharat is still building with NVIDIA's support.
The announcement does not name a license. The authors advise reading the dataset card and license before training on it.
How it was made
This is translated data, not text originally written in Indian languages. The source is English documents from 15 subsets of NVIDIA's Nemotron-CC-v2 corpus, covering web text, Q&A, maths, code and specialised content. Every output keeps its source ID, so each translated document can be traced back to the original.
There are two workflows:
- Twelve languages (Bengali, Gujarati, Hindi, Kannada, Malayalam, Marathi, Nepali, Odia, Punjabi, Tamil, Telugu, Urdu). Gemma-4-31B-it translates whole documents, then scores each output against the English source on an MQM-style rubric. Anything below 75 out of 100 is repaired using that feedback and scored again. Repair recovered 438.4 billion tokens, which are included in the total. These twelve also get romanized versions, made by converting accepted native-script translations.
- Ten languages (Assamese, Bodo, Dogri, Kashmiri, Konkani, Maithili, Manipuri, Sanskrit, Santali, Sindhi). Bodhan AI's 8B Indic Translate model translates, and Gemma-4-31B-it scores against the same 75-point bar. These are native script only. Santali was translated in overlapping windows, and Manipuri was translated into Bengali script and then converted to Meitei Mayek.
Does it help?
NVIDIA's first test is Nemotron 3.5 Super VL, which saw 10 billion Indic tokens during continued pretraining, drawn from the twelve-language subset plus natural Indic text. The configured budget was 100 billion tokens, so this is a small first dose.
That stage lifted Indic benchmarks by 4.98 to 9.82 percentage points across MILU, Indic MMLU-Pro, Indic ARC-Challenge, Indic ARC-Easy and Indic GSM8K. Against Nemotron 3 Super, the new model scored higher on five of six task benchmarks. The exception was IFEval 1.5, an instruction-following test, which dropped by 2.04 points.
What translation cannot do
The authors are direct about the limits:
- Translation does not replace text originally written in Indian languages, and does not guarantee coverage of local knowledge and culture.
- Quality checks are mostly automated, with sampled human review whose coverage varies by language.
- In the twelve-language workflow, the same model translates and judges its own output.
- The ten lower-resource languages need broader validation by native speakers.
- Deduplication is by source ID, not by meaning.
One more, which is our reading rather than theirs: the romanized data is converted from native-script translations, so it is transliteration. It is not the code-mixed Hinglish people actually type, and models that need to handle that will still need natural text.
Why it matters here
Most scheduled languages beyond Hindi and the large Dravidian languages have very little web text, which makes training data one of the hardest constraints on Indian-language models. A large, traceable, openly generated corpus lowers that floor for every team building Indic models, from startups to the foundation-model work funded under the IndiaAI Mission.
The open pipeline may matter more than the corpus. A team can run the same translate, score and repair loop over its own domain text, such as legal, agricultural or health documents, and keep the source trail.
What to watch
The actual dataset release and its license, the Indic-LLM-Eval-Suite, and whether models trained on natural Indian-language text keep an edge over translated corpora.
Source: Hugging Face blog, 16 October 2026. The wider picture: India's AI startup ecosystem in 2026.
Find these on the Radar
Every tool here lives on Kapyn Radar. Save the ones that fit into a Loadout and find them again.
