Nvidia's Nemotron 3.5 Transcribe: Open Model Covers 40 Languages and 8 Speakers

Nvidia released Nemotron 3.5 Transcribe on October 28, a free, open-weight speech model that transcribes recordings in 40 languages, translates 31 of them to or from English, and labels up to eight speakers in a single pass. The company says it ranks first among open-weight models on the Hugging Face Open ASR Leaderboard for English and can run on one GPU, from a Jetson AGX Thor module to Blackwell data-center cards.

The release matters because transcription is one of the most common AI jobs inside companies, from meeting notes to medical dictation, and most strong options are paid cloud APIs. Nvidia also published Granary v2, the open data pipeline and dataset behind the model, plus fine-tuning recipes, so developers can adapt it to their own vocabulary without sending audio to a third party.

Key facts at a glance

• 40 languages for transcription, 31 for translation to or from English, and speaker attribution in 13.
• Recordings up to one hour, a 65,536-token context, and up to eight labeled speakers.
• 30 billion-parameter language model, about 34 billion parameters including the audio encoder, with roughly 4.3 billion active per token.
• BF16, FP8 and NVFP4 checkpoints, released under the OpenMDW-1.1 license, which Nvidia says allows commercial use.
• Granary v2: 463,062 hours of speech across 48 languages and 16 source corpora in its public manifest, up from 25 languages in the first version.

Context: why speech models are suddenly busy

Voice AI has had a crowded month. Mistral shipped Voxtral 3 earlier in October, ElevenLabs launched Realtime for voice agents, and Salesforce released its Launa-Omni voice model. Nvidia already had small open speech models such as Parakeet, at 1.1 billion parameters, aimed at fast English transcription.

Transcribe is a different kind of model. Nvidia describes it as a Speech-Augmented Language Model (SALM): an audio encoder feeds a language model, so the same checkpoint can follow written instructions. Its language model is a hybrid that mixes Mamba-2 layers, which handle long sequences cheaply, with Transformer layers in a mixture-of-experts (MoE) design, where only some of the model's "experts" run for each token. It is built on Nvidia's Nemotron 3.5 Lightning 30B-A3B base.

That design is why Nvidia can claim high accuracy with a modest compute bill. Only about 4.3 billion parameters are active at a time, even though the full model is much larger.

One model, many transcription jobs

The headline feature is prompting. Instead of a fixed output, developers tell the model what they want in plain language, such as "Transcribe this recording and label each speaker," "Translate this audio into English," or "What decisions were made in this meeting?"

Nvidia's model card lists the supported tasks: verbatim or cleaned-up transcription, contextual transcription, speaker-attributed transcription, speech translation, language identification, word-level timestamps and audio question answering. Language support differs by task, and audio question answering is trained in English only.

Contextual prompts handle jargon without retraining. Nvidia's example for healthcare asks the model to transcribe a medical discussion with possible terms such as "HbA1c" and "metformin," used only when the audio supports them. The company's own advice is practical: as with any speech model, spot-check names, speaker changes and answers against the recording.

There is one important limit. Transcribe works on recorded audio, not live streams, so it fits meeting archives, podcasts, interviews and dictation files rather than real-time captions or voice agents that must respond instantly.

Benchmarks and how to read them

Nvidia's published results are strong, with the usual caveat that they are the company's own selection of tests. Lower word error rate (WER) is better.

Nvidia says the model card figures were captured on October 12. On the public English leaderboard itself, the company reports a 5.33% average WER, first among open-weight models when paid API models are excluded.

Independent leaderboards add some weight. Nvidia says Transcribe ties for first among open-weight models on Artificial Analysis's speech-to-text index at 2.8% WER, with about 4 billion active parameters against 24 billion for the model it ties. For speaker attribution, it reports the lowest diarization error rate, 18.26% across 21 languages, on David AI's multilingual leaderboard, narrowly ahead of Nvidia's own dedicated Nemotron 3 Diarization model at 18.38%.

A counterpoint is worth keeping in mind. The comparison table pits a 30 billion-parameter model against rivals as small as 0.6 billion and 1.7 billion parameters, which are far cheaper to run on modest hardware. For plain English transcription on a laptop, a smaller model may still be the more sensible choice.

What it takes to run it

The full-precision model needs serious hardware. Nvidia's quick-start example serves it on one H100 80 GB GPU with vLLM 0.32.0 and the NeMo Speech main branch, and SGLang-Omni also supports it from day one. Quantized FP8 and NVFP4 versions are meant to bring it to smaller machines, and Nvidia names RTX Spark, DGX Spark and Jetson AGX Thor as local targets. Local deployment through NeMo-Speech.cpp is listed alongside the quantized versions.

Throughput numbers depend on Nvidia's own software. The company says its optimized Nemotron Transcribe NIM, on one RTX PRO 6000 Blackwell Server Edition GPU, can process up to 2,238 seconds of audio per second and return a 30-second transcript in 0.13 seconds. Nvidia also says those serving optimizations are not yet part of the open-source release, and that the vLLM integration shipped on launch day is expected to be roughly two times slower.

For teams that do not want to host it, Nvidia lists hosted endpoints from Baseten, DeepInfra and Simplismart.

Granary v2 and fine-tuning

The second half of the release is about adaptation. Granary v2's public manifest provides three transcript forms for each clip, verbatim, written form with filler words, and clean written form, plus captions, question-and-answer pairs, translations and quality flags. Source audio has to be downloaded separately under each corpus's own license.

Nvidia's medical case study shows the payoff. On a benchmark of recordings from Indian doctors and medical students, a better prompt cut WER from 30.8% to 23.8%, and fine-tuning brought it to 11.8% on the full set. On held-out speakers, fine-tuning cut WER from 22.44% to 12.17%, a 45.8% relative reduction, using about 14 GPU-hours on one GB300. General English accuracy slipped slightly, from 1.54% to 1.69% on a standard test set.

What it means for you

For developers, this is a credible open alternative to paid transcription APIs for recorded audio, especially for multilingual meetings or speaker-labeled transcripts. Because the weights can run on your own hardware, audio does not have to leave your network, which helps with privacy reviews.

For teams in the US, UK, Canada and Australia handling sensitive recordings, such as clinics, law firms or HR departments, self-hosting can simplify data-handling questions. It does not remove them: you still need consent for recording, retention rules and human review of anything used for decisions.

For hobbyists, the quantized checkpoints are the ones to watch. Plan around the hardware first, since the reference setup is a data-center GPU.

What happens next

Nvidia says it aims to upstream its faster serving optimizations to NeMo Speech and vLLM later, and an NVFP4 version used in its throughput tests is described as forthcoming. Independent benchmark results from outside users will show whether the leaderboard lead holds on real-world audio. Early adopters are already building on it: Agora, which runs a real-time voice and video platform, used the model's word-level timestamps in a classroom demo that shows a transcript next to a recorded lesson. ModelScope also supports fine-tuning through its ms-swift framework.

Open questions remain. Nvidia's figures were captured on October 12 and selected by the company, the faster serving stack is not yet open source, and live-streaming use is out of scope for now. Developers comparing it with paid APIs should weigh hardware and hosting costs against per-minute pricing for their own volume.

Conclusion

Nemotron 3.5 Transcribe pushes open speech models from "good enough for English" toward a promptable, multilingual tool that also tells you who said what. The trade-off is size and hardware.

• Test it on your own audio before switching, and compare against a smaller model such as Parakeet for English-only jobs.
• Use contextual prompts first for jargon; fine-tune only if errors remain.
• Budget for hardware: start with FP8 or NVFP4 if you lack an 80 GB GPU.
• Keep humans in the loop for names, numbers and speaker changes in anything important.