NeMo

NVIDIA NeMo Speech is a scalable generative AI framework aimed at researchers and PyTorch developers working on speech model research, primarily covering areas such as automatic speech recognition (ASR), text-to-speech (TTS), and speech large models (Speech LLM). Its goal is to help users efficiently create, customize, and deploy new AI models by reusing existing code and pretrained model checkpoints.

Key capabilities and recent releases The repository has shifted to focus on audio, speech, and multimodal large models. Important releases listed in the materials include:

• 2026-07: MagpieTTS v2607 released, adding 3 new languages: Arabic, Korean, and Portuguese, bringing the total to 9 languages together with the existing English, Spanish, German, French, Vietnamese, Italian, Chinese, Hindi, and Japanese.
• 2026-06: Nemotron-3.5-ASR-Streaming-0.6B released, supporting 40 languages, with latency controllable between 80ms and 1s, and a single H100 supporting 240 to 2400 concurrent streams, based on the cache-aware Fastconformer architecture.
• 2026-04: Parakeet-unified-en-0.6b released, with a single model supporting both high-quality offline and streaming (minimum latency 160ms) English inference, and supporting punctuation and capitalization.
• 2026-03: Nemotron 3 VoiceChat early access released, based on the Nemotron Nano v2 LLM backbone with Nemotron speech and TTS decoders, providing full-duplex, natural, interruptible low-latency conversation.
• 2026-03: Nemotron-Speech-Streaming v2603 updated, trained on a larger and more diverse corpus, with lower WER in all latency modes.
• 2026-03: MagpieTTS v2602 released, supporting 9 languages.
• 2026-01: Nemotron-Speech-Streaming released, with a single checkpoint allowing users to choose the optimal point on the latency-accuracy Pareto curve.
• 2026-01: MagpieTTS v2512 released.
• 2025-08: Parakeet V3 and Canary V2 released, supporting speech recognition and translation for 25 European languages.
• 2025-06: Canary-Qwen-2.5B released, achieving a record 5.63% WER on the English Open ASR Leaderboard.

Versions and containers NeMo Speech 3.0 has been released as v3.0.0 and is provided in the 26.07.00 NeMo Speech NGC container. The last NeMo version before the repository split was v2.7.3, provided in the 26.04 NeMo NGC container.

Usage and installation NeMo Speech can be used with the user's choice of Python, PyTorch, and CUDA versions, with the minimum requirements being: Python 3.12 or above; PyTorch 2.7 or above (CPU, CUDA, etc. are all acceptable); training requires an NVIDIA GPU + CUDA, and inference is recommended. If you already have a Python/PyTorch/CUDA environment that meets the minimum requirements, NeMo Speech will install on top of it without replacing the existing environment. The version combinations pinned in uv.lock and provided in the official containers are Python 3.13, PyTorch 2.11 + CUDA 12.9, or PyTorch 2.12 + CUDA 13.2. These are the combinations officially tested and supported, but they are not hard requirements.

Installation methods include:

1. Install from source using uv (recommended): install a supported combination (Python 3.13, PyTorch 2.12, CUDA 13.2) into .venv/, and install NeMo Speech in editable mode. Add --group test to install the test suite, or add --group docs to build the documentation; run tools via uv run <cmd>, or activate with source .venv/bin/activate. On Linux, cu12 and cu13 are mutually exclusive, and only one of them can be passed (default cu13). To exactly reproduce the container baseline, add --locked --python 3.13.
2. Docker (ready to use, officially supported combination): the latest prebuilt image is the 26.07.00 NGC container. You can also build the container from source (default CUDA 13 / H100+); for A100, you can set GPU_TARGET=a100,A100 supports both CUDA 12 and CUDA 13 (the default CUDA 13 base image is recommended).
3. Install from PyPI using pip (fallback option, bring your own version): First install your own PyTorch (any version ≥ 2.7), then add NeMo Speech, and the existing build will be preserved. uv pip is used the same way as pip. Note: When bringing your own stack, do not use uv sync --locked, because it will apply uv.lock and replace your Python/PyTorch/CUDA; if you need to pull the official pinned PyTorch build, you can add the CUDA extra and the matching wheel index, and you need to use --extra-index-url.

About the Automodel backend: The Automodel backend can run without any compilation dependencies, and you can optionally use dedicated accelerated backends (Transformer Engine, FlashAttention, Mamba, grouped-GEMM/MoE, DeepEP) for better performance. These source-built kernels come from the compiled (Hopper/Blackwell) or compiled-a100 (A100) extras and are built by docker/Dockerfile (GPU_TARGET=h100plus / a100).

Notes Since PyTorch 2.6, torch.load uses weights_only=True by default. Some model checkpoints may require weights_only=False; in this case, you can set the environment variable TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=1 before running code that uses torch.load. However, you should do this only for trusted files, since loading such files from untrusted sources may pose an arbitrary code execution risk.

Applicable scenarios and limitations This framework is suitable for research and development related to speech recognition, speech synthesis, speech large models, and multimodal applications, and supports customization and deployment starting from pretrained checkpoints. For technical documentation, see the NeMo Speech Developer Documentation. The project welcomes community contributions; see CONTRIBUTING.md. It is licensed under the Apache License 2.0. The materials do not provide specific pricing, price, or commercial licensing details.