Granite-4.0-1b-speech - IBM's open-source multilingual speech model

What is Granite-4.0-1b-speech

Granite-4.0-1b-speech is IBM's open-source 1-billion-parameter multilingual speech model. It supports speech recognition in English, French, German, Spanish, Portuguese, and Japanese, as well as bidirectional translation with English, and supports unidirectional translation from English to Italian and Mandarin. The model is based on a 16-layer Conformer encoder and Q-Former projection layer architecture. On the HuggingFace Open ASR Leaderboard, its average word error rate is only 5.52%. It supports speculative decoding to accelerate inference, and its compact size makes it suitable for enterprise-level speech transcription and edge device deployment.

Main features of Granite-4.0-1b-speech

• Multilingual speech recognition: Supports automatic speech recognition in six languages: English, French, German, Spanish, Portuguese, and Japanese, and can convert speech input into corresponding text output.
• Bidirectional speech translation: The model implements bidirectional automatic speech translation between English and the six languages mentioned above, allowing users to conduct real-time speech interpretation and communication between different languages.
• Unidirectional speech translation: The model supports unidirectional speech translation from English to Italian and from English to Mandarin.
• Keyword bias recognition: The model has keyword list prompting capability. Users can add specific terms at the end of the prompt to enhance recognition accuracy for personal names, place names, and professional abbreviations.
• Safety protection mechanism: When receiving audio prompts in unfamiliar or abnormal formats, the model automatically falls back to the default transcription mode, effectively reducing security risks caused by adversarial input attacks.
• Efficient inference acceleration: The model supports speculative decoding technology and, combined with optimized Conformer encoder training, achieves high-speed inference with a 280x real-time factor.
• Edge device adaptation: Thanks to its compact architecture design of only 1 billion parameters, the model can be efficiently deployed and run on resource-constrained edge devices.

Key information and usage requirements for Granite-4.0-1b-speech

• Developer：IBM 。
• Core capabilities: Supports recognition of six languages: English, French, German, Spanish, Portuguese, and Japanese, and bidirectional translation with English, plus English-to-Italian and English-to-Mandarin translation.
• Environment requirements: Transformers>=4.52.1, torchaudio, soundfile; supports CUDA and Apple Silicon.
• Audio requirements: Mono, 16kHz sampling rate, introduced via the<|audio|>token.
• Safety recommendations: Use with Granite Guardian to detect risky content.

Core advantages and value of Granite-4.0-1b-speech

• Extreme efficiency: A lightweight architecture of only 1 billion parameters achieves inference speed with a 280x real-time factor, greatly reducing computational resource consumption while maintaining excellent recognition performance, making it especially suitable for deployment and operation on edge devices and in resource-constrained environments.
• Accurate recognition: The model achieved an average word error rate of 5.52% on the HuggingFace Open ASR Leaderboard benchmark, and reached an excellent performance of 1.42% on the Librispeech Clean dataset. Its accuracy is comparable to similar models with larger parameter counts.
• Multilingual coverage: A single model simultaneously supports speech recognition in six languages—English, French, German, Spanish, Portuguese, and Japanese—as well as bidirectional speech translation between these languages and English, meeting the multilingual processing needs of multinational enterprises in globalized operations.
• Enterprise Security: The model has built-in security protection mechanisms. When unfamiliar or abnormally formatted input prompts are detected, it automatically falls back to the default transcription mode, effectively avoiding the risk of adversarial attacks. Together with the Apache 2.0 open-source license, it provides legal protection for enterprise commercial use.
• Flexible and Easy to Use: The model natively supports multiple mainstream inference frameworks such as Transformers, vLLM, and MLX, and provides keyword list biasing functionality. It allows users to enhance the recognition accuracy of specific terms, names, and abbreviations through custom prompts, adapting to diverse business scenarios.

How to use Granite-4.0-1b-speech

• Install dependencies: Run pip install transformers torchaudio soundfile to install the necessary libraries. If using Apple Silicon, install mlx-audio。
• Load the model: Use AutoProcessor.from_pretrained and AutoModelForSpeechSeq2Seq.from_pretrained to load the processor and model respectively, and set torch_dtype=torch.bfloat16 to enable efficient inference.
• Prepare audio: Load a mono audio file with a 16kHz sampling rate, ensuring the audio dimensions meet the model input requirements.
• Build prompt: Use <|audio|> to mark the introduction of audio, and combine with apply_chat_template to generate a dialogue-format prompt. A keyword list can be added at the end to enable biased recognition.
• Run inference: Call the processor to convert the prompt and audio into model inputs, generate output through model.generate , and decode to obtain the final text result.
• Deployment methods: Choose vLLM for high-concurrency service-oriented deployment, or use MLX to run locally on Apple Silicon devices.

Project address of Granite-4.0-1b-speech

• HuggingFace Model Library：https://huggingface.co/ibm-granite/granite-4.0-1b-speech#granite-40-1b-speech

Comparison of Granite-4.0-1b-speech with similar competing products

Dimension	Granite-4.0-1b-speech	OpenAI Whisper
Language support	6 input languages, focusing on major European, American, and Asian languages	99 languages, covering a wider range including Chinese recognition
Model scale	1 billion parameters, lightweight and efficient	Multiple choices of parameters from tiny to large
Special features	Keyword biasing, speculative decoding acceleration	Strong general capabilities, multi-task end-to-end
Open-source license	Apache 2.0, commercial-friendly	MIT license, also open-source
Applicable scenarios	Enterprise-level edge deployment, real-time translation	Multilingual general recognition, research and exploration

Application scenarios of Granite-4.0-1b-speech

• Meeting transcription: The model can convert multilingual meeting speech into text records in real time, supporting recognition of participants' speech in English, French, German, Spanish, Portuguese, and Japanese, and automatically generating structured meeting minutes.
• Cross-border customer service support: Supports handling multilingual customer calls, enabling real-time speech-to-text transcription and bidirectional translation with English, helping customer service personnel understand customer needs in different languages and respond accurately.
• Video subtitle generation: Automatically generate accurate subtitles for multilingual video content. The keyword bias feature ensures accurate recognition of technical terms, personal names, and place names, improving subtitle quality.
• Real-time simultaneous interpretation: Provide speech-to-speech real-time translation assistance in international conferences or business negotiations, supporting mutual translation between six languages and English, lowering the threshold for cross-language communication.