whisper-ctranslate2 is a Whisper command-line client based on CTranslate2, compatible with the original OpenAI Whisper client, developed and open-sourced by Softcatala. It uses the Whisper implementation of CTranslate2 and faster-whisper, and is up to 4 times faster than openai/whisper at the same accuracy while using less memory; with batch inference it can be up to 16 times faster.
Main capabilities:
- Compatible with the OpenAI Whisper CLI, making migration easy.
- Provides a Docker image with preloaded models.
- Supports speaker diarization (speaker recognition).
- Supports voice activity detection (VAD) filtering to improve quality.
- Can load your own fine-tuned Whisper models.
- Supports real-time transcription from a microphone.
- Supports color-coded confidence visualization.
Installation methods: To install the latest stable version, simply enter the install command (the material does not give the specific command text). Using the prebuilt Docker image: first pull the image, which contains the small, medium, and large-v2 models. Use --gpus "device=0" to access the GPU; if there is no GPU, remove this parameter; "$(pwd)":/srv/files/ maps the current directory to /srv/files/ inside the container. If you need to use a model not included in the image, you can create a derived Docker image with that model preloaded, or use a Docker volume to persist and share the model files.
CPU and GPU support: GPU and CPU support is provided by CTranslate2. Compatible with x86-64 and AArch64/ARM64 CPUs, and integrates multiple backends optimized for these platforms: Intel MKL, oneDNN, OpenBLAS, Ruy, and Apple Accelerate. GPU execution requires the system to have the NVIDIA libraries cuBLAS for CUDA 12 and cuDNN 9 for CUDA 12 installed; please refer to the CTranslate2 documentation. By default, the best available hardware is selected for inference; the --device and --device_index options can be used to manually control the selection.
Usage: The command line is the same as OpenAI Whisper. The Whisper translation task translates transcription from the source language into English (the only supported target language). Use --help to view all supported options and their help.
CTranslate2-specific options: In addition to the OpenAI Whisper command-line options, options specific to CTranslate2 or whisper-ctranslate2 are also provided.
Batch inference: Batch inference transcribes each segment independently, and can provide an additional 2x to 4x speedup. --batch_size can be used to specify the maximum number of parallel requests for model decoding. Batch inference uses a voice activity detection (VAD) filter and ignores the following parameters: compression_ratio_threshold, logprob_threshold, no_speech_threshold, condition_on_previous_text, prompt_reset_on_temperature, prefix, hallucination_silence_threshold.
The --compute_type option accepts the values default, auto, int8, int8_float16, int16, float16, float32, indicating the quantization type to use. On CPU, int8 performs best.
Loading a model from a directory: The --model_directory option allows specifying the directory of the CTranslate2 Whisper model to load. For example, loading your own quantized version of a Whisper model or your own fine-tuned Whisper version. The model must be in CTranslate2 format.
Using a voice activity detection (VAD) filter: The --vad_filter option enables voice activity detection (VAD) to filter out parts of the audio that contain no speech. This step uses the Silero VAD model. The VAD filter accepts several additional options to determine the filtering behavior: a probability above a certain value is considered speech; final speech chunks shorter than min_speech_duration_ms are discarded; the maximum duration of speech chunks in seconds, with longer ones being split at the timestamp of the last silence.
The --print_colors True option prints the transcribed text using a color-coding strategy based on whisper.cpp to highlight highWords with confidence or low confidence.
Live transcription from microphone: --live_transcribe True option activates live transcription mode from the microphone.
Speaker diarization (speaker recognition): Supports speaker diarization using pyannote.audio to identify speakers. Currently supported at the segment level. To enable speaker diarization, follow these steps:
- Install pyannote.audio using pip install "pyannote.audio==4.0".
- Accept the pyannote/speaker-diarization-community-1 user conditions.
- Create an access token at hf.co/settings/tokens. Then, when executing, pass the HuggingFace API token as a parameter to enable speaker diarization. Speaker names will be added to the output files (such as JSON, VTT, and SRT files), for example: [SPEAKER_00]: There is a lot of people in this room. The --speaker_name SPEAKER_NAME option allows using your own string to identify speakers.
Use cases: Suitable for users who need local, fast, and free speech transcription, including meeting notes, subtitle generation, podcast transcription, live dictation, etc. Supports CPU and GPU, suitable for different hardware environments.
Limitations: Translation tasks only support English as the target language. Speaker diarization is currently only supported at the segment level. Batch inference ignores some parameters. GPU execution requires installing specific NVIDIA libraries. Basic transcription does not require PyTorch, but speaker diarization requires additionally installing pyannote.audio, accepting user conditions, and creating an access token.
