What is TIGER
TIGER (Time-frequency Interleaved Gain Extraction and Reconstruction Network) is a lightweight speech separation model proposed by a research team at Tsinghua University. Through a time-frequency interleaved modeling strategy, combined with band splitting and multi-scale attention mechanisms, it significantly improves speech separation performance while reducing the number of parameters and computational cost. The core of TIGER lies in the innovative Time-frequency Interleaved Gain Extraction module (FFI), which can efficiently integrate time and frequency information to better extract speech features. The model introduces the Multi-Scale Selective Attention module (MSA) and the Full-band/Frame Attention module (F³A), further optimizing feature extraction capability.
TIGER's Main Features
- Efficient Speech Separation: Through the innovative Time-frequency Interleaved Gain Extraction module (FFI) and multi-scale attention mechanism, TIGER can efficiently separate different speakers in mixed speech.
- Low Computational Cost and Low Parameter Count: After compressing 94.3% of the parameters and 95.3% of the computational cost, the model's performance remains comparable to the current state-of-the-art models.
- Adaptation to Complex Acoustic Environments: TIGER uses the EchoSet dataset to simulate noise and reverberation in real scenarios, improving the model's robustness in complex environments.
TIGER's Technical Principles
- Time-frequency Interleaved Modeling Strategy: The core of TIGER lies in the Time-frequency Interleaved Gain Extraction module (FFI), which effectively integrates time-frequency features by alternately processing time and frequency information. The module contains a frequency path and a frame path, each containing a Multi-Scale Selective Attention module (MSA) and a Full-band/Frame Attention module (F³A), capable of fusing local and global information to improve speech separation performance.
- Band Splitting: The energy of speech signals is unevenly distributed across different frequency bands. Mid-low frequency bands contain more speech information, while high frequency bands contain more noise and details. Through the band splitting strategy, TIGER divides the frequency band into sub-bands of different widths, reducing computational cost while allowing the model to focus on key frequency bands.
- Multi-Scale Attention Mechanism: TIGER introduces the Multi-Scale Selective Attention module (MSA), which fuses local and global information through multi-scale convolutional layers and selective attention mechanisms, enhancing the model's ability to extract multi-scale features.
- Overall Pipeline: TIGER's overall pipeline consists of five parts:
- Encoder: Converts the mixed audio signal into a time-frequency representation through the Short-Time Fourier Transform (STFT).
- Band Splitting Module: Divides the entire frequency band into multiple sub-bands, each converted to a unified feature dimension through one-dimensional convolution.
- Separator: Composed of multiple Time-frequency Interleaved Gain Extraction modules (FFI), used to extract the acoustic features of each speaker.
- Band Restoration Module: Restores the sub-bands to the full frequency band range.
- Decoder: Generates the clear speech signal of each speaker through the Inverse Short-Time Fourier Transform (iSTFT).
TIGER's Project Address
- Project Website：https://cslikai.cn/TIGER/
- Github Repository：https://github.com/JusperLee/TIGER
- arXiv Technical Paper：https://arxiv.org/pdf/2410.01469
TIGER's Application Scenarios
- Meeting and Speech Recording: In meeting or speech scenarios with multiple speakers, TIGER can efficiently separate the speech of different speakers, improving the efficiency and accuracy of meeting records.
- Video Editing and Production: In video content creation, TIGER can accurately separate the host's voice from background audio or the voices of other people, facilitating post-production and editing.
- Movie Audio Processing: TIGER performs excellently in movie audio separation tasks, capable of separating vocals, music, and sound effects, enhancing the flexibility and quality of audio processing.
- Intelligent Voice Assistants: In intelligent voice assistant applications, TIGER can help separate the user's voice from background noise, improving the voice interaction experience.