What is NovaSR
NovaSR is an open-source audio super-resolution model that, with only 52KB, can upgrade 16kHz low-sample-rate audio (such as telephone quality) to 48kHz high-sample-rate audio (such as studio-level quality). The model predicts and generates high-frequency information through a neural network, achieving significant improvement in sound quality. NovaSR processes extremely fast, reaching 3600x real-time on a single A100 GPU. The model is compact and can be embedded in edge devices such as TWS earbud chips. NovaSR is suitable for scenarios such as speech restoration, TTS output enhancement, and real-time communication, demonstrating the huge potential of small models in the field of audio processing.
Main features of NovaSR
-
Sound quality improvement: Converts telephone quality (16kHz) to studio-level quality (48kHz), enhancing the clarity and presence of the sound.
-
Real-time processing: Achieves 3600x real-time processing speed on a single A100 GPU, suitable for real-time audio enhancement scenarios.
-
Edge deployment: The model is only 52KB and can be easily embedded in devices such as TWS earbuds and smartwatches, enabling low-power, zero-latency sound quality enhancement.
-
Dataset enhancement: The model can be used to improve the sound quality of low-sample-rate audio datasets and unify audio standards.
-
Speech restoration: Suitable for scenarios such as renovating old podcast recordings and improving the quality of meeting recordings.
Technical principles of NovaSR
-
Spectrum learning: The model learns the spectral patterns of a large amount of high-quality audio to grasp the differences and correlations between audio at different sample rates.
-
High-frequency reconstruction: When low-sample-rate audio (such as 16kHz) is input, NovaSR uses a neural network to predict and generate the originally missing high-frequency part (16kHz~24kHz), upgrading the audio to 48kHz.
-
Lightweight architecture: NovaSR adopts a minimalist network architecture, containing fewer than 10 small convolutional layers and the Snake activation function, and is based on the BigVGAN design, balancing model size and sound quality performance.
-
Efficient inference: By optimizing the network structure and computation flow, NovaSR achieves extremely high inference speed and can run quickly on ordinary devices.
NovaSR project address
- GitHub repository：https://github.com/ysharma3501/NovaSR
- Online experience Demo：https://huggingface.co/spaces/YatharthS/NovaSR
Application scenarios of NovaSR
-
Speech and content creation: The model can improve the sound quality of podcasts, meeting recordings, and self-media speech, enhancing the user experience.
-
TTS and ASR engineering: Enhances the clarity of TTS output, improves the sound quality of low-sample-rate audio datasets, and optimizes speech recognition results.
-
Communications and real-time systems: Improves sound quality in real time in VoIP, customer service calls, and live streaming voice links, reducing computing costs.
-
Audio dataset enhancement: The model can upgrade low-sample-rate audio datasets to high definition, facilitating subsequent audio analysis and machine learning tasks.
-
Edge devices: The model can be integrated into devices such as TWS earbuds and smartwatches to achieve low-power, zero-latency sound quality enhancement.