Why Noise Reduction?
In many speech-related application scenarios, the presence of noise can seriously affect performance and user experience. For example:
- Speech Recognition: Noise reduces the accuracy of speech recognition, especially in low signal-to-noise ratio environments.
- Voice Cloning: Noise can degrade the naturalness and clarity of synthesized speech based on reference audio.
Speech noise reduction can solve these problems to some extent.
Common Noise Reduction Methods
Currently, there are several main methods for speech noise reduction:
- Spectral Subtraction: This is a classic noise reduction method with a simple principle.
- Wiener Filtering: This method works well for stable noise, but has limited effectiveness for variable noise.
- Deep Learning: This is currently the most advanced noise reduction method. By utilizing powerful deep learning models, such as Recurrent Neural Networks (RNN), Convolutional Neural Networks (CNN), and Generative Adversarial Networks (GAN), it learns the complex relationships between noise and speech, achieving more accurate and natural noise reduction effects.
ZipEnhancer Model: Deep Learning Noise Reduction
This tool is based on the Tongyi Lab's open-source ZipEnhancer model, and provides an easy-to-use interface and API, allowing everyone to easily experience the charm of deep learning noise reduction.
The project is open-sourced on GitHub
The core of the ZipEnhancer model is the Transformer network structure and multi-task learning strategy. It can not only remove noise, but also enhance speech quality and eliminate echo at the same time. The working principle is as follows:
- Self-Attention Mechanism: Captures important long-term relationships in speech signals and understands the contextual information of the sound.
- Multi-Head Attention Mechanism: Analyzes speech features from different perspectives to achieve more refined noise suppression and speech enhancement.
How to Use This Tool?
Windows Pre-packaged Version:
- Download and unzip the pre-packaged version (https://github.com/jianchang512/remove-noise/releases/download/v0.1/win-remove-noise-0.1.7z).
- Double-click the
runapi.bat
file, and the browser will automatically openhttp://127.0.0.1:5080
. - Select an audio or video file to start noise reduction.
Source Code Deployment:
- Environment Preparation: Ensure that Python 3.10 - 3.12 is installed.
- Install Dependencies: Run
pip install -r requirements.txt --no-deps
. - CUDA Acceleration (Optional): If you have an NVIDIA graphics card, you can install CUDA 12.1 to accelerate processing:bash
pip uninstall -y torch torchaudio torchvision pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
- Run the Program: Run
python api.py
.
Linux System:
- You need to install the
libsndfile
library:sudo apt-get update && sudo apt-get install libsndfile1
. - Note: Please ensure that the
datasets
library version is 3.0, otherwise errors may occur. You can use thepip list | grep datasets
command to view the version.
Interface Preview
API Usage
Interface Address: http://127.0.0.1:5080/api
Request Method: POST
Request Parameters:
stream
: 0 returns the audio URL, 1 returns the audio data.audio
: The audio or video file to be processed.
Return Result (JSON):
- Success (stream=0):
{"code": 0, "data": {"url": "Audio URL"}}
- Success (stream=1): WAV audio data.
- Failure:
{"code": -1, "msg": "Error Message"}
Sample Code (Python): (Optimized based on the original text)
import requests
url = 'http://127.0.0.1:5080/api'
file_path = './300.wav'
# Get the audio URL
try:
res = requests.post(url, data={"stream": 0}, files={"audio": open(file_path, 'rb')})
res.raise_for_status()
print(f"Noise-reduced audio URL: {res.json()['data']['url']}")
except requests.exceptions.RequestException as e:
print(f"Request failed: {e}")
# Get audio data
try:
res = requests.post(url, data={"stream": 1}, files={"audio": open(file_path, 'rb')})
res.raise_for_status()
with open("ceshi.wav", 'wb') as f:
f.write(res.content)
print("Noise-reduced audio saved as ceshi.wav")
except requests.exceptions.RequestException as e:
print(f"Request failed: {e}")