Leveraging Discriminative Latent Representations for Conditioning GAN-Based Speech Enhancement

Abstract

Generative speech enhancement methods based on generative adversarial networks (GANs) and diffusion models have shown promising results in various speech enhancement (SE) tasks. However, their performance in very low signal-to-noise ratio (SNR) scenarios remains underexplored and limited, as these conditions pose significant challenges to both discriminative and generative state-of-the-art methods. To address this, we propose a method that leverages latent features extracted from discriminative speech enhancement models as generic conditioning features to improve GAN-based speech enhancement. The proposed method, referred to as DisCoGAN, demonstrates performance improvements over baseline models, particularly in low-SNR scenarios, while also maintaining competitive or superior performance in high-SNR conditions and on real-world recordings. We also conduct a comprehensive evaluation of conventional GAN-based architectures, including GANs trained end-to-end, GANs as a first processing stage, and post-filtering GANs, as well as discriminative models under low-SNR conditions. We show that DisCoGAN consistently outperforms existing methods. Finally, we present an ablation study that investigates the contributions of individual components within DisCoGAN and analyzes the impact of the discriminative conditioning method on overall performance..

Evaluation Scenarios

In our work, we evaluate our proposed method with different SOTA generative and discriminative deep learning-based noise reduction methods in various SNR scenraios.

Following you can find some processed samples with different Methods:

1. Low SNR Dataset --> Go to the samples

2. DNS Challenge No-Reverb (High SNR Dataset) --> Go to the samples

3. Real Recordings -->Go to the samples

1. Low SNR Dataset

Item 1 (Speaker: Female)

Item 2 (Speaker: FeMale)

Item 3 (Speaker: Female)

Item 4 (Speaker: FeMale)

Item 5 (Speaker: Male)

Item 6 (Speaker: Male)

Item 7 (Speaker: Male)

Item 8 (Speaker: Male)

Item 9 (Speaker: Male)

Item 10 (Speaker: Male)

2. DNS Challenge No-Reverb

Item 1

Item 2

Item 3

Item 4

Item 5

Item 6

3. Real Recordings

Item 1

Item 2

Item 3

Item 4

Item 5

Item 6