Monaural Speech Enhancement with Full-Convolution Attention Module and Post-Processing Strategy

Abstract

The performance of phase-aware speech enhancement has improved dramatically in recent years. Combined with complex convolutions, deep complex U-Net and deep complex convolution recurrent network (DCCRN) have achieved superior performance in monaural phase-aware speech enhancement. However, these methods optimize the models with loss only in the time domain and ignore the global correlations along the frequency axis that capture the harmonic information between frequency bands. Also, the algorithms based on self-attention exhibit high computational complexity. To strike the balance between performance and computational cost, we propose a new monaural phase-aware method in the time-frequency domain on the deep complex U-Net structure. Specifically, this proposed method incorporates a dual-path recurrent neural network (DPRNN) block in the bottleneck to model both frequency-domain correlation and time-domain correlation. Additionally, attention modules are implemented between the complex encoder and decoder layers. This introduces a more effective way of enhancing the representation of the model, rather than directly concatenating their outputs. Finally, a post-processing module is introduced to mitigate the over-suppression of speech and residual noise. We conduct ablation studies to validate the effectiveness of the dual-path method and the post-processing module. Also, compared to several recent speech enhancement models, the proposed algorithm demonstrates remarkable improvements in terms of objective metrics.