Abstract: In order to obtain the clean speech from the noisy signal, a single-channel speech enhancement algorithm based on gender-related models is proposed. Specifically, in the training stage, Deep Neural Networks(DNN) and Nonnegative Matrix Factorization(NMF) are employed to train two gender-related DNN-NMF models using the genderspecific training data. In the test stage, an algorithm based on NMF and group sparsity penalty is proposed to identify the gender information of the speaker in the test signal. Then the corresponding DNN-NMF model is used to estimate the activations for speech enhancement. Experimental results show that the proposed algorithm performs better in suppressing the noises without decreasing the speech quality compared with other NMF-based and DNN-based methods.