Abstract: This paper introduced sparse representation on joint factor analysis to solve the channel mismatch problem and to improve system performance. This algorithm uses joint factor analysis to generate the speaker factors space and construct the over-complete dictionary to calculate speaker score by solving the optimization problem. The minimum detection cost function (minDCF) of the system with sparse representation on joint factor analysis gave good performance on NIST speaker recognition evaluation (SRE) 2008 telephone to telephone test corpus. Because the sparse representation algorithm and the support vector machine classification algorithm also have a good complementary, the fusion of JFA-SR and JFA-SVM can achieve 4.91% reduction in minDCF. The results of the experiments show that speaker verification using sparse representation on joint factor analysis is feasible and has a great future.