Abstract: In order to increase short time whispered speaker recognition rate in variable channel conditions, the hybrid compensation in model and feature domains is proposed. This method is based on joint factor analysis in training model stage. It extracts speaker factor and eliminate channel factor by estimating training speech speaker and channel spaces. Then in the test stage, the test speech channel factor is projected into feature space to engage in feature compensation, so it can remove channel information both in model and feature domains in order to improve recognition rate. The experiment result shows that the hybrid compensation can obtain the similar recognition rate in the three different training channel conditions and this method is more effective than joint factor analysis in the test of short whispered speech.