Abstract: In order to solve the problems of traditional deception detection using physiological indicators which only detected deception of present person under contacting condition, a new method combining phonetic features with deep learning is proposed. In the term of extracting features, the Hu moment of Mel-Frequency Cepstrum is calculated firstly,then discrete cosine transform is performed to remove correlation. This spectral feature makes use of the orthogonal and translational invariance of Hu moment, which could evaluate the degree how the local energy is concentrated to the center of energy gravity that has a close relationship with the energy changing caused by trembling in deception.Furthermore, a modified training algorithm for Restricted Boltzmann Machines is proposed to improve the exchange rate of adjacent temperature chains in the traditional parallel tempering, which partitions the state energy of Markov into several energy rings. In each ring, the states have similar energies. The performance of network will increase with the exchange rate. Experiments on Columbia-SRI-Colorado dataset show that the recognition rate is 71.47%, 7% higher than the experiments of Columbia University.