Abstract: The localization performance of matched-field processing decreases substantially for high-mismatch and high-frequency localization scenarios. Based on incoherent frequency-difference source localization technique and data-driven machine learning localization method, this paper proposes a convolutional neural network localization method based on the cross-spectral density matrix of coherent frequency-difference signal. This method utilizes dual-driven modeling and data to localize mid- and high-frequency underwater sources in mismatched and noisy scenarios. For frequency-difference signal, this method performs incoherent averaging along the central frequency dimension to reduce the impact of cross-term, and stacks along the frequency-difference dimension to preserve the correlation information between different frequencies. The resulting tensor data is then used as input to a convolutional neural network for underwater source localization. This paper conducts experiments using the SwellEx-96 simulation environment and sea trial data from the 2021 South China Sea Acoustic Tomography Long-Range Experiment. The experiments show that the proposed localization algorithm is suitable for localizing medium and high frequency sources in the noise-containing and mismatch-existing marine environment, showing favorable tolerance for underwater source localization.