The electrocardiogram (ECG) signal is widely used for diagnosis of heart disorders. However, ECG signal is a kind of weak signal to be interfered with heavy background interferences. Moreover, there are some overlaps between the interference frequency sub-bands and the ECG frequency sub-bands, so it is difficult to inhibit noise in the ECG signal. In this paper, the ECG signal in-band noise de-noising method based on empirical mode decomposition (EMD) is proposed. This method uses random permutation to process intrinsic mode functions (IMFs). It abstracts QRS complexes to separate them from noise so that the clean ECG signal is obtained. The method is validated through simulations on the MIT-BIH Arrhythmia Database and experiments on the measured test data. The results indicate that the proposed method can restrain noise, improve signal noise ratio (SNR) and reduce mean squared error (MSE) effectively.