For the first time, a significant noise source in the resonant fiber optic gyroscope (RFOG) called intermodulation induced error is proposed and deeply analyzed in this paper. The intermodulation error is produced by the laser frequency noise at even multiples of the modulation frequency due to an intermodulation effect, which will seriously limit the random noise performance of the RFOG. Experiments are designed and conducted to verify and measure the intermodulation induced error in the RFOG. The experimental results confirm the existence of intermodulation error, and fit well with the theory. As for the design of the RFOG, light sources with a narrow intrinsic linewidth and a high modulation frequency are preferable to achieve a high rotation-rate sensitivity.