Slope measuring deflectometry (SMD) systems are developing rapidly in testing freeform optics. They measure the surface slope using a camera and an incoherent source. The principle of the test is mainly discussed in geometric optic domain. The system response as a function of spatial frequency or instrument transfer function (ITF) has yet to be studied thoroughly. Through mathematical modeling, simulation, and experiment we show that the ITF of an SMD system is very close to the modulation transfer function of the camera used. Furthermore, the ITF can be enhanced using a deconvolution filter. This study will lead to more accurate measurements in SMD and will show the physical optics nature of these tests.