At present, there are two ways to obtain temperature information: contact type and nonintrusive type. As a nonintrusive temperature measurement method, ultrasonic thermometry can be used to acquire the temperature distribution of complex fields conveniently. By measuring the time-of-flight (TOF) between ultrasonic transmitters and receivers, and according to the relationship between temperature and ultrasonic velocity, the temperature distribution can be reconstructed. Among the existing algorithms, the least square method (LSM) will lose much information near the edges of the temperature field, and the algebra reconstruction technique (ART) is time-consuming with low reconstruction accuracy. In this article, an improved reconstruction algorithm based on an inverse quadratic function and singular value decomposition (IQ-SVD) is proposed, which can effectively increase the reconstruction accuracy. The simulations of the real temperature data are conducted in ideal and noisy environments, respectively. Moreover, the influence of region division and shape parameters on reconstruction accuracy is discussed. The simulation results indicate that, compared with conventional algorithms, the proposed algorithm can accurately reflect the temperature distribution, and the root mean square error in the central region and the edge region is reduced by 0.49% at least, and 1.28% at most.