Learning real-world heterogeneous noise models with a benchmark dataset

Noise modeling is an important research field in computer vision; the design of an accurate model for imaging sensor noise depends on not only a comprehensive benchmark dataset of the real world, but also a precise design of the noise modeling algorithm. However, due to the inaccurate estimation method of noise-free images and limited shooting scenes, the current realistic datasets could not describe the diverse noise properties sufficiently. Moreover, popular parametric noise models are not sophisticated enough to characterize the real-world noise exactly. In this work, we first construct a more comprehensive dataset of the real world by capturing more indoor and outdoor scenes under different lighting conditions using diverse smartphones, then we propose a non-parametric noise estimation method capable of modeling the spatial heterogeneity of real-world noise patterns. Specifically, in order to characterize the spatial heterogeneity of real-world noise, we assume a non-i.i.d Gaussian distribution and propose a deep convolutional neural network (DCNN)-based approach for learning pixel-wise noise variance maps. To learn the pixel-wise variance map, we have constructed a variance estimation network mapping from the conditional signals (clean image, ISO, and camera model) to surrogate labels obtained from the nonlocal search of similar patches from the clean-noisy image pair. Finally, we conducted denoising and classification experiments using different kinds of simulated noisy images, compared to the Poisson-Gaussian and Noise Flow noise models, the proposed method achieves denoising performance improvements (PSNR) of 1.13 dB and 2.51 dB respectively on the proposed real-world test dataset, denoising and classification results on the real noisy data captured by mobile phones have verified that our approach is more accurate than current noise modeling methods.