Abstract
Most digital cameras use sensors coated with a Color Filter Array (CFA) to capture channel components at every pixel location, resulting in a mosaic image that does not contain pixel values in all channels. Current research on reconstructing these missing channels, also known as demosaicing, introduces many artifacts, such as zipper effect and false color. Many deep learning demosaicing techniques outperform other classical techniques in reducing the impact of artifacts. However, most of these models tend to be over-parametrized. Consequently, edge implementation of the state-of-the-art deep learning-based demosaicing algorithms on low-end edge devices is a major challenge. We provide an exhaustive search of deep neural network architectures and obtain a Pareto front of Color Peak Signal to Noise Ratio (CPSNR) as the performance criterion versus the number of parameters as the model complexity that outperforms the state-of-the-art. Architectures on the Pareto front can then be used to choose the best architecture for a variety of resource constraints. Simple architecture search methods such as exhaustive search and grid search requires some conditions of the loss function to converge to the optimum. We clarify these conditions in a brief theoretical study.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Audet, C., Hare, W.: Derivative-Free and Blackbox Optimization. SSORFE. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68913-5
Gharbi, M., Chaurasia, G., Paris, S., Durand, F.: Deep joint demosaicking and denoising. ACM Trans. Graph. 35(6), 191 (2016)
Han, S., et al.: Eie: efficient inference engine on compressed deep neural network. SIGARCH Comput. Archit. News 44(3), 243–254 (2016)
Han, S., Pool, J., Tran, J., Dally, W.J.: Learning both weights and connections for efficient neural networks. CoRR abs/1506.02626 (2015)
Chang, L., Tan, Y.P.: Hybrid color filter array demosaicking for effective artifact suppression. J. Electron. Imaging 15, 15–17 (2006)
Li, X., Gunturk, B.K., Zhang, L.: Image demosaicing: a systematic survey (2007)
Loshchilov, I., Hutter, F.: SGDR: stochastic gradient descent with restarts. CoRR
Ma, N., Zhang, X., Zheng, H., Sun, J.: Shufflenet V2: practical guidelines for efficient CNN architecture design. CoRR abs/1807.11164 (2018)
Menon, D., Calvagno, G.: Color image demosaicking: an overview. Image Commun. 26(8–9), 518–533 (2011)
Sandler, M., Howard, A.G., Zhu, M., Zhmoginov, A., Chen, L.: Inverted residuals and linear bottlenecks: mobile networks for classification, detection and segmentation. CoRR abs/1801.04381 (2018)
Sifre, L., Mallat, S.: Rigid-motion scattering for texture classification. CoRR abs/1403.1687 (2014)
Snoek, J., Larochelle, H., Adams, R.P.: Practical bayesian optimization of machine learning algorithms. In: Advances in Neural Information Processing Systems, vol. 25, pp. 2951–2959. Curran Associates, Inc., New York (2012)
Syu, N., Chen, Y., Chuang, Y.: Learning deep convolutional networks for demosaicing. CoRR abs/1802.03769 (2018)
Tan, H., Xiao, H., Lai, S., Liu, Y., Zhang, M.: Deep residual learning for image demosaicing and blind denoising, August 2018
Tsirigotis, C., Bouthillier, X., Corneau-Tremblay, F., Henderson, P., Askari, et al.: Oríon: experiment version control for efficient hyperparameter optimization (2018)
Xie, J., Xu, L., Chen, E.: Image denoising and inpainting with deep neural networks. In: Advances in Neural Information Processing Systems, vol. 25 (2012)
Yu, K., Wang, C., Yang, S., et al.: An effective directional residual interpolation algorithm for color image demosaicking. Appl. Sci. 8, 680 (2018)
Yu, W.: Colour demosaicking method using adaptive cubic convolution interpolation with sequential averaging. IEEE Proc. - Vis. Image Sig. Process. 153(5), 666–676 (2006)
Zhen, R., Stevenson, R.L.: Image demosaicing. In: Celebi, M.E., Lecca, M., Smolka, B. (eds.) Color Image and Video Enhancement, pp. 13–54. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-09363-5_2
Acknowledgement
We want to acknowledge Jingwei Chen, Yanhui Geng, and Heng Liao for their support. Qiang Tang and Shao Hua Chen from HiSilicon Vancouver IC Lab kindly assisted us in initiating and developing demisaicing research direction. We appreciate the assistance of Huawei media research lab of Japan that provided us details about demosaicing and camera sensors. We declare our deep gratitude to Charles Audet and Sébastien Le Digabel who introduced to us the theory behind derivative-free optimization.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Ramakrishnan, R., Jui, S., Partovi Nia, V. (2019). Deep Demosaicing for Edge Implementation. In: Karray, F., Campilho, A., Yu, A. (eds) Image Analysis and Recognition. ICIAR 2019. Lecture Notes in Computer Science(), vol 11662. Springer, Cham. https://doi.org/10.1007/978-3-030-27202-9_25
Download citation
DOI: https://doi.org/10.1007/978-3-030-27202-9_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27201-2
Online ISBN: 978-3-030-27202-9
eBook Packages: Computer ScienceComputer Science (R0)