Abstract
Owing to recent advances in depth sensors and computer vision algorithms, depth images are often available with co-registered color images. In this paper, we propose a simple but effective method for obtaining an all-in-focus (AIF) color image from a database of color and depth image pairs. Since the defocus blur is inherently depth-dependent, the color pixels are first grouped according to their depth values. The defocus blur parameters are then estimated using the amount of the defocus blur of the grouped pixels. Given a defocused color image and its estimated blur parameters, the AIF image is produced by adopting the conventional pixel-wise mapping technique. In addition, the availability of the depth image disambiguates the objects located far or near from the in-focus object and thus facilitates image refocusing. We demonstrate the effectiveness of the proposed algorithm using both synthetic and real color and depth images.
Similar content being viewed by others
References
Ayatollahi S M, Moghadam A M E, Hosseini M S (2014) A taxonomy of depth map creation methods used in multiview video compression. Multimedia Tools Appl 72(2):1887–1909
Cao Y, Fang S, Wang Z (2013) Digital multi-focusing from a single photograph taken by an uncalibrated conventional camera. IEEE Trans Image Process 22(9):3703–3714
Fletcher R (1982) Second order corrections for non-differentiable optimization. Numer Anal 912:85–144
He L, Bleyer M, Gelautz M (2011) Object removal by depth-guided inpainting. In: Proceedings of AAPR Workshop, pp 1–8
Jung S-W, Ko S-J (2012) Depth map based image enhancement using color stereopsis. IEEE Signal Process Lett 19(5):303–306
Jung S-W, Choi O (2013) Color image enhancement using depth and intensity measurements of a time-of-flight depth camera. Opt Eng 52(10):1–11
Kodama K, Kubota A (2013) Efficient reconstruction of all-in-focus images through shifted pinholes from multi-focus images for dense light field synthesis and rendering. IEEE Trans Image Process 22(11):4407–4421
Lai K, Bo L, Ren X, Fox D (2011) A large-scale hierarchical multi-view RGB-D object dataset, pp 1817–1824
Pentland A P (1987) A new sense for depth of field. IEEE Trans Pattern Anal Mach Intell 9(4):523–531
Pertuz S, Puig D, Garcia M A, Fusiello A (2013) Generation of all-in-focus images by noise-robust selective fusion of limited depth-of-field images. IEEE Trans Image Process 22(3):1242–1251
Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1):7–42
Zhang W, Cham W-K (2012) Single-image refocusing and defocusing. IEEE Trans Image Process 21(2):873–882
Zhang X, Constable M, Chan KL (2011) Aesthetic enhancement of landscape photographs as informed by paintings across depth layers. In: Proceedings IEEE International Conference on Image Processing, pp 1113–1116
Zhou S, Sim T (2011) Defocus map estimation from a single image. Pattern Recognition 44(9):1852–1858
Acknowledgments
This research was supported by Basic Science Research Program through the National Research Foundation of Korea(NRF) funded by the Ministry of Science, ICT & Future Planning(NRF- 2014R1A1A2057970) and by the MSIP (Ministry of Science, ICT and Future Planning), Korea, under the ITRC(Information Technology Research Center) support program (NIPA-2015-H0301-15-1021) supervised by the NIPA (National IT Industry Promotion Agency).
Author information
Authors and Affiliations
Corresponding authors
Rights and permissions
About this article
Cite this article
Jung, SW., Park, J.H. & Jeong, YS. All-in-focus and multi-focus color image reconstruction from a database of color and depth image pairs. Multimed Tools Appl 75, 15493–15507 (2016). https://doi.org/10.1007/s11042-015-2535-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-015-2535-3