Abstract
Among the modern means of 3D geometry creation that exist in the literature, there are the Multi-View Stereo (MVS) reconstruction methods that received much attention from the research community and the multimedia industry. In fact, several methods showed that it is possible to recover geometry only from images with reconstruction accuracies paralleling that of excessively expensive laser scanners. The majority of these methods perform on images such as online community photo collection and estimate the surface position with its orientation by minimizing a matching cost function defined over a small local region. However, these datasets not only they are large but also contain more challenging scenes setups with different photometric effects; therefore fine-grained details of an object’s surface cannot be captured. This paper presents a robust multi-view stereo method based on metaheuristic optimization namely the Particle Swarm Optimization (PSO) in order to find the optimal depth, orientation, and surface roughness. To deal with the various shading and stereo mismatch problems caused by rough surfaces, shadows, and interreflections, we propose to use a robust matching/energy function which is a combination of two similarity measurements. Finally, our method computes individual depth maps that can be merged into compelling scene reconstructions. The proposed method is evaluated quantitatively using well-known Middlebury datasets and the obtained results show a high completeness score and comparable accuracy to those of the current top performing algorithms.
Similar content being viewed by others
References
Clerc M, Kennedy J (2002) The particle swarm - explosion, stability, and convergence in a multidimensional complex space. IEEE Trans Evol Comput 6(1):58–73
Esteban CH, Schmitt F (2004) Silhouette and stereo fusion for 3d object modeling. Comput Vis Image Underst 96(3):367–392
Esteban CH, Vogiatzis G, Cipolla R (2008) Multiview photometric stereo. IEEE Trans Pattern Anal Mach Intell 30(3):548–554
Faugeras OD, Keriven R (1998) Variational principles, surface evolution, pdes, level set methods, and the stereo problem. IEEE Trans Image Process 7(3):336–344
Fuhrmann S, Langguth F, Goesele M (2014) MVE - A multi-view reconstruction environment. In: 2014 eurographics workshop on graphics and cultural heritage, GCH 2014, Darmstadt, Germany, October 6-8, 2014, pp 11–18
Furukawa Y, Hernȧndez C. (2015) Multi-view stereo: A tutorial. Foundations and Trends in Computer Graphics and Vision 9(1-2):1–148
Furukawa Y, Ponce J (2007) Accurate, dense, and robust multi-view stereopsis. In: 2007 IEEE computer society conference on computer vision and pattern recognition (CVPR 2007), 18-23 June 2007. Minneapolis, Minnesota, USA
Galliani S, Lasinger K, Schindler K (2015) Massively parallel multiview stereopsis by surface normal diffusion. In: 2015 IEEE international conference on computer vision, ICCV 2015, Santiago, Chile, December 7-13, 2015, pp 873–881
Gallup D, Frahm J, Mordohai P, Yang Q, Pollefeys M (2007) Real-time plane-sweeping stereo with multiple sweeping directions. In: 2007 IEEE computer society conference on computer vision and pattern recognition (CVPR 2007), 18-23 June 2007. Minneapolis, Minnesota, USA
Goesele M, Curless B, Seitz SM (2006) Multi-view stereo revisited. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR 2006), 17-22 June 2006, New York, NY, USA, pp 2402–2409
Goesele M, Snavely N, Curless B, Hoppe H, Seitz SM (2007) Multi-view stereo for community photo collections. In: IEEE 11th international conference on computer vision, ICCV 2007, Rio de Janeiro, Brazil, October 14-20, 2007, pp 1–8
Harltey R, Zisserman A (2006) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge
Hertzmann A, Seitz SM (2005) Example-based photometric stereo: shape reconstruction with general, varying brdfs. IEEE Trans Pattern Anal Mach Intell 27(8):1254–1264
Hirschmu̇ller H., Scharstein D (2009) Evaluation of stereo matching costs on images with radiometric differences. IEEE Trans Pattern Anal Mach Intell 31(9):1582–1599
Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe RA, Kohli P, Shotton J, Hodges S, Freeman D, Davison AJ, Fitzgibbon AW (2011) Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, Santa Barbara, CA, USA, October 16-19, 2011, pp 559–568
Kazhdan MM, Hoppe H (2013) Screened poisson surface reconstruction. ACM Trans Graph 32(3):29,1–29,13
Langguth F, Sunkavalli K, Hadap S, Goesele M (2016) Shading-aware multi-view stereo. In: Computer Vision - ECCV 2016 - 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part III, pp 469–485
Li Z, Wang K, Zuo W, Meng D, Zhang L (2016) Detail-preserving and content-aware variational multi-view stereo reconstruction. IEEE Trans Image Process 25(2):864–877
Mei X, Sun X, Zhou M, Jiao S, Wang H, Zhang X (2011) On building an accurate stereo matching system on graphics hardware. In: IEEE international conference on computer vision workshops, ICCV 2011 Workshops, Barcelona, Spain, November 6-13, 2011, pp 467–474
Newcombe RA, Lovegrove S, Davison AJ (2011) DTAM: dense tracking and mapping in real-time. In: IEEE international conference on computer vision, ICCV 2011, Barcelona, Spain, November 6-13, 2011, pp 2320–2327
Oren M, Nayar SK (1993) Diffuse reflectance from rough surfaces. In: Conference on computer vision and pattern recognition, CVPR 1993, 15-17 June, 1993, New York, NY, USA, pp 763–764
Oren M, Nayar SK (1994) Generalization of lambert’s reflectance model. In: Proceedings of the 21st annual conference on computer graphics and interactive techniques, SIGGRAPH ’94, pp 239–246
Oxholm G, Nishino K (2014) Multiview shape and reflectance from natural illumination. In: 2014 IEEE conference on computer vision and pattern recognition, CVPR 2014, Columbus, OH, USA, June 23-28, 2014, pp 2163–2170
Ranftl R, Gehrig S, Pock T, Bischof H (2012) Pushing the limits of stereo using variational stereo estimation. In: 2012 IEEE intelligent vehicles symposium, IV 2012, Alcal de Henares, Madrid, Spain, June 3-7, 2012, pp 401–407
Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Comput Vis 47(1-3):7–42
Schlick C (1994) A survey of shading and reflectance models. Comput Graph Forum 13(2):121–131
Seitz SM, Curless B, Diebel J, Scharstein D, Szeliski R (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: 2006 IEEE computer society conference on computer vision and pattern recognition (CVPR 2006), 17-22 June 2006, New York, NY, USA, pp 519–528
Semerjian B (2014) A new variational framework for multiview surface reconstruction. In: 2014 - 13th European conference computer vision - ECCV, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI, pp 719–734
Shen S (2013) Accurate multiple view 3d reconstruction using patch-based stereo for large-scale scenes. IEEE Trans Image Process 22(5):1901–1914
Sinha SN, Pollefeys M (2005) Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation. In: 10th IEEE international conference on computer vision (ICCV 2005), 17-20 October 2005, Beijing, China, pp 349–356
Strecha C, von Hansen W, Gool LJV, Fua P, Thoennessen U (2008) On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: 2008 IEEE computer society conference on computer vision and pattern recognition CVPR 2008, 24-26 June 2008. Anchorage, Alaska, USA
Wei J, Resch B, Lensch HPA (2014) Multi-view depth map estimation with cross-view consistency. In: British machine vision conference, BMVC 2014, Nottingham, UK, September 1-5, 2014
Wen-Chao Che Nzen CHEN PYS (2015) Stochastic optimization based 3d dense reconstruction from multiple views with high accuracy and completeness. Journal of Information Science and Engineering
Wu C, Wilburn B, Matsushita Y, Theobalt C (2011) High-quality shape from multi-view stereo and shading under general illumination. In: The 24th IEEE conference on computer vision and pattern recognition, CVPR 2011, Colorado Springs, CO, USA, 20-25 June 2011, pp 969–976
Zhan Y, Gu Y, Huang K, Zhang C, Hu K (2016) Accurate image-guided stereo matching with efficient matching cost and disparity refinement. IEEE Trans Circuits Syst Video Technol 26(9):1632–1645
Zhu Z, Stamatopoulos C, Fraser CS (2015) Accurate and occlusion-robust multi-view stereo. ISPRS J Photogramm Remote Sens 109(Supplement C):47–61
Zollhȯfer M, Dai A, Innmann M, Wu C, Stamminger M, Theobalt C, Nießner M (2015) Shading-based refinement on volumetric signed distance functions. ACM Trans Graph 34(4):96,1–96,14
Acknowledgements
We thank all photographers who provided their images. We also like to thank Michael Goesele, Simon Fuhrmann and Romuald Perrot for their helpful comments and suggestions.
Funding
This study was not sponsored by any organization.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
This manuscript has not been published and is not under consideration for publication elsewhere.
This article does not contain any studies with human or animal subjects.
Conflict of interests
Saouli Abdelhak declares that he has no conflict of interest. Mohamed Chaouki Babahenini declares that he has no conflict of interest. Sofiane Medjram declares that he has no conflict of interest
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Saouli, A., Babahenini, M.C. & Medjram, S. Accurate, dense and shading-aware multi-view stereo reconstruction using metaheuritic optimization. Multimed Tools Appl 78, 15053–15077 (2019). https://doi.org/10.1007/s11042-018-6904-6
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-018-6904-6