Robustness of ToF and stereo fusion for high‐accuracy depth map
Authors: 
Zhenshan Bao, Bowen Li, Wenbo ZhangAuthors Info & Claims
Zhenshan Bao, Bowen Li, Wenbo ZhangAuthors Info & ClaimsAbstract
A depth map can be used in many applications such as robotic navigation, driverless, video production and 3D reconstruction. Both passive stereo and time‐of‐flight (ToF) cameras can provide the depth map for the captured real scenes, but they both have innate limitations. Since ToF cameras and passive stereo are intrinsically complementary for certain scenes, it is desirable to appropriately leverage all the available information by ToF cameras and passive stereo. As a result, this study proposes an approach to integrate ToF cameras and passive stereo to obtain high‐accuracy depth maps. The main contributions are: the first step is to design an energy cost function to utilise the depth map from ToF cameras to guide the stereo matching of passive stereo and the second step is to design their weight function for depth maps pixel‐level fusion. The experiments show that the proposed approach achieves the improved results with high accuracy and robustness.
6 References
[1]
Scharstein, D., Szeliski, R.: ‘A taxonomy and evaluation of dense two‐frame stereo correspondence algorithms’, Int. J. Comput. Vis., 2002, 47, (1–3), pp. 4–72
[2]
Veksler, O.: ‘Fast variable window for stereo correspondence using integral images’. IEEE Computer Society Conf. on Computer Vision & Pattern Recognition, Madison, WI, USA, 2003
[3]
Bobick, A.F., Intille, S.S.: ‘Large occlusion stereo’, Int. J. Comput. Vis., 1999, 33, (3), pp. 181–200
[4]
Yoon, K.J., Kweon, I.S.: ‘Adaptive support‐weight approach for correspondence search’, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI)., 2006, 28, (4), pp. 650–656
[5]
Sun, J., Zheng, N.N., Shum, H.Y.: ‘Stereo matching using belief propagation’, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI)., 2003, 25, (7), pp. 787–800
[6]
Anantrasirichai, N., Canagarajah, C.N., Redmil, D.W.: ‘Dynamic programming for multi‐view disparity/depth estimation’. IEEE Int. Conf. on Acoustics, Toulouse, France, 2006
[7]
‘PMD’, Available at http://www.pmdtec.com/, 2009
[8]
De‐Maeztu, L., Mattoccia, S., Villanueva, A.: ‘Linear stereo matching’, Int. Conf. Comput. Vis., 2013, 24, (4), pp. 1708–1715
[9]
Gudmundsson, S.A., Aanaes, H., Larsen, R.: ‘Fusion of stereo vision and time‐of‐flight imaging for improved 3d estimation’, Int. J. Intell. Syst. Technol. Appl., 2008, 5, (3–4), pp. 425–433
[10]
Zhu, J., Wang, L.: ‘Reliability fusion of time‐of‐flight depth and stereo for high quality depth maps’, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), 2011, 33, (7), pp. 1400–1414
[11]
Yang, Q., Tan, K.H., Culbertson, B.: ‘Fusion of active and passive sensor for fast 3D capture’. IEEE Int. Workshop on Multimedia Signal Processing (MMSP), Saint‐Malo, France, 2010, pp. 69–74
[12]
Oberle, W.F., Davis, L.: ‘Toward high resolution, ladar‐quality 3‐D world models using ladar‐stereo data integration and fusion’, Army Research Laboratory Aberdeen Proving Ground, 2005
[13]
Cheng, J., Leng, C.: ‘Fast and accurate image matching with cascade hashing for 3D reconstruction’. IEEE Conf. on Computer Vision and Pattern Recognition(CVPR), Columbus, Ohio, USA, 2014
[14]
Mutto, C.D., Zanuttigh, P.: ‘Locally consistent ToF and stereo data fusion’ (Springer, Berlin, Heidelberg, 2012), pp. 598–607
[15]
Fabien, C., Deepayan, B.: ‘Vision based technology for ambient assisted living: a review of the literature’, Environments, 2011, 3, pp. 253–269
[16]
Diebel, J., Thrun, S.: ‘An application of markov random fields to range sensing’, NIPS 2005, British Columbia, Canada, December 5‐8, 2005
[17]
Zhang, H., Wang, Y.: ‘SIFT flow for abrupt motion tracking via adaptive samples selection with sparse representation’, Neurocomputing. 2017, 249, pp. 253–265
[18]
Marin, G., Zanuttigh, P.: ‘Reliable fusion of ToF and stereo depth driven by confidence measures’, Comput. Vis.‐ECCV, 2016, 1, pp. 386–401
[19]
Gandhi, V., Cech, J.: ‘High‐resolution depth maps based on ToF‐stereo fusion’. IEEE Int. Conf. on Robotics and Automation., Saint Paul, MN, USA, 2012, vol. 20, no. 10, pp. 4742–4749
[20]
Agresti, G., Minto, L.: ‘Deep learning for confidence information in stereo and ToF data fusion’. IEEE Int. Conf. on Computer Vision Workshop (ICCVW)., Venice, Italy, 2017, pp. 697–705
[21]
Guo, N.: ‘Research on a joint calibration algorithm based on TOF camera and CCD camera’, Appl. Res. Comput., 2017, 35, (9), pp. 2838–2860
[22]
Zhang, Z.: ‘A flexible new technique for camera calibration’, IEEE Trans. Pattern Anal. Mach. Intell., 2000, 22, (11), pp. 1330–1334
[23]
Brown, M.Z.: ‘Advances in computational stereo’, IEEE Trans. Pattern Anal. Mach. Intell., 2003, 25, pp. 993–1008
[24]
Chen, K., Lai, Y.K.: ‘Automatic semantic modeling of indoor scenes from low‐quality RGB‐D data using contextual information’, ACM Trans. Graph., 2014, 33, (6), pp. 1–12 Article No. 208
[25]
Cheng, J., Leng, C.: ‘Fast and accurate image matching with cascade hashing for 3D reconstruction’. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, USA, 2014, pp. 23–28
[26]
Ann, N.Q., Bayuaji, L.: ‘Study on 3D scene reconstruction in robot navigation using stereo vision’. IEEE Int. Conf. on Automatic Control and Intelligent Systems, Selangor, Malaysia, 2016, pp. 72–77
[27]
Tebourbi, R., Belhadj, Z., Zribi, M.: ‘3‐D soil reconstruction from binocular disparity’. IEEE Int. Geoscience & Remote Sensing Symp., Honolulu, HI, USA, 2000, pp. 847–849
[28]
Hussmann, S.: ‘One‐phase algorithm for continuous wave TOF machine vision applications’, IEEE Trans. Instrum. Meas., 2013, 62, pp. 991–998
[29]
Guomundsson, S.R., Aans, H., Larsen, R.: ‘Environmental effects on measurement uncertainties of time‐of‐flight cameras’, Math. Comput., 2007, 1, (30), pp. 1–4
[30]
Ke, Z., Lu, J.: ‘Cross‐based local stereo matching using orthogonal integral images’, IEEE Trans. Circuits Syst. Video Technol., 2009, 19, (7), pp. 1073–1079
[31]
Zhang, J., Li, D.X., Zhang, M.: ‘Fast stereo matching algorithm based on adaptive window’. Int. Conf. on Audio Language & Image Processing., Shanghai, China, 2010, vol. 16, no. 6, pp. 138–142
Index Terms
- Robustness of ToF and stereo fusion for high‐accuracy depth map
Index terms have been assigned to the content through auto-classification.
Recommendations
Stereo fusion
A stereo fusion system that combines binocular and refractive stereo is presented.Our stereo fusion outperforms traditional binocular and refractive stereo.An efficient calibration method for refractive stereo is proposed. Display Omitted The ...
Stereo in the presence of specular reflection
ICCV '95: Proceedings of the Fifth International Conference on Computer VisionThe problem of accurate depth estimation using stereo in the presence of specular reflection is addressed. Specular reflection, a fundamental and ubiquitous reflection mechanism, is viewpoint dependent and can cause large intensity differences at ...
Real‐time depth enhancement by fusion for RGB‐D cameras
This study presents a real‐time refinement procedure for depth data acquired by RGB‐D cameras. Data from RGB‐D cameras suffer from undesired artefacts such as edge inaccuracies or holes owing to occlusions or low object remission. In this work, the ...
Comments
Information & Contributors
Information
Published In
© 2019 The Institution of Engineering and Technology.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 25 October 2019
Author Tags
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
View options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in