Cited By
View all- Guo PPan SGao WKhoshelham K(2024)Self-supervised monocular depth estimation via joint attention and intelligent mask lossMachine Vision and Applications10.1007/s00138-024-01640-136:1Online publication date: 28-Nov-2024
Robustness of ToF and stereo fusion for high‐accuracy depth map
Authors: 
Zhenshan Bao, Bowen Li, Wenbo ZhangAuthors Info & Claims
Zhenshan Bao, Bowen Li, Wenbo ZhangAuthors Info & ClaimsAbstract
A depth map can be used in many applications such as robotic navigation, driverless, video production and 3D reconstruction. Both passive stereo and time‐of‐flight (ToF) cameras can provide the depth map for the captured real scenes, but they both have innate limitations. Since ToF cameras and passive stereo are intrinsically complementary for certain scenes, it is desirable to appropriately leverage all the available information by ToF cameras and passive stereo. As a result, this study proposes an approach to integrate ToF cameras and passive stereo to obtain high‐accuracy depth maps. The main contributions are: the first step is to design an energy cost function to utilise the depth map from ToF cameras to guide the stereo matching of passive stereo and the second step is to design their weight function for depth maps pixel‐level fusion. The experiments show that the proposed approach achieves the improved results with high accuracy and robustness.
6 References
[1]
Scharstein, D., Szeliski, R.: ‘A taxonomy and evaluation of dense two‐frame stereo correspondence algorithms’, Int. J. Comput. Vis., 2002, 47, (1–3), pp. 4–72
[2]
Veksler, O.: ‘Fast variable window for stereo correspondence using integral images’. IEEE Computer Society Conf. on Computer Vision & Pattern Recognition, Madison, WI, USA, 2003
[3]
Bobick, A.F., Intille, S.S.: ‘Large occlusion stereo’, Int. J. Comput. Vis., 1999, 33, (3), pp. 181–200
[4]
Yoon, K.J., Kweon, I.S.: ‘Adaptive support‐weight approach for correspondence search’, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI)., 2006, 28, (4), pp. 650–656
[5]
Sun, J., Zheng, N.N., Shum, H.Y.: ‘Stereo matching using belief propagation’, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI)., 2003, 25, (7), pp. 787–800
[6]
Anantrasirichai, N., Canagarajah, C.N., Redmil, D.W.: ‘Dynamic programming for multi‐view disparity/depth estimation’. IEEE Int. Conf. on Acoustics, Toulouse, France, 2006
[7]
‘PMD’, Available at http://www.pmdtec.com/, 2009
[8]
De‐Maeztu, L., Mattoccia, S., Villanueva, A.: ‘Linear stereo matching’, Int. Conf. Comput. Vis., 2013, 24, (4), pp. 1708–1715
[9]
Gudmundsson, S.A., Aanaes, H., Larsen, R.: ‘Fusion of stereo vision and time‐of‐flight imaging for improved 3d estimation’, Int. J. Intell. Syst. Technol. Appl., 2008, 5, (3–4), pp. 425–433
[10]
Zhu, J., Wang, L.: ‘Reliability fusion of time‐of‐flight depth and stereo for high quality depth maps’, IEEE Trans. Pattern Anal. Mach. Intell. (PAMI), 2011, 33, (7), pp. 1400–1414
[11]
Yang, Q., Tan, K.H., Culbertson, B.: ‘Fusion of active and passive sensor for fast 3D capture’. IEEE Int. Workshop on Multimedia Signal Processing (MMSP), Saint‐Malo, France, 2010, pp. 69–74
[12]
Oberle, W.F., Davis, L.: ‘Toward high resolution, ladar‐quality 3‐D world models using ladar‐stereo data integration and fusion’, Army Research Laboratory Aberdeen Proving Ground, 2005
[13]
Cheng, J., Leng, C.: ‘Fast and accurate image matching with cascade hashing for 3D reconstruction’. IEEE Conf. on Computer Vision and Pattern Recognition(CVPR), Columbus, Ohio, USA, 2014
[14]
Mutto, C.D., Zanuttigh, P.: ‘Locally consistent ToF and stereo data fusion’ (Springer, Berlin, Heidelberg, 2012), pp. 598–607
[15]
Fabien, C., Deepayan, B.: ‘Vision based technology for ambient assisted living: a review of the literature’, Environments, 2011, 3, pp. 253–269
[16]
Diebel, J., Thrun, S.: ‘An application of markov random fields to range sensing’, NIPS 2005, British Columbia, Canada, December 5‐8, 2005
[17]
Zhang, H., Wang, Y.: ‘SIFT flow for abrupt motion tracking via adaptive samples selection with sparse representation’, Neurocomputing. 2017, 249, pp. 253–265
[18]
Marin, G., Zanuttigh, P.: ‘Reliable fusion of ToF and stereo depth driven by confidence measures’, Comput. Vis.‐ECCV, 2016, 1, pp. 386–401
[19]
Gandhi, V., Cech, J.: ‘High‐resolution depth maps based on ToF‐stereo fusion’. IEEE Int. Conf. on Robotics and Automation., Saint Paul, MN, USA, 2012, vol. 20, no. 10, pp. 4742–4749
[20]
Agresti, G., Minto, L.: ‘Deep learning for confidence information in stereo and ToF data fusion’. IEEE Int. Conf. on Computer Vision Workshop (ICCVW)., Venice, Italy, 2017, pp. 697–705
[21]
Guo, N.: ‘Research on a joint calibration algorithm based on TOF camera and CCD camera’, Appl. Res. Comput., 2017, 35, (9), pp. 2838–2860
[22]
Zhang, Z.: ‘A flexible new technique for camera calibration’, IEEE Trans. Pattern Anal. Mach. Intell., 2000, 22, (11), pp. 1330–1334
[23]
Brown, M.Z.: ‘Advances in computational stereo’, IEEE Trans. Pattern Anal. Mach. Intell., 2003, 25, pp. 993–1008
[24]
Chen, K., Lai, Y.K.: ‘Automatic semantic modeling of indoor scenes from low‐quality RGB‐D data using contextual information’, ACM Trans. Graph., 2014, 33, (6), pp. 1–12 Article No. 208
[25]
Cheng, J., Leng, C.: ‘Fast and accurate image matching with cascade hashing for 3D reconstruction’. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Columbus, Ohio, USA, 2014, pp. 23–28
[26]
Ann, N.Q., Bayuaji, L.: ‘Study on 3D scene reconstruction in robot navigation using stereo vision’. IEEE Int. Conf. on Automatic Control and Intelligent Systems, Selangor, Malaysia, 2016, pp. 72–77
[27]
Tebourbi, R., Belhadj, Z., Zribi, M.: ‘3‐D soil reconstruction from binocular disparity’. IEEE Int. Geoscience & Remote Sensing Symp., Honolulu, HI, USA, 2000, pp. 847–849
[28]
Hussmann, S.: ‘One‐phase algorithm for continuous wave TOF machine vision applications’, IEEE Trans. Instrum. Meas., 2013, 62, pp. 991–998
[29]
Guomundsson, S.R., Aans, H., Larsen, R.: ‘Environmental effects on measurement uncertainties of time‐of‐flight cameras’, Math. Comput., 2007, 1, (30), pp. 1–4
[30]
Ke, Z., Lu, J.: ‘Cross‐based local stereo matching using orthogonal integral images’, IEEE Trans. Circuits Syst. Video Technol., 2009, 19, (7), pp. 1073–1079
[31]
Zhang, J., Li, D.X., Zhang, M.: ‘Fast stereo matching algorithm based on adaptive window’. Int. Conf. on Audio Language & Image Processing., Shanghai, China, 2010, vol. 16, no. 6, pp. 138–142
Index terms have been assigned to the content through auto-classification.
Recommendations
Comments
Information & Contributors
Information
Published In
© 2019 The Institution of Engineering and Technology.
Publisher
John Wiley & Sons, Inc.
United States
Publication History
Published: 25 October 2019
Author Tags
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
Cited By
View all- Guo PPan SGao WKhoshelham K(2024)Self-supervised monocular depth estimation via joint attention and intelligent mask lossMachine Vision and Applications10.1007/s00138-024-01640-136:1Online publication date: 28-Nov-2024