research-article

Active Stereo: Integrating Disparity, Vergence, Focus, Aperture and Calibration for Surface Estimation

Authors:

A. L. AbbottAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 15, Issue 10

Pages 1007 - 1029

https://doi.org/10.1109/34.254059

Published: 01 October 1993 Publication History

Abstract

An approach to integrating stereo disparity, camera vergence, and lens focus to exploit their complementary strengths and weaknesses through active control of camera focus and orientations is presented. In addition, the aperture and zoom settings of the cameras are controlled. The result is an active vision system that dynamically and cooperatively interleaves image acquisition with surface estimation. A dense composite map of a single contiguous surface is synthesized by automatically scanning the surface and combining estimates of adjacent, local surface patches. This problem is formulated as one of minimizing a pair of objective functions. The first such function is concerned with the selection of a target for fixation. The second objective function guides the surface estimation process in the vicinity of the fixation point. Calibration parameters of the cameras are treated as variables during optimization, thus making camera calibration an integral, flexible component of surface estimation. An implementation of this method is described, and a performance evaluation of the system is presented. An average absolute error of less than 0.15% in estimated depth was achieved for a large surface having a depth of approximately 2 m.

References

[1]

{1} R. Bajcsy, "Active perception vs. passive perception," in Proc. Workshop Comput. Vision, Oct. 1985, pp. 55-59.

[2]

{2} R. Bajcsy, "Perception with feedback," in Proc. DARPA Image Understanding Workshop, Apr. 1988, pp. 279-288.

[3]

{3} G. Sperling, "Binocular vision: A physical and a neural theory," Amer. J. Psychol., vol. 83, pp. 461-534, 1970.

[4]

{4} E. P. Krotkov, Active Computer Vision by Cooperative Focus and Stereo. Springer-Verlag, 1989.

Digital Library

[5]

{5} D. H. Ballard and A. Ozcandarli, "Eye fixation and early vision: Kinetic depth," in Proc. Second Int. Conf. Comput. Vision, Dec. 1988, pp. 524-531.

[6]

{6} S. Das and N. Ahuja, "Multiresolution image acquisition and surface reconstruction," in Proc. Third Int. Conf. Comput. Vision, Dec. 1990.

[7]

{7} W. Hoff and N. Ahuja, "Surfaces from stereo," in Proc. DARPA Image Understanding Workshop, Dec. 1985, pp. 98-106.

[8]

{8} W. Hoff and N. Ahuja, "Surfaces from stereo: Integrating feature matching, disparity estimation and contour detection," IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-11, pp. 121-136, Feb. 1989.

Digital Library

[9]

{9} R. D. Eastman and A. M. Waxman, "Disparity functionals and stereo vision," in Proc. DARPA Image Understanding Workshop, Dec. 1985, pp. 245-254.

[10]

{10} T. E. Boult and L. -H. Chen, "Synergistic smooth surface stereo," in Proc. Second Int. Conf. Comput. Vision, Dec. 1988, pp. 118-122.

[11]

{11} A. L. Abbott and N. Ahuja, "Surface reconstruction by dynamic integration of focus, camera vergence, and stereo," in Proc. Second Int. Conf. Comput. Vision, Dec. 1988, pp. 532-543.

[12]

{12} E. Altman and N. Ahuja, "A dynamical systems approach to integration in stereo," in Proc. DARPA Image Understanding Workshop, Sept. 1990, pp. 423-427.

[13]

{13} W. Hoff and N. Ahuja, "Extracting surfaces from stereo images: An integrated approach," in Proc. First Int. Conf. Comput. Vision, June 1987, pp. 284-294.

[14]

{14} A. N. Choudhary, S. Das, N. Ahuja, and J. H. Patel, "Surface reconstruction from stereo images: An implementation on a hypercube multiprocessor," in Proc. Fourth Conf. Hypercube Concurrent Comput. Applications, Mar. 1989.

[15]

{15} W. E. L. Grimson, From Images to Surfaces: A Computational Study of the Human Early Visual System. Cambridge, MA: MIT Press, 1981.

Digital Library

[16]

{16} T. J. Olson and R. D. Potter, "Real-time vergence control," in Proc. IEEE Conf. Comput. Vision Patt. Recogn., 1989, pp. 404-409.

[17]

{17} B. K. P. Horn, "Focusing," Rep. No. 160, MIT Artificial Intell. Lab, 1968.

[18]

{18} J. M. Tenenbaum, "Accommodation in computer vision," Ph.D. Dissertation, Stanford Univ., 1971.

Digital Library

[19]

{19} E. P. Krotkov, "Focusing," Rep. No. MS-CIS-86-22, GRASP Lab., Univ. of Pennsylvania, Apr. 1986.

[20]

{20} R. A. Jarvis, "Focus optimisation criteria for computer image processing," Microscope, vol. 24, pp. 163-180, 1976.

[21]

{21} G. Ligthart and F. C. A. Groen, "A comparison of different autofocus algorithms," in Proc. Sixth Int. Conf. Patt. Recogn., Oct. 1982, pp. 597-600.

[22]

{22} S. Das and N. Ahuja, "Integrating multiresolution image acquisition and coarse-to-fine surface reconstruction from stereo," in Proc. IEEE Workshop Interpretation 3-D Scenes, Nov. 1989, pp. 9-15.

[23]

{23} D. Noton and L. Stark, "Scanpaths in saccadic eye movements while viewing and recognizing patterns," Vision Res., vol. 11, pp. 929-942, 1971.

[24]

{24} J. K. O'Regan and A. Lévy-Schoen, "Integrating visual information from successive fixations: Does trans-saccadic fusion exist?," Vision Res., vol. 23, no. 8, pp. 765-768, 1983.

[25]

{25} R. Groner, G. W. McConkie, and C. Menz, Eye Movements and Human Information Processing. Amsterdam: North-Holland, 1985.

[26]

{26} A. Lévy-Schoen, "Flexible and/or rigid control of oculomotor scanning behavior," in Eye Movements: Cognition and Visual Perception (D. F. Disher, R. A. Monty and J. W. Senders, Eds.). Hillsdale, NJ: Lawrence Erlbaum, 1981, pp. 299-314.

[27]

{27} J. M. Findlay, "Local and global influences on saccadic eye movements," in Eye Movements: Cognition and Visual Perception (D. F. Disher, R. A. Monty and J. W. Senders, Eds.). Hillsdale. NJ: Lawrence Erlbaum, 1981, pp. 171-179.

[28]

{28} V. Bozkov, Z. Bohdanecký, and T. Radil-Weiss, "Perception, Exploration and eye displacements," in Cognition and Eye Movements (R. Groner and P. Fraisse, Eds.). Amsterdam: North-Holland, 1982, pp. 24-33.

[29]

{29} N. H. Mackworth and A. J. Morandi, "The gaze selects informative details within picture," Perception Psychophys., vol. 2, pp. 547-552, 1967.

[30]

{30} P. J. Locher and C. F. Nodine, "Symmetry catches the eye," in Eye Movements: From Physiology to Cognition (J. K. O'Regan and A. Lévy-Schoen, Eds.). Amsterdam: North-Holland, 1987, pp. 353-361.

[31]

{31} C. Koch and S. Ullman, "Selecting one among the many: A simple network implementing shifts in selective visual attention," MIT AI Memo 770, 1984.

Digital Library

[32]

{32} J. J. Clark and N. J. Ferrier, "Modal control of an attentive vision system," in Proc. Second Int. Conf. Comput. Vision, Dec. 1988, pp. 514-513.

[33]

{33} D. J. Coombs and C. M. Brown, "Intelligent gaze control in binocular vision," in Proc. Fifth IEEE Int. Symp. Intell. Contr., Sept. 1990.

[34]

{34} P. J. Burt, "Algorithms and architectures for smart sensing," in Proc. DARPA Image Understanding Workshop, Apr. 1988, pp. 139-153.

[35]

{35} A. Shmuel and M. Werman, "Active vision: 3D from an image sequence," in Proc. 10th Int. Conf. Patt. Recogn., June 1990, pp. 48-54.

[36]

{36} C. M. Schor and L. B. Ciuffreda, Vergence Eye Movements: Basic and Clinical Aspects. Boston: Butterworths, 1983.

[37]

{37} J. M. Foley, "Primary distance perception," in Handbook of Sensory Physiology. Berlin: Springer-Verlag, 1978.

[38]

{38} V. V. Krishnan and L. Stark, "A heuristic model for the human vergence movement system," IEEE Trans. Biomed. Eng., vol. BME-24, no. 1, Jan. 1977.

[39]

{39} G. K. Hung and J. L. Semmlow, "Static behavior of accommodation and vergence: Computer simulation of an interactive dual-feedback system," IEEE Trans. Biomed. Eng., vol. BME-27, no. 8, Aug. 1980.

[40]

{40} C. M. Schor, "The relationship between fusional vergence eye movements and fixation disparity," Vision Res., vol. 19, no. 12, pp. 1359-1367, 1979.

[41]

{41} D. Marr and T. Poggio, "A computational theory of human stereo vision," in Proc. Royal Soc. London, vol. B, no. 204, pp. 301-328, 1979.

[42]

{42} D. Marr, Vision. San Francisco: Freeman, 1982.

[43]

{43} J. Aloimonos, I. Weiss, and A. Bandyopadhyay, "Active vision," in Proc. First Int. Conf. Comput. Vision, June 1987, pp. 35-54.

[44]

{44} A. Bandopadhay, B. Chandra, and D. H. Ballard, "Egomotion using active vision," in Proc. IEEE Conf. Comput. Vision Patt. Recogn., June 1986, pp. 498-503.

[45]

{45} D. Geiger and A. Yuille, "Stereopsis and eye-movement," in Proc. First Int. Conf. Comput. Vision, June 1987, pp. 306-314.

[46]

{46} F. P. Ferrie and M. D. Levine, "Integrating descriptions from multiple views," in Proc. Workshop Comput. Vision, Dec. 1987.

[47]

{47} B. Kamgar-Parsi, J. L. Jones, and A. Rosenfeld, "Registration of multiple overlapping range images: Scenes without distinctive features," in Proc. IEEE Conf. Comput. Vision Patt. Recogn., 1989, pp. 282-290.

[48]

{48} N. Ayache and O. D. Faugeras, "Building, registrating and fusing noisy visual maps," Int. J. Robotics Res., vol. 7, no. 6, Dec. 1988.

Digital Library

[49]

{49} H. Takahashi and F. Tomita, "Self-calibration of stereo cameras," in Proc. Second Int. Conf. Comput. Vision, Dec. 1988, pp. 123-128.

[50]

{50} A. L. Abbott, "Dynamic integration of depth cues for surface reconstruction from stereo images," Ph.D. Dissertation, Univ. of Illinois, 1990.

Digital Library

[51]

{51} M. A. Gennert and A. L. Yuille, "Determining the optimal weights in multiple objective function optimization," in Proc. Second Int. Conf. Comput. Vision, Dec. 1988, pp. 87-89.

Cited By

Baek SWalsh NChugunov IShi ZHeide F(2023)Centimeter-wave Free-space Neural Time-of-Flight ImagingACM Transactions on Graphics10.1145/352267142:1(1-18)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1145/3522671
Bhavsar ARajagopalan A(2012)Towards Unrestrained Depth Inference with Coherent Occlusion FillingInternational Journal of Computer Vision10.1007/s11263-011-0476-597:2(167-190)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1007/s11263-011-0476-5
Aydin TAkgul Y(2009)A Stereo Depth Recovery Method Using Layered Representation of the SceneProceedings of the 31st DAGM Symposium on Pattern Recognition - Volume 574810.5555/3089925.3089965(322-331)Online publication date: 9-Sep-2009
https://dl.acm.org/doi/10.5555/3089925.3089965
Show More Cited By

Active Stereo: Integrating Disparity, Vergence, Focus, Aperture and Calibration for Surface Estimation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
  2. Computer graphics
2. Networks
  1. Network protocols

Recommendations

Metric calibration of a stereo rig
VSR '95: Proceedings of the IEEE Workshop on Representation of Visual Scenes

Describes a method to determine affine and metric calibration for a stereo rig. The method does not involve the use of calibration objects or special motions, but simply a single general motion of the rig with fixed parameters (i.e. camera parameters ...
Assessment of stereo camera calibration techniques for a portable mobile mapping system

Mobile mapping systems that detect and geo‐reference road markings almost always consist of a stereo camera system integrated with a global positioning system/inertial navigation system. The data acquired by this navigational system allows features ...
Dynamic Stereo with Self-Calibration

An approach for incremental refinement of disparity maps obtained from a dynamic stereo sequence of a static scene is presented. The approach has been implemented using a binocular stereo vision system mounted on a mobile robot. A robust least median of ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 15, Issue 10

October 1993

129 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © Copyright © 1993 IEEE. All Rights Reserved.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 October 1993

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 10 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Baek SWalsh NChugunov IShi ZHeide F(2023)Centimeter-wave Free-space Neural Time-of-Flight ImagingACM Transactions on Graphics10.1145/352267142:1(1-18)Online publication date: 3-Mar-2023
https://dl.acm.org/doi/10.1145/3522671
Bhavsar ARajagopalan A(2012)Towards Unrestrained Depth Inference with Coherent Occlusion FillingInternational Journal of Computer Vision10.1007/s11263-011-0476-597:2(167-190)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1007/s11263-011-0476-5
Aydin TAkgul Y(2009)A Stereo Depth Recovery Method Using Layered Representation of the SceneProceedings of the 31st DAGM Symposium on Pattern Recognition - Volume 574810.5555/3089925.3089965(322-331)Online publication date: 9-Sep-2009
https://dl.acm.org/doi/10.5555/3089925.3089965
Samson ELaurendeau DParizeau MComtois SAllan JGosselin C(2006)The Agile Stereo Pair for active visionMachine Vision and Applications10.1007/s00138-006-0013-717:1(32-50)Online publication date: 27-Mar-2006
https://dl.acm.org/doi/10.1007/s00138-006-0013-7
Camacho PArrebola FSandoval F(2003)Multiresolution vision in autonomous systemsAutonomous robotic systems10.5555/860235.860254(451-470)Online publication date: 1-Jan-2003
https://dl.acm.org/doi/10.5555/860235.860254
Otero JOtero ASanchez L(2003)3D motion estimation of bubbles of gas in fluid glass, using an optical flow gradient technique extended to a third dimensionMachine Vision and Applications10.1007/s00138-002-0117-714:3(185-191)Online publication date: 1-Jul-2003
https://dl.acm.org/doi/10.1007/s00138-002-0117-7
Lin CShih SHung YTang G(2002)A new approach to automatic reconstruction of a 3-D world using active stereo visionComputer Vision and Image Understanding10.1006/cviu.2001.094385:2(117-143)Online publication date: 1-Feb-2002
https://dl.acm.org/doi/10.1006/cviu.2001.0943
Kopparapu SCorke P(2001)The Effect of Noise on Camera Calibration ParametersGraphical Models10.1006/gmod.2001.055163:5(277-303)Online publication date: 1-Sep-2001
https://dl.acm.org/doi/10.1006/gmod.2001.0551
Schechner YKiryati N(2000)Depth from Defocus vs. StereoInternational Journal of Computer Vision10.1023/A:100817512732739:2(141-162)Online publication date: 1-Sep-2000
https://dl.acm.org/doi/10.1023/A%3A1008175127327
Schechner YKiryati NBasri R(2000)Separation of Transparent Layers using FocusInternational Journal of Computer Vision10.1023/A:100816601746639:1(25-39)Online publication date: 31-Aug-2000
https://dl.acm.org/doi/10.1023/A%3A1008166017466
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents