research-article

Improving relative depth judgments in augmented reality with auxiliary augmentations

Authors:

Aleksi Mäkinen,

Jukka Häkkinen,

Pirkko OittinenAuthors Info & Claims

ACM Transactions on Applied Perception (TAP), Volume 10, Issue 1

Article No.: 6, Pages 1 - 21

https://doi.org/10.1145/2422105.2422111

Published: 04 March 2013 Publication History

Abstract

Significant depth judgment errors are common in augmented reality. This study presents a visualization approach for improving relative depth judgments in augmented reality. The approach uses auxiliary augmented objects in addition to the main augmentation to support ordinal and interval depth judgment tasks. The auxiliary augmentations are positioned spatially near real-world objects, and the location of the main augmentation can be deduced based on the relative depth cues between the augmented objects. In the experimental part, the visualization approach was tested in the “X-ray” visualization case with a video see-through system. Two relative depth cues, in addition to motion parallax, were used between graphical objects: relative size and binocular disparity. The results show that the presence of auxiliary objects significantly reduced errors in depth judgment. Errors in judging the ordinal location with respect to a wall (front, at, or behind) and judging depth intervals were reduced. In addition to reduced errors, the presence of auxiliary augmentation increased the confidence in depth judgments, and it was subjectively preferred. The visualization approach did not have an effect on the viewing time.

References

[1]

Allison, R., Gillam, B., and Vecellio, E. 2009. Binocular depth discrimination and estimation beyond interaction space. J. Vision 9, 1, 10, 1--14.

[2]

Avery, B., Sandor, C., and Thomas, B. H. 2009. Improving spatial perception for augmented reality X-ray vision. In Proceedings of the Virtual Reality Conference. IEEE, Los Alamitos, CA, 79--82.

[3]

Avery, B., Thomas, B. H., and Piekarski, W. 2008. User evaluation of see-through vision for mobile outdoor augmented reality. In Proceedings of the 7th IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '08) IEEE, Los Alamitos, CA, 69--72.

Digital Library

[4]

Bane, R. and Hollerer, T. 2004. Interactive tools for virtual X-ray vision in mobile augmented reality. In Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality (ISMAR '04), IEEE, Los Alamitos, CA, 231--239.

Digital Library

[5]

Birkfellner, W., Figl, M., Huber, K., Watzinger, F., Wanschitz, F., Hummel, J., Hanel, R., Greimel, W., Homolka, P., Ewers, R., and Bergmann, H. 2002. A head-mounted operating binocular for augmented reality visualization in medicine - Design and initial evaluation. IEEE Trans. Medical Imag. 21, 8, 991--997.

[6]

Collett, T. S. 1985. Extrapolating and interpolating surfaces in depth. In Proc. Royal Soc. London. Series B. Biol. Sci. 224, 1234, 43--56.

[7]

Cutting, J. E. and Vishton, P. M. 1995. Perceiving layout and knowing distances: The integration, relative potency, and contextual use of different information about depth. In Handbook of Perception and Cognition, W. Epstein and S. Rogers Eds., vol. 5, Academic Press, San Diego, CA., 69--117.

[8]

De Silva, V., Fernando, A., Worrall, S., Arachchi, H. K., and Kondoz, A. 2011. Sensitivity analysis of the human visual system for depth cues in stereoscopic 3-D displays. IEEE Trans. Multimedia 13, 3, 498--506.

Digital Library

[9]

Dey, A., Cunningham, A., and Sandor, C. 2010. Evaluating depth perception of photorealistic mixed reality visualizations for occluded objects in outdoor environments. In Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology (VRST '10). 211--218.

Digital Library

[10]

Drascic, D. and Milgram, P. 1991. Pointing accuracy of a virtual stereographic pointer in a real stereoscopic video world. In Proceedings of the SPIE Stereoscopic Displays and Applications II. Vol. 1457, 623--626.

[11]

Drascic, D. and Milgram, P. 1996. Perceptual Issues in augmented reality. In Proceedings of the SPIE Stereoscopic Displays and Applications. Vol. 2653, 123--134.

[12]

Ellis, S. R. and Menges, B. M. 1998. Localization of virtual objects in the near visual field. Hum. Factors 40, 3, 415--431.

[13]

Erkelens, C. J. and Collewijn, H. 1985. Motion perception during dichoptic viewing of moving random-dot stereograms. Vision Res. 25, 4, 583--588.

[14]

Faubert, J. 2001. Motion parallax, stereoscopy, and the perception of depth: Practical and theoretical issue. In Proceedings of the SPIE on Three-Dimensional Video and Display: Devices and Systems. 168--191.

[15]

Feiner, S., Webster, A., Krueger, T., MacIntyre, B., and Keller, E. 1995. Architectural Anatomy. Presence: Teleoper. Virtual Environ. 4, 3, 318--325.

Digital Library

[16]

Foley, J. M. 1991. Stereoscopic distance perception. In Pictorial Communication in Real and Virtual Environments, S. Ellis et al. Eds., Taylor&Francis.

Digital Library

[17]

Furmanski, C., Azuma, R., and Daily, M. 2002. Augmented-reality visualizations guided by cognition: perceptual heuristics for combining visible and obscured information. In Proceedings of the International Symposium on Mixed and Augmented Reality. IEEE, Los Alamitos, CA, 215--225.

Digital Library

[18]

Gilinsky, A. S. 1951. Perceived size and distance in visual space. Psychol. Rev. 58, 460--482.

[19]

Grechkin, T. Y., Nguyen, T. D., Plumert, J. M., Cremer, J. F., and Kearney, J. K. 2010. How does presentation method and measurement protocol affect distance estimation in real and virtual environments&quest; ACM Trans. Appl. Percept. 7, 4, 1--18.

Digital Library

[20]

Häkkinen, J. and Nyman, G. 1996. Depth asymmetry in Da Vinci stereopsis. Vision Res. 36, 23, 3815--3819.

[21]

Howard, H. J. 1919. A test for the judgment of distance. Trans. Amer. Ophthalmological Soc. 17, 195--235.

[22]

Hubona, G. S., Wheeler, P. N., Shirah, G. W., and Brandt, M. 1999. The relative contributions of stereo, lighting, and background scenes in promoting 3D depth visualization. ACM Trans. Comput.-Hum. Interact. 6, 3, 214--242.

Digital Library

[23]

Image Engineering. 2012. LED-PANEL. http://www.imageengineering.de/index.php&quest;option=com_content&view=article&id= 76&Itemid=68.

[24]

Johnston, E. B. 1991. Systematic distortions of shape from stereopsis. Vision Res. 31, 7--8, 1351--1360.

[25]

Jurgens, V., Cockburn, A., and Billinghurst, M. 2006. Depth cues for augmented reality stakeout. In Proceedings of the 7th ACM SIGCHI New Zealand Chapter International Conference on Computer-Human Interaction: Design Centered HCI (CHINZ '06). ACM, New York, 117--124.

Digital Library

[26]

Kanizsa, G. 1979. Organization in Vision: Essays on Gestalt Perception. Praeger, New York.

[27]

Kruijff, E., Swan, J., and Feiner, S. 2010. Perceptual issues in augmented reality revisited. In Proceedings of the 9th IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, Los Alamitos, CA, 3--12.

[28]

Kytö, M., Häkkinen, J., and Oittinen, P. 2011. Stereoscopic viewing facilitates the perception of crowds. In Proceedings of the IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS). IEEE, Los Alamitos, 49--53.

Digital Library

[29]

Kytö, M., Nuutinen, M., and Oittinen, P. 2011. Method for measuring stereo camera depth accuracy based on stereoscopic vision. In SPIE/IS&T Electronic Imaging Three-Dimensional Imaging, Interaction, and Measurement. Vol. 7864, 1--9.

[30]

Lambooij, M., IJsselsteijn, W., Fortuin, M., and Heynderickx, I. 2009. Visual discomfort and visual fatigue of stereoscopic displays: A review. J. Imag. Sci. Technol. 53, 3, 030201--14.

[31]

Landy, M. S., Maloney, L. T., Johnston, E. B., and Young, M. 1995. Measurement and modeling of depth cue combination: in defense of weak fusion. Vision Res. 35, 3, 389--412.

[32]

Liu, Y., Bovik, A. C., and Cormack, L. K. 2008. Disparity statistics in natural scenes. J. Vision 8, 11, 19, 1--14.

[33]

Livingston, M., Zhuming, A., and Decker, J. 2009. A user study towards understanding stereo perception in head-worn augmented reality displays. In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality. IEEE, Los Alamitos, CA, 53--56.

Digital Library

[34]

Livingston, M. A., Swan, J. E., II., Gabbard, J. L., Höllerer, T. H., Hix, D., Julier, S. J., Baillot, Y., and Brown, D. 2003. Resolving multiple occluded layers in augmented reality. In Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality. IEEE, Los Alamitos, CA, 56--65.

Digital Library

[35]

Loomis, J. M., da Silva, J. A., Fujita, N., and Fukusima, S. S. 1992. Visual space perception and visually directed action. J. Experiment. Psychol. Human Percept. Perform. 18, 4, 906--921.

[36]

McCandless, J. W., Ellis, S. R., and Adelstein, B. D. 2000. Localization of a time-delayed, monocular virtual object superimposed on a real environment. Presence 9, 1, 15--24.

Digital Library

[37]

McKee, S. P. and Taylor, D. G. 2010. The precision of binocular and monocular depth judgments in natural settings. J. Vision 10, 5, 1--13.

[38]

Nagata, S. 1991. How to reinforce perception of depth in single two-dimensional pictures. In Pictorial Communication in Virtual and Real Environments, S. R. Ellis Eds., Taylor&Francis, 527--544.

Digital Library

[39]

Nakayama, K. 1996. Binocular visual surface perception. In Proc. Nat. Acad. Sci. 93, 2, 634--639.

[40]

Nakayama, K., Shimojo, S., and Silverman, G. H. 1989. Stereoscopic depth: Its relation to image segmentation, grouping, and the recognition of occluded objects. Perception 18, 1, 55--68.

[41]

Napieralski, P. E., Altenhoff, B. M., Bertrand, J. W., Long, L. O., Babu, S. V., Pagano, C. C., Kern, J., and Davis, T. A. 2011. Near-field distance perception in real and virtual environments using both verbal and action responses. ACM Trans. Appl. Percept. 8, 3, 1--19.

Digital Library

[42]

Ooi, T. L., Wu, B., and He, Z. J. 2001. Distance determined by the angular declination below the horizon. Nature 414, 6860, 197--200.

[43]

OpenCV. 2011. Version 2.3. http://opencv.willowgarage.com/.

[44]

O'Shea, R. P., Blackburn, S. G., and Ono, H. 1994. Contrast as a depth cue. Vision Res. 34, 12, 1595--1604.

[45]

Palmisano, S., Gillam, B., Govan, D. G., Allison, R. S., and Harris, J. M. 2010. Stereoscopic perception of real depths at large distances. J. Vision 10, 6, 19, 1--16.

[46]

Peterson, S., Axholt, M., and Ellis, S. R. 2008. Managing visual clutter: A generalized technique for label segregation using stereoscopic disparity. In IEEE Virtual Reality, 169--176.

[47]

Peterson, S. D., Axholt, M., and Ellis, S. R. 2009. Objective and subjective assessment of stereoscopically separated labels in augmented reality. Comput. Graph. 33, 1, 23--33.

Digital Library

[48]

Philbeck, J. W. and Loomis, J. M. 1997. Comparison of two indicators of perceived egocentric distance under full-cue and reduced-cue conditions. J. Exper. Psychol. Hum. Percept. Perform. 23, 1, 72--85.

[49]

Polys, N. F., Bowman, D. A., and North, C. 2011. The role of depth and gestalt cues in information-rich virtual environments. Int. J. Hum.-Comput. Stud. 69, 1--2, 30--51.

Digital Library

[50]

Ramachandran, V. S. and Cavanagh, P. 1985. Subjective contours capture stereopsis. Nature 317, 527--530.

[51]

Reinhart, W. F. 1990. Comparison of depth cues for relative depth judgments. In Proceedings of the. SPIE on Stereoscopic Displays and Applications. Vol. 1256, 12--21.

[52]

Reinhart, W. F. 1991. Depth cueing for visual search and cursor positioning. In Proceedings of the SPIE on Stereoscopic Displays and Applications. Vol. 1457, 221--232.

[53]

Rogers, B. J. and Collett, T. S. 1989. The appearance of surfaces specified by motion parallax and binocular disparity. Quart. J. Hum. Exper. Psychol. 41, 4, 697--717.

[54]

Rolland, J. P. and Fuchs, H. 2000. Optical versus video see-through head-mounted displays in medical visualization. Presence 9, 3, 287--309.

Digital Library

[55]

Sandor, C., Cunningham, A., Dey, A., and Mattila, V. V. 2010. An augmented reality X-ray system based on visual saliency. In Proceedings of the IEEE International Symposium on Mixed and Augmented Reality. IEEE, Los Alamitos, CA, 27--36.

[56]

Sielhorst, T., Bichlmeier, C., Heining, S., and Navab, N. 2006. Depth perception - A major issue in medical AR: Evaluation study by twenty surgeons. In Proceedings of the International Conference on Medical Image Computing and Computer Assisted Intervention. 364--372.

Digital Library

[57]

Singh, G. J. S., II and Jones, J. 2010. Depth judgment measures and occluding surfaces in near-field augmented reality. In Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization. Vol. 1, 149--156.

Digital Library

[58]

Sugano, N., Kato, H., and Tachibana, K. 2003. The effects of shadow representation of virtual objects in augmented reality. In Proceedings of the Second IEEE and ACM International Symposium on Mixed and Augmented Reality. IEEE, Los Alamitos, CA, 76--83.

Digital Library

[59]

Sundet, J. M. 1978. Effects of colour on perceived depth: Review of experiments and evaluation of theories. Scandinavian J. Psychol. 19, 1, 133--143.

[60]

Suzuki, S. and Be, K. 1985. Topological structural analysis of digitized binary images by border following. Comput. Vision, Graph. Image Process. 30, 1, 32--46.

[61]

Swan, J. and Livingston, M. 2006. A perceptual matching technique for depth judgments in optical, see-through augmented reality. In Virtual Reality. 19--26.

Digital Library

[62]

Takashi, S., Kim, J., Hoffman, D. M., and Banks, M. S. 2011. The zone of comfort : Predicting visual discomfort with stereo displays. J. Vision 11, 8, 1--29.

[63]

Takeichi, H., Watanabe, T., and Shimojo, S. 1992. Illusory occluding contours and surface formation by depth propagation. Perception 21, 2, 177--184.

[64]

Tsuda, T., Yamamoto, H., Kameda, Y., and Ohta, Y. 2005. Visualization methods for outdoor see-through vision. In Proceedings of the International Conference on Augmented Tele-Existence. ACM, New York, 62--69.

Digital Library

[65]

Ware, C. 2004. Information Visualization: Perception for Design. Morgan Kaufmann, San Francisco, CA.

Digital Library

[66]

Willemsen, P., Gooch, A. A., Thompson, W. B., and Creem-Regehr, S. H. 2008. Effects of stereo viewing conditions on distance perception in virtual environments. Presence: Teleoper. Virtual Environ. 17, 1, 91--101.

Digital Library

[67]

Wither, J. and Hollerer, T. 2005. Pictorial depth cues for outdoor augmented reality. In Proceedings of the IEEE International Symposium on Wearable Computers. IEEE, Los Alamitos, CA, 92--99.

Digital Library

[68]

Yeh, Y.-Y. and Silverstein, L. 1990. Limits of fusion and depth judgment in stereoscopic color displays. Hum. Factors 32, 1, 45--60.

Digital Library

[69]

Zhai, S., Buxton, W., and Milgram, P. 1996. The partial-occlusion effect: Utilizing semitransparency in 3D human-computer interaction. ACM Trans. Comput. Hum. Interact. 3, 3, 254--284.

Digital Library

Cited By

Clarke TGwilt IZucco JMayer WSmith R(2024)Superpowers in the Metaverse: Augmented Reality Enabled X-Ray Vision in Immersive EnvironmentsAugmented and Virtual Reality in the Metaverse10.1007/978-3-031-57746-8_15(283-309)Online publication date: 12-May-2024
https://doi.org/10.1007/978-3-031-57746-8_15
Martin-Gomez AMerkl FWinkler AHeiliger CEck UKarcz KNavab N(2023)A Closer Look at Dynamic Medical Visualization Techniques2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00024(99-108)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00024
Muthalif MShojaei DKhoshelham K(2023)A review of augmented reality visualization methods for subsurface utilitiesAdvanced Engineering Informatics10.1016/j.aei.2021.10149851:COnline publication date: 15-Mar-2023
https://dl.acm.org/doi/10.1016/j.aei.2021.101498
Show More Cited By

Index Terms

Improving relative depth judgments in augmented reality with auxiliary augmentations

Recommendations

The effects of virtual reality, augmented reality, and motion parallax on egocentric depth perception
APGV '08: Proceedings of the 5th symposium on Applied perception in graphics and visualization

As the use of virtual and augmented reality applications becomes more common, the need to fully understand how observers perceive spatial relationships grows more critical. One of the key requirements in engineering a practical virtual or augmented ...
Visuotactile integration for depth perception in augmented reality
ICMI '16: Proceedings of the 18th ACM International Conference on Multimodal Interaction

Augmented reality applications using stereo head-mounted displays are not capable of perfectly blending real and virtual objects. For example, depth in the real world is perceived through cues such as accommodation and vergence. However, in stereo head-...
Perceiving depth: optical versus video see-through
VRST '16: Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology

Head-Mounted Displays (HMDs) and similar 3D visualization devices are becoming ubiquitous. Going a step forward, HMD see-through systems bring virtual objects to real world settings, allowing augmented reality to be used in complex engineering ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Applied Perception

ACM Transactions on Applied Perception Volume 10, Issue 1

February 2013

120 pages

ISSN:1544-3558

EISSN:1544-3965

DOI:10.1145/2422105

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 March 2013

Accepted: 01 October 2012

Revised: 01 October 2012

Received: 01 February 2012

Published in TAP Volume 10, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
771
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)3

Reflects downloads up to 18 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Clarke TGwilt IZucco JMayer WSmith R(2024)Superpowers in the Metaverse: Augmented Reality Enabled X-Ray Vision in Immersive EnvironmentsAugmented and Virtual Reality in the Metaverse10.1007/978-3-031-57746-8_15(283-309)Online publication date: 12-May-2024
https://doi.org/10.1007/978-3-031-57746-8_15
Martin-Gomez AMerkl FWinkler AHeiliger CEck UKarcz KNavab N(2023)A Closer Look at Dynamic Medical Visualization Techniques2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00024(99-108)Online publication date: 16-Oct-2023
https://doi.org/10.1109/ISMAR59233.2023.00024
Muthalif MShojaei DKhoshelham K(2023)A review of augmented reality visualization methods for subsurface utilitiesAdvanced Engineering Informatics10.1016/j.aei.2021.10149851:COnline publication date: 15-Mar-2023
https://dl.acm.org/doi/10.1016/j.aei.2021.101498
Zhang ZPan ZLi WSu Z(2022)X-Board: an egocentric adaptive AR assistant for perception in indoor environmentsVirtual Reality10.1007/s10055-022-00742-327:2(1327-1343)Online publication date: 26-Dec-2022
https://dl.acm.org/doi/10.1007/s10055-022-00742-3
Zollmann SLanglotz TGrasset RLo WMori SRegenbrecht H(2021)Visualization Techniques in Augmented Reality: A Taxonomy, Methods and PatternsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.298624727:9(3808-3825)Online publication date: 1-Sep-2021
https://doi.org/10.1109/TVCG.2020.2986247
Li HWang WMa WZhang GWang QQu J(2021)Design and analysis of depth cues on depth perception in interactive mixed reality simulation systemsJournal of the Society for Information Display10.1002/jsid.107430:2(87-102)Online publication date: 12-Aug-2021
https://doi.org/10.1002/jsid.1074
Garro VSundstedt VNavarro D(2020)A review of current trends on visual perception studies in virtual and augmented realitySIGGRAPH Asia 2020 Courses10.1145/3415263.3419144(1-97)Online publication date: 17-Nov-2020
https://dl.acm.org/doi/10.1145/3415263.3419144
Byvaltsev S(2020)Review of the features of augmented reality application in the training of operators and maintenance staffIOP Conference Series: Materials Science and Engineering10.1088/1757-899X/966/1/012121966(012121)Online publication date: 14-Nov-2020
https://doi.org/10.1088/1757-899X/966/1/012121
Byvaltsev S(2020)Augmented reality for SCADAIOP Conference Series: Materials Science and Engineering10.1088/1757-899X/862/5/052022862:5(052022)Online publication date: 1-May-2020
https://doi.org/10.1088/1757-899X/862/5/052022
Heinrich FBornemann KLawonn KHansen C(2019)Depth Perception in Projective Augmented Reality: An Evaluation of Advanced Visualization TechniquesProceedings of the 25th ACM Symposium on Virtual Reality Software and Technology10.1145/3359996.3364245(1-11)Online publication date: 12-Nov-2019
https://dl.acm.org/doi/10.1145/3359996.3364245
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents