research-article

Public Access

The Perception of Affordances in Mobile Augmented Reality

Authors:

Jeanine Stefanucci,

Sarah H. Creem-Regehr,

Bobby BodenheimerAuthors Info & Claims

SAP '21: ACM Symposium on Applied Perception 2021

Article No.: 12, Pages 1 - 10

https://doi.org/10.1145/3474451.3476239

Published: 16 September 2021 Publication History

Abstract

Today, augmented reality (AR) is most easily experienced through a mobile device such as a modern smartphone. For AR to be useful for applications such as training, it is important to understand how people perceive interactions with virtual objects presented to them via mobile AR. In this paper, we investigated two judgments of action capabilities (affordances) with virtual objects presented through smartphones: passing through an aperture and stepping over a gap. Our goals were to 1) determine if people can reliably scale these judgments to their body dimensions or capabilities and 2) explore whether cues presented in the context of the action could change their judgments. Assessments of perceived action capabilities were made in a pre/post-test design in which observers judged their affordances towards virtual objects prior to seeing an AR cue denoting their body dimension/capability, while viewing the cue, and after seeing the cue. Different patterns of results were found for the two affordances. For passing through, estimates became closer to shoulder width in the post-cue compared to the pre-cue block. For gap stepping, estimates were closer to actual stepping capability while viewing the cue, but did not persist when the cue was no longer present. Overall, our findings show that mobile smartphones can be used to assess perceived action capabilities with virtual targets and that AR cues can influence the perception of action capabilities in these devices. Our work provides a foundation for future studies investigating perception with the use of mobile AR with smartphones.

References

[1]

Nur Intan Adhani and Dayang Rohaya Awang Rambli. 2012. A survey of mobile augmented reality applications. In 1st International conference on future trends in computing and communication technologies. 89–96.

[2]

Matthias Berning, Daniel Kleinert, Till Riedel, and Michael Beigl. 2014. A Study of Depth Perception in Hand-Held Augmented Reality using Autostereoscopic Displays.

[3]

Ayush Bhargava, Kathryn M Lucaites, Leah S Hartman, Hannah Solini, Jeffrey W Bertrand, Andrew C Robb, Christopher C Pagano, and Sabarish V Babu. 2020a. Revisiting affordance perception in contemporary virtual reality. Virtual Reality (2020), 1–12.

[4]

Ayush Bhargava, Hannah Solini, Kathryn Lucaites, Jeffrey W Bertrand, Andrew Robb, Christopher C Pagano, and Sabarish V Babu. 2020b. Comparative Evaluation of Viewing and Self-Representation on Passability Affordances to a Realistic Sliding Doorway in Real and Immersive Virtual Environments. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 519–528.

[5]

Soumyajit Chakraborty, Jeanine K. Stefanucci, Sarah Creem-Regehr, and Bobby Bodenheimer. 2021. Distance Estimation with Mobile Augmented Reality in Action Space: Effects of Animated Cues. In 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 144–147. https://doi.org/10.1109/VRW52623.2021.00034

[6]

D. Chatzopoulos, C. Bermejo, Z. Huang, and P. Hui. 2017. Mobile Augmented Reality Survey: From Where We Are to Where We Go. IEEE Access 5(2017), 6917–6950.

[7]

Sarah H Creem-Regehr, Devin M Gill, Grant D Pointon, Bobby Bodenheimer, and Jeanine Stefanucci. 2019. Mind the gap: Gap affordance judgments of children, teens, and adults in an immersive virtual environment. Frontiers in Robotics and AI 6 (2019), 96.

[8]

James E. Cutting and Peter M. Vishton. 1995. Perceiving Layout and Knowing Distance: The Integration, Relative Potency and Contextual Use of Different Information about Depth. In Perception of Space and Motion, William Epstein and Sheena Rogers (Eds.). Academic Press, New York, 69–117.

[9]

Arindam Dey, Andrew Cunningham, and Christian Sandor. 2010. Evaluating depth perception of photorealistic mixed reality visualizations for occluded objects in outdoor environments. In Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology. ACM, 211–218.

Digital Library

[10]

Arindam Dey, Graeme Jarvis, Christian Sandor, and Gerhard Reitmayr. 2012. Tablet versus phone: Depth perception in handheld augmented reality. In 2012 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, 187–196.

Digital Library

[11]

Pei-Huang Diao and Nj Shih. 2018. MARINS: A mobile smartphone AR system for pathfinding in a dark environment. Sensors 18 (10 2018), 3442. https://doi.org/10.3390/s18103442

[12]

Tiffany Do, Jr LaViola, and Ryan McMahan. 2020. The Effects of Object Shape, Fidelity, Color, and Luminance on Depth Perception in Handheld Mobile Augmented Reality. ArXiv preprint (08 2020).

[13]

Elham Ebrahimi, Andrew Robb, Leah S Hartman, Christopher C Pagano, and Sabarish V Babu. 2018. Effects of anthropomorphic fidelity of self-avatars on reach boundary estimation in immersive virtual environments. In Proceedings of the 15th ACM Symposium on Applied Perception. 1–8.

Digital Library

[14]

John M Franchak and Karen E Adolph. 2012. What infants know and what they do: Perceiving possibilities for walking through openings.Developmental psychology 48, 5 (2012), 1254–1261.

[15]

J. M. Franchak and K. E. Adolph. 2014. Affordances as probabilistic functions: Implications for development, perception, and decisions for action. Ecological Psychology 26(2014), 109–124.

[16]

John M Franchak, Emma C Celano, and Karen E Adolph. 2012. Perception of passage through openings depends on the size of the body in motion. Experimental Brain Research 223, 2 (2012), 301–310.

[17]

Holly C. Gagnon, Dun Na, Keith Heiner, Jeanine Stefanucci, Sara Creem-Regehr, and Bobby Bodenheimer. 2020. The Role of Viewing Distance and Feedback on Affordance Judgments in Augmented Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 922–929.

[18]

Holly C. Gagnon, Yu Zhao, M. Richardson, Grant Pointon, Jeanine K Stefanucci, S. Creem-Regehr, and B. Bodenheimer. 2021. Gap Affordance Judgments in Mixed Reality: Testing the Role of Display Weight and Field of View. In Frontiers in Virtual Reality.

[19]

Michael Geuss, Jeanine Stefanucci, Sarah Creem-Regehr, and William B. Thompson. 2010. Can I Pass?: Using Affordances to Measure Perceived Size in Virtual Environments. In Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization (Los Angeles, California) (APGV ’10). ACM, New York, NY, USA, 61–64. https://doi.org/10.1145/1836248.1836259

Digital Library

[20]

Michael N. Geuss, Michael J. McCardell, and Jeanine K. Stefanucci. 2016. Fear similarly alters perceptual estimates of and actions over gaps. PLoS one 11, 7 (2016), e0158610.

[21]

James J. Gibson. 1979. The ecological approach to visual perception. Houghton Mifflin, Boston, MA.

[22]

O. Hasler, S. Blaser, and S. Nebiker. 2019. Implementation and First Evaluation of an Indoor Mapping Application Using Smartphones and AR Frameworks. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences XLII-2/W17 (2019), 135–141. https://doi.org/10.5194/isprs-archives-XLII-2-W17-135-2019

[23]

Takahiro Higuchi. 2013. Visuomotor control of human adaptive locomotion: understanding the anticipatory nature. Frontiers in Psychology 4, 277 (2013).

[24]

Lorenzo Jamone, Emre Ugur, Angelo Cangelosi, Luciano Fadiga, Alexandre Bernardino, Justus Piater, and José Santos-Victor. 2018. Affordances in Psychology, Neuroscience, and Robotics: A Survey. IEEE Transactions on Cognitive and Developmental Systems 10, 1(2018), 4–25. https://doi.org/10.1109/TCDS.2016.2594134

[25]

Yang Jiang and Leonard S Mark. 1994. The effect of gap depth on the perception of whether a gap is crossable. Perception & Psychophysics 56, 6 (1994), 691–700.

[26]

Eunice Jun, Jeanine K. Stefanucci, Sarah H. Creem-Regehr, Michael N. Geuss, and William B. Thompson. 2015. Big Foot: Using the Size of a Virtual Foot to Scale Gap Width. ACM Trans. Appl. Percept. 12, 4 (2015), 16:1–16:12.

Digital Library

[27]

Ernst Kruijff, J. Edward Swan, and Steven Feiner. 2010. Perceptual issues in augmented reality revisited. In 2010 IEEE International Symposium on Mixed and Augmented Reality. 3–12. https://doi.org/10.1109/ISMAR.2010.5643530

[28]

Gunnar Liestøl. 2009. Situated simulations: A prototyped augmented reality genre for learning on the iPhone. International Journal of Interactive Mobile Technologies (iJIM) 3 (2009), 24–28.

[29]

Jingjing (May) Liu, Gayathri Narasimham, Jeanine Stefanucci, Sarah Creem-Regehr, and Bobby Bodenheimer. 2020. Distance Perception in Modern Mobile Augmented Reality. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). 196–200.

[30]

Mark A Livingston, Zhuming Ai, J Edward Swan, and Harvey S Smallman. 2009. Indoor vs. outdoor depth perception for mobile augmented reality. In 2009 IEEE Virtual Reality Conference. IEEE, 55–62.

Digital Library

[31]

Leonard S Mark, James A Balliett, Kent D Craver, Stephen D Douglas, and Teresa Fox. 1990. What an actor must do in order to perceive the affordance for sitting. Ecological Psychology 2, 4 (1990), 325–366.

[32]

Donald A Norman. 2002. The design of everyday things. Basic books.

[33]

Paweł Nowacki and Marek Woda. 2020. Capabilities of ARCore and ARKit Platforms for AR/VR Applications. In Engineering in Dependability of Computer Systems and Networks, Wojciech Zamojski, Jacek Mazurkiewicz, Jarosław Sugier, Tomasz Walkowiak, and Janusz Kacprzyk (Eds.). Springer International Publishing, Cham, 358–370.

[34]

Jodie M Plumert. 1995. Relations between children’s overestimation of their physical abilities and accident proneness.Developmental Psychology 31, 5 (1995), 866.

[35]

Jodie M Plumert and David C Schwebel. 1997. Social and temperamental influences on children’s overestimation of their physical abilities: Links to accidental injuries. Journal of experimental child psychology 67, 3 (1997), 317–337.

[36]

Grant Pointon, Chelsey Thompson, Sarah Creem-Regehr, Jeanine Stefanucci, and Bobby Bodenheimer. 2018a. Affordances as a Measure of Perceptual Fidelity in Augmented Reality. In 2018 IEEE VR 2018 Workshop on Perceptual and Cognitive Issues in AR (PERCAR). 1–6.

[37]

Grant Pointon, Chelsey Thompson, Sarah Creem-Regehr, Jeanine Stefanucci, Miti Joshi, Richard Paris, and Bobby Bodenheimer. 2018b. Judging action capabilities in augmented reality. In Proceedings of the 15th ACM Symposium on Applied Perception. 1–8.

Digital Library

[38]

Horacio Rios, Eduardo González, Ciro Rodriguez, Hector R. Siller, and Manuel Contero. 2013. A Mobile Solution to Enhance Training and Execution of Troubleshooting Techniques of the Engine Air Bleed System on Boeing 737. Procedia Computer Science 25 (2013), 161 – 170. https://doi.org/10.1016/j.procs.2013.11.020 2013 International Conference on Virtual and Augmented Reality in Education.

[39]

Alberto Sanchez-Lite, Manuel Garcia, Rosario Domingo, and Miguel Angel Sebastian. 2013. Novel ergonomic postural assessment method (NERPA) using product-process computer aided engineering for ergonomic workplace design. PloS one 8, 8 (2013), e72703.

[40]

Yushan Siriwardhana, Pawani Porambage, Madhusanka Liyanage, and Mika Ylianttila. 2021. A Survey on Mobile Augmented Reality With 5G Mobile Edge Computing: Architectures, Applications, and Technical Aspects. IEEE Communications Surveys Tutorials 23, 2 (2021), 1160–1192. https://doi.org/10.1109/COMST.2021.3061981

[41]

Jeanine K Stefanucci, Sarah H Creem-Regehr, William B Thompson, David A Lessard, and Michael N Geuss. 2015. Evaluating the accuracy of size perception on screen-based displays: Displayed objects appear smaller than real objects. Journal of Experimental Psychology: Applied 21, 3 (2015), 215–223.

[42]

Jeanine K. Stefanucci and Michael N. Geuss. 2009. Big people, little world: The body influences size perception. Perception 38, 12 (2009), 1782–1795.

[43]

Thomas A Stoffregen, Chih-Mei Yang, M Russell Giveans, Moira Flanagan, and Benoît G Bardy. 2009. Movement in the perception of an affordance for wheelchair locomotion. Ecological Psychology 21, 1 (2009), 1–36.

[44]

J Edward Swan, Liisa Kuparinen, Scott Rapson, and Christian Sandor. 2017. Visually Perceived Distance Judgments: Tablet-Based Augmented Reality Versus the Real World. International Journal of Human–Computer Interaction 33, 7(2017), 576–591.

[45]

W. H. Warren and S. Whang. 1987. Visual Guidance of Walking Through Apertures: Body Scaled Information for Affordances. Journal of Experimental Psychology: Human Perception and Performance 13 (1987), 371–383.

[46]

Hansen Wu, Haley Adams, Grant Pointon, Jeanine Stefanucci, Sarah Creem-Regehr, and Bobby Bodenheimer. 2019. Danger from the deep: A gap affordance study in augmented reality. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE, 1775–1779.

[47]

Yawen Yu, Benoît G Bardy, and Thomas A Stoffregen. 2010. Influences of head and torso movement before and during affordance perception. Journal of Motor Behavior 43, 1 (2010), 45–54.

Cited By

Gagnon HStefanucci JCreem-Regehr SBodenheimer B(2023)Calibrated Passability Perception in Virtual Reality Transfers to Augmented RealityACM Transactions on Applied Perception10.1145/361345020:4(1-16)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1145/3613450
Shaer OOtiono JQian ZSeals ANov O(2022)Remote evaluation of augmented reality interaction with personal health informationFrontiers in Computer Science10.3389/fcomp.2022.9346944Online publication date: 15-Aug-2022
https://doi.org/10.3389/fcomp.2022.934694
Creem-Regehr SStefanucci JBodenheimer B(2022)Perceiving Affordances for Passing Through Apertures: A Discussion of Factors Influencing Replication Across Extended Reality2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00061(274-275)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00061

The Perception of Affordances in Mobile Augmented Reality
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction paradigms

Recommendations

Judging action capabilities in augmented reality
SAP '18: Proceedings of the 15th ACM Symposium on Applied Perception

The utility of mediated environments increases when environmental scale (size and distance) is perceived accurately. We present the use of perceived affordances---judgments of action capabilities---as an objective way to assess space perception in an ...
Calibrated Passability Perception in Virtual Reality Transfers to Augmented Reality
Special Issue on SAP 2023
As applications for virtual reality (VR) and augmented reality (AR) technology increase, it will be important to understand how users perceive their action capabilities in virtual environments. Feedback about actions may help to calibrate perception for ...
Multimodal augmented reality: the norm rather than the exception
MVAR '16: Proceedings of the 2016 workshop on Multimodal Virtual and Augmented Reality

Augmented reality (AR) is commonly seen as a technology that overlays virtual imagery onto a participant's view of the world. In line with this, most AR research is focused on what we see. In this paper, we challenge this focus on vision and make a case ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SAP '21: ACM Symposium on Applied Perception 2021

September 2021

109 pages

ISBN:9781450386630

DOI:10.1145/3474451

Editors:
Eakta Jain
University of Florida, USA
,
Anne-Hélène Olivier
University of Rennes 2, France
,
Massimiliano Di Luca
University of Birmingham, UK
,
Katja Zibrek
Inria Rennes, France
,
Rebecca Fribourg
Trinity College Dublin, Ireland

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 September 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Office of Naval Research

Conference

SAP '21

Sponsor:

SIGGRAPH

SAP '21: ACM Symposium on Applied Perception 2021

September 16 - 17, 2021

Virtual Event, France

Acceptance Rates

Overall Acceptance Rate 43 of 94 submissions, 46%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
527
Total Downloads

Downloads (Last 12 months)171
Downloads (Last 6 weeks)24

Reflects downloads up to 17 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Gagnon HStefanucci JCreem-Regehr SBodenheimer B(2023)Calibrated Passability Perception in Virtual Reality Transfers to Augmented RealityACM Transactions on Applied Perception10.1145/361345020:4(1-16)Online publication date: 25-Oct-2023
https://dl.acm.org/doi/10.1145/3613450
Shaer OOtiono JQian ZSeals ANov O(2022)Remote evaluation of augmented reality interaction with personal health informationFrontiers in Computer Science10.3389/fcomp.2022.9346944Online publication date: 15-Aug-2022
https://doi.org/10.3389/fcomp.2022.934694
Creem-Regehr SStefanucci JBodenheimer B(2022)Perceiving Affordances for Passing Through Apertures: A Discussion of Factors Influencing Replication Across Extended Reality2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00061(274-275)Online publication date: Oct-2022
https://doi.org/10.1109/ISMAR-Adjunct57072.2022.00061

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten