research-article

GazeForm: Dynamic Gaze-adaptive Touch Surface for Eyes-free Interaction in Airliner Cockpits

Authors:

Sylvain Pauchet,

Catherine Letondal,

Jean-Luc Vinot,

Mickaël Causse,

Valentin Becquet,

Guillaume CrouzetAuthors Info & Claims

DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference

Pages 1193 - 1205

https://doi.org/10.1145/3196709.3196712

Published: 08 June 2018 Publication History

Abstract

An increasing number of domains, including aeronautics, are adopting touchscreens. However, several drawbacks limit their operational use, in particular, eyes-free interaction is almost impossible making it difficult to perform other tasks simultaneously. We introduce GazeForm, an adaptive touch interface with shape-changing capacity that offers an adapted interaction modality according to gaze direction. When the user's eyes are focused on interaction, the surface is flat and the system acts as a touchscreen. When eyes are directed towards another area, physical knobs emerge from the surface. Compared to a touch only mode, experimental results showed that GazeForm generated a lower subjective mental workload and a higher efficiency of execution (20% faster). Furthermore, GazeForm required less visual attention and participants were able to concentrate more on a secondary monitoring task. Complementary interviews with pilots led us to explore timings and levels of control for using gaze to adapt modality.

Supplementary Material

suppl.mov (disfp139.mp4)

Supplemental video

Download
14.70 MB

References

[1]

Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, Jürgen Steimle, and Tino Weinkauf. 2015. Performance and Ergonomics of Touch Surfaces: A Comparative Study using Biomechanical Simulation. ACM Press, 1817--1826.

Digital Library

[2]

Jérôme Barbé, Nicolas Chatrenet, Régis Mollard, Marion Wolff, and Patxi Bérard. 2012. Physical ergonomics approach for touch screen interaction in an aircraft cockpit. In Proceedings of the 2012 Conference on Ergonomie et Interaction homme-machine (Ergo'IHM '12). ACM, New York, NY, USA, Pages 9, 8 pages.

Digital Library

[3]

Renaud Blanch, Yves Guiard, and Michel BeaudouinLafon. 2004. Semantic pointing: improving target acquisition with control-display ratio adaptation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, New York, NY, USA, 519--526.

Digital Library

[4]

David L. Brown, Holly S. Vitense, Paul A. Wetzel & Gretchen M. Anderson, (2002). Instrument scan strategies of F-117A Pilots. Aviation, Space, and Environmental Medicine, 73 (10), 1007--1013.

[5]

Tom Carter, Sue A. Seah, Ben Long, Bruce Drinkwater, and Siram Subramanian. 2013. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology (UIST '13). ACM, New York, NY, USA, 505--514.

Digital Library

[6]

Stéphane Chatty, Mathieu Magnaudet, and Daniel Prun. 2015. Verification of properties of interactive components from their executable code. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS '15). ACM, New York, NY, USA, 276--285.

Digital Library

[7]

Stéphane Chatty, Mathieu Magnaudet, Daniel Prun, Stéphane Conversy, Stéphanie Rey, and Mathieu Poirier. 2016. Designing, developing and verifying interactive components iteratively with djnn. 8th European Congress on Embedded Real Time Software and Systems (ERTS 2016).

[8]

Andy Cockburn, Carl Gutwin, Philippe Palanque, Yannick Deleris, Catherine Trask, Ashley Coveney, Marcus Yung, and Karon MacLean. 2017. Turbulent Touch: Touchscreen Input for Cockpit Flight Displays. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6742--6753.

Digital Library

[9]

Frédéric Dehais, Mickaël Causse, & Sébastien Tremblay. (2011). Mitigation of conflicts with automation: use of cognitive countermeasures. In Human Factors, 53(5), 448--460.

[10]

Sonia Dodd, Jeff Lancaster, Andrew Miranda, et al. 2014. Touch Screens on the Flight Deck: The Impact of Touch Target Size, Spacing, Touch Technology and Turbulence on Pilot Performance. In Pp. 6--10. SAGE Publications Sage CA: Los Angeles, CA.

[11]

Paul Dourish. 2004. What we talk about when we talk about context. Personal Ubiquitous Comput. 8, 1 (February 2004), 19--30.

Digital Library

[12]

George W. Fitzmaurice and William Buxton. 1997. An empirical evaluation of graspable user interfaces: towards specialized, space-multiplexed input. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '97). ACM, New York, NY, USA, 43--50.

Digital Library

[13]

George W. Fitzmaurice, Hiroshi Ishii, and William AS Buxton. 1995. Bricks: laying the foundations for graspable user interfaces. Proceedings of the SIGCHI conference on Human factors in computing systems, ACM Press/Addison-Wesley Publishing Co., 442--449.

Digital Library

[14]

Mackenzie G. Glaholt. 2014. Eye tracking in the cockpit: a review of the relationships between eye movements and the aviator's cognitive state. In Scientific report of Defence Research and Development Canada.

[15]

Fabian Göbel, Ioannis Giannopoulos, and Martin Raubal. 2016. The importance of visual attention for adaptive interfaces. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). ACM, New York, NY, USA, 930935.

Digital Library

[16]

Ken Funk. 1991. Cockpit task management: Preliminary definitions, normative theory, error taxonomy, and design recommendations. The International Journal of Aviation Psychology, 1(4), 271--285.

[17]

Chris Harrison and Scott E. Hudson. 2009. Providing dynamically changeable physical buttons on a visual display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 299--308.

Digital Library

[18]

Sandra G. Hart, Lowell E. Staveland, 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in psychology (Vol. 52, pp. 139--183). NorthHolland.

[19]

Sandra G. Hart, 2006. Nasa-Task Load Index (NASATLX); 20 Years Late. In Human Factors and Ergonomics Society Annual Meeting Proceedings vol. 50.

[20]

Fabian Hemmert, Gesche Joost, André Knörig, and Reto Wettach. 2008. Dynamic knobs: shape change as a means of interaction on a mobile phone. In CHI'08 Extended Abstracts on Human Factors in Computing Systems. ACM, 2309--2314.

Digital Library

[21]

Sylvain Hourlier, Sandra Guérard, Jean-Luc Barou and Xavier Servantie, 2015. Testing Touch Screens in Realistic Aeronautic Turbulent Conditions (Light to Severe). SAE International Journal of Aerospace 8

[22]

Ivy Software Bus http://www.eei.cena.fr/products/ivy/

[23]

Hiroo Iwata, Hiroaki Yano, Fumitaka Nakaizumi, and Ryo Kawamura. 2001. Project FEELEX: adding haptic surface to graphics. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques (SIGGRAPH '01). ACM, New York, NY, USA, 469--476.

Digital Library

[24]

Robert JK Jacob. 1993. Eye movement-based humancomputer interaction techniques: Toward noncommand interfaces. Advances in human-computer interaction 4: 151--190.

[25]

Yvonne Jansen, Pierre Dragicevic, and Jean-Daniel Fekete. 2012. Tangible remote controllers for wall-size displays. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2865-- 2874.

Digital Library

[26]

Sridher Kaminani. 2011. Human Computer Interaction Issues with Touch Screen Interfaces in the Flight Deck. In P. 6B4--1. IEEE.

[27]

Peter Kasarskis, Jennifer Stehwien, Joey Hickox, Anthony Aretz, & Chris Wickens, (2001). Comparison of expert and novice scan behaviors during VFR flight. Proceedings of the 11th International Symposium on Aviation Psychology, Columbus, OH: The Ohio State University.

[28]

Sofie Kinch, Erik Grönvall, Marianne Graves Petersen, and Majken Kirkegaard Rasmussen. 2014. Encounters on a shape-changing bench: exploring atmospheres and social behaviour in situ. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI '14). ACM, New York, NY, USA, 233--240.

Digital Library

[29]

Olivier Lefrancois, Nadine Matton, Yves Gourinat, Vsevolod Peysakhovich and Mickael Causse. The role of Pilots' monitoring strategies in flight performance. (2016) In: European Association for Aviation Psychology Conference EAAP32, 26 September 2016 -

[30]

September 2016 (Cascais, Portugal). 30. Catherine Letondal, Jean-Luc Vinot, Sylvain Pauchet, Caroline Boussiron, Stephanie Rey, Valentin Becquet, Claire Lavenir, 2018. Being in the Sky: Framing Tangible and Embodied Interaction for Future Airliner Cockpits. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction (TEI '18), to appear.

Digital Library

[31]

Loukia D. Loukopoulos, R. K. Dismukes, and I. Barshi. 2001. Cockpit interruptions and distractions: A line observation study. Proceedings of the 11th international symposium on aviation psychology, Ohio State University Press Columbus, 1--6.

[32]

Jan M. Noyes, and Alison F. Starr, 2007. A Comparison of Speech Input and Touch Screen for Executing Checklists in an Avionics Application. Int. J. Aviation Psychology 17

[33]

Sharon Oviatt and Philip Cohen. 2000. Perceptual user interfaces: multimodal interfaces that process what comes naturally. Commun. ACM 43, 3 (March 2000), 45--53.

Digital Library

[34]

Pernilla Qvarfordt. 2017. Gaze-informed multimodal interaction. In The Handbook of MultimodalMultisensor Interfaces, Sharon Oviatt, Björn Schuller, Philip R. Cohen, Daniel Sonntag, Gerasimos Potamianos, and Antonio Krüger (Eds.). Association for Computing Machinery and Morgan & Claypool, New York, NY, USA 365--402

Digital Library

[35]

Raf Ramakers, Johannes Schöning, and Kris Luyten. 2014. Paddle: highly deformable mobile devices with physical controls. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 2569--2578.

Digital Library

[36]

Majken K. Rasmussen, Esben W. Pedersen, Marianne G. Petersen, and Kasper Hornbæk. 2012. Shapechanging interfaces: a review of the design space and open research questions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 735--744.

Digital Library

[37]

Simon Robinson, Céline Coutrix, Jennifer Pearson, Juan Rosso, Matheus Fernandes Torquato, Laurence Nigay, and Matt Jones. 2016. Emergeables: Deformable Displays for Continuous Eyes-Free Mobile Interaction. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 3793--3805.

Digital Library

[38]

Dario D. Salvucci and John R. Anderson. 2000. Intelligent gaze-added interfaces. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems (CHI '00). ACM, New York, NY, USA, 273280.

Digital Library

[39]

Baris Serim, Khalil Klouche, and Giulio Jacucci. 2017. Gaze-Adaptive Above and On-Surface Interaction. In Proceedings of the 2017 Conference on Designing Interactive Systems (DIS '17). ACM, New York, NY, USA, 115--127.

Digital Library

[40]

Jihoon Suh, Wooshik Kim, and Andrea Bianchi. 2017. Button+: Supporting User and Context Aware Interaction through Shape-Changing Interfaces. In Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied Interaction (TEI '17). ACM, New York, NY, USA, 261--268.

Digital Library

[41]

Melanie Tory and Robert Kincaid. 2013. Comparing physical, overlay, and touch screen parameter controls. ACM Press, 91--100.

Digital Library

[42]

Jean-Luc Vinot, Catherine Letondal, Sylvain Pauchet, and Stéphane Chatty. 2016. Could tangibility improve the safety of touch-based interaction?: exploring a new physical design space for pilot-system interfaces. In Proceedings of the International Conference on Human-Computer Interaction in Aerospace (HCI-Aero '16). ACM, New York, NY, USA, Article 8, 8 pages.

Digital Library

[43]

Jean-Luc Vinot and Sylvie Athenes. 2012. Legible, are you sure?: an experimentation-based typographical design in safety-critical context. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, 2287--2296.

Digital Library

[44]

Jean-Luc Vinot and Stéphane Conversy. 2015. Concept of continuity, a "strong concept" to design graphical architecture of interactive systems. In Proceedings of the 27th Conference on l'Interaction Homme-Machine (IHM '15). ACM, New York, NY, USA, 15--10.

Digital Library

[45]

Simon Voelker, Kjell Ivar Øvergård, Chat Wacharamanotham, and Jan Borchers. 2015. Knobology Revisited: A Comparison of User Performance between Tangible and Virtual Rotary Knobs. In Proceedings of the 2015 International Conference on Interactive Tabletops & Surfaces (ITS '15). ACM, New York, NY, USA, 35--38.

Digital Library

[46]

Simon Voelker, Andrii Matviienko, Johannes Schöning, and Jan Borchers. 2015. Combining Direct and Indirect Touch Input for Interactive Workspaces using Gaze Input. In Proceedings of the 3rd ACM Symposium on Spatial User Interaction (SUI '15). ACM, New York, NY, USA, 79--88.

Digital Library

Cited By

Dubus RBrock AMackay W(2024)Developing a Design Space for Critiquing Autopilot InterfacesProceedings of the 35th Conference on l'Interaction Humain-Machine10.1145/3649792.3649805(1-12)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1145/3649792.3649805
Simon FImbert JCailleton PHublet LBrock A(2022)Application mobile d’aide à la navigation aérienne : une perspective de conception participativeProceedings of the 33rd Conference on l'Interaction Humain-Machine10.1145/3500866.3516373(1-13)Online publication date: 5-Apr-2022
https://dl.acm.org/doi/10.1145/3500866.3516373
De Pra YFontana FJarvelainen HPapetti SBianchi MSonego M(2022)Evaluation of rotation gestures in rotary vs. motionless knobs2022 IEEE Haptics Symposium (HAPTICS)10.1109/HAPTICS52432.2022.9765608(1-6)Online publication date: 21-Mar-2022
https://doi.org/10.1109/HAPTICS52432.2022.9765608
Show More Cited By

Index Terms

GazeForm: Dynamic Gaze-adaptive Touch Surface for Eyes-free Interaction in Airliner Cockpits
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Multi-plié: A Linear Foldable and Flattenable Interactive Display to Support Efficiency, Safety and Collaboration
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

We present the design concept of an accordion-fold interactive display to address the limits of touch-based interaction in airliner cockpits. Based on an analysis of pilot activity, tangible design principles for this design concept are identified. Two ...
Hover Cursor: Improving Touchscreen Acquisition Of Small Targets With Hover-enabled Pre-selection
CHI EA '15: Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems

Even with highly-sensitive touchscreens and emphasis on "designing for touch", small target selection remains difficult. Good touch performance cannot solve the "fat-finger" problem, which results from occlusion and the size disparity between fingers ...
Simulating an extendable tangible slider for eyes-free one-handed interaction on mobile devices
AVI '18: Proceedings of the 2018 International Conference on Advanced Visual Interfaces

Sliders are widely used on mobile devices. Envisioning mobile devices that can dynamically deform to raise tangible controls from the screen surface, tangible sliders offer the benefit of eyes-free interaction. However, reaching for distant values with ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

DIS '18: Proceedings of the 2018 Designing Interactive Systems Conference

June 2018

1418 pages

ISBN:9781450351980

DOI:10.1145/3196709

General Chairs:
Ilpo Koskinen
University of Twente
,
Youn-kyung Lim
KAIST
,
Program Chairs:
Teresa Cerratto-Pargman
Stockholm University
,
Kenny Chow
The Hong Kong Polytechnic University
,
William Odom
Simon Fraser University

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 June 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

DIS '18

Sponsor:

SIGCHI

DIS '18: Designing Interactive Systems Conference 2018

June 9 - 13, 2018

Hong Kong, China

Acceptance Rates

DIS '18 Paper Acceptance Rate 107 of 487 submissions, 22%;

Overall Acceptance Rate 1,158 of 4,684 submissions, 25%

Upcoming Conference

DIS '25

Sponsor:
sigchi

Designing Interactive Systems Conference

July 5 - 9, 2025

Funchal , Portugal

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
375
Total Downloads

Downloads (Last 12 months)41
Downloads (Last 6 weeks)4

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dubus RBrock AMackay W(2024)Developing a Design Space for Critiquing Autopilot InterfacesProceedings of the 35th Conference on l'Interaction Humain-Machine10.1145/3649792.3649805(1-12)Online publication date: 25-Mar-2024
https://dl.acm.org/doi/10.1145/3649792.3649805
Simon FImbert JCailleton PHublet LBrock A(2022)Application mobile d’aide à la navigation aérienne : une perspective de conception participativeProceedings of the 33rd Conference on l'Interaction Humain-Machine10.1145/3500866.3516373(1-13)Online publication date: 5-Apr-2022
https://dl.acm.org/doi/10.1145/3500866.3516373
De Pra YFontana FJarvelainen HPapetti SBianchi MSonego M(2022)Evaluation of rotation gestures in rotary vs. motionless knobs2022 IEEE Haptics Symposium (HAPTICS)10.1109/HAPTICS52432.2022.9765608(1-6)Online publication date: 21-Mar-2022
https://doi.org/10.1109/HAPTICS52432.2022.9765608
De Pra YFontana FPapetti S(2021)Interacting with Digital Audio Effects Through a Haptic Knob with Programmable Resistance2021 24th International Conference on Digital Audio Effects (DAFx)10.23919/DAFx51585.2021.9768295(113-120)Online publication date: 8-Sep-2021
https://doi.org/10.23919/DAFx51585.2021.9768295
Chau EYu JGoncu CWithana A(2021)Composite Line Designs and Accuracy Measurements for Tactile Line Tracing on Touch SurfacesProceedings of the ACM on Human-Computer Interaction10.1145/34885365:ISS(1-17)Online publication date: 5-Nov-2021
https://dl.acm.org/doi/10.1145/3488536
Pruszko LCoutrix CLaurillau YPiranda BBourgeois J(2021)Molecular HCIProceedings of the ACM on Human-Computer Interaction10.1145/34617335:EICS(1-33)Online publication date: 29-May-2021
https://dl.acm.org/doi/10.1145/3461733
Wambecke JGoguey ANigay LDargent LHauret DLafon Sde Visme J(2021)M[eye]croProceedings of the ACM on Human-Computer Interaction10.1145/34617325:EICS(1-22)Online publication date: 29-May-2021
https://dl.acm.org/doi/10.1145/3461732
Cantu AVinot JLetondal CPauchet SCausse M(2021)Does folding improve the usability of interactive surfaces in future airliner cockpits An evaluation under turbulent conditions and varying cognitive loadProceedings of the 32nd Conference on l'Interaction Homme-Machine10.1145/3450522.3451246(1-10)Online publication date: 13-Apr-2021
https://dl.acm.org/doi/10.1145/3450522.3451246
Simon FAlonso Duran LCarrau XDelacroix EGuilbert BIMBERT JCausse MBrock A(2021)BLEXVFR: Designing a Mobile Assistant for PilotsAdjunct Publication of the 23rd International Conference on Mobile Human-Computer Interaction10.1145/3447527.3474852(1-5)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3447527.3474852
De Pra YFontana FPapetti S(2021)Endless Knob with Programmable Resistive Force FeedbackHuman-Computer Interaction – INTERACT 202110.1007/978-3-030-85610-6_32(580-589)Online publication date: 26-Aug-2021
https://doi.org/10.1007/978-3-030-85610-6_32
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten