Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
10.1145/2858036.2858097acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Emergeables: Deformable Displays for Continuous Eyes-Free Mobile Interaction

Published: 07 May 2016 Publication History

Abstract

In this paper we present the concept of Emergeables -- mobile surfaces that can deform or 'morph' to provide fully-actuated, tangible controls. Our goal in this work is to provide the flexibility of graphical touchscreens, coupled with the affordance and tactile benefits offered by physical widgets. In contrast to previous research in the area of deformable displays, our work focuses on continuous controls (e.g., dials or sliders), and strives for fully-dynamic positioning, providing versatile widgets that can change shape and location depending on the user's needs. We describe the design and implementation of two prototype emergeables built to demonstrate the concept, and present an in-depth evaluation that compares both with a touchscreen alternative. The results show the strong potential of emergeables for on-demand, eyes-free control of continuous parameters, particularly when comparing the accuracy and usability of a high-resolution emergeable to a standard GUI approach. We conclude with a discussion of the level of resolution that is necessary for future emergeables, and suggest how high-resolution versions might be achieved.

Supplementary Material

suppl.mov (pn0421-file3.mp4)
Supplemental video
MP4 File (p3793-robinson.mp4)

References

[1]
Jason Alexander, John Hardy and Stephen Wattam. 2014. Characterising the physicality of everyday buttons. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (ITS '14). ACM, NY, NY, USA, 205--208. http: //dx.doi.org/10.1145/2669485.2669519.
[2]
Gilles Bailly, Thomas Pietrzak, Jonathan Deber and Daniel J. Wigdor. 2013. Métamorphe: augmenting hotkey usage with actuated keys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 563--572. http://dx.doi.org/10.1145/2470654.2470734.
[3]
Regina Bernhaupt, Marianna Obrist, Astrid Weiss, Elke Beck and Manfred Tscheligi. 2008. Trends in the living room and beyond: results from ethnographic studies using creative and playful probing. Comput. Entertain. 6, 1, 5:1--5:23. http://dx.doi.org/10.1145/ 1350843.1350848.
[4]
Stuart K. Card, Jock D. Mackinlay and George G. Robertson. 1991. A morphological analysis of the design space of input devices. ACM Trans. Inf. Syst. 9, 2, 99--122. http://dx.doi.org/10.1145/123078.128726.
[5]
Andy Cockburn, Carl Gutwin, Joey Scarr and Sylvain Malacria. 2014. Supporting novice to expert transitions in user interfaces. ACM Comput. Surv. 47, 2, 31:1--31:36. http://dx.doi.org/10.1145/2659796.
[6]
Céline Coutrix and Cédric Masclet. 2015. Shapechange for zoomable TUIs: opportunities and limits of a resizable slider. In Human-Computer Interaction - INTERACT 2015. Lecture Notes in Computer Science vol. 9296. Springer International Publishing, 349--366. http://dx.doi.org/10.1007/978--3--319--22701--6_27.
[7]
Panteleimon Dimitriadis and Jason Alexander. 2014. Evaluating the effectiveness of physical shape-change for in-pocket mobile device notifications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2589--2592. http://dx.doi.org/10.1145/2556288. 2557164.
[8]
George W. Fitzmaurice and William Buxton. 1997. An empirical evaluation of graspable user interfaces: towards specialized, space-multiplexed input. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '97). ACM, NY, NY, USA, 43--50. http://dx.doi.org/10.1145/ 258549.258578.
[9]
George W. Fitzmaurice, Hiroshi Ishii and William A. S. Buxton. 1995. Bricks: laying the foundations for graspable user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., NY, NY, USA, 442--449. http: //dx.doi.org/10.1145/223904.223964.
[10]
Sean Follmer, Daniel Leithinger, Alex Olwal, Akimitsu Hogge and Hiroshi Ishii. 2013. Inform: dynamic physical affordances and constraints through shape and object actuation. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, NY, NY, USA, 417--426. http://dx.doi.org/10.1145/2501988.2502032.
[11]
Antonio Gomes and Roel Vertegaal. 2015. Paperfold: evaluating shape changes for viewport transformations in foldable thin-film display devices. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15). ACM, NY, NY, USA, 153--160. http://dx.doi.org/10. 1145/2677199.2680572.
[12]
Sidhant Gupta, Tim Campbell, Jeffrey R. Hightower and Shwetak N. Patel. 2010. Squeezeblock: using virtual springs in mobile devices for eyes-free interaction. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, NY, NY, USA, 101--104. http: //dx.doi.org/10.1145/1866029.1866046.
[13]
John Hardy, Christian Weichel, Faisal Taher, John Vidler and Jason Alexander. 2015. Shapeclip: towards rapid prototyping with shape-changing displays for designers. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 19--28. http: //dx.doi.org/10.1145/2702123.2702599.
[14]
Chris Harrison and Scott E. Hudson. 2009. Providing dynamically changeable physical buttons on a visual display. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, NY, NY, USA, 299--308. http://dx.doi.org/10. 1145/1518701.1518749.
[15]
Fabian Hemmert, Susann Hamann, Matthias Löwe, Anne Wohlauf and Gesche Joost. 2010. Shape-changing mobiles: tapering in one-dimensional deformational displays in mobile phones. In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '10). ACM, NY, NY, USA, 249--252. http://dx.doi.org/10. 1145/1709886.1709936.
[16]
Fabian Hemmert, Gesche Joost, André Knörig and Reto Wettach. 2008. Dynamic knobs: shape change as a means of interaction on a mobile phone. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08). ACM, NY, NY, USA, 2309--2314. http://dx.doi.org/10.1145/1358628. 1358675.
[17]
David Holman and Roel Vertegaal. 2008. Organic user interfaces: designing computers in any way, shape, or form. Commun. ACM 51, 6, 48--55. http://dx.doi.org/10. 1145/1349026.1349037.
[18]
Jonathan Hook, Stuart Taylor, Alex Butler, Nicolas Villar and Shahram Izadi. 2009. A reconfigurable ferromagnetic input device. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST '09). ACM, NY, NY, USA, 51--54. http://dx.doi.org/10.1145/1622176. 1622186.
[19]
Oren Horev. 2006. Talking to the hand - the interactive potential of shape-change behavior in objects and tangible interfaces. In Proceedings of the 8th International Conference on Design and Semantics of Form and Movement (DeSForM '06), 68--83. http: //www.northumbria.ac.uk/media/6278714/desform2006-proceedings-v2.pdf.
[20]
Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '97). ACM, NY, NY, USA, 234--241. http://dx.doi.org/ 10.1145/258549.258715.
[21]
Yvonne Jansen, Pierre Dragicevic and Jean-Daniel Fekete. 2012. Tangible remote controllers for wall-size displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, NY, NY, USA, 2865--2874. http://dx.doi.org/10. 1145/2207676.2208691.
[22]
Yvonne Jansen, Thorsten Karrer and Jan Borchers. 2010. Mudpad: tactile feedback and haptic texture overlay for touch surfaces. In ACM International Conference on Interactive Tabletops and Surfaces (ITS '10). ACM, NY, NY, USA, 11--14. http: //dx.doi.org/10.1145/1936652.1936655.
[23]
Thorsten Karrer, Moritz Wittenhagen, Leonhard Lichtschlag, Florian Heller and Jan Borchers. 2011. Pinstripe: eyes-free continuous input on interactive clothing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, NY, NY, USA, 1313--1322. http://dx.doi.org/10. 1145/1978942.1979137.
[24]
Mohammadreza Khalilbeigi, Roman Lissermann, Wolfgang Kleine and Jürgen Steimle. 2012. Foldme: interacting with double-sided foldable displays. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI '12). ACM, NY, NY, USA, 33--40. http: //dx.doi.org/10.1145/2148131.2148142.
[25]
Konstantin Klamka and Raimund Dachselt. 2015. Elasticcon: elastic controllers for casual interaction. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '15). ACM, NY, NY, USA, 410--419. http://dx.doi.org/10.1145/2785830.2785849.
[26]
Georg Michelitsch, Jason Williams, Martin Osen, B. Jimenez and Stefan Rapp. 2004. Haptic chameleon: a new concept of shape-changing user interface controls with force feedback. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (CHI EA '04). ACM, NY, NY, USA, 1305--1308. http: //dx.doi.org/10.1145/985921.986050.
[27]
Adiyan Mujibiya, Xiang Cao, Desney S. Tan, Dan Morris, Shwetak N. Patel and Jun Rekimoto. 2013. The sound of touch: on-body touch and gesture sensing based on transdermal ultrasound propagation. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (ITS '13). ACM, NY, NY, USA, 189--198. http://dx.doi.org/10.1145/2512349.2512821.
[28]
Yusuke Nakagawa, Akiya Kamimura and Yoichiro Kawaguchi. 2012. Mimictile: a variable stiffness deformable user interface for mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, NY, NY, USA, 745--748. http://dx.doi.org/10.1145/ 2207676.2207782.
[29]
Esben W. Pedersen, Sriram Subramanian and Kasper Hornbæk. 2014. Is my phone alive?: a large-scale study of shape change in handheld devices using videos. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2579--2588. http://dx.doi.org/10.1145/ 2556288.2557018.
[30]
Simon T. Perrault, Eric Lecolinet, James Eagan and Yves Guiard. 2013. Watchit: simple gestures and eyes-free interaction for wristwatches and bracelets. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 1451--1460. http://dx.doi.org/10.1145/ 2470654.2466192.
[31]
Ivan Poupyrev, Tatsushi Nashida, Shigeaki Maruyama, Jun Rekimoto and Yasufumi Yamaji. 2004. Lumen: interactive visual and shape display for calm computing. In ACM SIGGRAPH 2004 Emerging Technologies (SIGGRAPH '04). ACM, NY, NY, USA, 17. http://dx.doi.org/10.1145/1186155.1186173.
[32]
Ivan Poupyrev, Tatsushi Nashida and Makoto Okabe. 2007. Actuation and tangible user interfaces: the vaucanson duck, robots, and shape displays. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07). ACM, NY, NY, USA, 205--212. http://dx.doi.org/10. 1145/1226969.1227012.
[33]
Ilya Rosenberg and Ken Perlin. 2009. The unmousepad: an interpolating multi-touch force-sensing input pad. In ACM SIGGRAPH 2009 Papers (SIGGRAPH '09). ACM, NY, NY, USA, 65:1--65:9. http://dx.doi. org/10.1145/1576246.1531371.
[34]
Anne Roudaut, Abhijit Karnik, Markus Löchtefeld and Sriram Subramanian. 2013. Morphees: toward high "shape resolution" in self-actuated flexible mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 593--602. http://dx.doi.org/10. 1145/2470654.2470738.
[35]
Jeremy Scott, David Dearman, Koji Yatani and Khai N. Truong. 2010. Sensing foot gestures from the pocket. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (UIST '10). ACM, NY, NY, USA, 199--208. http: //dx.doi.org/10.1145/1866029.1866063.
[36]
Marcos Serrano, Barrett M. Ens and Pourang P. Irani. 2014. Exploring the use of hand-to-face input for interacting with head-worn displays. In Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 3181--3190. http://dx.doi.org/10.1145/2556288. 2556984.
[37]
Orit Shaer and Eva Hornecker. 2010. Tangible user interfaces: past, present, and future directions. Found. Trends Hum.-Comput. Interact. 3, 1--2, 1--137. http: //dx.doi.org/10.1561/1100000026.
[38]
Han Sloetjes and Peter Wittenburg. 2008. Annotation by category: ELAN and ISO DCR. In Proceedings of the Sixth International Conference on Language Resources and Evaluation (LREC'08). European Language Resources Association (ELRA), Marrakech, Morocco.
[39]
Faisal Taher, John Hardy, Abhijit Karnik, Christian Weichel, Yvonne Jansen, Kasper Hornbæk and Jason Alexander. 2015. Exploring interactions with physically dynamic bar charts. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 3237--3246. http://dx.doi.org/10.1145/2702123.2702604.
[40]
Melanie Tory and Robert Kincaid. 2013. Comparing physical, overlay, and touch screen parameter controls. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (ITS '13). ACM, NY, NY, USA, 91--100. http: //dx.doi.org/10.1145/2512349.2512812.
[41]
Jessica Tsimeris, Colin Dedman, Michael Broughton and Tom Gedeon. 2013. Forceform: a dynamically deformable interactive surface. In Proceedings of the 2013 ACM International Conference on Interactive Tabletops and Surfaces (ITS '13). ACM, NY, NY, USA, 175--178. http://dx.doi.org/10.1145/2512349. 2512807.
[42]
Philip Tuddenham, David Kirk and Shahram Izadi. 2010. Graspables revisited: multi-touch vs. tangible input for tabletop displays in acquisition and manipulation tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, NY, NY, USA, 2223--2232. http://dx.doi. org/10.1145/1753326.1753662.
[43]
Marynel Vázquez, Eric Brockmeyer, Ruta Desai, Chris Harrison and Scott E. Hudson. 2015. 3d printing pneumatic device controls with variable activation force capabilities. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 1295--1304. http://dx.doi.org/10.1145/2702123. 2702569.
[44]
Lining Yao, Ryuma Niiyama, Jifei Ou, Sean Follmer, Clark Della Silva and Hiroshi Ishii. 2013. Pneui: pneumatically actuated soft composite materials for shape changing interfaces. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology (UIST '13). ACM, NY, NY, USA, 13--22. http://dx.doi.org/10.1145/2501988. 2502037.
[45]
Sang Ho Yoon, Ke Huo and Karthik Ramani. 2014. Plex: finger-worn textile sensor for mobile interaction during activities. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (UbiComp '14 Adjunct). ACM, NY, NY, USA, 191--194. http://dx.doi.org/10.1145/2638728.2638746.
[46]
Shumin Zhai, Jing Kong and Xiangshi Ren. 2004. Speed-accuracy tradeoff in Fitts' law tasks: on the equivalency of actual and nominal pointing precision. Int. J. Hum.-Comput. Stud. 61, 6, 823--856. http://dx.doi. org/10.1016/j.ijhcs.2004.09.007.

Cited By

View all
  • (2024)Popping-Up Poster: A Pin-Based Promotional Poster Device for Engaging Customers through Physical Shape TransformationProceedings of the ACM on Human-Computer Interaction10.1145/36981388:ISS(283-300)Online publication date: 24-Oct-2024
  • (2024)SHAPE-IT: Exploring Text-to-Shape-Display for Generative Shape-Changing Behaviors with LLMsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676348(1-29)Online publication date: 13-Oct-2024
  • (2024)ICY Interfaces: Exploration of Ice’s Ephemeral Features for Digital Game User ExperienceProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660725(2107-2124)Online publication date: 1-Jul-2024
  • Show More Cited By

Index Terms

  1. Emergeables: Deformable Displays for Continuous Eyes-Free Mobile Interaction

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
    May 2016
    6108 pages
    ISBN:9781450333627
    DOI:10.1145/2858036
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 May 2016

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    • Honorable Mention

    Author Tags

    1. continuous control
    2. deformable devices
    3. mobiles
    4. shape-changing
    5. tangiblity

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI'16
    Sponsor:
    CHI'16: CHI Conference on Human Factors in Computing Systems
    May 7 - 12, 2016
    California, San Jose, USA

    Acceptance Rates

    CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)63
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 31 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Popping-Up Poster: A Pin-Based Promotional Poster Device for Engaging Customers through Physical Shape TransformationProceedings of the ACM on Human-Computer Interaction10.1145/36981388:ISS(283-300)Online publication date: 24-Oct-2024
    • (2024)SHAPE-IT: Exploring Text-to-Shape-Display for Generative Shape-Changing Behaviors with LLMsProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676348(1-29)Online publication date: 13-Oct-2024
    • (2024)ICY Interfaces: Exploration of Ice’s Ephemeral Features for Digital Game User ExperienceProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660725(2107-2124)Online publication date: 1-Jul-2024
    • (2024)DeformIO: Dynamic Stiffness Control on a Deformable Force-sensing DisplayExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650772(1-8)Online publication date: 11-May-2024
    • (2024)Squishy, Yet Satisfying: Exploring Deformable Shapes' Cross-Modal Correspondences with Colours and EmotionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641952(1-20)Online publication date: 11-May-2024
    • (2023)Exploring Long-Term Mediated Relations with a Shape-Changing Thing: A Field Study of coMorphing StoolProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581140(1-14)Online publication date: 19-Apr-2023
    • (2023)Feel the Force, See the Force: Exploring Visual-tactile Associations of Deformable Surfaces with Colours and ShapesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580830(1-13)Online publication date: 19-Apr-2023
    • (2022)Investigating Pointing Performance for Tangible Surfaces with Physical 3D TargetsProceedings of the ACM on Human-Computer Interaction10.1145/35677366:ISS(617-639)Online publication date: 14-Nov-2022
    • (2022)ReCompFig: Designing Dynamically Reconfigurable Kinematic Devices Using Compliant Mechanisms and Tensioning CablesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502065(1-14)Online publication date: 29-Apr-2022
    • (2022)(Dis)Appearables: A Concept and Method for Actuated Tangible UIs to Appear and Disappear based on StagesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501906(1-13)Online publication date: 29-Apr-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media