research-article

MuscleIO: Muscle-Based Input and Output for Casual Notifications

Authors:

Justin Schulte,

Michael RohsAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 2, Issue 2

Article No.: 64, Pages 1 - 21

https://doi.org/10.1145/3214267

Published: 05 July 2018 Publication History

Abstract

Receiving and reacting to notifications on mobile devices can be cumbersome. We propose MuscleIO, the use of electrical muscle stimulation (EMS) for notification output and electromyography (EMG) for reacting to notifications. Our approach provides a one-handed, eyes-free, and low-effort way of dealing with notifications. We built a prototype that interleaves muscle input and muscle output signals using the same electrodes. EMS and EMG alternate such that the EMG input signal is measured in the gaps of the EMS output signal, so voluntary muscle contraction is measured during muscle stimulation.

Notifications are represented as EMS signals and are accepted or refused either by a directional or a time-based EMG response. A lab user study with 12 participants shows that the directional EMG response is superior to the time-based response in terms of reaction time, error rate, and user preference. Furthermore, the directional approach is the fastest and the most intuitive for users compared to a button-based smartwatch interface as a baseline.

Supplementary Material

duente (duente.zip)

Supplemental movie, appendix, image and software files for, MuscleIO: Muscle-Based Input and Output for Casual Notifications

Download
101.61 KB

References

[1]

Christoph Amma, Thomas Krings, Jonas Böer, and Tanja Schultz. 2015. Advancing Muscle-Computer Interfaces with High-Density Electromyography. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 929--938.

Digital Library

[2]

Anupama. Batra, Sangeeta. Vyas, and Dheeraj. Jeph. 2014. Color difference on simple visual reaction time in young volunteers. International Journal of Clinical and Experimental Physiology 1, 4 (2014), 311--313.

[3]

Edward Cutrell, Mary Czerwinski, and Eric Horvitz. 2001. Notification, Disruption, and Memory: Effects of Messaging Interruptions on Memory and Performance. In Proceedings of Interact 2001 (Interact '01). IFIP, 263--269.

[4]

Linda Di Geronimo, Marica Bertarini, Julia Badertscher, Maria Husmann, and Moira C. Norrie. 2017. MyoShare: Sharing Data Among Devices via Mid-air Gestures. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 48, 3 pages.

Digital Library

[5]

Tim Duente, Max Pfeiffer, and Michael Rohs. 2016. On-skin Technologies for Muscle Sensing and Actuation. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct (UbiComp '16). ACM, New York, NY, USA, 933--936.

Digital Library

[6]

Dylan Ebert and Fillia Makedon. 2017. MyoLearn: Using a Multimodal Armband Sensor for Vocational Safety Problem Identification. In Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments (PETRA '17). ACM, New York, NY, USA, 248--249.

Digital Library

[7]

Andrew J Elliot and Henk Aarts. 2011. Perception of the color red enhances the force and velocity of motor output. Emotion 11, 2 (2011), 445.

[8]

Frederik Funke and Ulf-Dietrich Reips. 2012. Why Semantic Differentials in Web-Based Research Should Be Made from Visual Analogue Scales and Not from 5-Point Scales. Field Methods 24, 3 (2012), 310--327. arXiv:https://doi.org/10.1177/1525822X12444061

[9]

Mahmoud Hassan, Florian Daiber, Frederik Wiehr, Felix Kosmalla, and Antonio Krüger. 2017. FootStriker: An EMS-based Foot Strike Assistant for Running. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 1, 1 (2017), 2:1--2:18.

Digital Library

[10]

Mariam Hassib, Max Pfeiffer, Stefan Schneegass, Michael Rohs, and Florian Alt. 2017. Emotion Actuator: Embodied Emotional Feedback Through Electroencephalography and Electrical Muscle Stimulation. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6133--6146.

Digital Library

[11]

Shamsi T. Iqbal and Eric Horvitz. 2010. Notifications and Awareness: A Field Study of Alert Usage and Preferences. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work (CSCW '10). ACM, New York, NY, USA, 27--30.

Digital Library

[12]

Mark I. Johnson. 2014. Transcutaneous Electrical Nerve Stimulation (TENS): Research to support clinical practice. Oxford University Press, Oxford, UK. http://oxfordmedicine.com/view/10.1093/med/9780199673278.001.0001/med-9780199673278

[13]

Hiroyuki Kajimoto, Naoki Kawakami, Taro Maeda, and Susumu Tachi. 1999. Tactile feeling display using functional electrical stimulation. In Proceedings of Ninth International Conference on Artificial reality and Telexistence (ICAT'99). Citeseer, 133.

[14]

Frederic Kerber, Pascal Lessel, and Antonio Krüger. 2015. Same-side Hand Interactions with Arm-placed Devices Using EMG. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, New York, NY, USA, 1367--1372.

Digital Library

[15]

Frederic Kerber, Michael Puhl, and Antonio Krüger. 2017. User-independent Real-time Hand Gesture Recognition Based on Surface Electromyography. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 36, 7 pages.

Digital Library

[16]

Jarrod Knibbe, Paul Strohmeier, Sebastian Boring, and Kasper Hornbæk. 2017. Automatic Calibration of High Density Electric Muscle Stimulation. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 3, Article 68 (Sept. 2017), 17 pages.

Digital Library

[17]

Pedro Lopes and Patrick Baudisch. 2013. Muscle-propelled Force Feedback: Bringing Force Feedback to Mobile Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2577--2580.

Digital Library

[18]

Pedro Lopes, Alexandra Ion, Willi Mueller, Daniel Hoffmann, Patrik Jonell, and Patrick Baudisch. 2015. Proprioceptive Interaction. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 939--948.

Digital Library

[19]

Pedro Lopes, Patrik Jonell, and Patrick Baudisch. 2015. Affordance++: Allowing Objects to Communicate Dynamic Use. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2515--2524.

Digital Library

[20]

Jess McIntosh, Charlie McNeill, Mike Fraser, Frederic Kerber, Markus Löchtefeld, and Antonio Krüger. 2016. EMPress: Practical Hand Gesture Classification with Wrist-Mounted EMG and Pressure Sensing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 2332--2342.

Digital Library

[21]

Abhinav Mehrotra, Robert Hendley, and Mirco Musolesi. 2016. PrefMiner: Mining User's Preferences for Intelligent Mobile Notification Management. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16). ACM, New York, NY, USA, 1223--1234.

Digital Library

[22]

Jun Nishida and Kenji Suzuki. 2017. bioSync: A Paired Wearable Device for Blending Kinesthetic Experience. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3316--3327.

Digital Library

[23]

Toni Pakkanen, Jani Lylykangas, Jukka Raisamo, Roope Raisamo, Katri Salminen, Jussi Rantala, and Veikko Surakka. 2008. Perception of Low-amplitude Haptic Stimuli when Biking. In Proceedings of the 10th International Conference on Multimodal Interfaces (ICMI '08). ACM, New York, NY, USA, 281--284.

Digital Library

[24]

Jerome Pasquero, Scott J. Stobbe, and Noel Stonehouse. 2011. A Haptic Wristwatch for Eyes-free Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 3257--3266.

Digital Library

[25]

Max Pfeiffer, Tim Duente, and Michael Rohs. 2016. Let Your Body Move: A Prototyping Toolkit for Wearable Force Feedback with Electrical Muscle Stimulation. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 418--427.

Digital Library

[26]

Max Pfeiffer, Tim Dünte, Stefan Schneegass, Florian Alt, and Michael Rohs. 2015. Cruise Control for Pedestrians: Controlling Walking Direction Using Electrical Muscle Stimulation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2505--2514.

Digital Library

[27]

Max Pfeiffer and Michael Rohs. 2017. Haptic Feedback for Wearables and Textiles Based on Electrical Muscle Stimulation. In Smart Textiles: Fundamentals, Design, and Interaction, Stefan Schneegass and Oliver Amft (Eds.). Springer International Publishing, Cham, 103--137.

[28]

Max Pfeiffer, Stefan Schneegass, Florian Alt, and Michael Rohs. 2014. Let Me Grab This: A Comparison of EMS and Vibration for Haptic Feedback in Free-Hand Interaction. In Augmented Human. ACM Press, New York, New York, USA, 1--8.

Digital Library

[29]

Martin Pielot, Karen Church, and Rodrigo de Oliveira. 2014. An In-situ Study of Mobile Phone Notifications. In Proceedings of the 16th International Conference on Human-computer Interaction with Mobile Devices & Services (MobileHCI '14). ACM, New York, NY, USA, 233--242.

Digital Library

[30]

Martin Pielot and Luz Rello. 2017. Productive, Anxious, Lonely: 24 Hours Without Push Notifications. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 11, 11 pages.

Digital Library

[31]

Henning Pohl, Peter Brandes, Hung Ngo Quang, and Michael Rohs. 2017. Squeezeback: Pneumatic Compression for Notifications. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5318--5330.

Digital Library

[32]

Henning Pohl, Justyna Medrek, and Michael Rohs. 2016. ScatterWatch: Subtle Notifications via Indirect Illumination Scattered in the Skin. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 7--16.

Digital Library

[33]

Henning Pohl and Roderick Murray-Smith. 2013. Focused and Casual Interactions: Allowing Users to Vary Their Level of Engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, New York, NY, USA, 2223--2232.

Digital Library

[34]

Thijs Roumen, Simon T. Perrault, and Shengdong Zhao. 2015. NotiRing: A Comparative Study of Notification Channels for Wearable Interactive Rings. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 2497--2500.

Digital Library

[35]

Alireza Sahami Shirazi, Niels Henze, Tilman Dingler, Martin Pielot, Dominik Weber, and Albrecht Schmidt. 2014. Large-scale Assessment of Mobile Notifications. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3055--3064.

Digital Library

[36]

T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, and James A. Landay. 2010. Making Muscle-computer Interfaces More Practical. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 851--854.

Digital Library

[37]

T. Schauer, R. C. Salbert, N.-O. Negård, and J. Raisch. 2004. Detection and Filtering of EMG for Assessing Voluntary Muscle Activity during FES. In Getting FES into clinical practice: Proceedings of IFESS - FESnet 2004 (2004), D. Wood and P. Taylor (Eds.). Bournemouth, United Kingdom, 185--187.

[38]

Stefan Schneegass and Rufat Rzayev. 2016. Embodied Notifications: Implicit Notifications Through Electrical Muscle Stimulation. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI '16). ACM, New York, NY, USA, 954--959.

Digital Library

[39]

Stefan Schneegass, Albrecht Schmidt, and Max Pfeiffer. 2016. Creating User Interfaces with Electrical Muscle Stimulation. interactions 24, 1 (2016), 74--77.

Digital Library

[40]

Raafat El-Sayed Shalaby. 2011. Ph.D. Development of an electromyography detection system for the control of functional electrical stimulation in neurological rehabilitation. Dissertation. Technische Universität Berlin. http://www.control.tu-berlin.de/images/d/d7/Shalaby2011b.pdf

[41]

Didier Staudenmann, Idsart Kingma, Andreas Daffertshofer, Dick F Stegeman, and Jaap H van Dieën. 2006. Improving EMG-based muscle force estimation by using a high-density EMG grid and principal component analysis. IEEE Transactions on Biomedical Engineering 53, 4 (2006), 712--719.

[42]

Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2010. PossessedHand: A Hand Gesture Manipulation System Using Electrical Stimuli. In Proceedings of the 1st Augmented Human International Conference (AH '10). ACM, New York, NY, USA, Article 2, 5 pages.

Digital Library

[43]

Markus Valtin, Thomas Schauer, Carsten Behling, Michael Daniel, and Matthias Weber. 2012. Combined Stimulation and Measurement System for Array Electrodes. In Proceedings of the International Conference on Biomedical Electronics and Devices - Volume 1: BIODEVICES, (BIOSTEC 2012). INSTICC, SciTePress, 345--349.

[44]

Dominik Weber. 2017. Towards Smart Notification Management in Multi-device Environments. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 68, 2 pages.

Digital Library

[45]

Katrin Wolf and Timm Bäder. 2015. Illusion of Surface Changes Induced by Tactile and Visual Touch Feedback. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA '15. ACM Press, New York, New York, USA, 1355--1360.

Digital Library

[46]

Fengpeng Yuan, Xianyi Gao, and Janne Lindqvist. 2017. How Busy Are You?: Predicting the Interruptibility Intensity of Mobile Users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5346--5360.

Digital Library

[47]

Xu Zhang, Xiang Chen, Wen-hui Wang, Ji-hai Yang, Vuokko Lantz, and Kong-qiao Wang. 2009. Hand Gesture Recognition and Virtual Game Control Based on 3D Accelerometer and EMG Sensors. In Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI '09). ACM, New York, NY, USA, 401--406.

Digital Library

Cited By

Stanke DSimon BLöwen SRohs M(2024)CaseTouch: Occlusion-Free Touch Input by adding a Thin Sensor Stripe to the Smartwatch CaseProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701583(172-183)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701583
Mazursky ABrooks JDesta BLopes P(2024)ThermalGrasp: Enabling Thermal Feedback even while Grasping and Walking2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00056(342-353)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00056
Ishimaru TSaga S(2023)Evaluation of Electric Muscle Stimulation Method for Haptic Augmented RealitySensors10.3390/s2304179623:4(1796)Online publication date: 5-Feb-2023
https://doi.org/10.3390/s23041796
Show More Cited By

Index Terms

MuscleIO: Muscle-Based Input and Output for Casual Notifications
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools
    2. Ubiquitous and mobile devices
      1. Mobile devices

Recommendations

Reducing Muscle Activity when Playing Tremolo by Using Electrical Muscle Stimulation to Learn Efficient Motor Skills

When beginners play the piano, the activity of the forearm muscles tends to be greater than that of experts because beginners move their fingers with more force than necessary. Reducing forearm muscle activity is important for pianists to prevent fatigue ...
Detection and Alleviation of Driving Fatigue Based on EMG and EMS/EEG Using Wearable Sensor
MOBIHEALTH'15: Proceedings of the 5th EAI International Conference on Wireless Mobile Communication and Healthcare

We present a new technique to detect and relieve drivers’ fatigue by using electromyography (EMG) and electrical muscle stimulation (EMS) respectively, while drivers are driving without any interruption. Driving fatigue was detected by EMG from the ...
On-skin technologies for muscle sensing and actuation
UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct

Electromyography (EMG) and electrical muscle stimulation (EMS) are promising technologies for muscle sensing and actuation in wearable interfaces. The required electrodes can be manufactured to form a thin layer on the skin. We discuss requirements and ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 2, Issue 2

June 2018

741 pages

EISSN:2474-9567

DOI:10.1145/3236498

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2018

Accepted: 01 April 2018

Revised: 01 April 2018

Received: 01 February 2018

Published in IMWUT Volume 2, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

23
Total Citations
View Citations
616
Total Downloads

Downloads (Last 12 months)91
Downloads (Last 6 weeks)3

Reflects downloads up to 03 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stanke DSimon BLöwen SRohs M(2024)CaseTouch: Occlusion-Free Touch Input by adding a Thin Sensor Stripe to the Smartwatch CaseProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701583(172-183)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701583
Mazursky ABrooks JDesta BLopes P(2024)ThermalGrasp: Enabling Thermal Feedback even while Grasping and Walking2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00056(342-353)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VR58804.2024.00056
Ishimaru TSaga S(2023)Evaluation of Electric Muscle Stimulation Method for Haptic Augmented RealitySensors10.3390/s2304179623:4(1796)Online publication date: 5-Feb-2023
https://doi.org/10.3390/s23041796
Okamoto MMurao K(2023)PPG2EMG: Estimating Upper-Arm Muscle Activities and EMG from Wrist PPG ValuesSensors10.3390/s2304178223:4(1782)Online publication date: 5-Feb-2023
https://doi.org/10.3390/s23041782
Seetohul VSchweitzer KBöhmer M(2023)Towards Improving Spatial Orientation using Electrical Muscle Stimulation as Tactile and Force FeedbackProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3631792(512-514)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3631792
Duente TSchulte JLucius MRohs M(2023)Colorful Electrotactile Feedback on the WristProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627800(172-184)Online publication date: 3-Dec-2023
https://dl.acm.org/doi/10.1145/3626705.3627800
Yamagami MPortnova-Fahreeva AKong JWobbrock JMankoff J(2023)How Do People with Limited Movement Personalize Upper-Body Gestures? Considerations for the Design of Personalized and Accessible Gesture InterfacesProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608430(1-15)Online publication date: 22-Oct-2023
https://dl.acm.org/doi/10.1145/3597638.3608430
Niijima AKubo Y(2023)Assisting with Fingertip Force Control by Active Bio-Acoustic Sensing and Electrical Muscle StimulationProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581192(1-13)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581192
Catană AVatavu R(2023)Fingerhints: Understanding Users’ Perceptions of and Preferences for On-Finger Kinesthetic NotificationsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581022(1-17)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581022
Eddy EScheme EBateman S(2023)A Framework and Call to Action for the Future Development of EMG-Based Input in HCIProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580962(1-23)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580962
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents