research-article

Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction

Authors:

Paul Strohmeier,

Sebastian Boring,

Kasper HornbækAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 408, Pages 1 - 12

https://doi.org/10.1145/3290605.3300638

Published: 02 May 2019 Publication History

Abstract

Around-device interaction methods expand the available interaction space for mobile devices; however, there is currently no way to simultaneously track a user's input and provide haptic feedback at the tracked point away from the device. We present Magnetips, a simple, mobile solution for around-device tracking and mid-air haptic feedback. Magnetips combines magnetic tracking and electromagnetic feedback that works regardless of visual occlusion, through most common materials, and at a size that allows for integration with mobile devices. We demonstrate: (1) high-frequency around-device tracking and haptic feedback; (2) the accuracy and range of our tracking solution which corrects for the effects of geomagnetism, necessary for enabling mobile use; and (3) guidelines for maximising strength of haptic feedback, given a desired tracking frequency. We present technical and usability evaluations of our prototype, and demonstrate four example applications of its use.

Supplementary Material

ZIP File (paper408.zip)

Complete field strength diagrams and confusion matrix of post-hoc analysis of haptic feedback strength ratings at distance.

Download
323.18 KB

MP4 File (paper408.mp4)

Supplemental video

Download
254.19 MB

MP4 File (paper408p.mp4)

Preview video

Download
1.98 MB

References

[1]

Daniel Ashbrook, Patrick Baudisch, and Sean White. 2011. Nenya: subtle and eyes-free mobile input with a magnetically-tracked finger ring. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2043--2046.

Digital Library

[2]

Matthias Baldauf, Sebastian Zambanini, Peter Fröhlich, and Peter Reichl. 2011. Markerless Visual Fingertip Detection for Natural Mobile Device Interaction. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '11). ACM, New York, NY, USA, 539--544.

Digital Library

[3]

Patrick Baudisch and Gerry Chu. 2009. Back-of-device Interaction Allows Creating Very Small Touch Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 1923--1932.

Digital Library

[4]

Sliman J Bensmaia and Mark Hollins. 2003. The vibrations of texture. Somatosensory and Motor Research 20, 1 (2003), 33--43.

[5]

Joanna Bergstrom-Lehtovirta, Sebastian Boring, and Kasper Hornbæk. 2017. Placing and Recalling Virtual Items on the Skin. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press, New York, New York, USA, 1497--1507.

Digital Library

[6]

Sebastian Boring, David Ledo, Xiang 'Anthony' Chen, Nicolai Marquardt, Anthony Tang, and Saul Greenberg. 2012. The Fat Thumb: Using the Thumb's Contact Size for Single-handed Mobile Interaction. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12). ACM, New York, NY, USA, 39--48.

Digital Library

[7]

Alex Butler, Shahram Izadi, and Steve Hodges. 2008. SideSight: Multi-touch- Interaction Around Small Devices. In Proceedings of the 21st annual ACM symposium on User interface software and technology - UIST '08. ACM Press, New York, New York, USA, 201.

Digital Library

[8]

Tom Carter, Sue Ann Seah, Benjamin Long, Bruce Drinkwater, and Sriram Subramanian. 2013. UltraHaptics: multi-point mid-air haptic feedback for touch surfaces. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 505--514.

Digital Library

[9]

Ke-Yu Chen, Kent Lyons, Sean White, and Shwetak Patel. 2013. uTrack: 3D input using two magnetic sensors. In Proceedings of the 26th annual ACM symposium on User interface software and technology. ACM, 237-- 244.

Digital Library

[10]

Ke-Yu Chen, Shwetak N Patel, and Sean Keller. 2016. Finexus: Tracking precise motions of multiple fingertips using magnetic sensing. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 1504--1514.

Digital Library

[11]

Euan Freeman, Stephen Brewster, and Vuokko Lantz. 2014. Tactile Feedback for Above-Device Gesture Interfaces: Adding Touch to Touchless Interactions. In Proceedings of the 16th International Conference on Multimodal Interaction (ICMI '14). ACM, New York, NY, USA, 419--426.

Digital Library

[12]

George A Gescheider. 1988. Psychophysical Schaling. Ann. Rev. Psychol 39 (1988), 169--200.

[13]

Xinying Han, Hiroaki Seki, and M Hikizu. 2007. Wearable handwriting input device using magnetic field. In SICE, 2007 Annual Conference. IEEE, 365--368.

[14]

Chris Harrison and Scott E. Hudson. 2009. Abracadabra. Proceedings of the 22nd annual ACM symposium on User interface software and technology - UIST '09 (2009), 121.

Digital Library

[15]

Ken Hinckley, Seongkook Heo, Michel Pahud, Christian Holz, Hrvoje Benko, Abigail Sellen, Richard Banks, Kenton O Hara, Gavin Smyth, and Bill Buxton. 2016. Pre-Touch Sensing for Mobile Interaction. Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (2016), 2869--2881. arXiv:1605.09782

Digital Library

[16]

Roland S Johansson and J Randall Flanagan. 2009. Coding and use of tactile signals from the fingertips in object manipulation tasks. Nature reviews. Neuroscience 10, 5 (2009), 345--59.

[17]

Azusa Kadomura and Itiro Siio. 2015. MagNail: user interaction with smart device through magnet attached to fingernail. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. ACM, 309--312.

Digital Library

[18]

Hamed Ketabdar, Mehran Roshandel, and Kamer Ali Yüksel. 2010. Towards using embedded magnetic field sensor for around mobile device 3D interaction. In Proceedings of the 12th international conference on Human computer interaction with mobile devices and services. ACM, 153--156.

Digital Library

[19]

Jarrod Knibbe, Diego Martinez Plasencia, Christopher Bainbridge, Chee-Kin Chan, Jiawei Wu, Thomas Cable, Hassan Munir, and David Coyle. 2014. Extending interaction for smart watches. In Proceedings of the extended abstracts of the 32nd annual ACM conference on Human factors in computing systems - CHI EA '14. ACM Press, New York, New York, USA, 1891--1896.

Digital Library

[20]

Mathieu Le Goc, Stuart Taylor, Shahram Izadi, and Cem Keskin. 2014. A low-cost transparent electric field sensor for 3d interaction on mobile devices. In Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, 3167--3170.

Digital Library

[21]

Sebastian OH Madgwick, Andrew JL Harrison, and Ravi Vaidyanathan. 2011. Estimation of IMU and MARG orientation using a gradient descent algorithm. In Rehabilitation Robotics (ICORR), 2011 IEEE International Conference on. IEEE, 1--7.

[22]

Ian Oakley and Doyoung Lee. 2014. Interaction on the Edge: Offset Sensing for Small Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 169--178.

Digital Library

[23]

Masa Ogata and Michita Imai. 2015. SkinWatch: Skin Gesture Interaction for Smart Watch. Proceedings of the 6th Augmented Human International Conference on - AH '15 (2015), 21--24.

Digital Library

[24]

Simon T Perrault, Eric Lecolinet, James Eagan, and Yves Guiard. 2013. WatchIt : Simple Gestures and Eyes-free Interaction for Wristwatches and Bracelets. In Proc. CHI. 1451--1460.

Digital Library

[25]

Robert F. Schmidt. 1986. Fundamentals of Sensory Physiology. Vol. 3. Berlin Heidelberg New York Tokyo. 25--33 pages. arXiv:arXiv:1011.1669v3

[26]

Stefan Schneegass and Alexandra Voit. 2016. GestureSleeve: using touch sensitive fabrics for gestural input on the forearm for controlling smartwatches. Proceedings of the 2016 ACM International Symposium on Wearable Computers - ISWC '16 (2016), 108--115.

Digital Library

[27]

Rajinder Sodhi, Ivan Poupyrev, Matthew Glisson, and Ali Israr. 2013. AIREAL: interactive tactile experiences in free air. ACM Transactions on Graphics (TOG) 32, 4 (2013), 134.

Digital Library

[28]

Daniel Spelmezan, Deepak Ranjan Sahoo, and Sriram Subramanian. 2017. Sparkle: hover feedback with touchable electric arcs. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 3705--3717.

Digital Library

[29]

Srinath Sridhar, Anders Markussen, Antti Oulasvirta, Christian Theobalt, and Sebastian Boring. 2017. WatchSense: On- and AboveSkin Input Sensing through aWearable Depth Sensor Srinath. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press, New York, New York, USA, 3891--3902.

Digital Library

[30]

Paul Strohmeier. 2013. Magnetic Implant. http://fkeel.blogspot.de/ 2013/01/magnetic-implant-sensing.html

[31]

Paul Strohmeier, Jesse Burstyn, and Roel Vertegaal. 2015. Effects of Display Sizes on a Scrolling Task using a Cylindrical Smartwatch. Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct - MobileHCI '15 (2015), 846--853.

Digital Library

[32]

Paul Strohmeier and Kasper Hornbæk. 2017. Generating Haptic Textures with a Vibrotactile Actuator. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems - CHI '17. ACM Press, New York, New York, USA, 4994--5005.

Digital Library

[33]

Yuriko Suzuki and Minoru Kobayashi. 2005. Air jet driven force feedback in virtual reality. IEEE computer graphics and applications 25, 1 (2005), 44--47.

Digital Library

[34]

Radu-Daniel Vatavu, Annette Mossel, and Christian Schönauer. 2016. Digital Vibrons: Understanding Users' Perceptions of Interacting with Invisible, Zero-weight Matter. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16). ACM, New York, NY, USA, 217--226.

Digital Library

[35]

Malte Weiss, Chat Wacharamanotham, Simon Voelker, and Jan Borchers. 2011. FingerFlux: near-surface haptic feedback on tabletops. In Proceedings of the 24th annual ACM symposium on User interface software and technology. ACM, 615--620.

Digital Library

[36]

Graham Wilson, Thomas Carter, Sriram Subramanian, and Stephen A. Brewster. 2014. Perception of Ultrasonic Haptic Feedback on the Hand: Localisation and Apparent Motion. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 1133--1142.

Digital Library

[37]

Junhan Zhou, Yang Zhang, Gierad Laput, and Chris Harrison. 2016. AuraSense: enabling expressive around-smartwatch interactions with electric field sensing. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology. ACM, 81--86.

Digital Library

Cited By

Stanke DSimon BLöwen SRohs M(2024)CaseTouch: Occlusion-Free Touch Input by adding a Thin Sensor Stripe to the Smartwatch CaseProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701583(172-183)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701583
Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2024)Above-Screen Fingertip Tracking and Hand Representation for Precise Touch Input with a Phone in Virtual RealityProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670961(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670961
Nishida JTajima DNamikawa YKasahara S(2024)Demonstrating Magnus: A Magnetic Hand Exoskeleton for Fast and Dexterous Finger ActuationACM SIGGRAPH 2024 Emerging Technologies10.1145/3641517.3664390(1-2)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641517.3664390
Show More Cited By

Index Terms

Magnetips: Combining Fingertip Tracking and Haptic Feedback for Around-Device Interaction
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices

Recommendations

Reflective haptics: enhancing stylus-based interactions on touch screens
EuroHaptics'10: Proceedings of the 2010 international conference on Haptics: generating and perceiving tangible sensations, Part I

In this paper, we introduce the prototype of a low cost haptically augmented stylus for pen computing on touch screens. The stylus supports human-computer interaction through a dynamic haptic feedback. This reflective feedback is generated by a ...
Reflective Haptics: Enhancing Stylus-Based Interactions on Touch Screens
Proceedings, Part I, of the International Conference on Haptics: Generating and Perceiving Tangible Sensations - Volume 6191

In this paper, we introduce the prototype of a low cost haptically augmented stylus for pen computing on touch screens. The stylus supports human-computer interaction through a dynamic haptic feedback. This reflective feedback is generated by a ...
Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices
UIST '09: Proceedings of the 22nd annual ACM symposium on User interface software and technology

We present Abracadabra, a magnetically driven input technique that offers users wireless, unpowered, high fidelity finger input for mobile devices with very small screens. By extending the input area to many times the size of the device's screen, our ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

45
Total Citations
View Citations
1,291
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)9

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Stanke DSimon BLöwen SRohs M(2024)CaseTouch: Occlusion-Free Touch Input by adding a Thin Sensor Stripe to the Smartwatch CaseProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701583(172-183)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701583
Matulic FKashima TBeker DSuzuo DFujiwara HVogel D(2024)Above-Screen Fingertip Tracking and Hand Representation for Precise Touch Input with a Phone in Virtual RealityProceedings of the 50th Graphics Interface Conference10.1145/3670947.3670961(1-15)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3670947.3670961
Nishida JTajima DNamikawa YKasahara S(2024)Demonstrating Magnus: A Magnetic Hand Exoskeleton for Fast and Dexterous Finger ActuationACM SIGGRAPH 2024 Emerging Technologies10.1145/3641517.3664390(1-2)Online publication date: 13-Jul-2024
https://dl.acm.org/doi/10.1145/3641517.3664390
Vega GMartinez-Missir VWittchen DSabnis NGirouard ACochrane KStrohmeier P(2024)vARitouch: Back of the Finger Device for Adding Variable Compliance to Rigid ObjectsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642828(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642828
Deshpande HHan BMoon KBianchi AZheng CKim J(2024)Reconfigurable Interfaces by Shape Change and Embedded MagnetsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642802(1-12)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642802
Ma YXie TZhang PKim HJe S(2024)AirPush: A Pneumatic Wearable Haptic Device Providing Multi-Dimensional Force Feedback on a FingertipProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642536(1-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642536
Wu MLangerak THilliges OZarate J(2024)Using Synthetic Data in Supervised Learning for Robust 5-DoF Magnetic Marker LocalizationIEEE Transactions on Magnetics10.1109/TMAG.2024.341672960:8(1-11)Online publication date: Aug-2024
https://doi.org/10.1109/TMAG.2024.3416729
Yamamoto SIkematsu KKato KSugiura Y(2024)Pinch Force Measurement Using a Geomagnetic Sensor2024 IEEE/SICE International Symposium on System Integration (SII)10.1109/SII58957.2024.10417164(284-287)Online publication date: 8-Jan-2024
https://doi.org/10.1109/SII58957.2024.10417164
Yu JPerera PPerera RValashani MWithana A(2024)Fabricating Customizable 3-D Printed Pressure Sensors by Tuning Infill CharacteristicsIEEE Sensors Journal10.1109/JSEN.2024.335833024:6(7604-7613)Online publication date: 15-Mar-2024
https://doi.org/10.1109/JSEN.2024.3358330
Heo JChoi HLee YKim HJi HPark HLee YJung CNguyen HLee D(2024)Hand Tracking: SurveyInternational Journal of Control, Automation and Systems10.1007/s12555-024-0298-122:6(1761-1778)Online publication date: 28-May-2024
https://doi.org/10.1007/s12555-024-0298-1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten