research-article

TwistIn: Tangible Authentication of Smart Devices via Motion Co-analysis with a Smartwatch

Authors:

Ho-Man Colman Leung,

Pheng-Ann HengAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 2, Issue 2

Article No.: 72, Pages 1 - 24

https://doi.org/10.1145/3214275

Published: 05 July 2018 Publication History

Abstract

Smart devices contain sensitive information that has to be guarded against unauthorized access through authentication. Existing authentication methods become obsolete as they are designed either for logging-in one device at a time or are ineffective in a multi-user multi-device environment. This paper presents TwistIn, a simple gesture that takes a smartwatch as an authentication token for fast access and control of other smart devices. Our mechanism is particularly useful for devices such as smartphones, smart glasses, and small IoT objects. To log in a device, one simply need to pick it up and twist it a few times. Then, by co-analyzing the motion data from the device and the watch, our method can extend the user authentication on the watch to the device. This is a simple and tangible interaction that takes only one to two seconds to perform. Furthermore, to account for user variation in wrist bending, we decompose wrist and forearm rotations via an optimization to improve the method accuracy. We implemented TwistIn, collected thousands of gesture samples, and conducted various experiments to evaluate our prototype system and show that it achieved over 95% detection accuracy.

Supplementary Material

leung (leung.zip)

Supplemental movie, appendix, image and software files for, TwistIn: Tangible Authentication of Smart Devices via Motion Co-analysis with a Smartwatch

Download
23.78 MB

References

[1]

Ron Amadeo. 2015. Meet Google's "Eddystone" - a flexible, open source iBeacon fighter: The Ars Technica review. https://arstechnica.com/gadgets/2015/07/meet-googles-eddystone-a-flexible-open-source-ibeacon-fighter/ {Online; accessed 17-April-2018}.

[2]

Daniel Amitay. 2011. Most common iPhone passcodes. Retrieved June 15 (2011), 2011.

[3]

Inc. Apple. 2017. About iBeacon on your iPhone, iPad, and iPod touch. https://support.apple.com/en-gb/HT202880 {Online; accessed 17-April-2018}.

[4]

Adam J Aviv, Katherine L Gibson, Evan Mossop, Matt Blaze, and Jonathan M Smith. 2010. Smudge Attacks on Smartphone Touch Screens. Woot 10 (2010), 1--7.

Digital Library

[5]

Noam Ben-Asher, Niklas Kirschnick, Hanul Sieger, Joachim Meyer, Asaf Ben-Oved, and Sebastian Möller. 2011. On the need for different security methods on mobile phones. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services. ACM, 465--473.

Digital Library

[6]

Manuele Bicego, Andrea Lagorio, Enrico Grosso, and Massimo Tistarelli. 2006. On the use of SIFT features for face authentication. In Computer Vision and Pattern Recognition Workshop, 2006. CVPRW'06. Conference on. IEEE, 35--35.

Digital Library

[7]

Jonathan Camhi. 2015. BI Intelligence projects 34 billion devices will be connected by 2020. http://www.businessinsider.com/bi-intelligence-34-billion-connected-devices-2020-2015-11 {Online; accessed 27-February-2017}.

[8]

Bing Chang, Ximing Liu, Yingjiu Li, Pingjian Wang, Wen-Tao Zhu, and Zhan Wang. 2017. Employing Smartwatch for Enhanced Password Authentication. In International Conference on Wireless Algorithms, Systems, and Applications. Springer, 691--703.

[9]

Xiang Anthony Chen, Tovi Grossman, Daniel J Wigdor, and George Fitzmaurice. 2014. Duet: exploring joint interactions on a smart phone and a smart watch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 159--168.

Digital Library

[10]

Andrew Cunningham and Lee Hutchinson. 2016. macOS 10.12 Sierra: The Ars Technica review. https://arstechnica.com/gadgets/2016/09/macos-10-12-sierra-the-ars-technica-review/4/#h4 {Online; accessed 16-April-2018}.

[11]

Yuan Du, Haoyi Ren, Gang Pan, and Shjian Li. 2011. Tilt 8 touch: Mobile phone for 3D interaction. In Proceedings of the 13th international conference on Ubiquitous computing. ACM, 485--486.

Digital Library

[12]

Serge Egelman, Sakshi Jain, Rebecca S Portnoff, Kerwell Liao, Sunny Consolvo, and David Wagner. 2014. Are you ready to lock?. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. ACM, 750--761.

Digital Library

[13]

Rainhard Dieter Findling, Muhammad Muaaz, Daniel Hintze, and René Mayrhofer. 2014. Shakeunlock: Securely unlock mobile devices by shaking them together. In Proceedings of the 12th International Conference on Advances in Mobile Computing and Multimedia. ACM, 165--174.

Digital Library

[14]

Mario Frank, Ralf Biedert, Eugene Ma, Ivan Martinovic, and Dawn Song. 2013. Touchalytics: On the applicability of touchscreen input as a behavioral biometric for continuous authentication. IEEE transactions on information forensics and security 8, 1 (2013), 136--148.

Digital Library

[15]

Inc. Gartner. 2017. Gartner Says 8.4 Billion Connected "Things" Will Be in Use in 2017, Up 31 Percent From 2016. http://www.gartner.com/newsroom/id/3598917 {Online; accessed 10-May-2017}.

[16]

Avik Ghose, Chirabrata Bhaumik, and Tapas Chakravarty. 2013. BlueEye: A system for proximity detection using bluetooth on mobile phones. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication. ACM, 1135--1142.

Digital Library

[17]

Carles Gomez, Joaquim Oller, and Josep Paradells. 2012. Overview and evaluation of bluetooth low energy: An emerging low-power wireless technology. Sensors 12, 9 (2012), 11734--11753.

[18]

John J Guiry, Pepijn van de Ven, and John Nelson. 2014. Multi-sensor fusion for enhanced contextual awareness of everyday activities with ubiquitous devices. Sensors 14, 3 (2014), 5687--5701.

[19]

Ken Hinckley. 2003. Synchronous gestures for multiple persons and computers. In Proceedings of the 16th annual ACM symposium on User interface software and technology. ACM, 149--158.

Digital Library

[20]

Ken Hinckley and Hyunyoung Song. 2011. Sensor synaesthesia: touch in motion, and motion in touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 801--810.

Digital Library

[21]

Berthold KP Horn. 1987. Closed-form solution of absolute orientation using unit quaternions. JOSA A 4, 4 (1987), 629--642.

[22]

Chao Hu, Max Q-H Meng, Mrinal Mandal, and Peter Xiaoping Liu. 2006. Robot rotation decomposition using quaternions. In Mechatronics and Automation, Proceedings of the 2006 IEEE International Conference on. IEEE, 1158--1163.

[23]

William Hutama, Peng Song, Chi-Wing Fu, and Wooi Boon Goh. 2011. Distinguishing multiple smart-phone interactions on a multi-touch wall display using tilt correlation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 3315--3318.

Digital Library

[24]

Andrew H Johnston and Gary M Weiss. 2015. Smartwatch-based biometric gait recognition. In Biometrics Theory, Applications and Systems (BTAS), 2015 IEEE 7th International Conference on. IEEE, 1--6.

[25]

Ju-Whan Kim, Han-Jong Kim, and Tek-Jin Nam. 2016. M. Gesture: An Acceleration-Based Gesture Authoring System on Multiple Handheld and Wearable Devices. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2307--2318.

Digital Library

[26]

Florian Klompmaker, Karsten Nebe, and Julien Eschenlohr. 2013. Towards multimodal 3d tabletop interaction using sensor equipped mobile devices. Mobile Computing, Applications, and Services (2013), 100--114.

[27]

Sven Kratz, Michael Rohs, and Georg Essl. 2013. Combining acceleration and gyroscope data for motion gesture recognition using classifiers with dimensionality constraints. In Proceedings of the 2013 international conference on Intelligent user interfaces. ACM, 173--178.

Digital Library

[28]

Jennifer R Kwapisz, Gary M Weiss, and Samuel A Moore. 2011. Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter 12, 2 (2011), 74--82.

Digital Library

[29]

Gierad Laput, Yang Zhang, and Chris Harrison. 2017. Synthetic Sensors: Towards General-Purpose Sensing. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 3986--3999.

Digital Library

[30]

Jonathan Lester, Blake Hannaford, and Gaetano Borriello. 2004. "Are You With Me?" ï£¡-- Using Accelerometers to Determine if Two Devices are Carried by the Same Person. In International Conference on Pervasive Computing. Springer, 33--50.

[31]

Kent Lyons. 2015. What can a dumb watch teach a smartwatch?: Informing the design of smartwatches. In Proceedings of the 2015 ACM International Symposium on Wearable Computers. ACM, 3--10.

Digital Library

[32]

Takuya Maekawa, Daisuke Nakai, Kazuya Ohara, and Yasuo Namioka. 2016. Toward practical factory activity recognition: unsupervised understanding of repetitive assembly work in a factory. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1088--1099.

Digital Library

[33]

Simon Olberding, Sergio Soto Ortega, Klaus Hildebrandt, and Jürgen Steimle. 2015. Foldio: Digital fabrication of interactive and shape-changing objects with foldable printed electronics. In Proceedings of the 28th Annual ACM Symposium on User Interface Software 8 Technology. ACM, 223--232.

Digital Library

[34]

Mahsan Rofouei, Andrew Wilson, AJ Brush, and Stewart Tansley. 2012. Your phone or mine?: fusing body, touch and device sensing for multi-user device-display interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1915--1918.

Digital Library

[35]

Jaime Ruiz and Yang Li. 2011. DoubleFlip: a motion gesture delimiter for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2717--2720.

Digital Library

[36]

Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 197--206.

Digital Library

[37]

Florian Schaub, Ruben Deyhle, and Michael Weber. 2012. Password entry usability and shoulder surfing susceptibility on different smartphone platforms. In Proceedings of the 11th international conference on mobile and ubiquitous multimedia. ACM, 13.

Digital Library

[38]

Dominik Schmidt, Fadi Chehimi, Enrico Rukzio, and Hans Gellersen. 2010. PhoneTouch: a technique for direct phone interaction on surfaces. In Proceedings of the 23nd annual ACM symposium on User interface software and technology. ACM, 13--16.

Digital Library

[39]

Corey Shemelya, Fernando Cedillos, Efrian Aguilera, David Espalin, Danny Muse, Ryan Wicker, and Eric MacDonald. 2015. Encapsulated copper wire and copper mesh capacitive sensing for 3-D printing applications. IEEE Sensors Journal 15, 2 (2015), 1280--1286.

[40]

Muhammad Shoaib, Stephan Bosch, Ozlem Durmaz Incel, Hans Scholten, and Paul JM Havinga. 2016. Complex human activity recognition using smartphone and wrist-worn motion sensors. Sensors 16, 4 (2016), 426.

[41]

Peng Song, Wooi Boon Goh, Chi-Wing Fu, Qiang Meng, and Pheng-Ann Heng. 2011. WYSIWYF: exploring and annotating volume data with a tangible handheld device. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1333--1342.

Digital Library

[42]

Xing Su, Hanghang Tong, and Ping Ji. 2014. Activity recognition with smartphone sensors. Tsinghua Science and Technology 19, 3 (2014), 235--249.

[43]

Vu H Tran, Kenny TW Choo, Youngki Lee, Richard C Davis, and Archan Misra. 2016. MAGI: Enabling multi-device gestural applications. In Pervasive Computing and Communication Workshops (PerCom Workshops), 2016 IEEE International Conference on. IEEE, 1--6.

[44]

Sebastian Uellenbeck, Markus Dürmuth, Christopher Wolf, and Thorsten Holz. 2013. Quantifying the security of graphical passwords: the case of android unlock patterns. In Proceedings of the 2013 ACM SIGSAC conference on Computer 8 communications security. ACM, 161--172.

Digital Library

[45]

Dirk Van Bruggen, Shu Liu, Mitch Kajzer, Aaron Striegel, Charles R Crowell, and John D'Arcy. 2013. Modifying smartphone user locking behavior. In Proceedings of the Ninth Symposium on Usable Privacy and Security. ACM, 10.

Digital Library

[46]

Alex Varshavsky, Adin Scannell, Anthony LaMarca, and Eyal De Lara. 2007. Amigo: Proximity-based authentication of mobile devices. In International Conference on Ubiquitous Computing. Springer, 253--270.

Digital Library

[47]

Gerard Wilkinson, Ahmed Kharrufa, Jonathan Hook, Bradley Pursglove, Gavin Wood, Hendrik Haeuser, Nils Y Hammerla, Steve Hodges, and Patrick Olivier. 2016. Expressy: Using a Wrist-worn Inertial Measurement Unit to Add Expressiveness to Touch-based Interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 2832--2844.

Digital Library

[48]

Katrin Wolf and Jonas Willaredt. 2015. PickRing: seamless interaction through pick-up detection. In Proceedings of the 6th Augmented Human International Conference. ACM, 13--20.

Digital Library

[49]

Junshuang Yang, Yanyan Li, and Mengjun Xie. 2015. MotionAuth: Motion-based authentication for wrist worn smart devices. In Pervasive Computing and Communication Workshops (PerCom Workshops), 2015 IEEE International Conference on. IEEE, 550--555.

[50]

Hongzi Zhu, Jingmei Hu, Shan Chang, and Li Lu. 2017. ShakeIn: Secure User Authentication of Smartphones with Habitual Single-handed Shakes. IEEE Transactions on Mobile Computing 16, 10 (2017), 2901--2912.

Cited By

Xu ZJin WYao CLiu XMa SLiu YQin ZVakilinia ILiu D(2024)EM-Rhythm: An Authentication Method for Heterogeneous IoT DevicesACM Transactions on Sensor Networks10.1145/370044120:6(1-21)Online publication date: 16-Oct-2024
https://dl.acm.org/doi/10.1145/3700441
Maeda SOkano MNishigaki MOhki T(2024)DAP PLXENT#x1D107; : Develop pair-Authentication Protocol with DAPAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680449(1-6)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680449
Liu YXu ZQin ZOu LJin W(2024)TouchAccess: Unlock IoT Devices on Touching by Leveraging Human-Induced EM EmanationsIEEE Transactions on Mobile Computing10.1109/TMC.2024.341499223:12(12587-12602)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TMC.2024.3414992
Show More Cited By

Index Terms

TwistIn: Tangible Authentication of Smart Devices via Motion Co-analysis with a Smartwatch
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input
2. Security and privacy
  1. Security services
    1. Authentication

Recommendations

SenseBelt: A Belt-Worn Sensor to Support Cross-Device Interaction
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Mobile interaction is shifting from a single device to simultaneous interaction with ensembles of devices such as phones, tablets, or watches. Spatially-aware cross-device interaction between mobile devices typically requires a fixed tracking ...
Investigating Cross-Device Interaction between a Handheld Device and a Large Display
CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

There is a growing interest in HCI research to explore cross-device interaction, giving rise to an interest in different approaches facilitating interaction between handheld devices and large displays. Contributing to this, we have investigated the use ...
Using Personal Devices to Facilitate Multi-user Interaction with Large Display Walls
UIST '15 Adjunct: Adjunct Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology

Large display walls and personal devices such as Smartphones have complementary characteristics. While large displays are well-suited to multi-user interaction (potentially with complex data), they are inherently public and generally cannot present an ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 2, Issue 2

June 2018

741 pages

EISSN:2474-9567

DOI:10.1145/3236498

Issue’s Table of Contents

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2018

Accepted: 01 April 2018

Revised: 01 February 2018

Received: 01 August 2017

Published in IMWUT Volume 2, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
600
Total Downloads

Downloads (Last 12 months)66
Downloads (Last 6 weeks)11

Reflects downloads up to 08 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu ZJin WYao CLiu XMa SLiu YQin ZVakilinia ILiu D(2024)EM-Rhythm: An Authentication Method for Heterogeneous IoT DevicesACM Transactions on Sensor Networks10.1145/370044120:6(1-21)Online publication date: 16-Oct-2024
https://dl.acm.org/doi/10.1145/3700441
Maeda SOkano MNishigaki MOhki T(2024)DAP PLXENT#x1D107; : Develop pair-Authentication Protocol with DAPAdjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction10.1145/3640471.3680449(1-6)Online publication date: 21-Sep-2024
https://dl.acm.org/doi/10.1145/3640471.3680449
Liu YXu ZQin ZOu LJin W(2024)TouchAccess: Unlock IoT Devices on Touching by Leveraging Human-Induced EM EmanationsIEEE Transactions on Mobile Computing10.1109/TMC.2024.341499223:12(12587-12602)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TMC.2024.3414992
Yu XZhou ZZhang LLi X(2024)ThumbUp: Secure Smartwatch Controller for Smart Homes Using Simple Hand GesturesIEEE Transactions on Mobile Computing10.1109/TMC.2022.321692723:1(865-878)Online publication date: Jan-2024
https://doi.org/10.1109/TMC.2022.3216927
Park SWang XChen KLee Y(2024)Station: Gesture-Based Authentication for Voice InterfacesIEEE Internet of Things Journal10.1109/JIOT.2024.338272111:12(22668-22683)Online publication date: 15-Jun-2024
https://doi.org/10.1109/JIOT.2024.3382721
Azim MRahman AHeo S(2023)UnifiedSense: Enabling Without-Device Gesture Interactions Using Over-the-shoulder Training Between Redundant Wearable SensorsProceedings of the ACM on Human-Computer Interaction10.1145/36042777:MHCI(1-25)Online publication date: 13-Sep-2023
https://dl.acm.org/doi/10.1145/3604277
Zhang YHan DLi AZhang LLi TZhang Y(2023)MagAuth: Secure and Usable Two-Factor Authentication With Magnetic Wrist WearablesIEEE Transactions on Mobile Computing10.1109/TMC.2021.307259822:1(311-327)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TMC.2021.3072598
Qiu HChen YChen ZYang CWu SLi FXie L(2023)Feasibility of Wrist-Worn, Cancelable, Real-Time Biometric Authentication via HD-sEMG and Dynamic GesturesIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.332749072(1-11)Online publication date: 2023
https://doi.org/10.1109/TIM.2023.3327490
Trnka MAbdelfattah AShrestha ACoffey MCerny T(2022)Systematic Review of Authentication and Authorization Advancements for the Internet of ThingsSensors10.3390/s2204136122:4(1361)Online publication date: 10-Feb-2022
https://doi.org/10.3390/s22041361
Shi CXie LWang CYang PSong YLu S(2021)Just Shake Them TogetherWireless Communications & Mobile Computing10.1155/2021/66684782021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/6668478
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents