research-article

EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables

Authors:

Jie YangAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 5, Issue 1

Article No.: 39, Pages 1 - 27

https://doi.org/10.1145/3448098

Published: 30 March 2021 Publication History

Abstract

Biometric-based authentication is gaining increasing attention for wearables and mobile applications. Meanwhile, the growing adoption of sensors in wearables also provides opportunities to capture novel wearable biometrics. In this work, we propose EarDynamic, an ear canal deformation based user authentication using in-ear wearables. EarDynamic provides continuous and passive user authentication and is transparent to users. It leverages ear canal deformation that combines the unique static geometry and dynamic motions of the ear canal when the user is speaking for authentication. It utilizes an acoustic sensing approach to capture the ear canal deformation with the built-in microphone and speaker of the in-ear wearable. Specifically, it first emits well-designed inaudible beep signals and records the reflected signals from the ear canal. It then analyzes the reflected signals and extracts fine-grained acoustic features that correspond to the ear canal deformation for user authentication. Our extensive experimental evaluation shows that EarDynamic can achieve a recall of 97.38% and an F1 score of 96.84%. Results also show that our system works well under different noisy environments with various daily activities.

References

[1]

2017. HUAWEI FreeBuds 3. https://consumer.huawei.com/en/audio/freebuds3/

[2]

2020. Wireless Bluetooth Sports Headphones. https://www.jabra.com/sports-headphones

[3]

Aditya Abhyankar and Stephanie Schuckers. 2009. Integrating a wavelet based perspiration liveness check with fingerprint recognition. Pattern Recognition 42, 3 (2009), 452--464.

Digital Library

[4]

Amazon. 2020. Amazon Echo Buds. https://www.amazon.com/Echo-Buds/dp/B07F6VM1S3/

[5]

Takashi Amesaka, Hiroki Watanabe, and Masanori Sugimoto. 2019. Facial expression recognition using ear canal transfer function. In Proceedings of the 23rd International Symposium on Wearable Computers. 1--9.

Digital Library

[6]

Apple. 2020. Apple AirPods Pro. https://www.apple.com/airpods-pro/

[7]

Takayuki Arakawa, Takafumi Koshinaka, Shohei Yano, Hideki Irisawa, Ryoji Miyahara, and Hitoshi Imaoka. 2016. Fast and accurate personal authentication using ear acoustics. In 2016 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). IEEE, 1--4.

[8]

Hind Baqeel and Saqib Saeed. 2019. Face Detection Authentication on Smartphones: End Users Usability Assessment Experiences. In 2019 International Conference on Computer and Information Sciences (ICCIS). IEEE, 1--6.

[9]

Abdelkareem Bedri, David Byrd, Peter Presti, Himanshu Sahni, Zehua Gue, and Thad Starner. 2015. Stick it in your ear: Building an in-ear jaw movement sensor. In Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers. 1333--1338.

Digital Library

[10]

Johan Carioli, Aidin Delnavaz, Ricardo J Zednik, and Jérémie Voix. 2017. Piezoelectric earcanal bending sensor. IEEE Sensors Journal 18, 5 (2017), 2060--2067.

[11]

Phillip L De Leon, Michael Pucher, Junichi Yamagishi, Inma Hernaez, and Ibon Saratxaga. 2012. Evaluation of speaker verification security and detection of HMM-based synthetic speech. IEEE Transactions on Audio, Speech, and Language Processing 20, 8 (2012), 2280--2290.

Digital Library

[12]

Aidin Delnavaz and Jérémie Voix. 2013. Energy harvesting for in-ear devices using ear canal dynamic motion. IEEE Transactions on Industrial Electronics 61, 1 (2013), 583--590.

[13]

Jianjiang Feng, Anil K Jain, and Arun Ross. 2009. Fingerprint alteration. submitted to IEEE TIFS (2009).

[14]

Matteo Ferrara, Annalisa Franco, and Davide Maltoni. 2016. On the effects of image alterations on face recognition accuracy. In Face recognition across the imaging spectrum. Springer, 195--222.

[15]

Yang Gao, Wei Wang, Vir V Phoha, Wei Sun, and Zhanpeng Jin. 2019. EarEcho: Using Ear Canal Echo for Wearable Authentication. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1--24.

Digital Library

[16]

Valentin Goverdovsky, David Looney, Preben Kidmose, and Danilo P Mandic. 2015. In-ear EEG from viscoelastic generic earpieces: Robust and unobtrusive 24/7 monitoring. IEEE Sensors Journal 16, 1 (2015), 271--277.

[17]

Malcolm J Grenness, Jon Osborn, and W Lee Weller. 2002. Mapping ear canal movement using area-based surface matching. The Journal of the Acoustical Society of America 111, 2 (2002), 960--971.

[18]

Rosa González Hautamäki, Tomi Kinnunen, Ville Hautamäki, Timo Leino, and Anne-Maria Laukkanen. 2013. I-vectors meet imitators: on vulnerability of speaker verification systems against voice mimicry. In Interspeech. 930--934.

[19]

Mahdieh Izadpanahkakhk, Seyyed Mohammad Razavi, Mehran Taghipour-Gorjikolaie, Seyyed Hamid Zahiri, and Aurelio Uncini. 2018. Deep region of interest and feature extraction models for palmprint verification using convolutional neural networks transfer learning. Applied Sciences 8, 7 (2018), 1210.

[20]

Amin Jalali, Rommohan Mallipeddi, and Minho Lee. 2015. Deformation invariant and contactless palmprint recognition using convolutional neural network. In Proceedings of the 3rd International Conference on Human-Agent Interaction. 209--212.

Digital Library

[21]

Artur Janicki, Federico Alegre, and Nicholas Evans. 2016. An assessment of automatic speaker verification vulnerabilities to replay spoofing attacks. Security and Communication Networks 9, 15 (2016), 3030--3044.

Digital Library

[22]

Tomi Kinnunen, Md Sahidullah, Héctor Delgado, Massimiliano Todisco, Nicholas Evans, Junichi Yamagishi, and Kong Aik Lee. 2017. The ASVspoof 2017 challenge: Assessing the limits of replay spoofing attack detection. (2017).

[23]

Andreas Kipp, M-B Wesenick, and Florian Schiel. 1996. Automatic detection and segmentation of pronunciation variants in German speech corpora. In Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP'96, Vol. 1. IEEE, 106--109.

[24]

Thomas Kisler, Florian Schiel, and Han Sloetjes. 2012. Signal processing via web services: the use case WebMAUS. In Digital Humanities Conference 2012.

[25]

Rajamani Santhosh Kumar, KR Jothi Kumar, D Saravana Bhavan, and A Anandaraj. [n.d.]. Variations in the External Auditory Canal of 185 Adult Individuals: A Clinico-Morphological Study. International Journal of Scientific and Research Publications ([n. d.]), 305.

[26]

Nianfeng Liu, Man Zhang, Haiqing Li, Zhenan Sun, and Tieniu Tan. 2016. DeepIris: Learning pairwise filter bank for heterogeneous iris verification. Pattern Recognition Letters 82 (2016), 154--161.

Digital Library

[27]

Davide Maltoni, Dario Maio, Anil K Jain, and Salil Prabhakar. 2009. Handbook of fingerprint recognition. Springer Science & Business Media.

Digital Library

[28]

Emanuela Marasco and Arun Ross. 2014. A survey on antispoofing schemes for fingerprint recognition systems. ACM Computing Surveys (CSUR) 47, 2 (2014), 1--36.

Digital Library

[29]

Gian Luca Marcialis, Fabio Roli, and Alessandra Tidu. 2010. Analysis of fingerprint pores for vitality detection. In 2010 20th international conference on pattern recognition. IEEE, 1289--1292.

Digital Library

[30]

Amir Mohammadi, Sushil Bhattacharjee, and Sébastien Marcel. 2017. Deeply vulnerable: a study of the robustness of face recognition to presentation attacks. IET Biometrics 7, 1 (2017), 15--26.

[31]

Jang-Ho Park, Dae-Geun Jang, Jung Wook Park, and Se-Kyoung Youm. 2015. Wearable sensing of in-ear pressure for heart rate monitoring with a piezoelectric sensor. Sensors 15, 9 (2015), 23402--23417.

[32]

Chester Pirzanski and Brenda Berge. 2005. Ear canal dynamics: Facts versus perception. The Hearing Journal 58, 10 (2005), 50--52.

[33]

Kiran B Raja, Ramachandra Raghavendra, Vinay Krishna Vemuri, and Christoph Busch. 2015. Smartphone based visible iris recognition using deep sparse filtering. Pattern Recognition Letters 57 (2015), 33--42.

Digital Library

[34]

Daniel M Rasetshwane and Stephen T Neely. 2011. Inverse solution of ear-canal area function from reflectance. The Journal of the Acoustical Society of America 130, 6 (2011), 3873--3881.

[35]

Mosur K Ravishankar. 1996. Efficient Algorithms for Speech Recognition. Technical Report. Carnegie-Mellon Univ Pittsburgh pa Dept of Computer Science.

[36]

U Rosenhall. 1996. The Human Ear Canal. Theoretical Considerations and Clinical Applications including Cerumen Management by BB Ballachandra. JOURNAL OF AUDIOLOGICAL MEDICINE 5 (1996), 176--177.

[37]

Robert E Schapire and Yoav Freund. 2013. Boosting: Foundations and algorithms. Kybernetes (2013).

[38]

Ulrich Scherhag, Christian Rathgeb, Johannes Merkle, Ralph Breithaupt, and Christoph Busch. 2019. Face recognition systems under morphing attacks: A survey. IEEE Access 7 (2019), 23012--23026.

[39]

Florian Schiel, Christoph Draxler, and Jonathan Harrington. 2011. Phonemic segmentation and labelling using the MAUS technique. (2011).

[40]

Stephanie AC Schuckers. 2002. Spoofing and anti-spoofing measures. Information Security technical report 7, 4 (2002), 56--62.

[41]

Chao Shen, Yuanxun Li, Yufei Chen, Xiaohong Guan, and Roy A Maxion. 2017. Performance analysis of multi-motion sensor behavior for active smartphone authentication. IEEE Transactions on Information Forensics and Security 13, 1 (2017), 48--62.

[42]

Kaavya Sriskandaraja, Vidhyasaharan Sethu, Eliathamby Ambikairajah, and Haizhou Li. 2016. Front-end for antispoofing counter-measures in speaker verification: Scattering spectral decomposition. IEEE Journal of Selected Topics in Signal Processing 11,4 (2016), 632--643.

[43]

Michael R Stinson and BW Lawton. 1989. Specification of the geometry of the human ear canal for the prediction of sound-pressure level distribution. The Journal of the Acoustical Society of America 85, 6 (1989), 2492--2503.

[44]

Boudewijn Venema, Johannes Schiefer, Vladimir Blazek, Nikolai Blanik, and Steffen Leonhardt. 2013. Evaluating innovative in-ear pulse oximetry for unobtrusive cardiovascular and pulmonary monitoring during sleep. IEEE journal of translational engineering in health and medicine 1 (2013), 2700208--2700208.

[45]

Rudolf M Verdaasdonk and Niels Liberton. 2019. The Iphone X as 3D scanner for quantitative photography of faces for diagnosis and treatment follow-up (Conference Presentation). In Optics and Biophotonics in Low-Resource Settings V, Vol. 10869. International Society for Optics and Photonics, 1086902.

[46]

Jérémie Voix. 2017. The ear beyond hearing: From smart earplug to in-ear brain computer interfaces. In Proc. 24th Int. Congr. Sound Vibrat.(ICSV). 1--11.

[47]

Chen Wang, Junping Zhang, Jian Pu, Xiaoru Yuan, and Liang Wang. 2010. Chrono-gait image: A novel temporal template for gait recognition. In European Conference on Computer Vision. Springer, 257--270.

[48]

Mao-Che Wang, Chia-Yu Liu, An-Suey Shiao, and Tyrone Wang. 2005. Ear problems in swimmers. Journal of the chinese medical association 68, 8 (2005), 347--352.

[49]

Zhizheng Wu, Tomi Kinnunen, Nicholas Evans, Junichi Yamagishi, Cemal Hanilçi, Md Sahidullah, and Aleksandr Sizov. 2015. ASVspoof 2015: the first automatic speaker verification spoofing and countermeasures challenge. In Sixteenth Annual Conference of the International Speech Communication Association.

Cited By

Chen YYu JChen YKong LZhu YChen YOkoshi TKo JLiKamWa R(2024)RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFIDProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661887(169-182)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661887
Duan DSun ZNi TLi SJia XXu WLi TOkoshi TKo JLiKamWa R(2024) F 2 Key: Dynamically Converting Your Face into a Private Key Based on COTS Headphones for Reliable Voice Interaction Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661860(127-140)Online publication date: 4-Jun-2024
https://doi.org/10.1145/3643832.3661860
Wang CFeng YZhong LZhu SZhang CZheng SLiang CWang YHe CYu CShi Y(2024)UbiPhysioProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435528:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643552
Show More Cited By

Index Terms

EarDynamic: An Ear Canal Deformation Based Continuous User Authentication Using In-Ear Wearables
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools
2. Security and privacy
  1. Security services
    1. Authentication
      1. Biometrics

Recommendations

ToothSonic: Earable Authentication via Acoustic Toothprint

Earables (ear wearables) are rapidly emerging as a new platform encompassing a diverse range of personal applications. The traditional authentication methods hence become less applicable and inconvenient for earables due to their limited input interface. ...
An ear canal deformation based continuous user authentication using earables
MobiCom '21: Proceedings of the 27th Annual International Conference on Mobile Computing and Networking

Biometric-based authentication is gaining increasing attention for wearables and mobile applications. Meanwhile, the growing adoption of sensors in wearables also provides opportunities to capture novel wearable biometrics. In this work, we propose ...
EarEcho: Using Ear Canal Echo for Wearable Authentication

Smart wearable devices have recently become one of the major technological trends and been widely adopted by the general public. Wireless earphones, in particular, have seen a skyrocketing growth due to its great usability and convenience. With the goal ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 5, Issue 1

March 2021

1272 pages

EISSN:2474-9567

DOI:10.1145/3459088

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2021

Published in IMWUT Volume 5, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

72
Total Citations
View Citations
1,007
Total Downloads

Downloads (Last 12 months)194
Downloads (Last 6 weeks)13

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen YYu JChen YKong LZhu YChen YOkoshi TKo JLiKamWa R(2024)RFSpy: Eavesdropping on Online Conversations with Out-of-Vocabulary Words by Sensing Metal Coil Vibration of Headsets Leveraging RFIDProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661887(169-182)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661887
Duan DSun ZNi TLi SJia XXu WLi TOkoshi TKo JLiKamWa R(2024) F 2 Key: Dynamically Converting Your Face into a Private Key Based on COTS Headphones for Reliable Voice Interaction Proceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661860(127-140)Online publication date: 4-Jun-2024
https://doi.org/10.1145/3643832.3661860
Wang CFeng YZhong LZhu SZhang CZheng SLiang CWang YHe CYu CShi Y(2024)UbiPhysioProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435528:1(1-27)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643552
Wang ZWang YYang J(2024)EarSlideProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435158:1(1-29)Online publication date: 6-Mar-2024
https://doi.org/10.1145/3643515
Ahmad NLeung H(2024)HyperHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435118:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643511
Xu ZLiu TJiang RHu PGuo ZLiu C(2024)AFaceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435108:1(1-33)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643510
Jiang FWang HLi Y(2024)VesNet: A Vessel Network for Jointly Learning Route Pattern and Future TrajectoryACM Transactions on Intelligent Systems and Technology10.1145/363937015:2(1-25)Online publication date: 28-Mar-2024
https://dl.acm.org/doi/10.1145/3639370
Duan DChen YXu WLi T(2024)EarSEProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314477:4(1-33)Online publication date: 12-Jan-2024
https://doi.org/10.1145/3631447
Xia KLi WGan SLu S(2024)TS2ACTProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314457:4(1-22)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631445
Chen WHu YSong WLiu YTorralba AMatusik W(2024)CAvatarProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314247:4(1-24)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631424
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents