research-article

Open access

A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-based Pathogens

Authors:

Mileva Van Tuyl,

Zipporah Cohen,

Fadel AdibAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 5, Issue 1

Article No.: 31, Pages 1 - 22

https://doi.org/10.1145/3448121

Published: 30 March 2021 Publication History

Abstract

Hand-to-Face transmission has been estimated to be a minority, yet non-negligible, vector of COVID-19 transmission and a major vector for multiple other pathogens. At the same time, as it cannot be effectively addressed with mainstream protection measures, such as wearing masks or tracing contacts, it remains largely untackled. To help address this issue, we have developed Saving Face - an app that alerts users when they are about to touch their faces, by analyzing the distortion patterns in the ultrasound signal emitted by their earphones. The system only relies on pre-existing hardware (a smartphone with generic earphones), which allows it to be rapidly scalable to billions of smartphone users worldwide. This paper describes the design, implementation and evaluation of the system, as well as the results of a user study testing the solution's accuracy, robustness, and user experience during various day-to-day activities (93.7% Sensitivity and 91.5% Precision, N=10). While this paper focuses on the system's application to detecting hand-to-face gestures, the technique can also be applicable to other types of gestures and gesture-based applications.

Supplementary Material

rojas (rojas.zip)

Supplemental movie, appendix, image and software files for, A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-based Pathogens

Download
24.70 MB

References

[1]

Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C. Miller. 2014. 3D Tracking via Body Radio Reflections. In 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14). USENIX Association, Seattle, WA, 317--329. https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/adib

[2]

Farzaneh Ahmadi, Ian V McLoughlin, Sunita Chauhan, and Gail Ter-Haar. 2012. Bio-effects and safety of low-intensity, low-frequency ultrasonic exposure. Progress in biophysics and molecular biology 108, 3 (2012), 119--138.

[3]

Asaph Azaria., Brian Mayton., and Joseph Paradiso. 2016. Thumbs-Up - Wearable Sensing Device for Detecting Hand-to-Mouth Compulsive Habits. (2016), 54--65. https://doi.org/10.5220/0005680900540065

[4]

Karina S Bate, John M Malouff, Einar T Thorsteinsson, and Navjot Bhullar. 2011. The efficacy of habit reversal therapy for tics, habit disorders, and stuttering: a meta-analytic review. Clinical Psychology Review 31, 5 (2011), 865--871.

[5]

Mike Bodge, Brian Moore, and Isaac Blankensmith. 2020. Do Not Touch Your Face. https://donottouchyourface.com/

[6]

William J Bradshaw, Ethan C Alley, Jonathan H Huggins, Alun L Lloyd, and Kevin M Esvelt. 2020. Bidirectional contact tracing is required for reliable COVID-19 control. medRxiv (2020).

[7]

Xiang'Anthony' Chen. 2020. FaceOff: Detecting Face Touching with a Wrist-Worn Accelerometer. arXiv preprint arXiv:2008.01769 (2020).

[8]

Apple Co. 2020. vDSP: Perform basic arithmetic operations and common digital signal processing routines on large vectors. https://developer.apple.com/documentation/accelerate/vdsp

[9]

DocuSign. 2020. Electronic Signature and Agreement. https://www.docusign.com/

[10]

Luca Ferretti, Chris Wymant, Michelle Kendall, Lele Zhao, Anel Nurtay, Lucie Abeler-Dörner, Michael Parker, David Bonsall, and Christophe Fraser. 2020. Quantifying SARS-CoV-2 transmission suggests epidemic control with digital contact tracing. Science 368, 6491 (2020).

[11]

U.S. Center for Disease Control. 2019. What Noises Cause Hearing Loss? | NCEH | CDC. https://www.cdc.gov/nceh/hearing_loss/what_noises_cause_hearing_loss.html

[12]

Sidhant Gupta, Daniel Morris, Shwetak Patel, and Desney Tan. 2012. SoundWave: Using the Doppler Effect to Sense Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI '12). Association for Computing Machinery, New York, NY, USA, 1911--1914. https://doi.org/10.1145/2207676.2208331

Digital Library

[13]

Megan R Heinicke, Jordan T Stiede, Raymond G Miltenberger, and Douglas W Woods. 2020. Reducing risky behavior with habit reversal: A review of behavioral strategies to reduce habitual hand-to-head behavior. Journal of applied behavior analysis 53, 3 (2020), 1225--1236.

[14]

Joseph A Himle, Deborah Bybee, Lisa A O'Donnell, Addie Weaver, Sarah Vlnka, Daniel T DeSena, and Jessica M Rimer. 2018. Awareness enhancing and monitoring device plus habit reversal in the treatment of trichotillomania: An open feasibility trial. Journal of obsessive-compulsive and related disorders 16 (2018), 14--20.

[15]

Joseph A Himle, David M Perlman, and Laura M Lokers. 2008. Prototype awareness enhancing and monitoring device for trichotillomania. Behaviour research and therapy 46, 10 (2008), 1187--1191.

[16]

Immutouch. 2020. Protect yourself from germs break bad habits. https://immutouch.com/

[17]

Günter Kampf, Daniel Todt, Stephanie Pfaender, and Eike Steinmann. 2020. Persistence of coronaviruses on inanimate surfaces and their inactivation with biocidal agents. Journal of Hospital Infection 104, 3 (2020), 246--251.

[18]

Bryce Kellogg, Vamsi Talla, and Shyamnath Gollakota. 2014. Bringing Gesture Recognition to All Devices. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (Seattle, WA) (NSDI'14). USENIX Association, USA, 303--316.

Digital Library

[19]

National Health System United Kingdom. 2018. Skin picking disorder. https://www.nhs.uk/conditions/skin-picking-disorder/

[20]

T. G. Leighton. 2017. Comment on 'Are some people suffering as a result of increasing mass exposure of the public to ultrasound in air?'. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences 473, 2199 (March 2017), 20160828. https://doi.org/10.1098/rspa.2016.0828 Publisher: Royal Society.

[21]

X. Li, H. Dai, L. Cui, and Y. Wang. 2017. SonicOperator: Ultrasonic gesture recognition with deep neural network on mobiles. In 2017 IEEE SmartWorld, Ubiquitous Intelligence Computing, Advanced Trusted Computed, Scalable Computing Communications, Cloud Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI). 1--7.

[22]

Ethan S Long, Raymond G Miltenberger, Sherry A Ellingson, and Shelley M Ott. 1999. Augmenting simplified habit reversal in the treatment of oral-digital habits exhibited by individuals with mental retardation. Journal of Applied Behavior Analysis 32, 3 (1999), 353--365.

[23]

Wenguang Mao, Jian He, and Lili Qiu. 2016. CAT: high-precision acoustic motion tracking. In Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking. 69--81.

Digital Library

[24]

Wenguang Mao, Mei Wang, Wei Sun, Lili Qiu, Swadhin Pradhan, and Yi-Chao Chen. 2019. RNN-Based Room Scale Hand Motion Tracking. In The 25th Annual International Conference on Mobile Computing and Networking. 1--16.

[25]

Rafael Martinez. [n.d.]. JPL COVID-19 PULSE Pendant. https://medeng.jpl.nasa.gov/covid-19/pulse/

[26]

American Society For Microbiology. 2003. Survey: Many Travelers Coast Through U.S. Airports Without Washing Their Hands. https://www.sciencedaily.com/releases/2003/09/030916074111.html

[27]

Rajalakshmi Nandakumar, Vikram Iyer, Desney Tan, and Shyamnath Gollakota. 2016. Fingerio: Using active sonar for fine-grained finger tracking. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. 1515--1525.

Digital Library

[28]

Minister of National Health and Government of Canada Welfare. 1991. Guidelines for the Safe Use of Ultrasound - Safety Code 24. https://www.canada.ca/content/dam/hc-sc/documents/services/environmental-workplace-health/reports-publications/radiation/safety-code_24-securite-eng.pdf

[29]

Pavlok. 2020. The Most Effective Solution To Living Life Smoke Free For Those Who Have Tried Everything Else - Guaranteed. https://pavlok.com/quit-smoking-with-pavlok/

[30]

Domenico Prattichizzo, Tommaso Lisini, Gianluca Paolocci, Nicole D'Aurizio, Sara Marullo, and Annino De Petra. 2020. https://sites.google.com/unisi.it/noface-touchapp/

[31]

Aurelius Prochazka. 2020. AudioKit Developer Documentation Site. https://audiokit.io/

[32]

Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole-Home Gesture Recognition Using Wireless Signals. In Proceedings of the 19th Annual International Conference on Mobile Computing amp; Networking (Miami, Florida, USA) (MobiCom '13). Association for Computing Machinery, New York, NY, USA, 27--38. https://doi.org/10.1145/2500423.2500436

Digital Library

[33]

Dale Purves, George J. Augustine, David Fitzpatrick, Lawrence C. Katz, Anthony-Samuel LaMantia, James O. McNamara, and S. Mark Williams. 2001. The Audible Spectrum. Neuroscience. 2nd edition (2001). https://www.ncbi.nlm.nih.gov/books/NBK10924/ Publisher: Sinauer Associates.

[34]

Yang Qifan, Tang Hao, Zhao Xuebing, Yin Li, and Zhang Sanfeng. 2014. Dolphin: Ultrasonic-Based Gesture Recognition on Smartphone Platform. 1461--1468. https://doi.org/10.1109/CSE.2014.273

[35]

John T Rapp, Raymond G Miltenberger, and Ethan S Long. 1998. Augmenting simplified habit reversal with an awareness enhancement device: Preliminary findings. Journal of Applied Behavior Analysis 31, 4 (1998), 665--668.

[36]

Wenjie Ruan, Quan Z. Sheng, Lei Yang, Tao Gu, Peipei Xu, and Longfei Shangguan. 2016. AudioGest: Enabling Fine-Grained Hand Gesture Detection by Decoding Echo Signal. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (Heidelberg, Germany) (UbiComp '16). Association for Computing Machinery, New York, NY, USA, 474--485. https://doi.org/10.1145/2971648.2971736

Digital Library

[37]

Dan Russel. [n.d.]. Absorption and Attenuation of Sound in Air. https://www.acs.psu.edu/drussell/Demos/Absorption/Absorption.html

[38]

Sheng Shen, Daguan Chen, Yu-Lin Wei, Zhijian Yang, and Romit Roy Choudhury. 2020. Voice localization using nearby wall reflections. In Proceedings of the 26th Annual International Conference on Mobile Computing and Networking. 1--14.

Digital Library

[39]

Joren Six, Olmo Cornelis, and Marc Leman. 2014. TarsosDSP, a real-time audio processing framework in Java. In Audio Engineering Society Conference: 53rd International Conference: Semantic Audio. Audio Engineering Society.

[40]

Sam Vafaei. 2018. Raw Frequency Response. https://www.rtings.com/headphones/tests/sound-quality/raw-frequency-response

[41]

Neeltje Van Doremalen, Trenton Bushmaker, Dylan H Morris, Myndi G Holbrook, Amandine Gamble, Brandi N Williamson, Azaibi Tamin, Jennifer L Harcourt, Natalie J Thornburg, Susan I Gerber, et al. 2020. Aerosol and surface stability of SARS-CoV-2 as compared with SARS-CoV-1. New England Journal of Medicine 382, 16 (2020), 1564--1567.

[42]

Chris Velazco. [n.d.]. Ultrasound and software could replace a phone's proximity sensor. https://www.engadget.com/2016-01-19-elliptic-labs-beauty-ultrasound.html

[43]

Z. Wang, Y. Hou, K. Jiang, C. Zhang, W. Dou, Z. Huang, and Y. Guo. 2019. A Survey on Human Behavior Recognition Using Smartphone-Based Ultrasonic Signal. IEEE Access 7 (2019), 100581--100604.

[44]

Hiroki Watanabe and Tsutomu Terada. 2018. Improving Ultrasound-Based Gesture Recognition Using a Partially Shielded Single Microphone. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (Singapore, Singapore) (ISWC '18). Association for Computing Machinery, New York, NY, USA, 9--16. https://doi.org/10.1145/3267242.3267274

Digital Library

[45]

Douglas Woods and Raymond Miltenberger. 2007. Tic disorders, trichotillomania, and other repetitive behavior disorders: Behavioral approaches to analysis and treatment. Springer Science & Business Media.

[46]

Chaojun Xie, Hongjun Zhao, Kuibiao Li, Zhoubin Zhang, Xiaoxiao Lu, Huide Peng, Dahu Wang, Jin Chen, Xiao Zhang, Di Wu, Yuzhou Gu, Jun Yuan, Lin Zhang, and Jiachun Lu. 2020. The evidence of indirect transmission of SARS-CoV-2 reported in Guangzhou, China. BMC Public Health 20, 1 (2020), 1--9. https://doi.org/10.1186/s12889-020-09296-y

[47]

Sangki Yun, Yi-Chao Chen, Huihuang Zheng, Lili Qiu, and Wenguang Mao. 2017. Strata: Fine-grained acoustic-based device-free tracking. In Proceedings of the 15th annual international conference on mobile systems, applications, and services. 15--28.

Digital Library

[48]

Junbo Zhang and Swarun Kumar. 2020. NoFaceContact: stop touching your face with NFC. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services. 468--469.

Digital Library

[49]

Chen Zhao, Ke-Yu Chen, Md Tanvir Islam Aumi, Shwetak Patel, and Matthew S. Reynolds. 2014. SideSwipe: Detecting in-Air Gestures around Mobile Devices Using Actual GSM Signal. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (Honolulu, Hawaii, USA) (UIST '14). Association for Computing Machinery, New York, NY, USA, 527--534. https://doi.org/10.1145/2642918.2647380

Cited By

Zhang QGuo KYang YWang D(2025)WearSE: Enabling Streaming Speech Enhancement on Eyewear Using Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122889:1(1-30)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712288
Pour Rezaei PJokela TVetek ASalmimaa M(2024)Informing the Design of Intervention Solutions for Body-Focused Repetitive BehaviorsProceedings of the ACM on Human-Computer Interaction10.1145/36765138:MHCI(1-15)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676513
Zhang XWang YHan YLiang CChatterjee ITang JYi XPatel SShi Y(2024)The EarSAVAS DatasetProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596168:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659616
Show More Cited By

Index Terms

A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-based Pathogens
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools

Recommendations

Automated Static Code Analysis for Classifying Android Applications Using Machine Learning
CIS '10: Proceedings of the 2010 International Conference on Computational Intelligence and Security

In this paper we apply Machine Learning (ML) techniques on static features that are extracted from Android’s application files for the classification of the files. Features are extracted from Android’s Java byte-code (i.e.,.dex files) and other file ...
Smart smartphone development: iOS versus android
SIGCSE '11: Proceedings of the 42nd ACM technical symposium on Computer science education

In a remarkably short timeframe, developing apps for smartphones has gone from an arcane curiosity to an essential skill set. Employers are scrambling to find developers capable of transforming their ideas into apps. Educators interested in filling that ...
Built-in face recognition for smart photo sharing in mobile devices
ICME '11: Proceedings of the 2011 IEEE International Conference on Multimedia and Expo

This paper presents a smart photo sharing application for mo bile devices based on an efficient yet powerful face recognition engine. The system allows registering subjects either from live video (mobile camera) or from still images, linking the ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 5, Issue 1

March 2021

1272 pages

EISSN:2474-9567

DOI:10.1145/3459088

Issue’s Table of Contents

Copyright © 2021 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2021

Published in IMWUT Volume 5, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

NSF (National Science Foundation)
Swiss National Science Foundation

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
709
Total Downloads

Downloads (Last 12 months)134
Downloads (Last 6 weeks)14

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang QGuo KYang YWang D(2025)WearSE: Enabling Streaming Speech Enhancement on Eyewear Using Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/37122889:1(1-30)Online publication date: 3-Mar-2025
https://dl.acm.org/doi/10.1145/3712288
Pour Rezaei PJokela TVetek ASalmimaa M(2024)Informing the Design of Intervention Solutions for Body-Focused Repetitive BehaviorsProceedings of the ACM on Human-Computer Interaction10.1145/36765138:MHCI(1-15)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676513
Zhang XWang YHan YLiang CChatterjee ITang JYi XPatel SShi Y(2024)The EarSAVAS DatasetProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596168:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659616
Yi EZhang FXiong JNiu KYao ZZhang D(2024)Enabling WiFi Sensing on New-generation WiFi CardsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36338077:4(1-26)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3633807
Yang YChen THuang YGuo XShangguan L(2024)MAF: Exploring Mobile Acoustic Field for Hand-to-Face Gesture InteractionsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642437(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642437
Fathian RPhan SHo CRouhani H(2023)Face touch monitoring using an instrumented wristband using dynamic time warping and k-nearest neighboursPLOS ONE10.1371/journal.pone.028177818:2(e0281778)Online publication date: 17-Feb-2023
https://doi.org/10.1371/journal.pone.0281778
Liu LCao ZLi T(2023)FaceTouch: Practical Face Touch Detection with a Multimodal Wearable System for Epidemiological SurveillanceProceedings of the 8th ACM/IEEE Conference on Internet of Things Design and Implementation10.1145/3576842.3582368(13-26)Online publication date: 9-May-2023
https://dl.acm.org/doi/10.1145/3576842.3582368
Röddiger TClarke CBreitling PSchneegans TZhao HGellersen HBeigl M(2022)Sensing with EarablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35503146:3(1-57)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550314
Kakaraparthi VShao QCarver CPham TBui NNguyen PZhou XVu T(2021)FaceSenseProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34781295:3(1-27)Online publication date: 14-Sep-2021
https://dl.acm.org/doi/10.1145/3478129
D'Aurizio NBaldi TMarullo SPaolocci GPrattichizzo D(2021)Reducing Face-Touches to Limit COVID-19 Outbreak: an Overview of Solutions2021 29th Mediterranean Conference on Control and Automation (MED)10.1109/MED51440.2021.9480242(645-650)Online publication date: 22-Jun-2021
https://doi.org/10.1109/MED51440.2021.9480242

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents