research-article

Open access

FaceSense: Sensing Face Touch with an Ear-worn System

Authors:

Vimal Kakaraparthi,

Charles J. Carver,

Tam VuAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 5, Issue 3

Article No.: 110, Pages 1 - 27

https://doi.org/10.1145/3478129

Published: 14 September 2021 Publication History

Abstract

Face touch is an unconscious human habit. Frequent touching of sensitive/mucosal facial zones (eyes, nose, and mouth) increases health risks by passing pathogens into the body and spreading diseases. Furthermore, accurate monitoring of face touch is critical for behavioral intervention. Existing monitoring systems only capture objects approaching the face, rather than detecting actual touches. As such, these systems are prone to false positives upon hand or object movement in proximity to one's face (e.g., picking up a phone). We present FaceSense, an ear-worn system capable of identifying actual touches and differentiating them between sensitive/mucosal areas from other facial areas. Following a multimodal approach, FaceSense integrates low-resolution thermal images and physiological signals. Thermal sensors sense the thermal infrared signal emitted by an approaching hand, while physiological sensors monitor impedance changes caused by skin deformation during a touch. Processed thermal and physiological signals are fed into a deep learning model (TouchNet) to detect touches and identify the facial zone of the touch. We fabricated prototypes using off-the-shelf hardware and conducted experiments with 14 participants while they perform various daily activities (e.g., drinking, talking). Results show a macro-F1-score of 83.4% for touch detection with leave-one-user-out cross-validation and a macro-F1-score of 90.1% for touch zone identification with a personalized model.

References

[1]

[n. d.]. AMG8833 Thermal Sesnor. https://media.digikey.com/pdf/Data%20Sheets/Panasonic%20Sensors%20PDFs/Grid-EYE_AMG88.pdf. ([n. d.]). Accessed: 2021-05-10.

[2]

[n. d.]. HC-06 Bluetooth Transceiver Module. http://wiki.sunfounder.cc/index.php?title=Bluetooth_Transceiver_Module_HC-06. ([n. d.]). Accessed: 2021-05-10.

[3]

[n. d.]. WIRELINGZERO PROCESSOR. https://tinycircuits.com/products/wirelingzero. ([n. d.]). Accessed: 2021-05-10.

[4]

04/01/2020. Magnet-based face touch detection app for coronavirus: Use your phone's compass and a magnet on your wrist to detect face touching, including audio and vibration alerts. (04/01/2020). https://matter.childmind.org/face-guardian.html

[5]

2020. Custom Vision: An AI service and end-to-end platform for applying computer vision to your specific scenario. (2020). https://azure.microsoft.com/en- us/services/cognitive-services/custom-vision-service/

[6]

April 2, 2020. How YOU Can Use Computer Vision to Avoid Touching Your Face! (April 2, 2020). https://medium.com/microsoftazure/how-you-can-use-computer-vision-to-avoid-touching-your-face-34a426ffddfd

[7]

Nathan H Azrin, R Gregg Nunn, and SE Frantz. 1980. Treatment of hairpulling (trichotillomania): a comparative study of habit reversal and negative practice training. Journal of Behavior Therapy and Experimental Psychiatry 11, 1 (1980), 13--20.

[8]

Felix Barroso, Norbert Freedman, and Stanley Grand. 1980. Self-touching, performance, and attentional processes. Perceptual and Motor Skills 50, 3_suppl (1980), 1083--1089.

[9]

Mikhail Belkin, Irina Matveeva, and Partha Niyogi. 2004. Regularization and Semi-supervised Learning on Large Graphs. In Learning Theory, John Shawe-Taylor and Yoram Singer (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 624--638.

[10]

Guillermo Bernal, Tao Yang, Abhinandan Jain, and Pattie Maes. 2018. PhysioHMD: a conformable, modular toolkit for collecting physiological data from head-mounted displays. In Proceedings of the 2018 ACM International Symposium on Wearable Computers (ISWC '18). Association for Computing Machinery, New York, NY, USA, 160--167. https://doi.org/10.1145/3267242.3267268

Digital Library

[11]

David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin Raffel. 2019. Mixmatch: A holistic approach to semi-supervised learning. arXiv preprint arXiv:1905.02249 (2019).

[12]

Cigdem Beyan, Matteo Bustreo, Muhammad Shahid, Gian Luca Bailo, Nicolo Carissimi, and Alessio Del Bue. 2020. Analysis of Face-Touching Behavior in Large Scale Social Interaction Dataset. In Proceedings of the 2020 International Conference on Multimodal Interaction (ICMI '20). Association for Computing Machinery, New York, NY, USA, 24--32. https://doi.org/10.1145/3382507.3418876

Digital Library

[13]

Avrim Blum and Tom Mitchell. 1998. Combining Labeled and Unlabeled Data with Co-Training. In Proceedings of the Eleventh Annual Conference on Computational Learning Theory (COLT' 98). Association for Computing Machinery, New York, NY, USA, 92--100. https://doi.org/10.1145/279943.279962

Digital Library

[14]

John R Buck, Michael M Daniel, and Andrew C Singer. 1997. Computer explorations in signals and systems using MATLAB. Prentice-Hall, Inc.

[15]

Xiang 'Anthony' Chen. 2020. FaceOff: Detecting Face Touching with a Wrist-Worn Accelerometer. (2020). arXiv:cs.HC/2008.01769

[16]

Yinbo Chen, Xiaolong Wang, Zhuang Liu, Huijuan Xu, and Trevor Darrell. 2020. A new meta-baseline for few-shot learning. arXiv preprint arXiv:2003.04390 (2020).

[17]

Chip. April 10, 2014. OpenBCI: Measuring Electrode Impedance. (April 10, 2014). http://eeghacker.blogspot.com/2014/04/openbci-measuring-electrode-impedance.html

[18]

Youngjun Cho, Nadia Bianchi-Berthouze, Nicolai Marquardt, and Simon J. Julier. 2018. Deep Thermal Imaging: Proximate Material Type Recognition in the Wild through Deep Learning of Spatial Surface Temperature Patterns. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). Association for Computing Machinery, New York, NY, USA, 1--13. https://doi.org/10.1145/3173574.3173576

[19]

François Chollet. 2017. Xception: Deep Learning with Depthwise Separable Convolutions. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1800--1807. https://doi.org/10.1109/CVPR.2017.195

[20]

EJ Clar, CP Her, and CG Sturelle. 1975. Skin impedance and moisturization. J Soc Cosmet Chem 26 (1975), 337--353.

[21]

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 (2019).

[22]

N. H. Dardas and N. D. Georganas. 2011. Real-Time Hand Gesture Detection and Recognition Using Bag-of-Features and Support Vector Machine Techniques. IEEE Transactions on Instrumentation and Measurement 60, 11 (2011), 3592--3607. https://doi.org/10.1109/TIM.2011.2161140

[23]

Scott Day. 2002. Important factors in surface EMG measurement. Bortec Biomedical Ltd publishers (2002), 1--17.

[24]

Kyra Densing, Hippokrates Konstantinidis, and Melanie Seiler. 2018. Effect of stress level on different forms of self-touch in pre-and postadolescent girls. Journal of motor behavior 50, 5 (2018), 475--485.

[25]

N. D'Aurizio, T. L. Baldi, G. Paolocci, and D. Prattichizzo. 2020. Preventing Undesired Face-Touches With Wearable Devices and Haptic Feedback. IEEE Access 8 (2020), 139033--139043. https://doi.org/10.1109/ACCESS.2020.3012309

[26]

Paul Ekman and Wallace V Friesen. 1972. Hand movements. Journal of communication 22, 4 (1972), 353--374.

[27]

Paul Ekman, Joseph C Hager, and Wallace V Friesen. 1981. The symmetry of emotional and deliberate facial actions. Psychophysiology 18, 2 (1981), 101--106.

[28]

Nir Friedman and Stuart Russell. 2013. Image Segmentation in Video Sequences: A Probabilistic Approach. (2013). arXiv:cs.CV/1302.1539

[29]

Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-Adversarial Training of Neural Networks. 17, 1 (Jan. 2016), 2096--2030.

[30]

Norm Gitis and Raja Sivamani. 2004. Tribometrology of skin. Tribology Transactions 47, 4 (2004), 461--469.

[31]

Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep Sparse Rectifier Neural Networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research), Geoffrey Gordon, David Dunson, and Miroslav Dudík (Eds.), Vol. 15. PMLR, Fort Lauderdale, FL, USA, 315--323. http://proceedings.mlr.press/v15/glorot11a.html

[32]

Jun Gong, Yang Zhang, Xia Zhou, and Xing-Dong Yang. 2017. Pyro: Thumb-Tip Gesture Recognition Using Pyroelectric Infrared Sensing. In Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology (UIST '17). Association for Computing Machinery, New York, NY, USA, 553--563. https://doi.org/10.1145/3126594.3126615

Digital Library

[33]

T. Guo, C. P. Huynh, and M. Solh. 2019. Domain-Adaptive Pedestrian Detection in Thermal Images. In 2019 IEEE International Conference on Image Processing (ICIP). 1660--1664. https://doi.org/10.1109/ICIP.2019.8803104

[34]

Jinni A Harrigan, John R Kues, John J Steffen, and Robert Rosenthal. 1987. Self-touching and impressions of others. Personality and Social Psychology Bulletin 13, 4 (1987), 497--512.

[35]

Chad Hart. March 19, 2020. webrtcH4cKS: Stop touching your face using a browser and TensorFlow.js. (March 19, 2020). https://webrtchacks.com/stop-touching-your-face-with-browser-tensorflow-js/

[36]

Yoram Harth and Daniel Lischinsky. 2011. A novel method for real-time skin impedance measurement during radiofrequency skin tightening treatments: Multisource radiofrequency skin tightening and skin impedance. Journal of Cosmetic Dermatology 10, 1 (March 2011), 24--29. https://doi.org/10.1111/j.1473-2165.2010.00535.x

[37]

Megan R Heinicke, Jordan T Stiede, Raymond G Miltenberger, and Douglas W Woods. 2020. Reducing risky behavior with habit reversal: A review of behavioral strategies to reduce habitual hand-to-head behavior. Journal of applied behavior analysis 53, 3 (2020), 1225--1236.

[38]

Joseph A. Himle, Deborah Bybee, Lisa A. O'Donnell, Addie Weaver, Sarah Vlnka, Daniel T. DeSena, and Jessica M. Rimer. 2018. Awareness enhancing and monitoring device plus habit reversal in the treatment of trichotillomania: An open feasibility trial. Journal of Obsessive-Compulsive and Related Disorders 16 (2018), 14--20. https://doi.org/10.1016/j.jocrd.2017.10.007

[39]

Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. 2020. Meta-learning in neural networks: A survey. arXiv preprint arXiv:2004.05439 (2020).

[40]

M. Shamim Hossain, Syed Umar Amin, Mansour Alsulaiman, and Ghulam Muhammad. 2019. Applying Deep Learning for Epilepsy Seizure Detection and Brain Mapping Visualization. ACM Transactions on Multimedia Computing, Communications, and Applications 15, 1s (Feb. 2019), 1--17. https://doi.org/10.1145/3241056

Digital Library

[41]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger (Eds.), Vol. 25. Curran Associates, Inc., 1097--1105. https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

[42]

Maurice H. Krout. 1954. An Experimental Attempt to Produce Unconscious Manual Symbolic Movements. The Journal of General Psychology 51, 1 (1954), 93--120. https://doi.org/10.1080/00221309.1954.9920209 arXiv:https://doi.org/10.1080/00221309.1954.9920209

[43]

Yen Lee Angela Kwok, Jan Gralton, and Mary-Louise McLaws. 2015. Face touching: a frequent habit that has implications for hand hygiene. American journal of infection control 43, 2 (2015), 112--114.

[44]

O. Köpüklü, A. Gunduz, N. Kose, and G. Rigoll. 2019. Real-time Hand Gesture Detection and Classification Using Convolutional Neural Networks. In 2019 14th IEEE International Conference on Automatic Face Gesture Recognition (FG 2019). 1--8. https://doi.org/10.1109/FG.2019.8756576

[45]

Zakareya Lasefr, Ramasani Rakesh Reddy, and Khaled Elleithy. 2017. Smart phone application development for monitoring epilepsy seizure detection based on EEG signal classification. In 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON). IEEE, New York City, NY, 83--87. https://doi.org/10.1109/UEMCON.2017.8248992

[46]

Vernon J Lawhern, Amelia J Solon, Nicholas R Waytowich, Stephen M Gordon, Chou P Hung, and Brent J Lance. 2018. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces. Journal of Neural Engineering 15, 5 (jul 2018), 056013. https://doi.org/10.1088/1741-2552/aace8c

[47]

James F Leckman, Dorothy E Grice, James Boardman, Heping Zhang, Amy Vitale, Colin Bondi, John Alsobrook, Bradley S Peterson, Donald J Cohen, Steven A Rasmussen, et al. 1997. Symptoms of obsessive-compulsive disorder. American Journal of Psychiatry 154, 7 (1997), 911--917.

[48]

Yann LeCun and Yoshua Bengio. 1998. Convolutional Networks for Images, Speech, and Time Series. MIT Press, Cambridge, MA, USA, 255--258.

[49]

Jiseon Lee, Junhee Park, Sejung Yang, Hani Kim, Yun Seo Choi, Hyeon Jin Kim, Hyang Woon Lee, and Byung-Uk Lee. 2017. Early Seizure Detection by Applying Frequency-Based Algorithm Derived from the Principal Component Analysis. 11 (2017), 52. https://doi.org/10.3389/fninf.2017.00052

[50]

Haili Liu, Ya Wang, Kevin Wang, and Hanbing Lin. 2017. Turning a pyroelectric infrared motion sensor into a high-accuracy presence detector by using a narrow semi-transparent chopper. Applied Physics Letters 111, 24 (2017), 243901. https://doi.org/10.1063/1.4998430 arXiv:https://doi.org/10.1063/1.4998430

[51]

Xuefeng Liu, Tianye Yang, Shaojie Tang, Peng Guo, and Jianwei Niu. 2020. From Relative Azimuth to Absolute Location: Pushing the Limit of PIR Sensor Based Localization. In Proc. of MobiCom. Association for Computing Machinery, New York, NY, USA, Article 1, 14 pages. https://doi.org/10.1145/3372224.3380878

Digital Library

[52]

Christine Lochner, Annerine Roos, and Dan J Stein. 2017. Excoriation (skin-picking) disorder: a systematic review of treatment options. Neuropsychiatric disease and treatment 13 (2017), 1867.

[53]

Melika Lotfi, Michael R Hamblin, and Nima Rezaei. 2020. COVID-19: Transmission, prevention, and potential therapeutic opportunities. Clinica chimica acta (2020).

[54]

Fei Lu, Chenshuo Wang, Rongjian Zhao, Lidong Du, Zhen Fang, Xiuhua Guo, and Zhan Zhao. 2018. Review of stratum corneum impedance measurement in non-invasive penetration application. Biosensors 8, 2 (2018), 31.

[55]

Charles W McMonnies. 2008. Management of chronic habits of abnormal eye rubbing. Contact Lens and Anterior Eye 31, 2 (2008), 95--102.

[56]

Allan Michael Michelin, Georgios Korres, Sara Ba'ara, Hadi Assadi, Haneen Alsuradi, Rony R Sayegh, Antonis Argyros, and Mohamad Eid. 2021. FaceGuard: A Wearable System To Avoid Face Touching. Frontiers in Robotics and AI 8 (2021), 47.

[57]

Inc. 2021 Monsoon Solutions. 2021. High Voltage Power Monitor. (2021). https://www.msoon.com/high-voltage-power-monitor

[58]

Stephanie Margarete Mueller, Sven Martin, and Martin Grunwald. 2019. Self-touch: contact durations and point of touch of spontaneous facial self-touches differ depending on cognitive and emotional load. PloS one 14, 3 (2019), e0213677.

[59]

Raphael Mun. July 13, 2020. Face Touch Detection with TensorFlow.js Part 1: Using Real-Time Webcam Data With Deep Learning. (July 13, 2020). https://www.codeproject.com/Articles/5272773/Face-Touch-Detection-with-TensorFlow-js-Part-1-Usi

[60]

Katrin Müller, Stephanie Fröhlich, Andresa M. C. Germano, Jyothsna Kondragunta, Maria Fernanda del Carmen Agoitia Hurtado, Julian Rudisch, Daniel Schmidt, Gangolf Hirtz, Peter Stollmann, and Claudia Voelcker-Rehage. 2020. Sensor-based systems for early detection of dementia (SENDA): a study protocol for a prospective cohort sequential study. BMC Neurology 20, 1 (Dec. 2020), 84. https://doi.org/10.1186/s12883-020-01666-8

[61]

Phuc Nguyen, Nam Bui, Anh Nguyen, Hoang Truong, Abhijit Suresh, Matt Whitlock, Duy Pham, Thang Dinh, and Tam Vu. 2018. TYTH-Typing On Your Teeth: Tongue-Teeth Localization for Human-Computer Interface. In Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services. ACM, Munich Germany, 269--282. https://doi.org/10.1145/3210240.3210322

Digital Library

[62]

Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 8024--8035. http://papers.neurips.cc/paper/9015-pytorch-an-imperative-style-high-performance-deep-learning-library.pdf

Digital Library

[63]

Xiachuan Pei, Hao Jin, Shurong Dong, Dong Lou, Lie Ma, Xingang Wang, Weiwei Cheng, and Hei Wong. 2019. Flexible wireless skin impedance sensing system for wound healing assessment. Vacuum 168 (2019), 108808.

[64]

Valentin Radu, Catherine Tong, Sourav Bhattacharya, Nicholas D. Lane, Cecilia Mascolo, Mahesh K. Marina, and Fahim Kawsar. 2018. Multimodal Deep Learning for Activity and Context Recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 157 (Jan. 2018), 27 pages. https://doi.org/10.1145/3161174

Digital Library

[65]

Rajiv Ranganathan, Rui Wang, Rani Gebara, and Subir Biswas. 2017. Detecting Compensatory Trunk Movements in Stroke Survivors using a Wearable System. In Proceedings of the 2017 Workshop on Wearable Systems and Applications. ACM, Niagara Falls New York USA, 29--32. https://doi.org/10.1145/3089351.3089353

Digital Library

[66]

M. Raza, M. Awais, W. Ellahi, N. Aslam, H.X. Nguyen, and H. Le-Minh. 2019. Diagnosis and monitoring of Alzheimer's patients using classical and deep learning techniques. Expert Systems with Applications 136 (Dec. 2019), 353--364. https://doi.org/10.1016/j.eswa.2019.06.038

[67]

C. D. Rodin, L. N. de Lima, F. A. de Alcantara Andrade, D. B. Haddad, T. A. Johansen, and R. Storvold. 2018. Object Classification in Thermal Images using Convolutional Neural Networks for Search and Rescue Missions with Unmanned Aerial Systems. In 2018 International Joint Conference on Neural Networks (IJCNN). 1--8. https://doi.org/10.1109/IJCNN.2018.8489465

[68]

Camilo Rojas, Niels Poulsen, Mileva Van Tuyl, Daniel Vargas, Zipporah Cohen, Joe Paradiso, Pattie Maes, Kevin Esvelt, and Fadel Adib. 2021. A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-Based Pathogens. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 5, 1, Article 31 (March 2021), 22 pages. https://doi.org/10.1145/3448121

Digital Library

[69]

Steven Schlosser, Donald W Black, Nancee Blum, and Rise B Goldstein. 1994. The demography, phenomenology, and family history of 22 persons with compulsive hair pulling. Annals of Clinical Psychiatry 6, 3 (1994), 147--152.

[70]

Md. Haidar Sharif. 2021. Laser-Based Algorithms Meeting Privacy in Surveillance: A Survey. IEEE Access 9 (2021), 92394--92419. https://doi.org/10.1109/ACCESS.2021.3092687

[71]

Ramin Shiraly, Zahra Shayan, and Mary-Louise McLaws. 2020. Face touching in the time of COVID-19 in Shiraz, Iran. American Journal of Infection Control 48, 12 (2020), 1559 - 1561. https://doi.org/10.1016/j.ajic.2020.08.009

[72]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. (2015). arXiv:cs.CV/1409.1556

[73]

Kihyuk Sohn, David Berthelot, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D Cubuk, Alex Kurakin, Han Zhang, and Colin Raffel. 2020. Fixmatch: Simplifying semi-supervised learning with consistency and confidence. arXiv preprint arXiv:2001.07685 (2020).

[74]

Melinda A. Stanley and Suzanne G. Mouton. 1996. Trichotillomania Treatment Manual. Springer US, Boston, MA, 657--687. https://doi.org/10.1007/978-1-4899-1528-3_17

[75]

Tanuja Subba and Tejbanta Singh Chingtham. 2019. A Survey: EMG Signal-Based Controller for Human-Computer Interaction. In Advances in Communication, Cloud, and Big Data (Lecture Notes in Networks and Systems), Hiren Kumar Deva Sarma, Samarjeet Borah, and Nitul Dutta (Eds.). Springer, Singapore, 117--125. https://doi.org/10.1007/978-981-10-8911-4_13

[76]

Bahareh Taji, Adrian DC Chan, and Shervin Shirmohammadi. 2018. Effect of pressure on skin-electrode impedance in wearable biomedical measurement devices. IEEE Transactions on Instrumentation and Measurement 67, 8 (2018), 1900--1912.

[77]

Kaat Vandecasteele, Thomas De Cooman, Jonathan Dan, Evy Cleeren, Sabine Van Huffel, Borbála Hunyadi, and Wim Van Paesschen. 2020. Visual seizure annotation and automated seizure detection using behind-the-ear electroencephalographic channels. Epilepsia 61, 4 (April 2020), 766--775. https://doi.org/10.1111/epi.16470

[78]

A.T. Vehkaoja, J.A. Verho, M.M. Puurtinen, N.M. Nojd, J.O. Lekkala, and J.A. Hyttinen. 2005. Wireless Head Cap for EOG and Facial EMG Measurements. In 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference. 5865--5868. https://doi.org/10.1109/IEMBS.2005.1615824

[79]

Alexander von Luhmann, Jessica Addesa, Sourav Chandra, Abhijit Das, Mitsuhiro Hayashibe, and Anirban Dutta. 2017. Neural interfacing non-invasive brain stimulation with NIRS-EEG joint imaging for closed-loop control of neuroenergetics in ischemic stroke. In 2017 8th International IEEE/EMBS Conference on Neural Engineering (NER). IEEE, Shanghai, China, 349--353. https://doi.org/10.1109/NER.2017.8008362

[80]

Kyle B Walsh. 2019. Non-invasive sensor technology for prehospital stroke diagnosis: Current status and future directions. International Journal of Stroke 14, 6 (Aug. 2019), 592--602. https://doi.org/10.1177/1747493019866621

[81]

W. Wang, J. Zhang, and C. Shen. 2010. Improved human detection and classification in thermal images. In 2010 IEEE International Conference on Image Processing. 2313--2316. https://doi.org/10.1109/ICIP.2010.5649946

[82]

John G Webster. 1984. Reducing motion artifacts and interference in biopotential recording. IEEE transactions on biomedical engineering 12 (1984), 823--826.

[83]

Wikipedia contributors. 2021. Room temperature --- Wikipedia, The Free Encyclopedia. (2021). https://en.wikipedia.org/w/index.php?title=Room_temperature&oldid=1016698070 [Online; accessed 5-May-2021].

[84]

C. R. Wren, A. Azarbayejani, T. Darrell, and A. P. Pentland. 1997. Pfinder: real-time tracking of the human body. IEEE Transactions on Pattern Analysis and Machine Intelligence 19, 7 (1997), 780--785. https://doi.org/10.1109/34.598236

Digital Library

[85]

Chaojun Xie, Hongjun Zhao, Kuibiao Li, Zhoubin Zhang, Xiaoxiao Lu, Huide Peng, Dahu Wang, Jin Chen, Xiao Zhang, Di Wu, et al. 2020. The evidence of indirect transmission of SARS-CoV-2 reported in Guangzhou, China. BMC public health 20, 1 (2020), 1--9.

[86]

Hongfei Xue, Wenjun Jiang, Chenglin Miao, Fenglong Ma, Shiyang Wang, Ye Yuan, Shuochao Yao, Aidong Zhang, and Lu Su. 2020. DeepMV: Multi-View Deep Learning for Device-Free Human Activity Recognition. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 1, Article 34 (March 2020), 26 pages. https://doi.org/10.1145/3380980

Digital Library

[87]

D. Yang, W. Sheng, and R. Zeng. 2015. Indoor human localization using PIR sensors and accessibility map. In 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems (CYBER). 577--581. https://doi.org/10.1109/CYBER.2015.7288004

[88]

Uldis Zarins. 2018. Anatomy of Facial Expressions. Exonicus, Incorporated.

[89]

Fan Zhang, Tongyu Jin, Qingqing Hu, and Pingang He. 2018. Distinguishing skin cancer cells and normal cells using electrical impedance spectroscopy. Journal of Electroanalytical Chemistry 823 (Aug. 2018), 531--536. https://doi.org/10.1016/j.jelechem.2018.06.021

[90]

Junbo Zhang and Swarun Kumar. 2020. NoFaceContact: Stop Touching Your Face with NFC. In Proceedings of the 18th International Conference on Mobile Systems, Applications, and Services (MobiSys '20). Association for Computing Machinery, New York, NY, USA, 468--469. https://doi.org/10.1145/3386901.3396603

Digital Library

[91]

camilorq Zhi Wei Gan. 2020. Saving Face App. https://github.com/camilorq/SavingFaceApp. (2020).

[92]

Z. Zivkovic. 2004. Improved adaptive Gaussian mixture model for background subtraction. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., Vol. 2. 28-31 Vol.2. https://doi.org/10.1109/ICPR.2004.1333992

Cited By

Ibrahim MLyons T(2024)FaceTouch: Detecting hand-to-face touch with supervised contrastive learning to assist in tracing infectious diseasesPLOS ONE10.1371/journal.pone.028867019:6(e0288670)Online publication date: 13-Jun-2024
https://doi.org/10.1371/journal.pone.0288670
Liao ZJin MAn SNiu CWu FDeng TChen G(2024)Waving Hand as Infrared Source for Ubiquitous Gas SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596058:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659605
Shao QOkoshi TKo JLiKamWa R(2024)Weaving Physical and Physiological Sensing with Computational FabricsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661385(748-750)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661385
Show More Cited By

Index Terms

FaceSense: Sensing Face Touch with an Ear-worn System
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Embedded systems
2. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing systems and tools

Recommendations

Emotion Recognition Using Multimodal Deep Learning
Proceedings of the 23rd International Conference on Neural Information Processing - Volume 9948

To enhance the performance of affective models and reduce the cost of acquiring physiological signals for real-world applications, we adopt multimodal deep learning approach to construct affective models with SEED and DEAP datasets to recognize ...
Read More
Multimodal Deep Convolutional Neural Network for Audio-Visual Emotion Recognition
ICMR '16: Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval

Emotion recognition is a challenging task because of the emotional gap between subjective emotion and the low-level audio-visual features. Inspired by the recent success of deep learning in bridging the semantic gap, this paper proposes to bridge the ...
Read More
Modeling multimodal cues in a deep learning-based framework for emotion recognition in the wild
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal Interaction

In this paper, we propose a multimodal deep learning architecture for emotion recognition in video regarding our participation to the audio-video based sub-challenge of the Emotion Recognition in the Wild 2017 challenge. Our model combines cues from ...
Read More

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 5, Issue 3

Sept 2021

1443 pages

EISSN:2474-9567

DOI:10.1145/3486621

Issue’s Table of Contents

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2021

Published in IMWUT Volume 5, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

NSF
Alfred P. Sloan Research Fellowship

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
1,428
Total Downloads

Downloads (Last 12 months)383
Downloads (Last 6 weeks)54

Other Metrics

View Author Metrics

Citations

Cited By

Ibrahim MLyons T(2024)FaceTouch: Detecting hand-to-face touch with supervised contrastive learning to assist in tracing infectious diseasesPLOS ONE10.1371/journal.pone.028867019:6(e0288670)Online publication date: 13-Jun-2024
https://doi.org/10.1371/journal.pone.0288670
Liao ZJin MAn SNiu CWu FDeng TChen G(2024)Waving Hand as Infrared Source for Ubiquitous Gas SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36596058:2(1-26)Online publication date: 15-May-2024
https://dl.acm.org/doi/10.1145/3659605
Shao QOkoshi TKo JLiKamWa R(2024)Weaving Physical and Physiological Sensing with Computational FabricsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661385(748-750)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661385
Hu ZRadmehr AZhang YPan SNguyen P(2024)IOTeethProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435168:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643516
Shimon SNeshati ASun JXu QZhao J(2024)Exploring Uni-manual Around Ear Off-Device Gestures for EarablesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435138:1(1-29)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643513
Duan DChen YXu WLi T(2024)EarSEProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314477:4(1-33)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631447
Yang YChen THuang YGuo XShangguan L(2024)MAF: Exploring Mobile Acoustic Field for Hand-to-Face Gesture InteractionsProceedings of the CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642437(1-20)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642437
Nirmal IKhamis AHassan MHu WLi RKalyanaraman A(2024)WiFi2Radar: Orientation-Independent Single-Receiver WiFi Sensing via WiFi to Radar TranslationIEEE Internet of Things Journal10.1109/JIOT.2023.334937811:9(15750-15766)Online publication date: 1-May-2024
https://doi.org/10.1109/JIOT.2023.3349378
Zhao CChen DWu QWang WZhao ZLiu X(2023)Application of Anti-wear Technology for Water-cooled Wall of CFB BoilersHighlights in Science, Engineering and Technology10.54097/hset.v41i.682041(232-241)Online publication date: 30-Mar-2023
https://doi.org/10.54097/hset.v41i.6820
Pawlicki ŁFotyga ARewieński JGroth MKulas ŁFotyga G(2023)Wireless Body Area Network for Preventing Self-Inoculation Transmission of Respiratory Viral DiseasesSensors10.3390/s2304204223:4(2042)Online publication date: 11-Feb-2023
https://doi.org/10.3390/s23042042
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents