Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
skip to main content
research-article
Public Access

mTeeth: Identifying Brushing Teeth Surfaces Using Wrist-Worn Inertial Sensors

Published: 24 June 2021 Publication History

Abstract

Ensuring that all the teeth surfaces are adequately covered during daily brushing can reduce the risk of several oral diseases. In this paper, we propose the mTeeth model to detect teeth surfaces being brushed with a manual toothbrush in the natural free-living environment using wrist-worn inertial sensors. To unambiguously label sensor data corresponding to different surfaces and capture all transitions that last only milliseconds, we present a lightweight method to detect the micro-event of brushing strokes that cleanly demarcates transitions among brushing surfaces. Using features extracted from brushing strokes, we propose a Bayesian Ensemble method that leverages the natural hierarchy among teeth surfaces and patterns of transition among them. For training and testing, we enrich a publicly-available wrist-worn inertial sensor dataset collected from the natural environment with time-synchronized precise labels of brushing surface timings and moments of transition. We annotate 10,230 instances of brushing on different surfaces from 114 episodes and evaluate the impact of wide between-person and within-person between-episode variability on machine learning model's performance for brushing surface detection.

References

[1]
Accessed August, 2020. Bass tooth brushing technique. https://tube.medchrome.com/2013/04/brushing-technique-bass-and-modified.html
[2]
Accessed August, 2020. ELAN software. https://archive.mpi.nl/tla/elan/
[3]
Accessed August, 2020. Oral B Genius brush. https://oralb.com/en-us/products/electric-toothbrushes/genius-8000-rechargeable-electric-toothbrush/
[4]
Accessed August, 2020. Sonic-powered automatic toothbrush. https://bfbe9e.kckb.st
[5]
Sayma Akther, Nazir Saleheen, Shahin Alan Samiei, Vivek Shetty, Emre Ertin, and Santosh Kumar. 2019. mORAL: An mHealth model for inferring Oral Hygiene Behaviors in-the-wild using wrist-worn inertial sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 1 (2019), 1--25.
[6]
Miguel Bruns Alonso, Jelle Stienstra, and Rob Dijkstra. 2014. Brush and learn: transforming tooth brushing behavior through interactive materiality, a design exploration. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction. ACM, 113--120.
[7]
Thomas Attin and E Hornecker. 2005. Tooth brushing and oral health: how frequently and when should tooth brushing be performed? Oral Health & Preventive Dentistry 3, 3 (2005).
[8]
Barbara Bruno, Fulvio Mastrogiovanni, Antonio Sgorbissa, Tullio Vernazza, and Renato Zaccaria. 2012. Human motion modelling and recognition: A computational approach. In 2012 IEEE International Conference on Automation Science and Engineering (CASE 2012). IEEE, 156--161.
[9]
Yu-Chen Chang, Jin-Ling Lo, Chao-Ju Huang, Nan-Yi Hsu, Hao-Hua Chu, Hsin-Yen Wang, Pei-Yu Chi, and Ya-Lin Hsieh. 2008. Playful toothbrush: ubicomp technology for teaching tooth brushing to kindergarten children. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 363--372.
[10]
Renate Deinzer, Stefanie Ebel, Helen Blättermann, Ulrike Weik, and Jutta Margraf-Stiksrud. 2018. Toothbrushing: to the best of one's abilities is possibly not good enough. BMC Oral Health 18, 1 (2018), 1--7.
[11]
Christian Graetz, Jule Bielfeldt, Lars Wolff, Claudia Springer, Karim M Fawzy El-Sayed, Sonja Sälzer, Sabah Badri-Höher, and Christof E Dörfer. 2013. Toothbrushing education via a smart software visualization system. Journal of Periodontology 84, 2 (2013), 186--195.
[12]
Taku Hachisu and Hiroyuki Kajimoto. 2015. Modulating tooth brushing sounds to affect user impressions. International Journal of Arts and Technology 8, 4 (2015), 307--324.
[13]
Takashi Hamatani, Moustafa Elhamshary, Akira Uchiyama, and Teruo Higashino. 2018. FluidMeter: Gauging the human daily fluid intake using smartwatches. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1--25.
[14]
Peng-Ju Hsieh, Yen-Liang Lin, Yu-Hsiu Chen, and Winston Hsu. 2016. Egocentric activity recognition by leveraging multiple mid-level representations. In Multimedia and Expo (ICME), International Conference on. IEEE, 1--6.
[15]
Hua Huang and Shan Lin. 2016. Toothbrushing monitoring using wrist watch. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems. ACM, 202--215.
[16]
Kee-Deog Kim, Jin-Sun Jeong, Hae Na Lee, Yu Gu, Kyeong-Seop Kim, Jeong-Whan Lee, and Wonse Park. 2015. Efficacy of computer-assisted, 3D motion-capture toothbrushing instruction. Clinical Oral Investigations 19, 6 (2015), 1389--1394.
[17]
Kyeong-Seop Kim, Tae-Ho Yoon, Jeong-Whan Lee, and Dong-Jun Kim. 2009. Interactive toothbrushing education by a smart toothbrush system via 3D visualization. Computer Methods and Programs in Biomedicine 96, 2 (2009), 125--132.
[18]
Joseph Korpela, Ryosuke Miyaji, Takuya Maekawa, Kazunori Nozaki, and Hiroo Tamagawa. 2015. Evaluating tooth brushing performance with smartphone sound data. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 109--120.
[19]
Joseph Korpela, Ryosuke Miyaji, Takuya Maekawa, Kazunori Nozaki, and Hiroo Tamagawa. 2016. Toothbrushing performance evaluation using smartphone audio based on hybrid HMM-recognition/SVM-regression model. Journal of Information Processing 24, 2 (2016), 302--313.
[20]
Kang-Hwi Lee, Jeong-Whan Lee, Kyeong-Seop Kim, Dong-Jun Kim, Kyungho Kim, Heui-Kyung Yang, Keesam Jeong, and Byungchae Lee. 2007. Tooth brushing pattern classification using three-axis accelerometer and magnetic sensor for smart toothbrush. In 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE, 4211--4214.
[21]
Nahyeon Lee, Doyoung Jang, Yeji Kim, Byung-Cull Bae, and Jun-Dong Cho. 2016. Denteach: A device for fostering children's good tooth-brushing habits. In Proceedings of the The 15th International Conference on Interaction Design and Children. ACM, 619--624.
[22]
Young-Jae Lee, Pil-Jae Lee, Kyeong-Seop Kim, Wonse Park, Kee-Deog Kim, Dosik Hwang, and Jeong-Whan Lee. 2011. Toothbrushing region detection using three-axis accelerometer and magnetic sensor. IEEE Transactions on Biomedical Engineering 59, 3 (2011), 872--881.
[23]
Young-Jae Lee, Pil-Jae Lee, Jeong-Whan Lee, Kyeong-Seop Kim, Jin-Sun Jeong, Wonse Park, and Kee-Deog Kim. 2011. Quantitative assessment of toothbrushing education efficacy using smart toothbrush. In 2011 4th International Conference on Biomedical Engineering and Informatics (BMEI), Vol. 2. IEEE, 1160--1164.
[24]
Hong Li, Shishir Chawla, Richard Li, Sumeet Jain, Gregory D Abowd, Thad Starner, Cheng Zhang, and Thomas Plötz. 2018. Wristwash: towards automatic handwashing assessment using a wrist-worn device. In Proceedings of the 2018 ACM International Symposium on Wearable Computers. ACM, 132--139.
[25]
Yuan Liang, Hsuan Wei Fan, Zhujun Fang, Leiying Miao, Wen Li, Xuan Zhang, Weibin Sun, Kun Wang, Lei He, and Xiang'Anthony' Chen. 2020. OralCam: Enabling Self-Examination and Awareness of Oral Health Using a Smartphone Camera. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1--13.
[26]
Harald Löe. 2000. Oral hygiene in the prevention of caries and periodontal disease. International Dental Journal 50, 3 (2000), 129--139.
[27]
Chengwen Luo, Xingyu Feng, Junliang Chen, Jianqiang Li, Weitao Xu, Wei Li, Li Zhang, Zahir Tari, and Albert Y Zomaya. 2019. Brush like a Dentist: Accurate Monitoring of Toothbrushing via Wrist-Worn Gesture Sensing. In IEEE INFOCOM 2019. IEEE, 1234--1242.
[28]
Marco Marcon, Augusto Sarti, and Stefano Tubaro. 2016. Toothbrush motion analysis to help children learn proper tooth brushing. Computer Vision and Image Understanding 148 (2016), 34--45.
[29]
Tatsuo Nakajima, Vili Lehdonvirta, Eiji Tokunaga, and Hiroaki Kimura. 2008. Reflecting human behavior to motivate desirable lifestyle. In Proceedings of the 7th ACM conference on Designing interactive systems. ACM, 405--414.
[30]
Zhenchao Ouyang, Jingfeng Hu, Jianwei Niu, and Zhiping Qi. 2017. An asymmetrical acoustic field detection system for daily tooth brushing monitoring. In IEEE Global Communications Conference. IEEE, 1--6.
[31]
Abhinav Parate, Meng-Chieh Chiu, Chaniel Chadowitz, Deepak Ganesan, and Evangelos Kalogerakis. 2014. Risq: Recognizing smoking gestures with inertial sensors on a wristband. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services. 149--161.
[32]
Nazir Saleheen, Amin Ahsan Ali, Syed Monowar Hossain, Hillol Sarker, Soujanya Chatterjee, Benjamin Marlin, Emre Ertin, Mustafa Al'Absi, and Santosh Kumar. 2015. puffMarker: a multi-sensor approach for pinpointing the timing of first lapse in smoking cessation. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 999--1010.
[33]
Edison Thomaz, Irfan Essa, and Gregory D Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1029--1040.
[34]
Lahiru NS Wijayasingha and Benny Lo. 2016. A wearable sensing framework for improving personal and oral hygiene for people with developmental disabilities. In 2016 IEEE Wireless Health (WH). IEEE, 1--7.
[35]
Tobias Winterfeld, N Schlueter, Daniela Harnacke, Jörg Illig, Jutta Margraf-Stiksrud, Renate Deinzer, and Carolina Ganss. 2015. Tooth-brushing and flossing behaviour in young adults---a video observation. Clinical Oral Investigations 19, 4 (2015), 851--858.
[36]
Takuma Yoshitani, Masa Ogata, and Koji Yatani. 2016. LumiO: a plaque-aware toothbrush. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 605--615.

Cited By

View all
  • (2024)ToMoBrush: Exploring Dental Health Sensing Using a Sonic ToothbrushProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785058:3(1-27)Online publication date: 9-Sep-2024
  • (2024)BrushBuds: Toothbrushing Tracking Using Earphone IMUsCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3680521(655-660)Online publication date: 5-Oct-2024
  • (2024)A Smartwatch-based Approach for Oral Health Monitoring using Deep LearningProceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3652037.3663912(343-350)Online publication date: 26-Jun-2024
  • Show More Cited By

Index Terms

  1. mTeeth: Identifying Brushing Teeth Surfaces Using Wrist-Worn Inertial Sensors

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
    Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 5, Issue 2
    June 2021
    932 pages
    EISSN:2474-9567
    DOI:10.1145/3472726
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 24 June 2021
    Published in IMWUT Volume 5, Issue 2

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. brushing detection
    2. flossing detection
    3. hand-to-mouth gestures
    4. mHealth

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)266
    • Downloads (Last 6 weeks)40
    Reflects downloads up to 09 Nov 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)ToMoBrush: Exploring Dental Health Sensing Using a Sonic ToothbrushProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785058:3(1-27)Online publication date: 9-Sep-2024
    • (2024)BrushBuds: Toothbrushing Tracking Using Earphone IMUsCompanion of the 2024 on ACM International Joint Conference on Pervasive and Ubiquitous Computing10.1145/3675094.3680521(655-660)Online publication date: 5-Oct-2024
    • (2024)A Smartwatch-based Approach for Oral Health Monitoring using Deep LearningProceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3652037.3663912(343-350)Online publication date: 26-Jun-2024
    • (2024)Poster: An Automated Method to Detect Tooth Brushing Activity with Smartwatch SensorsProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661417(676-677)Online publication date: 3-Jun-2024
    • (2024)Hygiea+: Toward Energy-Efficient and Highly Accurate Toothbrushing Monitoring via Wrist-Worn Gesture SensingIEEE Internet of Things Journal10.1109/JIOT.2024.343919411:20(32670-32686)Online publication date: 15-Oct-2024
    • (2024)Gesture Recognition Dynamics: Unveiling Video Patterns with Deep Learning2024 International Conference on Data Science and Network Security (ICDSNS)10.1109/ICDSNS62112.2024.10691103(1-7)Online publication date: 26-Jul-2024
    • (2023)Application of Anti-wear Technology for Water-cooled Wall of CFB BoilersHighlights in Science, Engineering and Technology10.54097/hset.v41i.682041(232-241)Online publication date: 30-Mar-2023
    • (2022)ActiveSense: A Novel Active Learning Framework for Human Activity Recognition2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops53856.2022.9767388(224-229)Online publication date: 21-Mar-2022
    • (2022)Stepping Into the Next Decade of Ubiquitous and Pervasive Computing: UbiComp and ISWC 2021IEEE Pervasive Computing10.1109/MPRV.2022.316006321:2(87-99)Online publication date: 1-Apr-2022
    • (2022)A Wearable-based Preventive Model To Promote Oral Health Through Personalized Notification2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)10.1109/EMBC48229.2022.9871128(4282-4285)Online publication date: 11-Jul-2022

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Get Access

    Login options

    Full Access

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media