research-article

Public Access

mORAL: An mHealth Model for Inferring Oral Hygiene Behaviors in-the-wild Using Wrist-worn Inertial Sensors

Authors:

Nazir Saleheen,

Shahin Alan Samiei,

Santosh KumarAuthors Info & Claims

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, Volume 3, Issue 1

Article No.: 1, Pages 1 - 25

https://doi.org/10.1145/3314388

Published: 29 March 2019 Publication History

Abstract

We address the open problem of reliably detecting oral health behaviors passively from wrist-worn inertial sensors. We present our model named mORAL (pronounced em oral) for detecting brushing and flossing behaviors, without the use of instrumented toothbrushes so that the model is applicable to brushing with still prevalent manual toothbrushes. We show that for detecting rare daily events such as toothbrushing, adopting a model that is based on identifying candidate windows based on events, rather than fixed-length timeblocks, leads to significantly higher performance. Trained and tested on 2,797 hours of sensor data collected over 192 days on 25 participants (using video annotations for ground truth labels), our brushing model achieves 100% median recall with a false positive rate of one event in every nine days of sensor wearing. The average error in estimating the start/end times of the detected event is 4.1% of the interval of the actual toothbrushing event.

References

[1]

Regina M Benjamin. 2010. Oral health: the silent epidemic. Public health reports 125, 2 (2010), 158--159.

[2]

Lal Gamze Bozgeyikli, Evren Can Bozgeyikli, and Andrew Raij. {n. d.}. Keep Brushing! Developing Healthy Oral Hygiene Habits in Young Children with an Interactive Toothbrush. ({n. d.}).

[3]

Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5--32.

Digital Library

[4]

Barbara Bruno, Fulvio Mastrogiovanni, and Antonio Sgorbissa. 2014. A public domain dataset for ADL recognition using wrist-placed accelerometers. In RO-MAN. 738--743.

[5]

Barbara Bruno, Fulvio Mastrogiovanni, Antonio Sgorbissa, Tullio Vernazza and Renato Zaccaria. 2012. Human motion modelling and recognition: A computational approach. In 2012 IEEE International Conference on Automation Science and Engineering (CASE 2012). IEEE, 156--161.

[6]

Barbara Bruno, Fulvio Mastrogiovanni, Antonio Sgorbissa, Tullio Vernazza, and Renato Zaccaria. 2013. Analysis of human behavior recognition algorithms based on acceleration data. In Robotics and Automation (ICRA), 2013 IEEE International Conference on. IEEE, 1602--1607.

[7]

Rich Caruana, Alexandru Niculescu-Mizil, Geoff Crew, and Alex Ksikes. 2004. Ensemble selection from libraries of models. In Proceedings of the twenty-first international conference on Machine learning. ACM, 18.

Digital Library

[8]

CDC et al. 2010. Oral health: Preventing cavities, gum disease, tooth loss, and oral cancers. At a glance 2011. Atlanta: Division of Oral Health, National Center for Chronic Disease Prevention and Health Promotion (2010).

[9]

Centers for Medicare Medicaid Services 2018. Centers for Medicare Medicaid Services. https://www.cms.gov/Research-Statistics-Data-and-Systems/Statistics-Trends-and-Reports/NationalHealthExpendData/NationalHealthAccountsProjected.html.

[10]

Barbara Lesley Chadwick, Deborah White, Deborah Lader, and Nigel Pitts. 2011. Preventive behaviour and risks to oral health-a report from the Adult Dental Health Survey 2009. (2011).

[11]

Yu-Chen Chang, Jin-Ling Lo, Chao-Ju Huang, Nan-Yi Hsu, Hao-Hua Chu, Hsin-Yen Wang, Pei-Yu Chi, and Ya-Lin Hsieh. 2008. Playful toothbrush: ubicomp technology for teaching tooth brushing to kindergarten children. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 363--372.

Digital Library

[12]

CHIIZ {n. d.}. Sonic-powered automatic toothbrush. https://bfbe9e.kckb.st.

[13]

Waltenegus Dargie. 2009. Analysis of Time and Frequency Domain Features of Accelerometer Measurements. In ICCCN, Vol. 9. 1--6.

Digital Library

[14]

Thomas G Dietterich. 2000. Ensemble methods in machine learning. In International workshop on multiple classifier systems. Springer, 1--15.

Digital Library

[15]

Katherine Ellis, Jacqueline Kerr, Suneeta Godbole, Gert Lanckriet, David Wing, and Simon Marshall. 2014. A random forest classifier for the prediction of energy expenditure and type of physical activity from wrist and hip accelerometers. Physiological measurement 35, 11 (2014), 2191.

[16]

Andrew Estabrooks, Taeho Jo, and Nathalie Japkowicz. 2004. A multiple resampling method for learning from imbalanced data sets. Computational intelligence 20, 1 (2004), 18--36.

[17]

Yoav Freund, Robert E Schapire, et al. 1996. Experiments with a new boosting algorithm. In Icml, Vol. 96. Citeseer, 148--156.

Digital Library

[18]

E Grossman, M Cronin, W Dembling, and H Proskin. 1996. A comparative clinical study of extrinsic tooth stain removal with two electric toothbrushes {Braun D7 and D9} and a manual brush. American journal of dentistry 9 (1996), S25--9.

[19]

Taku Hachisu and Hiroyuki Kajimoto. 2012. Augmentation of toothbrush by modulating sounds resulting from brushing. In Advances in Computer Entertainment. Springer, 31--43.

Digital Library

[20]

Taku Hachisu and Hiroyuki Kajimoto. 2015. Modulating tooth brushing sounds to affect user impressions. International Journal of Arts and Technology 8, 4 (2015), 307--324.

[21]

Mark Andrew Hall. 1999. Correlation-based feature selection for machine learning. (1999).

[22]

Haibo He and Edwardo A Garcia. 2008. Learning from imbalanced data. IEEE Transactions on Knowledge & Data Engineering 9 (2008), 1263--1284.

Digital Library

[23]

Timothy Hnat, Syed Monowar Hossain, Nasir Ali, Simona Carini, Tyson Condie, Ida Sim, Mani B. Srivastava, and Santosh Kumar. 2017. mCerebrum and Cerebral Cortex: A Real-time Collection, Analytic, and Intervention Platform for High-frequency Mobile Sensor Data. In AMIA 2017, American Medical Informatics Association Annual Symposium, Washington, DC, USA, November 4-8, 2017. http://knowledge.amia.org/65881-amia-1.3897810/t006-1.3900376/f006-1.3900377/2732068-1.3900423/2727701-1.3900420

[24]

Syed Monowar Hossain, Timothy Hnat, Nazir Saleheen, Nusrat Jahan Nasrin, Joseph Noor, Bo-Jhang Ho, Tyson Condie, Mani Srivastava, and Santosh Kumar. 2017. mCerebrum: A Mobile Sensing Software Platform for Development and Validation of Digital Biomarkers and Interventions. In Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems. ACM, 7.

Digital Library

[25]

Peng-Ju Hsieh, Yen-Liang Lin, Yu-Hsiu Chen, and Winston Hsu. 2016. Egocentric activity recognition by leveraging multiple mid-level representations. In Multimedia and Expo (ICME), 2016 IEEE International Conference on. IEEE, 1--6.

[26]

Hua Huang and Shan Lin. 2016. Toothbrushing monitoring using wrist watch. In Proceedings of the 14th ACM Conference on Embedded Network Sensor Systems CD-ROM. ACM, 202--215.

Digital Library

[27]

Shubham Jain, Carlo Borgiattino, Yanzhi Ren, Marco Gruteser, Yingying Chen, and Carla Fabiana Chiasserini. 2015. Lookup: Enabling pedestrian safety services via shoe sensing. In Proceedings of the 13th Annual International Conference on Mobile Systems, Applications, and Services. ACM, 257--271.

Digital Library

[28]

Kyeong-Seop Kim, Tae-Ho Yoon, Jeong-Whan Lee, and Dong-Jun Kim. 2009. Interactive toothbrushing education by a smart toothbrush system via 3D visualization. Computer methods and programs in biomedicine 96, 2 (2009), 125--132.

Digital Library

[29]

Joseph Korpela, Ryosuke Miyaji, Takuya Maekawa, Kazunori Nozaki, and Hiroo Tamagawa. 2015. Evaluating tooth brushing performance with smartphone sound data. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 109--120.

Digital Library

[30]

Joseph Korpela, Ryosuke Miyaji, Takuya Maekawa, Kazunori Nozaki, and Hiroo Tamagawa. 2016. Toothbrushing performance evaluation using smartphone audio based on hybrid HMM-recognition/SVM-regression model. Journal of Information Processing 24, 2 (2016), 302--313.

[31]

Sotiris B Kotsiantis, I Zaharakis, and P Pintelas. 2007. Supervised machine learning: A review of classification techniques. Emerging artificial intelligence applications in computer engineering 160 (2007), 3--24.

Digital Library

[32]

Young-Jae Lee, Pil-Jae Lee, Kyeong-Seop Kim, Wonse Park, Kee-Deog Kim, Dosik Hwang, and Jeong-Whan Lee. 2012. Toothbrushing region detection using three-axis accelerometer and magnetic sensor. IEEE Transactions on Biomedical Engineering 59, 3 (2012), 872--881.

[33]

Cheng-Yuan Li, Yen-Chang Chen, Wei-Ju Chen, Polly Huang, and Hao-hua Chu. 2013. Sensor-embedded teeth for oral activity recognition. In Proceedings of the 2013 international symposium on wearable computers. ACM, 41--44.

Digital Library

[34]

Andrea Mannini, Stephen S Intille, Mary Rosenberger, Angelo M Sabatini, and William Haskell. 2013. Activity recognition using a single accelerometer placed at the wrist or ankle. Medicine and science in sports and exercise 45, 11 (2013), 2193.

[35]

Marco Marcon, Augusto Sarti, and Stefano Tubaro. 2016. Toothbrush motion analysis to help children learn proper tooth brushing. Computer Vision and Image Understanding 148 (2016), 34--45.

Digital Library

[36]

Hyung Gi Min and Eun Tae Jeung. 2015. Complementary filter design for angle estimation using mems accelerometer and gyroscope. Department of Control and Instrumentation, Changwon National University, Changwon, Korea (2015), 641--773.

[37]

Tatsuo Nakajima, Vili Lehdonvirta, Eiji Tokunaga, Masaaki Ayabe, Hiroaki Kimura, and Yohei Okuda. 2007. Lifestyle ubiquitous gaming: making daily lives more plesurable. In Embedded and Real-Time Computing Systems and Applications, 2007. RTCSA 2007. 13th IEEE International Conference on. IEEE, 257--266.

Digital Library

[38]

Zhenchao Ouyang, Jingfeng Hu, Jianwei Niu, and Zhiping Qi. 2017. An Asymmetrical Acoustic Field Detection System for Daily Tooth Brushing Monitoring. In GLOBECOM 2017-2017 IEEE Global Communications Conference. IEEE, 1--6.

[39]

Abhinav Parate, Meng-Chieh Chiu, Chaniel Chadowitz, Deepak Ganesan, and Evangelos Kalogerakis. 2014. Risq: Recognizing smoking gestures with inertial sensors on a wristband. In Proceedings of the 12th annual international conference on Mobile systems, applications, and services. ACM, 149--161.

Digital Library

[40]

Toby G Pavey, Nicholas D Gilson, Sjaan R Gomersall, Bronwyn Clark, and Stewart G Trost. 2017. Field evaluation of a random forest activity classifier for wrist-worn accelerometer data. Journal of science and medicine in sport 20, 1 (2017), 75--80.

[41]

Mahbubur Rahman, Nasir Ali, Rummana Bari, Nazir Saleheen, Mustafa alâĂ&Zacute;bsi, Emre Ertin, Ashley Kennedy, Kenzie L Preston, and Santosh Kumar. 2017. mDebugger: Assessing and Diagnosing the Fidelity and Yield of Mobile Sensor Data. In Mobile Health. Springer, 121--143.

[42]

Tauhidur Rahman, Mary Czerwinski, Ran Gilad-Bachrach, and Paul Johns. 2016. Predicting about-to-eat moments for just-in-time eating intervention. In Proceedings of the 6th International Conference on Digital Health Conference. ACM, 141--150.

Digital Library

[43]

Lior Rokach. 2010. Ensemble-based classifiers. Artificial Intelligence Review 33, 1--2 (2010), 1--39.

Digital Library

[44]

Nazir Saleheen, Amin Ahsan Ali, Syed Monowar Hossain, Hillol Sarker, Soujanya Chatterjee, Benjamin Marlin, Emre Ertin, Mustafa Al'Absi, and Santosh Kumar. 2015. puffMarker: a multi-sensor approach for pinpointing the timing of first lapse in smoking cessation. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 999--1010.

Digital Library

[45]

Hillol Sarker, Moushumi Sharmin, Amin Ahsan Ali, Md Mahbubur Rahman, Rummana Bari, Syed Monowar Hossain, and Santosh Kumar. 2014. Assessing the availability of users to engage in just-in-time intervention in the natural environment. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 909--920.

Digital Library

[46]

Vivek Shetty, John Yamamoto, and Kenneth Yale. 2018. Re-architecting oral healthcare for the 21st century. Journal of dentistry 74 (2018), S10--S14.

[47]

Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. 2015. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. ACM, 127--140.

Digital Library

[48]

Edison Thomaz, Irfan Essa, and Gregory D Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1029--1040.

Digital Library

[49]

Adrian Yap. 2017. Oral health equals total health: A brief review. Journal of Dentistry Indonesia 24, 2 (2017), 59--62.

[50]

Takuma Yoshitani, Masa Ogata, and Koji Yatani. 2016. LumiO: a plaque-aware toothbrush. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 605--615.

Digital Library

[51]

Shaoyan Zhang, Alex V Rowlands, Peter Murray, Tina L Hurst, et al. 2012. Physical activity classification using the GENEA wrist-worn accelerometer. (2012).

Cited By

Casilari EGonzalez-Cañete F(2024)A Smartwatch-based Approach for Oral Health Monitoring using Deep LearningProceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3652037.3663912(343-350)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1145/3652037.3663912
Cui JWu DHe YOuyang Z(2024)Dentists who can auscultate: Microphone-based toothbrushing quality monitoring system for electronic toothbrushExpert Systems with Applications10.1016/j.eswa.2024.124817255(124817)Online publication date: Dec-2024
https://doi.org/10.1016/j.eswa.2024.124817
Jeyakumar JSarker AGarcia LSrivastava M(2023)X-CHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808047:1(1-28)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580804
Show More Cited By

Index Terms

mORAL: An mHealth Model for Inferring Oral Hygiene Behaviors in-the-wild Using Wrist-worn Inertial Sensors
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing design and evaluation methods

Recommendations

mTeeth: Identifying Brushing Teeth Surfaces Using Wrist-Worn Inertial Sensors

Ensuring that all the teeth surfaces are adequately covered during daily brushing can reduce the risk of several oral diseases. In this paper, we propose the mTeeth model to detect teeth surfaces being brushed with a manual toothbrush in the natural free-...
When generalized eating detection machine learning models fail in the field
UbiComp '17: Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers

Problematic eating behaviors are a major cause of obesity. To improve our understanding of these eating behaviors, we need to be able to first reliably detect them. In this paper we use a wrist-worn sensor to test a generalized machine learning models' ...
Alarm Notification of Body Sensors Utilising Activity Recognition and Smart Device Application
MoMM '16: Proceedings of the 14th International Conference on Advances in Mobile Computing and Multi Media

Smart device and body sensors have become innovative and converged with health informatics. To reduce the battery power consumption and bandwidth usage in wireless body area networks, it is proposed to infer sensed data in sensor nodes, which can raise ...

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies Volume 3, Issue 1

March 2019

786 pages

EISSN:2474-9567

DOI:10.1145/3323054

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 March 2019

Accepted: 01 January 2019

Revised: 01 November 2018

Received: 01 August 2018

Published in IMWUT Volume 3, Issue 1

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
1,337
Total Downloads

Downloads (Last 12 months)171
Downloads (Last 6 weeks)23

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Casilari EGonzalez-Cañete F(2024)A Smartwatch-based Approach for Oral Health Monitoring using Deep LearningProceedings of the 17th International Conference on PErvasive Technologies Related to Assistive Environments10.1145/3652037.3663912(343-350)Online publication date: 26-Jun-2024
https://dl.acm.org/doi/10.1145/3652037.3663912
Cui JWu DHe YOuyang Z(2024)Dentists who can auscultate: Microphone-based toothbrushing quality monitoring system for electronic toothbrushExpert Systems with Applications10.1016/j.eswa.2024.124817255(124817)Online publication date: Dec-2024
https://doi.org/10.1016/j.eswa.2024.124817
Jeyakumar JSarker AGarcia LSrivastava M(2023)X-CHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808047:1(1-28)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580804
Chen KWang LHuang YWu KWang LCosta XAl Hassanieh HAsadi ACox LPerino DWidmer JGiustiniano D(2023)LiT: Fine-grained Toothbrushing Monitoring with Commercial LED ToothbrushProceedings of the 29th Annual International Conference on Mobile Computing and Networking10.1145/3570361.3613287(1-16)Online publication date: 2-Oct-2023
https://dl.acm.org/doi/10.1145/3570361.3613287
Wang D(2023)Analysis of A Mahony Algorithm-Based System for Monitoring Swimming 3D Motion Data2023 IEEE International Conference on Integrated Circuits and Communication Systems (ICICACS)10.1109/ICICACS57338.2023.10100053(1-5)Online publication date: 24-Feb-2023
https://doi.org/10.1109/ICICACS57338.2023.10100053
Thurzo AUrbanová WNovák BCzako LSiebert TStano PMareková SFountoulaki GKosnáčová HVarga I(2022)Where Is the Artificial Intelligence Applied in Dentistry? Systematic Review and Literature AnalysisHealthcare10.3390/healthcare1007126910:7(1269)Online publication date: 8-Jul-2022
https://doi.org/10.3390/healthcare10071269
Nthubu B(2021)An Overview of Sensors, Design and Healthcare Challenges in Smart Homes: Future Design QuestionsHealthcare10.3390/healthcare91013299:10(1329)Online publication date: 5-Oct-2021
https://doi.org/10.3390/healthcare9101329
Adler DTseng VQi GScarpa JSen SChoudhury T(2021)Identifying Mobile Sensing Indicators of Stress-ResilienceProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34635285:2(1-32)Online publication date: 24-Jun-2021
https://dl.acm.org/doi/10.1145/3463528
Chang YZhao YDong MWang YLu YLv QDick RLu TGu NShang L(2021)MemXProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34635095:2(1-23)Online publication date: 24-Jun-2021
https://dl.acm.org/doi/10.1145/3463509
Wang CGao YMathur ADe C. Williams ALane NBianchi-Berthouze N(2021)Leveraging Activity Recognition to Enable Protective Behavior Detection in Continuous DataProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34635085:2(1-27)Online publication date: 24-Jun-2021
https://dl.acm.org/doi/10.1145/3463508
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents