Article

Analysis of chewing sounds for dietary monitoring

Authors:

Mathias Stäger,

Gerhard TrösterAuthors Info & Claims

UbiComp'05: Proceedings of the 7th international conference on Ubiquitous Computing

Pages 56 - 72

https://doi.org/10.1007/11551201_4

Published: 11 September 2005 Publication History

Abstract

The paper reports the results of the first stage of our work on an automatic dietary monitoring system. The work is part of a large European project on using ubiquitous systems to support healthy lifestyle and cardiovascular disease prevention. We demonstrate that sound from the user's mouth can be used to detect that he/she is eating. The paper also shows how different kinds of food can be recognized by analyzing chewing sounds. The sounds are acquired with a microphone located inside the ear canal. This is an unobtrusive location widely accepted in other applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing data from four subjects on four different food types typically found in a meal. Up to 99% accuracy is achieved on eating recognition and between 80% to 100% on food type classification.

References

[1]

Mynatt, E., Melenhorst, A.S., Fisk, A.D., Rogers, W.: Aware technologies for aging in place: understanding user needs and attitudes. In: IEEE Pervasive Computing. Volume 3. (2004) 36-41.

Digital Library

[2]

Gellersen, H.W., Schmidt, A., Beigl, M.: Multi-sensor context-awareness in mobile devices and smart artifacts. Mobile Networks and Applications 7 (2002) 341-351.

Digital Library

[3]

Philipose, M., Fishkin, K., Perkowitz, M., Patterson, D., Fox, D., Kautz, H., Hahnel, D.: Inferring activities from interactions with objects. IEEE Pervasive Computing 3 (2004) 50-57.

Digital Library

[4]

Mihailidis, A., Carmichael, B., Boger, J.: The use of computer vision in an intelligent environment to support aging-in-place, safety, and independence in the home. IEEE Transactions on Information Technology in Biomedicine 8 (2004) 238-247.

Digital Library

[5]

Römer, K., Schoch, T., Mattern, F., Dübendorfer, T.: Smart identification frameworks for ubiquitous computing applications. In: PerCom 2003: Proc. of the First IEEE Int'l Conference Pervasive Computing and Communications. (2003).

Digital Library

[6]

Beigl, M., Gellersen, H.W., Schmidt, A.: MediaCups: Experience with design and use of computer-augmented everyday artefacts. Computer Networks, Special Issue on Pervasive Computing 35 (2001) 401-409.

Digital Library

[7]

Vickers, Z., Christensen, C.: Relationships between sensory crispness and other sensory and instrumental parameters. Journal of Texture Studies (1980) 291-307.

[8]

Vickers, Z.: Relationships of chewing sounds to judgements of crispness crunchiness and hardness. Journal of Texture Studies (1981) 121-124.

[9]

Drake, B.: Food crushing sounds. an introductory study. Journal of Food Science (1963) 233-241.

[10]

Lee, W., Schweitzer, G., Morgan, G., Shepherd, D.: Analysis of food crushing sounds during mastication: Total sound level studies. Journal of Texture Studies (1990) 156-178.

[11]

Kapur, K.: Frequency spectrographic analysis of bone conducted chewing sounds in persons with natural and artifcial dentitions. Journal of Texture Studies (1971) 50-61.

[12]

Dacremont, C., Colas, B., Sauvageot, F.: Contribution of air- and bone-conduction to the creation of sounds perceived during sensory evaluation of foods. Journal of Texture Studies (1991) 443-456.

[13]

Vickers, Z.: Sensory acoustical and force? Deformation measurements of potato chip crispness. Journal of Texture Studies (1987) 138-140.

[14]

Szczesniak, A.: Texture: Is it still an overlooked food attribute? Food Technology (1990) 86-95.

[15]

AlChakra, W., Allaf, K., Jemai, A.: Characterization of brittle food products: Application of the acoustical emission method. Journal of Texture Studies (1996) 327-348.

[16]

Edmister, J., Vickers, Z.: Instrumental acoustical measures of crispness in foods. Journal of Texture Studies (1985) 153-167.

[17]

Vincent, J.F.V.: The quantification of crispness. Journal of Science in Food Argiculture (1998) 162-168.

[18]

DeBelie, N., De Smedt, V., J., D.B.: Principal component analysis of chewing sounds to detect differences in apple crispness. Journal of Postharvest Biology and Technology (2000) 109-119.

[19]

Brenman, H., Weiss, R., Black, M.: Sound as a diagnostic aid in the detection of occlusion discrepancies. Penn Dental Journal (1966).

[20]

Watt, D.: Gnathosonics - a study of sound produced by the masticatory mechanism. Journal of Prosthetic Dentistry (1966).

[21]

Prinz, J.: Computer aided gnathosonic analysis: distinguishing between single and multiple tooth impact sounds. Journal of Oral Rehabilitation (2000) 682-689.

[22]

Widmalm, S., Williams, W., Zengh, C.: Time frequency distribution of tmj sounds. Journal of Oral Rehabilitation (1991).

[23]

Tyson, K.: Monitoring the state of occlusion - gnathosonics can be reliable. Journal of Oral Rehabilitation (1998) 395-402.

[24]

Hutchings, J., Lillford, D.: The perception of food texture - the philosophy of the breakdown path. Journal of Texture Studies (1988) 103-115.

[25]

Brochetti, D., Penfield, M., Burchfield, S.: Speech analysis techniques: A potential model for the study of mastication sounds. Journal of Texture Studies (1992) 111-138.

[26]

Watt, D.: Gnathosonics and Occlusal Dynamics. Praeger New York (1981).

[27]

Stäger, M., Lukowicz, P., Perera, N., von Büren, T., Tröster, G., Starner, T.: SoundButton: Design of a Low Power Wearable Audio Classification System. In: ISWC 2003: Proc. of the 7th IEEE Int'l Symposium on Wearable Computers. (2003) 12-17.

Digital Library

[28]

Li, S.Z.: Content-based audio classification and retrieval using the nearest feature line method. IEEE Transactions on Speech and Audio Processing 8 (2000) 619-625.

[29]

Li, D., Sethi, I., Dimitrova, N., McGee, T.: Classification of general audio data for content-based retrieval. Pattern Recognition Letters 22 (2001) 533-544.

Digital Library

[30]

Peltonen, V., Tuomi, J., Klapuri, A., Huopaniemi, J., Sorsa, T.: Computational auditory scene recognition. In: IEEE Int'l Conf. on Acoustics, Speech, and Signal Processing. Volume 2. (2002) 1941-1944.

[31]

Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann (1999).

Digital Library

[32]

Provost, F., Fawcett, T.: Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In: KDD '97: Proc. of the 3rd Int'l Conference on Knowledge Discovery and Data Mining. (1997) 43-48.

Cited By

Mamish JAlharbi RSen SHolla SKamath PSangar YAlshurafa NHester J(2024)NIR-sighted: A Programmable Streaming Architecture for Low-Energy Human-Centric Vision ApplicationsACM Transactions on Embedded Computing Systems10.1145/367207623:6(1-26)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3672076
Zhang RZhang JGade NCao PKim SYan JZhang C(2022)EatingTrakProceedings of the ACM on Human-Computer Interaction10.1145/35467496:MHCI(1-22)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1145/3546749
Bin Morshed MHaresamudram HBandaru DAbowd GPloetz T(2022)A Personalized Approach for Developing a Snacking Detection System using Earbuds in a Semi-Naturalistic SettingProceedings of the 2022 ACM International Symposium on Wearable Computers10.1145/3544794.3558469(11-16)Online publication date: 11-Sep-2022
https://dl.acm.org/doi/10.1145/3544794.3558469
Show More Cited By

Recommendations

Auditory Seasoning Filters: Altering Food Perception via Augmented Sonic Feedback of Chewing Sounds
CHI '23: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

The experience of what we eat depends not only on the taste of the food, but also on other modalities of sensory feedback. Perceptual research has shown the potential of altering visual, olfactory, and textural food cues to affect flavor, texture, and ...
Fractal Nature of Chewing Sounds
New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops
Abstract
In the battle against Obesity as well as Eating Disorders, non-intrusive dietary monitoring has been investigated by many researchers. For this purpose, one of the most promising modalities is the acoustic signal captured by a common microphone ...
Chewing jockey: augmented food texture by using sound based on the cross-modal effect
ACE '11: Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology

We focus on the dining and show how to improve dining experience. We use sound effects to augment food texture, creating a cross-modal illusion.

Our system is composed of a bone-conduction speaker, a microphone, a photoreflector to measure the motion of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

UbiComp'05: Proceedings of the 7th international conference on Ubiquitous Computing

September 2005

394 pages

ISBN:9783540287605

Editors:
Michael Beigl
Distributed and Ubiquitous Systems, Muehlenpfordtstr. 23, Braunschweig, Germany
,
Stephen Intille
House_n, Massachusetts Institute of Technology, One Cambridge Center, 4FL, Cambridge, MA, Germany
,
Jun Rekimoto
Interaction Laboratory, Sony Computer Science Laboratories, Inc., 3-14-13 Higashigotanda, Shinagawa-ku, Tokyo, MA, Japan
,
Hideyuki Tokuda
Faculty of Environment and Information Studies, Keio University, 5322, Endo, Fujisawa, Kanagawa, MA, Japan

Sponsors

Fujitsu
NEC
KDDI R&D Laboratories Inc.
NTT DoCoMo
Fuji Xerox: Fuji Xerox

In-Cooperation

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 11 September 2005

Check for updates

Qualifiers

Article

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

65
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mamish JAlharbi RSen SHolla SKamath PSangar YAlshurafa NHester J(2024)NIR-sighted: A Programmable Streaming Architecture for Low-Energy Human-Centric Vision ApplicationsACM Transactions on Embedded Computing Systems10.1145/367207623:6(1-26)Online publication date: 11-Sep-2024
https://dl.acm.org/doi/10.1145/3672076
Zhang RZhang JGade NCao PKim SYan JZhang C(2022)EatingTrakProceedings of the ACM on Human-Computer Interaction10.1145/35467496:MHCI(1-22)Online publication date: 20-Sep-2022
https://dl.acm.org/doi/10.1145/3546749
Bin Morshed MHaresamudram HBandaru DAbowd GPloetz T(2022)A Personalized Approach for Developing a Snacking Detection System using Earbuds in a Semi-Naturalistic SettingProceedings of the 2022 ACM International Symposium on Wearable Computers10.1145/3544794.3558469(11-16)Online publication date: 11-Sep-2022
https://dl.acm.org/doi/10.1145/3544794.3558469
Jin YGao YGuo XWen JLi ZJin ZBulusu NAryafar EBalasubramanian ASong J(2022)EarHealthProceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services10.1145/3498361.3538935(397-408)Online publication date: 27-Jun-2022
https://dl.acm.org/doi/10.1145/3498361.3538935
M. Silva LA. Epstein D(2021)Investigating Preferred Food Description Practices in Digital Food JournalingProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462145(589-605)Online publication date: 28-Jun-2021
https://dl.acm.org/doi/10.1145/3461778.3462145
Kopyto DZhang RAmft O(2021)Audio-Based Onset Detection applied to Chewing Cycle SegmentationProceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460421.3478819(124-128)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460421.3478819
Kamachi HKondo THossain TYokokubo ALopez G(2021)Automatic Segmentation Method of Bone Conduction Sound for Eating Activity Detailed DetectionAdjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460418.3479353(310-315)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460418.3479353
Ahuja AFerlini AMascolo C(2021)PilotEar: Enabling In-ear Inertial NavigationAdjunct Proceedings of the 2021 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460418.3479326(139-145)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460418.3479326
Röddiger TClarke CWolffram DBudde MBeigl MKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)EarRumble: Discreet Hands- and Eyes-Free Input by Voluntary Tensor Tympani Muscle ContractionProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445205(1-14)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445205
Pan SRen XVos SBrombacher A(2021)Design Opportunities of Digital Tools for Promoting Healthy Eating Routines Among Dutch Office WorkersHCI International 2021 - Late Breaking Papers: HCI Applications in Health, Transport, and Industry10.1007/978-3-030-90966-6_8(94-110)Online publication date: 24-Jul-2021
https://dl.acm.org/doi/10.1007/978-3-030-90966-6_8
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents