Abstract
The paper reports the results of the first stage of our work on an automatic dietary monitoring system. The work is part of a large European project on using ubiquitous systems to support healthy lifestyle and cardiovascular disease prevention. We demonstrate that sound from the user’s mouth can be used to detect that he/she is eating. The paper also shows how different kinds of food can be recognized by analyzing chewing sounds. The sounds are acquired with a microphone located inside the ear canal. This is an unobtrusive location widely accepted in other applications (hearing aids, headsets). To validate our method we present experimental results containing 3500 seconds of chewing data from four subjects on four different food types typically found in a meal. Up to 99% accuracy is achieved on eating recognition and between 80% to 100% on food type classification.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Mynatt, E., Melenhorst, A.S., Fisk, A.D., Rogers, W.: Aware technologies for aging in place: understanding user needs and attitudes. IEEE Pervasive Computing 3, 36–41 (2004)
Gellersen, H.W., Schmidt, A., Beigl, M.: Multi-sensor context-awareness in mobile devices and smart artifacts. Mobile Networks and Applications 7, 341–351 (2002)
Philipose, M., Fishkin, K., Perkowitz, M., Patterson, D., Fox, D., Kautz, H., Hahnel, D.: Inferring activities from interactions with objects. IEEE Pervasive Computing 3, 50–57 (2004)
Mihailidis, A., Carmichael, B., Boger, J.: The use of computer vision in an intelligent environment to support aging-in-place, safety, and independence in the home. IEEE Transactions on Information Technology in Biomedicine 8, 238–247 (2004)
Römer, K., Schoch, T., Mattern, F., Dübendorfer, T.: Smart identification frameworks for ubiquitous computing applications. In: PerCom 2003: Proc. of the First IEEE Int’l Conference Pervasive Computing and Communications (2003)
Beigl, M., Gellersen, H.W., Schmidt, A.: MediaCups: Experience with design and use of computer-augmented everyday artefacts. Computer Networks, Special Issue on Pervasive Computing 35, 401–409 (2001)
Vickers, Z., Christensen, C.: Relationships between sensory crispness and other sensory and instrumental parameters. Journal of Texture Studies, 291–307 (1980)
Vickers, Z.: Relationships of chewing sounds to judgements of crispness crunchiness and hardness. Journal of Texture Studies, 121–124 (1981)
Drake, B.: Food crushing sounds. an introductory study. Journal of Food Science, 233–241 (1963)
Lee, W., Schweitzer, G., Morgan, G., Shepherd, D.: Analysis of food crushing sounds during mastication: Total sound level studies. Journal of Texture Studies, 156–178 (1990)
Kapur, K.: Frequency spectrographic analysis of bone conducted chewing sounds in persons with natural and artifcial dentitions. Journal of Texture Studies, 50–61 (1971)
Dacremont, C., Colas, B., Sauvageot, F.: Contribution of air- and bone-conduction to the creation of sounds perceived during sensory evaluation of foods. Journal of Texture Studies, 443–456 (1991)
Vickers, Z.: Sensory acoustical and force? Deformation measurements of potato chip crispness. Journal of Texture Studies, 138–140 (1987)
Szczesniak, A.: Texture: Is it still an overlooked food attribute? Food Technology, 86–95 (1990)
AlChakra, W., Allaf, K., Jemai, A.: Characterization of brittle food products: Application of the acoustical emission method. Journal of Texture Studies, 327–348 (1996)
Edmister, J., Vickers, Z.: Instrumental acoustical measures of crispness in foods. Journal of Texture Studies, 153–167 (1985)
Vincent, J.F.V.: The quantification of crispness. Journal of Science in Food Argiculture, 162–168 (1998)
DeBelie, N., De Smedt, V.J. D.B.: Principal component analysis of chewing sounds to detect differences in apple crispness. Journal of Postharvest Biology and Technology, 109–119 (2000)
Brenman, H., Weiss, R., Black, M.: Sound as a diagnostic aid in the detection of occlusion discrepancies. Penn Dental Journal (1966)
Watt, D.: Gnathosonics - a study of sound produced by the masticatory mechanism. Journal of Prosthetic Dentistry (1966)
Prinz, J.: Computer aided gnathosonic analysis: distinguishing between single and multiple tooth impact sounds. Journal of Oral Rehabilitation, 682–689 (2000)
Widmalm, S., Williams, W., Zengh, C.: Time frequency distribution of tmj sounds. Journal of Oral Rehabilitation (1991)
Tyson, K.: Monitoring the state of occlusion - gnathosonics can be reliable. Journal of Oral Rehabilitation, 395–402 (1998)
Hutchings, J., Lillford, D.: The perception of food texture - the philosophy of the breakdown path. Journal of Texture Studies, 103–115 (1988)
Brochetti, D., Penfield, M., Burchfield, S.: Speech analysis techniques: A potential model for the study of mastication sounds. Journal of Texture Studies, 111– 138 (1992)
Watt, D.: Gnathosonics and Occlusal Dynamics. Praeger New York (1981)
Stäger, M., Lukowicz, P., Perera, N., von Büren, T., Tröster, G., Starner, T.: SoundButton: Design of a Low Power Wearable Audio Classification System. In: ISWC 2003: Proc. of the 7th IEEE Int’l Symposium on Wearable Computers, pp.12–17 (2003)
Li, S.Z.: Content-based audio classification and retrieval using the nearest feature line method. IEEE Transactions on Speech and Audio Processing 8, 619–625 (2000)
Li, D., Sethi, I., Dimitrova, N., McGee, T.: Classification of general audio data for content-based retrieval. Pattern Recognition Letters 22, 533–544 (2001)
Peltonen, V., Tuomi, J., Klapuri, A., Huopaniemi, J., Sorsa, T.: Computational auditory scene recognition. In: IEEE Int’l Conf. on Acoustics, Speech, and Signal Processing, vol. 2, pp. 1941–1944 (2002)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Francisco (1999)
Provost, F., Fawcett, T.: Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In: KDD 1997: Proc. of the 3rd Int’l Conference on Knowledge Discovery and Data Mining, pp.43–48 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Amft, O., Stäger, M., Lukowicz, P., Tröster, G. (2005). Analysis of Chewing Sounds for Dietary Monitoring. In: Beigl, M., Intille, S., Rekimoto, J., Tokuda, H. (eds) UbiComp 2005: Ubiquitous Computing. UbiComp 2005. Lecture Notes in Computer Science, vol 3660. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11551201_4
Download citation
DOI: https://doi.org/10.1007/11551201_4
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28760-5
Online ISBN: 978-3-540-31941-2
eBook Packages: Computer ScienceComputer Science (R0)