research-article

Sparse natural gesture spotting in free living to monitor drinking with wrist-worn inertial sensors

Authors:

Giovanni Schiboni,

Oliver AmftAuthors Info & Claims

ISWC '18: Proceedings of the 2018 ACM International Symposium on Wearable Computers

Pages 140 - 147

https://doi.org/10.1145/3267242.3267253

Published: 08 October 2018 Publication History

Get Access

Abstract

We present a spotting network composed of Gaussian Mixture Hidden Markov Models (GMM-HMMs) to detect sparse natural gestures in free living. The key technical features of our approach are (1) a method to mine non-gesture patterns that deals with the arbitrary data (Null Class), and (2) an optimisation based on multipopulation genetic programming to approximate spotting network's parameters across target and non-target models. We evaluate our GMM-HMMs spotting network in a novel free living dataset, including totally 35 days of annotated inertial sensor's recordings from seven participants. Drinking was chosen as target gesture. Our method reached an average F1-score of over 74% and clearly outperformed an HMM-based threshold model approach. The results suggest that our spotting network approach is viable for sparse natural pattern spotting.

References

[1]

Oliver Amft. 2010. Adaptive activity spotting based on event rates. In Sensor Networks, Ubiquitous, and Trustworthy Computing (SUTC), 2010 IEEE International Conference on. IEEE, 169--176.

Digital Library

Google Scholar

[2]

Oliver Amft, David Bannach, Gerald Pirkl, Matthias Kreil, and Paul Lukowicz. 2010. Towards wearable sensing-based assessment of fluid intake. In Pervasive Computing and Communications Workshops (PERCOM Workshops), 2010 8th IEEE International Conference on. IEEE, 298--303.

Crossref

Google Scholar

[3]

Mahmoud Elmezain, Ayoub Al-Hamadi, and Bernd Michaelis. 2009. Hand trajectory-based gesture spotting and recognition using HMM. In Image Processing (ICIP), 2009 16th IEEE International Conference on. IEEE, 3577--3580.

Digital Library

Google Scholar

[4]

Francisco Fernández, Marco Tomassini, and Leonardo Vanneschi. 2003. An empirical study of multipopulation genetic programming. Genetic Programming and Evolvable Machines 4, 1 (2003), 21--51.

Digital Library

Google Scholar

[5]

Simon Günter and Horst Bunke. 2003. Fast feature selection in an HMM-based multiple classifier system for handwriting recognition. In Joint Pattern Recognition Symposium. Springer, 289--296.

Crossref

Google Scholar

[6]

Mithun George Jacob and Juan Pablo Wachs. 2014. Context-based hand gesture recognition for the operating room. Pattern Recognition Letters 36 (2014), 196--203.

Digital Library

Google Scholar

[7]

Holger Junker, Oliver Amft, Paul Lukowicz, and Gerhard Tröster. 2008. Gesture spotting with body-worn inertial sensors to detect user activities. Pattern Recognition 41, 6 (2008), 2010--2024.

Digital Library

Google Scholar

[8]

Hyeon-Kyu Lee and Jin-Hyung Kim. 1999. An HMM-based threshold model approach for gesture recognition. IEEE Transactions on pattern analysis and machine intelligence 21, 10 (1999), 961--973.

Digital Library

Google Scholar

[9]

Jerry Mannil, Mohammad-Mahdi Bidmeshki, and Roozbeh Jafari. 2011. Rejection of irrelevant human actions in real-time hidden Markov model based recognition systems for wearable computers. In Proceedings of the 2nd Conference on Wireless Health. ACM, 8.

Digital Library

Google Scholar

[10]

Bo Peng and Gang Qian. 2011. Online gesture spotting from visual hull data. IEEE transactions on pattern analysis and machine intelligence 33, 6 (2011), 1175--1188.

Digital Library

Google Scholar

[11]

LR Rabiner and BH Juang. 1992. Hidden Markov models for speech recognition - Strengths and limitations. In Speech Recognition and Understanding. Springer, 3--29.

Google Scholar

[12]

Michael N Sawka, Samuel N Cheuvront, and Robert Carter. 2005. Human water needs. Nutrition reviews 63, s1 (2005).

Google Scholar

[13]

Edison Thomaz, Irfan Essa, and Gregory D Abowd. 2015. A practical approach for recognizing eating moments with wrist-mounted inertial sensing. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1029--1040.

Digital Library

Google Scholar

[14]

Shilei Zhang, Zhiwei Shuang, Qin Shi, and Yong Qin. 2010. Improved mandarin keyword spotting using confusion garbage model. In Pattern Recognition (ICPR), 2010 20th International Conference on. IEEE, 3700--3703.

Digital Library

Google Scholar

Cited By

View all

Wang CKumar TDe Raedt WCamps GHallez HVanrumste B(2024)Eating Speed Measurement Using Wrist-Worn IMU Sensors Towards Free-Living EnvironmentsIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2024.342287528:10(5816-5828)Online publication date: Oct-2024
https://doi.org/10.1109/JBHI.2024.3422875
Hiraguchi HPerone PToet ACamps GBrouwer A(2023)Technology to Automatically Record Eating Behavior in Real Life: A Systematic ReviewSensors10.3390/s2318775723:18(7757)Online publication date: 8-Sep-2023
https://doi.org/10.3390/s23187757
Wang CKong JCai YKumar TDe Raedt WCamps GHallez HVanrumste B(2023)Intake Gesture Detection With IMU Sensor in Free-Living Environments: The Effects of Measuring Two-Hand Intake and Down-Sampling2023 IEEE 19th International Conference on Body Sensor Networks (BSN)10.1109/BSN58485.2023.10331032(1-4)Online publication date: 9-Oct-2023
https://doi.org/10.1109/BSN58485.2023.10331032
Show More Cited By

Recommendations

Gesture spotting with body-worn inertial sensors to detect user activities

We present a method for spotting sporadically occurring gestures in a continuous data stream from body-worn inertial sensors. Our method is based on a natural partitioning of continuous sensor signals and uses a two-stage approach for the spotting task. ...
Fall Detection Using Kinematic Features from a Wrist-Worn Inertial Sensor
2019 IEEE International Symposium on Medical Measurements and Applications (MeMeA)
The threats and injuries caused by falling are significant and pose risks to life among the elderly population. Detecting falls and providing prompt care helps in reducing the risks associated with falling. Technology solutions are being employed for the ...
Physical activity recognition in free-living from body-worn sensors
SenseCam '13: Proceedings of the 4th International SenseCam & Pervasive Imaging Conference

Machine learning techniques are used to improve accelerometer-based measures of physical activity. Most studies have used laboratory-collected data to develop algorithms to classify behaviors, but studies of free-living activity are needed to improve ...

Comments

Information & Contributors

Information

Published In

ISWC '18: Proceedings of the 2018 ACM International Symposium on Wearable Computers

October 2018

307 pages

ISBN:9781450359672

DOI:10.1145/3267242

General Chairs:
Kai Kunze
Keio University
,
Rajesh K. Balan
Singapore Management University
,
Youngki Lee
Singapore Management University
,
Program Chairs:
Michael Beigl
Karlsruhe Institute of Technology
,
Roshan Peiris
Keio University

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UbiComp '18

Sponsor:

UbiComp '18: The 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing

October 8 - 12, 2018

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 38 of 196 submissions, 19%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
281
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)5

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Wang CKumar TDe Raedt WCamps GHallez HVanrumste B(2024)Eating Speed Measurement Using Wrist-Worn IMU Sensors Towards Free-Living EnvironmentsIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2024.342287528:10(5816-5828)Online publication date: Oct-2024
https://doi.org/10.1109/JBHI.2024.3422875
Hiraguchi HPerone PToet ACamps GBrouwer A(2023)Technology to Automatically Record Eating Behavior in Real Life: A Systematic ReviewSensors10.3390/s2318775723:18(7757)Online publication date: 8-Sep-2023
https://doi.org/10.3390/s23187757
Wang CKong JCai YKumar TDe Raedt WCamps GHallez HVanrumste B(2023)Intake Gesture Detection With IMU Sensor in Free-Living Environments: The Effects of Measuring Two-Hand Intake and Down-Sampling2023 IEEE 19th International Conference on Body Sensor Networks (BSN)10.1109/BSN58485.2023.10331032(1-4)Online publication date: 9-Oct-2023
https://doi.org/10.1109/BSN58485.2023.10331032
Sharma SHoover A(2022)Top-Down Detection of Eating Episodes by Analyzing Large Windows of Wrist Motion Using a Convolutional Neural NetworkBioengineering10.3390/bioengineering90200709:2(70)Online publication date: 11-Feb-2022
https://doi.org/10.3390/bioengineering9020070
Yang YWang HJiang RGuo XCheng JChen Y(2022)A Review of IoT-Enabled Mobile Healthcare: Technologies, Challenges, and Future TrendsIEEE Internet of Things Journal10.1109/JIOT.2022.31444009:12(9478-9502)Online publication date: 15-Jun-2022
https://doi.org/10.1109/JIOT.2022.3144400
Wang CKumar TDe Raedt WCamps GHallez HVanrumste B(2022)Drinking Gesture Detection Using Wrist-Worn IMU Sensors with Multi-Stage Temporal Convolutional Network in Free-Living Environments2022 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)10.1109/EMBC48229.2022.9871817(1778-1782)Online publication date: 11-Jul-2022
https://doi.org/10.1109/EMBC48229.2022.9871817
Baronetto AAmft O(2022)AIM in Wearable and Implantable ComputingArtificial Intelligence in Medicine10.1007/978-3-030-64573-1_299(1187-1201)Online publication date: 18-Feb-2022
https://doi.org/10.1007/978-3-030-64573-1_299
Kopyto DUhlenberg LZhang RStonawski VHorndasch SAmft O(2022)AIM in Eating DisordersArtificial Intelligence in Medicine10.1007/978-3-030-64573-1_213(1643-1661)Online publication date: 18-Feb-2022
https://doi.org/10.1007/978-3-030-64573-1_213
Cohen RFernie GRoshan Fekr A(2021)Fluid Intake Monitoring Systems for the Elderly: A Review of the LiteratureNutrients10.3390/nu1306209213:6(2092)Online publication date: 19-Jun-2021
https://doi.org/10.3390/nu13062092
Kopyto DZhang RAmft O(2021)Audio-Based Onset Detection applied to Chewing Cycle SegmentationProceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460421.3478819(124-128)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460421.3478819
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Recommendations

Gesture spotting with body-worn inertial sensors to detect user activities

Fall Detection Using Kinematic Features from a Wrist-Worn Inertial Sensor

Physical activity recognition in free-living from body-worn sensors