research-article

Wearable sensor based multimodal human activity recognition exploiting the diversity of classifier ensemble

Authors:

Liangying Peng,

Gencai ChenAuthors Info & Claims

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Pages 1112 - 1123

https://doi.org/10.1145/2971648.2971708

Published: 12 September 2016 Publication History

Abstract

Effectively utilizing multimodal information (e.g., heart rate and acceleration) is a promising way to achieve wearable sensor based human activity recognition (HAR). In this paper, an activity recognition approach MARCEL (<u>M</u>ultimodal <u>A</u>ctivity <u>R</u>ecognition with <u>C</u>lassifier <u>E</u>nsemble) is proposed, which exploits the diversity of base classifiers to construct a good ensemble for multimodal HAR, and the diversity measure is obtained from both labeled and unlabeled data. MARCEL uses neural network (NN) as base classifiers to construct the HAR model, and the diversity of classifier ensemble is embedded in the error function of the model. In each iteration, the error of the model is decomposed and back-propagated to base classifiers. To ensure the overall accuracy of the model, the weights of base classifiers are learnt in the classifier fusion process with sparse group lasso. Extensive experiments show that MARCEL is able to yield a competitive HAR performance, and has its superiority on exploiting multimodal signals.

References

[1]

Bao, L., Intille, S. Activity recognition from user-annotated acceleration data. In Proceedings of International Conference on Pervasive Computing (2004), 1--17.

[2]

Kwapisz, J., Weiss, G., Moore, S. Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter (2011), 12(2), 74--82.

Digital Library

[3]

Ravi, N., Dandekar, N., Mysore, P., Littman, M.L. Activity recognition from accelerometer data. In Proceedings of AAAI (2005), 1541--1546.

Digital Library

[4]

Pawar, T., Chaudhuri, S., Duttagupta, S. P. Body movement activity recognition for ambulatory cardiac monitoring. IEEE Transactions on Biomedical Engineering (2007), 54(5), 874--882.

[5]

Guo, H., Chen, L., Chen, G., Lv, M. Smartphone-based activity recognition independent of device orientation and placement. International Journal of Communication Systems (2015).

[6]

Kunze, K., Lukowicz, P. Dealing with sensor displacement in motion-based onbody activity recognition systems. In Proceedings of International Conference on Ubiquitous Computing (2008), 20--29.

Digital Library

[7]

Lara, Ó.D., Pérez, A.J., Labrador, M.A., Posada, J.D. Centinela: A human activity recognition system based on acceleration and vital sign data. Pervasive and Mobile Computing (2012), 8(5), 717--729.

Digital Library

[8]

Parkka, J., Ermes, M., Korpipaa, P., Mantyjarvi, J., Peltola, J., Korhonen, I. Activity classification using realistic data from wearable sensors. IEEE Transactions on Information Technology in Biomedicine (2006), 10(1), 119--128.

Digital Library

[9]

Tapia, E.M., Intille, S.S., Haskell, W., Larson, K., Wright, J., King, A., Friedman, R. Real-time recognition of physical activities and their intensities using wireless accelerometers and a heart rate monitor. In Proceedings of IEEE International Symposium on Wearable Computers (2007), 37--40.

Digital Library

[10]

Li, M., Rozgic, V., Thatte, G., Lee, S., Emken, B.A., Annavaram, M., Narayanan, S. Multimodal physical activity recognition by fusing temporal and cepstral information. IEEE Transactions on Neural Systems and Rehabilitation Engineering (2010), 18(4), 369--380.

[11]

Guo, H., Chen, L., Shen, Y., Chen, G. Activity recognition exploiting classifier level fusion of acceleration and physiological signals. In Proceedings of ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication (2014), 63--66.

Digital Library

[12]

Goswami, G., Mittal, P., Majumdar, A., Vatsa, M., Singh, R. Group sparse representation based classification for multi-feature multimodal biometrics. Information Fusion (2015).

Digital Library

[13]

Zhang, M.L., Zhou, Z.H. Exploiting unlabeled data to enhance ensemble diversity. Data Mining and Knowledge Discovery (2013), 26(1): 98--129.

Digital Library

[14]

Subramanya, A., Raj, A., Bilmes, J., Fox, D. Recognizing activities and spatial context using wearable sensors. In Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence (2006).

[15]

Plötz, T, Hammerla, N., Olivier, P. Feature learning for activity recognition in ubiquitous computing. In Proceedings of International Joint Conference on Artificial Intelligence (2011), 22(1): 1729.

Digital Library

[16]

Ladha, C., Hammerla, N. Y., Olivier, P., Plötz, T. ClimbAX: skill assessment for climbing enthusiasts. In Proceedings of ACM International Joint Conference on Pervasive and Ubiquitous Computing (2013), 235--244.

Digital Library

[17]

Yang, J. B., Nguyen, M. N., San, P. P., Li, X. L., Krishnaswamy, S. Deep convolutional neural networks on multichannel time series for human activity recognition. In Proceedings of International Joint Conference on Artificial Intelligence (2015), 25--31.

Digital Library

[18]

Hu, Q., Li, L., Wu, X., Schaefer, G., Yu, D. Exploiting diversity for optimizing margin distribution in ensemble learning. Knowledge-Based Systems (2014), 67, 90--104.

Digital Library

[19]

Yin, X. C., Huang, K., Yang, C., Hao, H. W. Convex ensemble learning with sparsity and diversity. Information Fusion (2014), 20, 49--59.

[20]

Díez-Pastor, J. F., Rodríguez, J. J., García-Osorio, C. I., Kuncheva, L. I. Diversity techniques improve the performance of the best imbalance learning ensembles. Information Sciences (2015), 325, 98--117.

Digital Library

[21]

Zhang, M., Zhou, Z. Exploiting unlabeled data to enhance ensemble diversity. In Proceedings of IEEE International Conference on Data Mining (2010), 619--628.

Digital Library

[22]

Krishnan, N., Colbry, D., Juillard, C., Panchanathan, S. Real time human activity recognition using tri-axial accelerometers. In Proceedings of Sensors, Signals and Information Processing Workshop (2008).

[23]

Miluzzo, E., Lane, N. D., Fodor, K., Peterson, R., Lu, H. Sensing meets mobile social networks: The design, implementation and evaluation of the CenceMe application. In Proceedings of ACM Conference on Embedded Network Sensor Systems (2008), 337--350.

Digital Library

[24]

Gupta, P., Dallas, T. Feature Selection and Activity Recognition System using a Single Tri-axial Accelerometer. IEEE Transactions on Biomedical Engineering (2014), 61(6), 1780--1786.

[25]

Zhu, C., Sheng, W. Realtime recognition of complex human daily activities using human motion and location data. IEEE Transactions on Biomedical Engineering (2012), 59(9), 2422--2430.

[26]

Althloothi, S., Mahoor, M. H., Zhang, X., Voyles, R. M. Human activity recognition using multi-features and multiple kernel learning. Pattern Recognition (2014), 47(5), 1800--1812.

Digital Library

[27]

Yin, X., Huang, K., Hao, H., Iqbal, K., Wang, Z. A novel classifier ensemble method with sparsity and diversity. Neurocomputing (2014), 134, 214--221.

Digital Library

[28]

Chen, H. Diversity and regularization in neural network ensembles. School of Computer Science University of Birmingham, PhD Thesis October (2008).

[29]

Yu, Y., Li, Y.F., Zhou, Z.H. Diversity regularized machine. In Proceedings of International Joint Conference on Artificial Intelligence (2011), 1603--1608.

Digital Library

[30]

Li, N., Yu, Y., Zhou, Z.H. Diversity regularized ensemble pruning. In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (2012), 330--345.

[31]

Raina, R., Battle, A., Lee, H., Packer, B., Ng, A. Y. Self-taught learning: transfer learning from unlabeled data. In Proceedings of International Conference on Machine Learning (2007), 759--766.

Digital Library

[32]

Bhattacharya, S., Nurmi, P., Hammerla, N., Plötz, T. Using unlabeled data in a sparse-coding framework for human activity recognition. Pervasive and Mobile Computing (2014), 15, 242--262.

Digital Library

[33]

Tibshirani, R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 1996, 267--288.

[34]

Liu, J., Ji, S., Ye, J. Multi-task feature learning via efficient l 2, 1-norm minimization. In Proceedings of the twenty-fifth Conference on Uncertainty in Artificial Intelligence (2009), 339--348.

Digital Library

[35]

Yuan, M., Lin, Y. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 2006, 68(1), 49--67.

[36]

Zhao, L., Hu, Q., Wang, W. Heterogeneous Feature Selection with Multi-Modal Deep Neural Networks and Sparse Group Lasso. IEEE Transactions on Multimedia (2015), 17(11), 1936--1948.

Digital Library

[37]

Lane, N D., Georgiev, P., Qendro, L. DeepEar: robust smartphone audio sensing in unconstrained acoustic environments using deep learning. In Proceedings of ACM International Joint Conference on Pervasive and Ubiquitous Computing (2015), 283--294.

Digital Library

[38]

Deng, L., Platt, J C. Ensemble deep learning for speech recognition. In Proceedings of INTERSPEECH (2014), 1915--1919.

[39]

Romaszko, L. A deep learning approach with an ensemble-based neural network classifier for black box icml 2013 contest. Workshop on Challenges in Representation Learning, ICML (2013).

[40]

Zhang, X., Povey, D., Khudanpur, S. A Diversity-Penalizing Ensemble Training Method for Deep Learning. In Proceedings of INTERSPEECH (2015).

[41]

Opitz, D.W. Feature selection for ensembles. In Proceedings of AAAI/IAAI (1999), 379--384.

Digital Library

[42]

Skalak, D.B. The sources of increased accuracy for two proposed Boosting algorithms. In Proceedings of AAAI, Integrating Multiple Learned Models Workshop (1996), 1129, 1133--1133.

[43]

Shipp, C.A., Kuncheva, L. Relationship between combination methods and measures of diversity in combining classifiers. Information Fusion (2002), 3, 135--148.

[44]

Kuncheva, L., Whitaker, C. Measures of diversity in classifier ensembles. Machine Learning (2003), 51(2), 181--207.

Digital Library

[45]

LeCun, Y., Bottou, L., Bengio, Y., Haffner P. Gradient-Based Learning Applied to Document Recognition. In Proceedings of the IEEE (1998), 86(11), 2278--2324.

[46]

LeCun, Y., Bottou, L., Orr, G., Müller, K. Efficient backprop. Neural networks: Tricks of the Trade (2012), 9--48.

Digital Library

[47]

Liu, J., Ye, J. Moreau-Yosida regularization for grouped tree structure learning. In Proceedings of Advances in Neural Information Processing Systems (2010), 1459--1467.

Digital Library

[48]

Reiss, A., Stricker, D. Introducing a new benchmarked dataset for activity monitoring. In Proceedings of IEEE International Symposium on Wearable Computers (2012), 108--109.

Digital Library

[49]

Banos, O., Garcia, R., Holgado-Terriza, J. A., Damas, M., Pomares, H., Rojas, I., ... Villalonga, C. mHealthDroid: A Novel Framework for Agile Development of Mobile Health Applications. In Proceedings of Ambient Assisted Living and Daily Activities (2014), 91--98.

[50]

Liu, J., Ji, S., Ye, J. SLEP: Sparse learning with efficient projections. Arizona State University (2009), 6, 491.

[51]

Pudil, P., Ferri, F. J., Novovicova, J., Kittler, J. Floating search methods for feature selection with nonmonotonic criterion functions. In Proceedings of IAPR International Conference on Pattern Recognition (1994), 2, 279--283.

Cited By

Kayan HHeartfield RRana OBurnap PPerera C(2024)CASPER: Context-Aware IoT Anomaly Detection System for Industrial Robotic ArmsACM Transactions on Internet of Things10.1145/36704145:3(1-36)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1145/3670414
Kaur HRani VKumar M(2024)Human activity recognition: A comprehensive reviewExpert Systems10.1111/exsy.13680Online publication date: 27-Jul-2024
https://doi.org/10.1111/exsy.13680
Sun HChen Y(2024)A Rapid Response System for Elderly Safety Monitoring Using Progressive Hierarchical Action RecognitionIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2024.340919732(2134-2142)Online publication date: 2024
https://doi.org/10.1109/TNSRE.2024.3409197
Show More Cited By

Index Terms

Wearable sensor based multimodal human activity recognition exploiting the diversity of classifier ensemble
1. Human-centered computing

Recommendations

Classifier ensemble optimization for human activity recognition in smart homes
ICUIMC '13: Proceedings of the 7th International Conference on Ubiquitous Information Management and Communication

Recognizing human activities is an active research area due to its applicability in many applications, such as assistive living and healthcare. Currently, the major challenges in activity recognition include the reliability of prediction of each ...
Building Locally Discriminative Classifier Ensemble Through Classifier Fusion Among Nearest Neighbors
PCM 2016: 17th Pacific-Rim Conference on Advances in Multimedia Information Processing - Volume 9916

Many studies on ensemble learning that combines multiple classifiers have shown that, it is an effective technique to improve accuracy and stability of a single classifier. In this paper, we propose a novel discriminative classifier fusion method, which ...
A novel classifier ensemble method with sparsity and diversity

We consider the classifier ensemble problem in this paper. Due to its superior performance to individual classifiers, class ensemble has been intensively studied in the literature. Generally speaking, there are two prevalent research directions on this, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '16: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 2016

1288 pages

ISBN:9781450344616

DOI:10.1145/2971648

General Chairs:
Paul Lukowicz
DFKI and University of Kaiserslautern
,
Antonio Krüger
DFKI and Saarland University
,
Program Chairs:
Andreas Bulling
Max Planck Institute for Informatics
,
Youn-Kyung Lim
KAIST
,
Shwetak N. Patel
University of Washington

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 September 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

UbiComp '16

Sponsor:

UbiComp '16: The 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing

September 12 - 16, 2016

Germany, Heidelberg

Acceptance Rates

UbiComp '16 Paper Acceptance Rate 101 of 389 submissions, 26%;

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

UBICOMP '24

Sponsor:
sigchi
sigchi

The 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing

October 5 - 9, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

58
Total Citations
View Citations
1,175
Total Downloads

Downloads (Last 12 months)55
Downloads (Last 6 weeks)3

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kayan HHeartfield RRana OBurnap PPerera C(2024)CASPER: Context-Aware IoT Anomaly Detection System for Industrial Robotic ArmsACM Transactions on Internet of Things10.1145/36704145:3(1-36)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1145/3670414
Kaur HRani VKumar M(2024)Human activity recognition: A comprehensive reviewExpert Systems10.1111/exsy.13680Online publication date: 27-Jul-2024
https://doi.org/10.1111/exsy.13680
Sun HChen Y(2024)A Rapid Response System for Elderly Safety Monitoring Using Progressive Hierarchical Action RecognitionIEEE Transactions on Neural Systems and Rehabilitation Engineering10.1109/TNSRE.2024.340919732(2134-2142)Online publication date: 2024
https://doi.org/10.1109/TNSRE.2024.3409197
Cooper GMarculescu R(2024)Beyond Thresholds: A General Approach to Sensor Selection for Practical Deep Learning-based HAR2024 IEEE/ACM Ninth International Conference on Internet-of-Things Design and Implementation (IoTDI)10.1109/IoTDI61053.2024.00005(1-12)Online publication date: 13-May-2024
https://doi.org/10.1109/IoTDI61053.2024.00005
Carratù MGallo VLaino VLiguori CPietrosanto A(2024)Including Measurement Uncertainty to Improve the Reliability of Classification ANN2024 IEEE International Instrumentation and Measurement Technology Conference (I2MTC)10.1109/I2MTC60896.2024.10561217(1-6)Online publication date: 20-May-2024
https://doi.org/10.1109/I2MTC60896.2024.10561217
Imanzadeh STanha JJalili M(2024)Ensemble of deep learning techniques to human activity recognition using smart phone signalsMultimedia Tools and Applications10.1007/s11042-024-18935-0Online publication date: 1-Apr-2024
https://doi.org/10.1007/s11042-024-18935-0
Dixon SYao LDavidson R(2024)Modality aware contrastive learning for multimodal human activity recognitionConcurrency and Computation: Practice and Experience10.1002/cpe.802036:16Online publication date: 25-Apr-2024
https://doi.org/10.1002/cpe.8020
Chen LHu RWu MZhou X(2023)HMGANProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109097:3(1-27)Online publication date: 27-Sep-2023
https://dl.acm.org/doi/10.1145/3610909
Hu CHu LYuan LLu DLyu LChen Y(2023)FedIERF: Federated Incremental Extremely Random Forest for Wearable Health MonitoringJournal of Computer Science and Technology10.1007/s11390-023-3009-038:5(970-984)Online publication date: 30-Sep-2023
https://doi.org/10.1007/s11390-023-3009-0
Baltabay MYazici ASterling MEver E(2023)Designing Efficient and Lightweight Deep Learning Models for Healthcare AnalysisNeural Processing Letters10.1007/s11063-023-11246-955:6(6947-6977)Online publication date: 23-Mar-2023
https://doi.org/10.1007/s11063-023-11246-9
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents