research-article

Using unlabeled data in a sparse-coding framework for human activity recognition

Authors:

Sourav Bhattacharya,

Thomas PlötzAuthors Info & Claims

Pervasive and Mobile Computing, Volume 15, Issue C

Pages 242 - 262

https://doi.org/10.1016/j.pmcj.2014.05.006

Published: 01 December 2014 Publication History

Abstract

We propose a sparse-coding framework for activity recognition in ubiquitous and mobile computing that alleviates two fundamental problems of current supervised learning approaches. (i) It automatically derives a compact, sparse and meaningful feature representation of sensor data that does not rely on prior expert knowledge and generalizes well across domain boundaries. (ii) It exploits unlabeled sample data for bootstrapping effective activity recognizers, i.e., substantially reduces the amount of ground truth annotation required for model estimation. Such unlabeled data is easy to obtain, e.g., through contemporary smartphones carried by users as they go about their everyday activities.Based on the self-taught learning paradigm we automatically derive an over-complete set of basis vectors from unlabeled data that captures inherent patterns present within activity data. Through projecting raw sensor data onto the feature space defined by such over-complete sets of basis vectors effective feature extraction is pursued. Given these learned feature representations, classification backends are then trained using small amounts of labeled training data.We study the new approach in detail using two datasets which differ in terms of the recognition tasks and sensor modalities. Primarily we focus on a transportation mode analysis task, a popular task in mobile-phone based sensing. The sparse-coding framework demonstrates better performance than the state-of-the-art in supervised learning approaches. More importantly, we show the practical potential of the new approach by successfully evaluating its generalization capabilities across both domain and sensor modalities by considering the popular Opportunity dataset. Our feature learning approach outperforms state-of-the-art approaches to analyzing activities of daily living.

References

[1]

L. Atallah, G.-Z. Yang, The use of pervasive sensing for behaviour profiling-a survey, Pervasive and Ubiquitous Computing, 5 (2009) 447-464.

Digital Library

[2]

N.D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, A.T. Campbell, A survey of mobile phone sensing, IEEE Commun. Mag., 48 (2010) 140-150.

Digital Library

[3]

L. Bao, S.S. Intille, Activity recognition from user-annotated acceleration data, in: A. Ferscha, F. Mattern (Eds.), Proc. Int. Conf. Pervasive Comp. (Pervasive), 2004.

[4]

B. Logan, J. Healey, M. Philipose, E.M. Tapia, S. Intille, A long-term evaluation of sensing modalities for activity recognition, in: Proc. ACM Conf. Ubiquitous Comp. (UbiComp), 2007.

[5]

C. Pham, P. Olivier, Slice&dice: Recognizing food preparation activities using embedded accelerometers, in: Proc. Int. Conf. Ambient Intell. (AmI), 2009.

[6]

J. Hoey, T. Plötz, D. Jackson, A. Monk, C. Pham, P. Olivier, Rapid specification and automated generation of prompting systems to assist people with dementia, Pervasive Ubiquitous Comput., 7 (2011) 299-318.

Digital Library

[7]

S. Consolvo, D.W. McDonald, T. Toscos, M.Y. Chen, J. Froehlich, B. Harrison, P. Klasnja, A. LaMarca, L. LeGrand, R. Libby, I. Smith, J.A. Landay, Activity sensing in the wild: a field trial of ubifit garden, in: Proc. ACM SIGCHI Conf. on Human Factors in Comp. Systems (CHI), 2008.

[8]

M. Rabbi, S. Ali, T. Choudhury, E. Berke, Passive and in-situ assessment of mental and physical well-being using mobile sensors, in: Proc. ACM Conf. Ubiquitous Comp. (UbiComp), 2011.

[9]

Thomas Plötz, Nils Y. Hammerla, Agata Rozga, Andrea Reavis, Nathan Call, Gregory D. Abowd, Automatic assessment of problem behavior in individuals with developmental disabilities, in: Proceedings of the 2012 ACM Conference on Ubiquitous Computing, ACM, New York, NY, USA, 2012, pp. 391-400.

Digital Library

[10]

J. Lester, T. Choudhury, G. Borriello, A practical approach to recognizing physical activities, in: Proc. Int. Conf. Pervasive Comp. (Pervasive), 2006.

[11]

J. Pärkkä, M. Ermes, P. Korpipää, J. Mäntyjärvi, J. Peltola, I. Korhonen, Activity classification using realistic data from wearable sensors, Biomedicine, 10 (2006) 119-128.

Digital Library

[12]

C.M. Bishop, Pattern Recognition and Machine Learning, Springer, 2007.

[13]

D. Figo, P. Diniz, D. Ferreira, J. Cardoso, Preprocessing techniques for context recognition from accelerometer data, Pervasive Mobile Comput., 14 (2010) 645-662.

Digital Library

[14]

T. Huynh, B. Schiele, Analyzing features for activity recognition, in: Proc. Joint Conf. on Smart objects and Ambient Intell. (sOc-EUSAI), 2005.

[15]

V. Könönen, J. Mäntyjärvi, H. Similä, J. Pärkkä, M. Ermes, Automatic feature selection for context recognition in mobile devices, Pervasive Ubiquitous Comput., 6 (2010) 181-197.

Digital Library

[16]

T. Huynh, M. Fritz, B. Schiele, Discovery of activity patterns using topic models, in: Proc. ACM Conf. Ubiquitous Comp. (UbiComp), 2008, pp. 10-19. http://doi.acm.org/10.1145/1409635.1409638.

[17]

M. Stikic, D. Larlus, S. Ebert, B. Schiele, Weakly supervised recognition of daily life activities with wearable sensors, IEEE Trans. Pattern Anal. Mach. Intell., 33 (2011) 2521-2537.

Digital Library

[18]

R. Raina, A. Battle, H. Lee, B. Packer, A.Y. Ng, Self-taught learning: transfer learning from unlabeled data, in: Proc. Int. Conf. on Machine Learning (ICML), 2007.

[19]

R. Grosse, R. Raina, H. Kwong, A.Y. Ng, Shift-invariance sparse coding for audio classification, in: Proc. Int. Conf. Uncertainty Art. Intell. (UAI), 2007.

[20]

D. Roggen, A. Calatroni, M. Rossi, T. Holleczek, K. Förster, G. Tröster, P. Lukowicz, D. Bannach, G. Pirkl, A. Ferscha, J. Doppler, C. Holzmann, M. Kurz, G. Holl, R. Chavarriaga, H. Sagha, H. Bayati, M. Creatura, J. del R. Millán, Collecting complex activity datasets in highly rich networked sensor environments, in: Networked Sensing Systems (INSS), 2010 Seventh International Conference on, 2010, pp. 233-240. http://dx.doi.org/10.1109/INSS.2010.5573462.

[21]

O. Amft, Self-taught learning for activity spotting in on-body motion sensor data, in: Proc. Int. Symp. Wearable Comp. (ISWC), 2011.

Digital Library

[22]

Semi-Supervised Learning, in: Semi-Supervised Learning, MIT Press, 2010.

[23]

D. Guan, W.Y. Lee, Y.-K. Lee, A. Gavrilov, S. Lee, Activity recognition based on semi-supervised learning, in: Proc. IEEE Int. Conf. on Embedded and Real-Time Comp. Systems and Applications (RTCSA), 2007.

[24]

M. Stikic, K.B. Schiele, Exploring semi-supervised and active learning for activity recognition, in: Proc. Int. Symp. Wearable Comp. (ISWC), 2008.

Digital Library

[25]

K. Nigam, A.K. McCallum, S. Thrun, T. Mitchell, Text classification from labeled and unlabeled documents using EM, Mach. Learn., 39 (2000) 103-134.

[26]

T. Stiefmeier, D. Roggen, G. Tröster, G. Ogris, P. Lukowicz, Wearable activity tracking in car manufacturing, IEEE Pervasive Comput., 7 (2008) 42-50.

Digital Library

[27]

H. Alemar, T.L.M. van Kasteren, C. Ersoy, Using active learning to allow activity recognition on a large scale, in: Proc. Int. Joint Conf. Ambient Intell. (AmI), Springer, 2011.

[28]

R. Caruana, Multitask learning, Mach. Learn., 28 (1997) 41-75.

Digital Library

[29]

D.H. Hu, V.W. Zheng, Q. Yang, Cross-domain activity recognition via transfer learning, Pervasive Ubiquitous Comput., 7 (2011) 344-358.

Digital Library

[30]

T.L.M. van Kasteren, G. Englebienne, B.J.A. Kröse, Transferring knowledge of activity recognition across sensor networks, in: Proc. Int. Conf. Pervasive Comp. (Pervasive), 2010.

[31]

Nicholas D. Lane, Ye Xu, Hong Lu, Shaohan Hu, Tanzeem Choudhury, Andrew T. Campbell, Feng Zhao, Enabling large-scale human activity inference on smartphones using community similarity networks (CSN), in: Proceedings of the 13th International Conference on Ubiquitous Computing, ACM, New York, NY, USA, 2011, pp. 355-364.

Digital Library

[32]

R.A. Amar, D.R. Dooly, S.A. Goldman, Q. Zhang, Multiple-instance learning of real-valued data, in: Proc. Int. Conf. on Machine Learning (ICML), 2001.

[33]

M. Stikic, B. Schiele, Activity recognition from sparsely labeled data using multi-instance learning, in: Proc. Int. Symp. on Location and Context-Awareness (LoCA), 2009.

Digital Library

[34]

A. Coates, H. Lee, A.Y. Ng, An analysis of single-layer networks in unsupervised feature learning, in: Proc. Int. Conf. Art. Intell. and Statistics (AISTAT), 2011.

[35]

G.E. Hinton, S. Osindero, Y.-W. Teh, A fast learning algorithm for deep belief nets, Neural Comput., 18 (2006) 1527-1554.

Digital Library

[36]

J. Mantyjärvi, J. Himberg, T. Seppänen, Recognizing human motion with multiple acceleration sensors, in: 2001 IEEE International Conference on Systems, Man, and Cybernetics, vol. 2, 2001, pp. 747-752.

[37]

T. Plötz, N.Y. Hammerla, P. Olivier, Feature learning for activity recognition in ubiquitous computing, in: Proc. Int. Joint Conf. Art. Intell. (IJCAI), 2011.

[38]

N. Hammerla, R. Kirkham, P. Andras, T. Plötz, On preserving statistical characteristics of accelerometry data using their empirical cumulative distribution, in: Proc. Int. Symp. Wearable Computing (ISWC), 2013.

[39]

D. Minnen, T. Starner, I. Essa, C. Isbell, Discovering characteristic actions from on-body sensor data, in: Proc. Int. Symp. Wearable Comp. (ISWC), 2006.

[40]

J. Frank, S. Mannor, D. Precup, Activity and gait recognition with time-delay embeddings, in: Proc. AAAI Conf. Art. Intell. (AAAI), 2010.

[41]

P.O. Hoyer, Non-negative sparse coding, in: Proc. IEEE Workshop on Neural Networks for Signal Processing, 2002.

[42]

H. Lee, A. Battle, R. Raina, A.Y. Ng, Efficient sparse coding algorithms, in: Proc. Int. Conf. Neural Information Proc. Systems (NIPS), 2007.

[43]

Bruno A. Olshausen, David J. Field, Sparse coding with an overcomplete basis set: a strategy employed by V1, Vis. Res., 37 (1997) 3311-3325.

[44]

P. Berkhin, Survey of clustering data mining techniques, in: Grouping Multidimensional Data, Springer, 2006, pp. 25-71.

[45]

David Lazer, Alex S. Pentland, Lada Adamic, Sinan Aral, Albert L. Barabasi, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, Yony Jebara, Gary King, Michael Macy, Deb Roy, Marshall V. Alstyne, Life in the network: the coming age of computational social science, Science, 323 (2009) 721-723.

[46]

L. Liao, D.J. Patterson, D. Fox, H. Kautz, Learning and inferring transportation routines, Artificial Intelligence, 171 (2007).

[47]

Y. Zheng, Y. Liu, J. Yuan, X. Xie, Urban computing with taxicabs, in: Proc. ACM Conf. Ubiquitous Comp. (UbiComp), 2011.

[48]

D. Soper, Is human mobility tracking a good idea?, Commun. ACM, 55 (2012) 35-37.

Digital Library

[49]

T. Brezmes, J.-L. Gorricho, J. Cotrina, Activity recognition from accelerometer data on a mobile phone, in: Workshop Proc. of 10th Int. Work-Conference on Artificial Neural Networks (IWANN), 2009.

[50]

Sasank Reddy, Min Mun, Jeff Burke, Deborah Estrin, Mark Hansen, Mani Srivastava, Using mobile phones to determine transportation modes, ACM Trans. on Sensor Networks, 6 (2010) 13:1-13:27.

Digital Library

[51]

S. Wang, C. Chen, J. Ma, Accelerometer based transportation mode recognition on mobile phones, in: Proc. Asia-Pacific Conf. on Wearable Computing Systems.

[52]

S. Hemminki, P. Nurmi, S. Tarkoma, Accelerometer-based transportation mode detection on smartphones, in: Embedded Networked Sensor Systems (SenSys), 2013.

[53]

Hong Lu, Jun Yang, Zhigang Liu, Nicholas D. Lane, Tanzeem Choudhury, Andrew T. Campbell, The Jigsaw continuous sensing engine for mobile phone applications, in: Proceedings of the 8th ACM Conference on Embedded Networked Sensor Systems, ACM, New York, NY, USA, 2010, pp. 71-84.

Digital Library

[54]

T. Plötz, P. Moynihan, C. Pham, P. Olivier, Activity recognition and healthier food preparation, in: Activity Recognition in Pervasive Intelligent Environments, Vol. 4, Atlantis Press, 2011, pp. 313-329.

[55]

I. Joliffe, Principal Component Analysis, Springer, 1986.

[56]

H. Sagha, S. Digumarti, J. del R. Millan, R. Chavarriaga, A. Calatroni, D. Roggen, G. Tröster, Benchmarking classification techniques using the opportunity human activity dataset, in: IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2011.

[57]

Q. McNemar, Note on the sampling error of the difference between correlated proportions or percentages, Psychometrika, 12 (1947) 153-157.

[58]

P.-N. Tan, M. Steinbach, V. Kumar, Introduction to Data Mining, Addison-Wesley Longman Publishing Co., Inc., 2005.

[59]

M.B. Kjærgaard, S. Bhattacharya, H. Blunck, P. Nurmi, Energy-efficient trajectory tracking for mobile devices, in: The 9th International Conference on Mobile Systems, Applications and Services, 2011, pp. 307-320.

[60]

S. Bhattacharya, H. Blunck, M. Kjærgaard, P. Nurmi, Robust and energy-efficient trajectory tracking for mobile devices, IEEE Trans. Mobile Comput., PP (2014) 1-1.

[61]

P. Lukowicz, G. Pirkl, D. Bannach, F. Wagner, A. Calatroni, K. Förster, T. Holleczek, M. Rossi, D. Roggen, G. Tröster, J. Doppler, C. Holzmann, A. Riener, A. Ferscha, R. Chavarriaga, Recording a complex, multi modal activity data set for context recognition, in: ARCS Workshops, 2010, pp. 161-166.

[62]

A. Bulling, U. Blanke, B. Schiele, A tutorial on human activity recognition using body-worn inertial sensors, ACM Comput. Surv., 46 (2014) 33:1-33:33.

Digital Library

Cited By

Gong TKim YOrzikulova ALiu YHwang SShin JLee S(2023)DAPPERProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962567:2(1-27)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596256
Ige AMohd Noor M(2023)A deep local-temporal architecture with attention for lightweight human activity recognitionApplied Soft Computing10.1016/j.asoc.2023.110954149:PAOnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.asoc.2023.110954
Basly HOuarda WSayadi FOuni BAlimi A(2022)DTR-HAR: deep temporal residual representation for human activity recognitionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-021-02064-y38:3(993-1013)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s00371-021-02064-y
Show More Cited By

Using unlabeled data in a sparse-coding framework for human activity recognition
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning

Recommendations

Activity discovering and modelling with labelled and unlabelled data in smart environments

We propose an activity recognition model balancing accuracy, overhead, data labelling.We propose a similarity measurement method to effectively discover activity patterns.We perform comprehensive experimental and comparison studies to validate our ...
Exploiting unlabeled data to enhance ensemble diversity

Ensemble learning learns from the training data by generating an ensemble of multiple base learners. It is well-known that to construct a good ensemble with strong generalization ability, the base learners are deemed to be accurate as well as diverse. ...
SelfPAB: large-scale pre-training on accelerometer data for human activity recognition
Abstract
Annotating accelerometer-based physical activity data remains a challenging task, limiting the creation of robust supervised machine learning models due to the scarcity of large, labeled, free-living human activity recognition (HAR) datasets. ...

Comments

Information & Contributors

Information

Published In

cover image Pervasive and Mobile Computing

Pervasive and Mobile Computing Volume 15, Issue C

December 2014

262 pages

ISSN:1574-1192

Issue’s Table of Contents

Copyright © Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 December 2014

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

28
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Gong TKim YOrzikulova ALiu YHwang SShin JLee S(2023)DAPPERProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962567:2(1-27)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3596256
Ige AMohd Noor M(2023)A deep local-temporal architecture with attention for lightweight human activity recognitionApplied Soft Computing10.1016/j.asoc.2023.110954149:PAOnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.asoc.2023.110954
Basly HOuarda WSayadi FOuni BAlimi A(2022)DTR-HAR: deep temporal residual representation for human activity recognitionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-021-02064-y38:3(993-1013)Online publication date: 1-Mar-2022
https://dl.acm.org/doi/10.1007/s00371-021-02064-y
Lago PMatsuki MAdachi KInoue S(2021)Using additional training sensors to improve single-sensor complex activity recognitionProceedings of the 2021 ACM International Symposium on Wearable Computers10.1145/3460421.3480421(18-22)Online publication date: 21-Sep-2021
https://dl.acm.org/doi/10.1145/3460421.3480421
Tang CPerez-Pozuelo ISpathis DBrage SWareham NMascolo C(2021)SelfHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34481125:1(1-30)Online publication date: 30-Mar-2021
https://dl.acm.org/doi/10.1145/3448112
Ma HZhang ZLi WLu S(2021)Unsupervised Human Activity Representation Learning with Multi-task Deep ClusteringProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34480745:1(1-25)Online publication date: 30-Mar-2021
https://dl.acm.org/doi/10.1145/3448074
Li CTong CNiu DJiang BZuo XCheng LXiong JYang J(2021)Similarity Embedding Networks for Robust Human Activity RecognitionACM Transactions on Knowledge Discovery from Data10.1145/344802115:6(1-17)Online publication date: 19-May-2021
https://dl.acm.org/doi/10.1145/3448021
Faridee AKhan MPathak NRoy NPoor VHan Z(2019)AugToActProceedings of the 16th EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services10.1145/3360774.3360831(162-171)Online publication date: 12-Nov-2019
https://dl.acm.org/doi/10.1145/3360774.3360831
Nair HTan CZeng MMengshoel OShen JHarle RFarrahi KLane N(2019)AttriNetAdjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341162.3345600(510-517)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341162.3345600
Saeed AOzcelebi TLukkien J(2019)Multi-task Self-Supervised Learning for Human Activity DetectionProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33289323:2(1-30)Online publication date: 21-Jun-2019
https://dl.acm.org/doi/10.1145/3328932
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents