research-article

Zero-shot human activity recognition via nonlinear compatibility based method

Authors:

Shuji HaoAuthors Info & Claims

WI '17: Proceedings of the International Conference on Web Intelligence

Pages 322 - 330

https://doi.org/10.1145/3106426.3106526

Published: 23 August 2017 Publication History

Abstract

Human activity recognition aims to recognize human activities from sensor readings. Most of existing methods in this area can only recognize activities contained in training dataset. However, in practical applications, previously unseen activities are often encountered. In this paper, we propose a new zero-shot learning method to solve the problem of recognizing previously unseen activities. The proposed method learns a nonlinear compatibility function between feature space instances and semantic space prototypes. With this function, testing instances are classified to unseen activities with highest compatibility scores. To evaluate the effectiveness of the proposed method, we conduct extensive experiments on three public datasets. Experimental results show that our proposed method consistently outperforms state-of-the-art methods in human activity recognition problems.

References

[1]

Zeynep Akata, Mateusz Malinowski, Mario Fritz, and Bernt Schiele. 2016. Multi-cue zero-shot learning with strong supervision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 59--68.

[2]

Zeynep Akata, Florent Perronnin, Zaid Harchaoui, and Cordelia Schmid. 2013. Label-embedding for attribute-based classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 819--826.

Digital Library

[3]

Zeynep Akata, Scott Reed, Daniel Walter, Honglak Lee, and Bernt Schiele. 2015. Evaluation of output embeddings for fine-grained image classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2927--2936.

[4]

Lei Jimmy Ba, Kevin Swersky, Sanja Fidler, and others. 2015. Predicting deep zero-shot convolutional neural networks using textual descriptions. In Proceedings of the IEEE International Conference on Computer Vision. 4247--4255.

Digital Library

[5]

Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46, 3 (2014), 33.

Digital Library

[6]

José Carlos Castillo, Davide Carneiro, Juan Serrano-Cuerda, Paulo Novais, Antonio Fernández-Caballero, and José Neves. 2014. A multi-modal approach for activity classification and fall detection. International Journal of Systems Science 45, 4 (2014), 810--824.

Digital Library

[7]

Soravit Changpinyo, Wei-Lun Chao, Boqing Gong, and Fei Sha. 2016. Synthesized classifiers for zero-shot learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5327--5336.

[8]

Liming Chen, Jesse Hoey, Chris D Nugent, Diane J Cook, and Zhiwen Yu. 2012. Sensor-based activity recognition. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 42, 6 (2012), 790--808.

Digital Library

[9]

Heng-Tze Cheng, Martin Griss, Paul Davis, Jianguo Li, and Di You. 2013. Towards zero-shot learning for human activity recognition using semantic attribute sequence model. In Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing. ACM, 355--358.

Digital Library

[10]

Heng-Tze Cheng, Feng-Tso Sun, Martin Griss, Paul Davis, Jianguo Li, and Di You. 2013. Nuactiv: Recognizing unseen new activities using semantic attribute-based learning. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services. ACM, 361--374.

Digital Library

[11]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research 12, Jul (2011), 2121--2159.

Digital Library

[12]

Mohamed Elhoseiny, Babak Saleh, and Ahmed Elgammal. 2013. Write a classifier: Zero-shot learning using purely textual descriptions. In Proceedings of the IEEE International Conference on Computer Vision. 2584--2591.

Digital Library

[13]

Nils Y. Hammerla, Shane Halloran, and Thomas Plötz. 2016. Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. 1533--1540.

Digital Library

[14]

Tâm Huynh, Mario Fritz, and Bernt Schiele. 2008. Discovery of activity patterns using topic models. In Proceedings of the 10th international conference on Ubiquitous computing. ACM, 10--19.

Digital Library

[15]

Eunju Kim, Sumi Helal, and Diane Cook. 2010. Human activity recognition and pattern discovery. IEEE Pervasive Computing 9, 1 (2010), 48--53.

Digital Library

[16]

Yongjin Kwon, Kyuchang Kang, and Changseok Bae. 2014. Unsupervised learning for human activity recognition using smartphone sensors. Expert Systems with Applications 41, 14 (2014), 6067--6074.

[17]

Hugo Larochelle, Dumitru Erhan, and Yoshua Bengio. 2008. Zero-data Learning of New Tasks. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence. 646--651.

Digital Library

[18]

M. Lichman. 2013. UCI Machine Learning Repository. (2013). http://archive.ics.uci.edu/ml

[19]

Jingen Liu, Benjamin Kuipers, and Silvio Savarese. 2011. Recognizing human actions by attributes. In The 24th IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 3337--3344.

Digital Library

[20]

Le T Nguyen, Ming Zeng, Patrick Tague, and Joy Zhang. 2015. I did not smoke 100 cigarettes today!: avoiding false positives in real-world activity recognition. In Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 1053--1063.

Digital Library

[21]

Mark Palatucci, Dean Pomerleau, Geoffrey E Hinton, and Tom M Mitchell. 2009. Zero-shot learning with semantic output codes. In Advances in neural information processing systems. 1410--1418.

Digital Library

[22]

Vijay Rajanna, Raniero Lara-Garduno, Dev Jyoti Behera, Karthic Madanagopal, Daniel Goldberg, and Tracy Hammond. 2014. Step up life: a context aware health assistant. In Proceedings of the Third ACM SIGSPATIAL International Workshop on the Use of GIS in Public Health. ACM, 21--30.

Digital Library

[23]

Scott Reed, Zeynep Akata, Honglak Lee, and Bernt Schiele. 2016. Learning deep representations of fine-grained visual descriptions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 49--58.

[24]

Attila Reiss and Didier Stricker. 2012. Introducing a new benchmarked dataset for activity monitoring. In 2012 16th International Symposium on Wearable Computers. IEEE, 108--109.

Digital Library

[25]

Daniel Roggen, Alberto Calatroni, Mirco Rossi, Thomas Holleczek, Kilian Förster, Gerhard Tröster, Paul Lukowicz, David Bannach, Gerald Pirkl, Alois Ferscha, and others. 2010. Collecting complex activity datasets in highly rich networked sensor environments. In Seventh International Conference on Networked Sensing Systems. IEEE, 233--240.

[26]

Marcus Rohrbach, Sandra Ebert, and Bernt Schiele. 2013. Transfer learning in a transductive setting. In Advances in neural information processing systems. 46--54.

Digital Library

[27]

Bernardino Romera-Paredes and Philip Torr. 2015. An embarrassingly simple approach to zero-shot learning. In Proceedings of the 32nd International Conference on Machine Learning. 2152--2161.

Digital Library

[28]

Nagender Kumar Suryadevara and Subhas Chandra Mukhopadhyay. 2012. Wireless sensor network based home monitoring system for wellness determination of elderly. IEEE Sensors Journal 12, 6 (2012), 1965--1972.

[29]

Theano Development Team. 2016. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints abs/1605.02688 (May 2016). http://arxiv.org/abs/1605.02688

[30]

Jie Wan, Michael J O'grady, and Gregory M O'hare. 2015. Dynamic sensor event segmentation for real-time activity recognition in a smart home context. Personal and Ubiquitous Computing 19, 2 (2015), 287--301.

Digital Library

[31]

Yongqin Xian, Zeynep Akata, Gaurav Sharma, Quynh Nguyen, Matthias Hein, and Bernt Schiele. 2016. Latent embeddings for zero-shot classification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 69--77.

[32]

Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiaoli Li, and Shonali Krishnaswamy. 2015. Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence. 3995--4001.

Digital Library

[33]

Yongxin Yang and Timothy M Hospedales. 2014. A unified perspective on multi-domain and multi-task learning. In International Conference on Learning Representations.

[34]

Lina Yao, Feiping Nie, Quan Z Sheng, Tao Gu, Xue Li, and Sen Wang. 2016. Learning from less for better: semi-supervised activity recognition via shared structure discovery. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM, 13--24.

Digital Library

[35]

Vincent Wenchen Zheng, Derek Hao Hu, and Qiang Yang. 2009. Cross-domain activity recognition. In Proceedings of the 11th international conference on Ubiquitous computing. ACM, 61--70.

Digital Library

Cited By

Zhang XChowdhury RZhang JHong DGupta RShang JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Unleashing the Power of Shared Label Structures for Human Activity RecognitionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615101(3340-3350)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615101
Wang WLi Q(2023)Generalized Zero-Shot Activity Recognition with Embedding-Based MethodACM Transactions on Sensor Networks10.1145/358269019:3(1-25)Online publication date: 5-Apr-2023
https://dl.acm.org/doi/10.1145/3582690
Fdez-Díaz MMontañés EQuevedo J(2023)Direct side information learning for zero-shot regressionNeurocomputing10.1016/j.neucom.2023.126873561(126873)Online publication date: Dec-2023
https://doi.org/10.1016/j.neucom.2023.126873
Show More Cited By

Recommendations

Zero-Shot Learning for IMU-Based Activity Recognition Using Video Embeddings

The Activity Recognition Chain generally precludes the challenging scenario of recognizing new activities that were unseen during training, despite this scenario being a practical and common one as users perform diverse activities at test time. A few ...
Generalized Zero-Shot Activity Recognition with Embedding-Based Method
Sensor-based human activity recognition aims to recognize the activities performed by people with the sensor readings. Most of existing works in this area rely on supervised classification algorithms, and can only recognize activities covered by the ...
Towards zero-shot learning for human activity recognition using semantic attribute sequence model
UbiComp '13: Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing

Understanding human activities is important for user-centric and context-aware applications. Previous studies showed promising results using various machine learning algorithms. However, most existing methods can only recognize the activities that were ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WI '17: Proceedings of the International Conference on Web Intelligence

August 2017

1284 pages

ISBN:9781450349512

DOI:10.1145/3106426

Conference Chair:
Amit Sheth
Wright State University GuoROLE@GENERAL CHAIR
,
General Chairs:
Axel Ngonga
Leipzig University, Germany
,
yin Wang
Chongqing University of Posts and Telecommunications, China
,
Elizabeth Chang
The University of New South Wales, Australia
,
Dominik Ślęzak
Infobright Inc. & University of Warsaw, Poland
,
Bogdan Franczyk
Leipzig University, Germany
,
Program Chairs:
Rainer Alt
Leipzig University, Germany
,
Xiaohui Tao
University of Southern Queensland, Australia

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence
TCII: IEEE Computer Society Technical Committee on Intelligent Informatics
Web Intelligence Consortium

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

National Research Foundation, Prime Minister's Office, Singapore

Conference

WI '17

Sponsor:

SIGAI
TCII

WI '17: International Conference on Web Intelligence 2017

August 23 - 26, 2017

Leipzig, Germany

Acceptance Rates

WI '17 Paper Acceptance Rate 118 of 178 submissions, 66%;

Overall Acceptance Rate 118 of 178 submissions, 66%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
342
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)7

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhang XChowdhury RZhang JHong DGupta RShang JFrommholz IHopfgartner FLee MOakes MLalmas MZhang MSantos R(2023)Unleashing the Power of Shared Label Structures for Human Activity RecognitionProceedings of the 32nd ACM International Conference on Information and Knowledge Management10.1145/3583780.3615101(3340-3350)Online publication date: 21-Oct-2023
https://dl.acm.org/doi/10.1145/3583780.3615101
Wang WLi Q(2023)Generalized Zero-Shot Activity Recognition with Embedding-Based MethodACM Transactions on Sensor Networks10.1145/358269019:3(1-25)Online publication date: 5-Apr-2023
https://dl.acm.org/doi/10.1145/3582690
Fdez-Díaz MMontañés EQuevedo J(2023)Direct side information learning for zero-shot regressionNeurocomputing10.1016/j.neucom.2023.126873561(126873)Online publication date: Dec-2023
https://doi.org/10.1016/j.neucom.2023.126873
Yang ZGu YWeng WGuo SChen Y(2022)Dual-Alignment Based Generalized Zero-shot Learning for Human Activity Recognition2022 IEEE Smartworld, Ubiquitous Intelligence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld/UIC/ScalCom/DigitalTwin/PriComp/Meta)10.1109/SmartWorld-UIC-ATC-ScalCom-DigitalTwin-PriComp-Metaverse56740.2022.00071(351-359)Online publication date: Dec-2022
https://doi.org/10.1109/SmartWorld-UIC-ATC-ScalCom-DigitalTwin-PriComp-Metaverse56740.2022.00071
Chatterjee SMitra BChakraborty S(2022)AmicroN: Framework for Generating Micro-Activity Annotations for Human Activity Recognition2022 IEEE International Conference on Smart Computing (SMARTCOMP)10.1109/SMARTCOMP55677.2022.00019(26-31)Online publication date: Jun-2022
https://doi.org/10.1109/SMARTCOMP55677.2022.00019
Fdez-Díaz MQuevedo JMontañés E(2022)Target inductive methods for zero-shot regressionInformation Sciences: an International Journal10.1016/j.ins.2022.03.075599:C(44-63)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1016/j.ins.2022.03.075
Yang XHe SWang BTabatabaie M(2021)Spatio-Temporal Graph Attention Embedding for Joint Crowd Flow and Transition PredictionsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34950035:4(1-24)Online publication date: 30-Dec-2021
https://dl.acm.org/doi/10.1145/3495003
Tabbaa LSearle RBafti SHossain MIntarasisrisawat JGlancy MAng C(2021)VREEDProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34950025:4(1-20)Online publication date: 30-Dec-2021
https://dl.acm.org/doi/10.1145/3495002
Huynh SBalan RKo J(2021)iMonProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34949995:4(1-26)Online publication date: 30-Dec-2021
https://dl.acm.org/doi/10.1145/3494999
Meyer JFrank ASchlebusch TKasneci E(2021)A CNN-based Human Activity Recognition System Combining a Laser Feedback Interferometry Eye Movement Sensor and an IMU for Context-aware Smart GlassesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/34949985:4(1-24)Online publication date: 30-Dec-2021
https://dl.acm.org/doi/10.1145/3494998
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents