research-article

Self-Gated Recurrent Neural Networks for Human Activity Recognition on Wearable Devices

Authors:

Jia-Ching WangAuthors Info & Claims

Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017

Pages 179 - 185

https://doi.org/10.1145/3126686.3126764

Published: 23 October 2017 Publication History

Abstract

This paper develops a self-gated recurrent neural network (SGRNN), and applies it to human activity recognition (HAR), using time-series signals collected from embedded sensors of wearable devices. Recurrent neural networks (RNNs) are very powerful for time-series signal analysis. Especially, by integrating gates into recurrent units, gated RNNs such as LSTM and GRU are more complexity, and do not suffer from the vanishing gradient problem, so can learn very long-term dependencies. However, for use on wearable devices, RNNs must be simplified to reduce resource consumption, including memory usage and computational cost. The proposed model is approximately the same size and burdensome computation as that of a standard RNN, but exhibits explicit properties of the gating mechanism, so it is unaffected by the problem of vanishing gradients. Experimental results on the HAR problem not only demonstrate that the accuracy of our model is superior to that of the standard RNN, and is comparable with that of LSTM and GRU, but the model is low in resource consumption.

References

[1]

Mohammad Abu Alsheikh, Ahmed Selim, Dusit Niyato, Linda Doyle, Shaowei Lin, and Hwee-Pink Tan. 2016. Deep Activity Recognition Models with Triaxial Accelerometers. In Artificial Intelligence Applied to Assistive Technologies and Smart Environments, Papers from the 2016 AAAI Workshop, Phoenix, Arizona, USA, February 12, 2016. http://www.aaai.org/ocs/index.php/WS/AAAIW16/paper/view/12627

[2]

Snehal Kumar Chennuru, Peng-Wen Chen, Jiang Zhu, and Joy Ying Zhang. 2010. Mobile Lifelogger - Recording, Indexing, and Understanding a Mobile User's Life. In Mobile Computing, Applications, and Services - Second International ICST Conference, MobiCASE 2010, Santa Clara, CA, USA, October 25-28, 2010, Revised Selected Papers. 263--281.

[3]

Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1724--1734. arXiv:1406.1078

[4]

Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR abs/1412.3555 (2014). http://arxiv.org/abs/1412.3555

[5]

Hanjun Dai, Bo Dai, Yan-Ming Zhang, and Le Song Shuang Li. 2017. Recurrent hidden semi-Markov model. ICLR. (2017).

[6]

Davide Figo, Pedro C. Diniz, Diogo R. Ferreira, and João M. Cardoso. 2010. Preprocessing Techniques for Context Recognition from Accelerometer Data. Personal Ubiquitous Comput. 14, 7 (Oct. 2010), 645--662.

Digital Library

[7]

Yuan Gao and Dorota Glowacka. 2016. Deep Gate Recurrent Neural Network. In Proceedings of The 8th Asian Conference on Machine Learning, ACML 2016, Hamilton, New Zealand, November 16-18, 2016. 350--365. http://jmlr.org/proceedings/papers/v63/gao30.html

[8]

Alex Graves, Abdel-Rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In International Conference on Acoustics, Speech and Signal Processing (ICASSP). 6645--6649. arXiv:arXiv:1303.5778v1

[9]

S. Ha and S. Choi. 2016. Convolutional neural networks for human activity recognition using multiple accelerometer and gyroscope sensors. In 2016 International Joint Conference on Neural Networks (IJCNN). 381--388.

[10]

Nils Y. Hammerla, Shane Halloran, and Thomas Plötz. 2016. Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables. In Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI'16). AAAI Press, 1533--1540. http://dl.acm.org/citation.cfm?id=3060832.3060835

Digital Library

[11]

Sepp Hochreiter, S. Hochreiter, Jürgen Schmidhuber, and J. Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--80. arXiv:1206.2944

Digital Library

[12]

Tâm Huynh and Bernt Schiele. 2005. Analyzing Features for Activity Recognition. In Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-aware Services: Usages and Technologies. ACM, New York, NY, USA, 159--163.

Digital Library

[13]

Masaya Inoue, Sozo Inoue, and Takeshi Nishida. 2016. Deep Recurrent Neural Network for Mobile Human Activity Recognition with High Throughput. CoRR abs/1611.03607 (2016). http://arxiv.org/abs/1611.03607

[14]

Wenchao Jiang and Zhaozheng Yin. 2015. Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks. In Proceedings of the 23rd ACM International Conference on Multimedia (MM '15). ACM, New York, NY, USA, 1307--1310.

Digital Library

[15]

Rafal Józefowicz, Wojciech Zaremba, and Ilya Sutskever. 2015. An Empirical Exploration of Recurrent Network Architectures. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015. 2342--2350.

Digital Library

[16]

Quoc V. Le, Navdeep Jaitly, and Geoffrey E. Hinton. 2015. A Simple Way to Initialize Recurrent Networks of Rectified Linear Units. arXiv preprint arXiv:1504.00941 (2015), 1--9. arXiv:arXiv:1504.00941v1

[17]

R. Pacanu, T. Mikolov, and Y. Bengio. 2013. On the difficulties of training recurrent neural networks. In Proceedings of the 30th International Conference on Machine Learning, ICML 2013. 1310--1318. arXiv:arXiv:1211.5063v2

Digital Library

[18]

Thomas Plötz, Nils Y. Hammerla, and Patrick Olivier. 2011. Feature Learning for Activity Recognition in Ubiquitous Computing. In IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011. 1729--1734.

Digital Library

[19]

D. Ravi, C. Wong, B. Lo, and G. Z. Yang. 2016. Deep learning for human activity recognition: A resource efficient implementation on low-power devices. In 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN). 71--76.

[20]

Jorge Luis Reyes-Ortiz, Luca Oneto, Albert Samà, Xavier Parra, and Davide Anguita. 2016. Transition-Aware Human Activity Recognition Using Smartphones. Neurocomputing 171 (2016), 754--767.

Digital Library

[21]

T. Tieleman and G. Hinton. 2012. Lecture 6.5 - RmsProp: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning. (2012).

[22]

Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2015, Boston, MA, USA, June 7-12, 2015. 3156--3164.

[23]

Toan H. Vu, Le Dung, and Jia-Ching Wang. 2016. Transportation Mode Detection on Mobile Devices Using Recurrent Nets. In Proceedings of the 2016 ACM Conference on Multimedia Conference, MM 2016, Amsterdam, The Netherlands, October 15-19, 2016. 392--396.

Digital Library

[24]

Ronald J. Williams and David Zipser. 1995. Gradient-Based Learning Algorithms for Recurrent Networks and Their Computational Complexity. (1995).

[25]

Pang Wu, Huan-Kai Peng, Jiang Zhu, and Ying Zhang. 2011. SensCare: Semiautomatic Activity Summarization System for Elderly Care. In Mobile Computing, Applications, and Services - Third International Conference, MobiCASE 2011, Los Angeles, CA, USA, October 24-27, 2011. Revised Selected Papers. 1--19.

[26]

Pang Wu, Jiang Zhu, and Joy Ying Zhang. 2013. MobiSens: A Versatile Mobile Sensing Platform for Real-World Applications. MONET 18, 1 (2013), 60--80.

Digital Library

[27]

Yuhuai Wu, Saizheng Zhang, Ying Zhang, Yoshua Bengio, and Ruslan Salakhutdinov. 2016. On Multiplicative Integration with Recurrent Neural Networks. CoRR abs/1606.06630 (2016). http://arxiv.org/abs/1606.06630

[28]

Meng-Chieh Yu, Tong Yu, Shao-Chen Wang, Chih-Jen Lin, and Edward Y. Chang. 2014. Big data small footprint: The design of a low-power classifier for detecting transportation modes. In Proceedings of the VLDB Endowment (PVLDB), Vol. 7. 1429--1440.

Digital Library

[29]

Piero Zappi, Clemens Lombriser, Thomas Stiefmeier, Elisabetta Farella, Daniel Roggen, Luca Benini, and Gerhard Tröster. 2008. Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection. In Wireless Sensor Networks, 5th European Conference, EWSN 2008, Bologna, Italy, January 30 - February 1, 2008, Proceedings. 17--33.

Digital Library

[30]

Matthew D. Zeiler, Marc'Aurelio Ranzato, Rajat Monga, Mark Z. Mao, K. Yang, Quoc Viet Le, Patrick Nguyen, Andrew W. Senior, Vincent Vanhoucke, Jeffrey Dean, and Geoffrey E. Hinton. 2013. On rectified linear units for speech processing. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2013, Vancouver, BC, Canada, May 26-31, 2013. 3517--3521.

[31]

M. Zeng, L. T. Nguyen, B. Yu, O. J. Mengshoel, J. Zhu, P. Wu, and J. Zhang. 2014. Convolutional Neural Networks for human activity recognition using mobile sensors. In 6th International Conference on Mobile Computing, Applications and Services. 197--205.

[32]

Mi Zhang and Alexander A. Sawchuk. 2011. A Feature Selection-based Framework for Human Activity Recognition Using Wearable Multimodal Sensors. In Proceedings of the 6th International Conference on Body Area Networks (BodyNets '11). ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), ICST, Brussels, Belgium, Belgium, 92--98. http://dl.acm.org/citation.cfm?id=2318776.2318798

Digital Library

Cited By

Li YWu JLi WFang A(2024)DWOSC: Dynamic Weight Optimization and Smoothness Constraint for Sensor-Based Human Activity RecognitionIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.336627773(1-11)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3366277
Wang XLi YFang AHe PGuo Y(2024)A Dual Pipeline With Spatio-Temporal Attention Fusion Approach for Human Activity RecognitionIEEE Sensors Journal10.1109/JSEN.2024.341629524:15(25150-25162)Online publication date: 1-Aug-2024
https://doi.org/10.1109/JSEN.2024.3416295
Thakur DFortino G(2024)A Comprehensive Review of Deep Learning for Activity RecognitionActivity Recognition and Prediction for Smart IoT Environments10.1007/978-3-031-60027-2_4(67-95)Online publication date: 27-May-2024
https://doi.org/10.1007/978-3-031-60027-2_4
Show More Cited By

Index Terms

Self-Gated Recurrent Neural Networks for Human Activity Recognition on Wearable Devices
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Clustering and classification
  2. Information systems applications
    1. Mobile information processing systems

Recommendations

Minimal gated unit for recurrent neural networks

Recurrent neural networks (RNN) have been very successful in handling sequence data. However, understanding RNN and finding the best practices for RNN learning is a difficult task, partly because there are many competing and complex hidden units, such ...
A stable online self-constructing recurrent neural network
ISNN'11: Proceedings of the 8th international conference on Advances in neural networks - Volume Part III

A new online self-constructing recurrent neural network (SCRNN) model is proposed, of which the network structure could adjust according to the specific problem in real time. If the approximation performance of SCRNN is insufficient, SCRNN can create ...
A new type of recurrent neural network for handwritten character recognition
ICDAR '95: Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 1) - Volume 1

The authors propose a new type of recurrent neural network for handwritten character recognition. The proposed recurrent neural network differs from Jordan and Elman recurrent neural networks in view of functions and architectures because it was ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Thematic Workshops '17: Proceedings of the on Thematic Workshops of ACM Multimedia 2017

October 2017

558 pages

ISBN:9781450354165

DOI:10.1145/3126686

Program Chairs:
Wanmin Wu
Google, USA
,
Jianchao Yang
Snap Inc., USA
,
Qi Tian
The University of Texas at San Antonio, USA
,
Roger Zimmermann
National University of Singapore, Singapore

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '17

Sponsor:

SIGMM

MM '17: ACM Multimedia Conference

October 23 - 27, 2017

California, Mountain View, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
300
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li YWu JLi WFang A(2024)DWOSC: Dynamic Weight Optimization and Smoothness Constraint for Sensor-Based Human Activity RecognitionIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.336627773(1-11)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3366277
Wang XLi YFang AHe PGuo Y(2024)A Dual Pipeline With Spatio-Temporal Attention Fusion Approach for Human Activity RecognitionIEEE Sensors Journal10.1109/JSEN.2024.341629524:15(25150-25162)Online publication date: 1-Aug-2024
https://doi.org/10.1109/JSEN.2024.3416295
Thakur DFortino G(2024)A Comprehensive Review of Deep Learning for Activity RecognitionActivity Recognition and Prediction for Smart IoT Environments10.1007/978-3-031-60027-2_4(67-95)Online publication date: 27-May-2024
https://doi.org/10.1007/978-3-031-60027-2_4
Li YWu JFang ALi W(2023)A Simple Optimization Strategy via Contrastive Loss for Recognizing Human Activity Using Wearable SensorsIEEE Sensors Journal10.1109/JSEN.2023.330321423:18(21588-21598)Online publication date: 15-Sep-2023
https://doi.org/10.1109/JSEN.2023.3303214
Alawneh LAl-Ayyoub MAl-Sharif ZShatnawi A(2022)Personalized human activity recognition using deep learning and edge-cloud architectureJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-022-03752-w14:9(12021-12033)Online publication date: 18-Feb-2022
https://doi.org/10.1007/s12652-022-03752-w
Valarezo Añazco EHan SKim KLopez PKim TLee S(2021)Hand Gesture Recognition Using Single Patchable Six-Axis Inertial Measurement Unit via Recurrent Neural NetworksSensors10.3390/s2104140421:4(1404)Online publication date: 17-Feb-2021
https://doi.org/10.3390/s21041404
Kumar PChauhan S(2021)Human activity recognition with deep learning: overview, challenges and possibilitiesCCF Transactions on Pervasive Computing and Interaction10.1007/s42486-021-00063-5Online publication date: 9-Apr-2021
https://doi.org/10.1007/s42486-021-00063-5
Zhang FYang Y(2021)Trust model simulation of cross border e-commerce based on machine learning and Bayesian networkJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-021-03066-3Online publication date: 13-Mar-2021
https://doi.org/10.1007/s12652-021-03066-3
Alawneh LAlsarhan TAl-Zinati MAl-Ayyoub MJararweh YLu H(2021)Enhancing human activity recognition using deep learning and time series augmented dataJournal of Ambient Intelligence and Humanized Computing10.1007/s12652-020-02865-4Online publication date: 7-Jan-2021
https://doi.org/10.1007/s12652-020-02865-4
Gholamrezaii MAlModarresi S(2021)A time-efficient convolutional neural network model in human activity recognitionMultimedia Tools and Applications10.1007/s11042-020-10435-1Online publication date: 26-Feb-2021
https://doi.org/10.1007/s11042-020-10435-1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents