research-article

Public Access

Deep Convolutional Bidirectional LSTM Based Transportation Mode Recognition

Authors:

Jeya Vikranth Jeyakumar,

Sandeep Singh Sandha,

Mani SrivastavaAuthors Info & Claims

UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers

Pages 1606 - 1615

https://doi.org/10.1145/3267305.3267529

Published: 08 October 2018 Publication History

Abstract

Traditional machine learning approaches for recognizing modes of transportation rely heavily on hand-crafted feature extraction methods which require domain knowledge. So, we propose a hybrid deep learning model: Deep Convolutional Bidirectional-LSTM (DCBL) which combines convolutional and bidirectional LSTM layers and is trained directly on raw sensor data to predict the transportation modes. We compare our model to the traditional machine learning approaches of training Support Vector Machines and Multilayer Perceptron models on extracted features. In our experiments, DCBL performs better than the feature selection methods in terms of accuracy and simplifies the data processing pipeline. The models are trained on the Sussex-Huawei Locomotion-Transportation (SHL) dataset. The submission of our team, Vahan, to SHL recognition challenge uses an ensemble of DCBL models trained on raw data using the different combination of sensors and window sizes and achieved an F1-score of 0.96 on our test data.

References

[1]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, and others. 2016. Tensorflow: a system for large-scale machine learning. In OSDI, Vol. 16. 265--283.

Digital Library

[2]

Oresti Banos, Juan-Manuel Galvez, Miguel Damas, Hector Pomares, and Ignacio Rojas. 2014. Window size impact in human activity recognition. Sensors 14, 4 (2014), 6474--6499.

[3]

Ling Bao and Stephen S Intille. 2004. Activity recognition from user-annotated acceleration data. In International Conference on Pervasive Computing. Springer, 1--17.

[4]

Andreas Bulling, Ulf Blanke, and Bernt Schiele. 2014. A tutorial on human activity recognition using body-worn inertial sensors. ACM Computing Surveys (CSUR) 46, 3 (2014), 33.

Digital Library

[5]

François Chollet and others. 2015. Keras: Deep learning library for theano and tensorflow. (2015).

[6]

Mathias Ciliberto, Francisco Javier Ordoñez Morales, Hristijan Gjoreski, Daniel Roggen, Sami Mekki, and Stefan Valentin. 2017. High reliability Android application for multidevice multimodal mobile data acquisition and annotation. In Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems. ACM, 62.

Digital Library

[7]

Ç Berke Erdaş, Işıl Atasoy, Koray Açıcı, and Hasan Oğul. 2016. Integrating features for accelerometer-based activity recognition. Procedia Computer Science 98 (2016), 522--527.

Digital Library

[8]

Shih-Hau Fang, Yu-Xaing Fei, Zhezhuang Xu, and Yu Tsao. 2017. Learning transportation modes from smartphone sensors based on deep neural network. IEEE Sensors Journal 17, 18 (2017), 6111--6118.

[9]

Shih-Hau Fang, Hao-Hsiang Liao, Yu-Xiang Fei, Kai-Hsiang Chen, Jen-Wei Huang, Yu-Ding Lu, and Yu Tsao. 2016. Transportation modes classification using sensors on smartphones. Sensors 16, 8 (2016), 1324.

[10]

Hristijan Gjoreski, Jani Bizjak, Martin Gjoreski, and Matjaž Gams. 2016. Comparing deep and classical machine learning methods for human activity recognition using wrist accelerometer. In Proceedings of the IJCAI 2016 Workshop on Deep Learning for Artificial Intelligence, New York, NY, USA, Vol. 10.

[11]

Hristijan Gjoreski, Mathias Ciliberto, Francisco Javier Ordoñez Morales, Daniel Roggen, Sami Mekki, and Stefan Valentin. 2017. A versatile annotated dataset for multimodal locomotion analytics with mobile devices. In Proceedings of the 15th ACM Conference on Embedded Network Sensor Systems. ACM, 61.

Digital Library

[12]

Hristijan Gjoreski, Mathias Ciliberto, Lin Wang, Francisco Javier Ordonez Morales, Sami Mekki, Stefan Valentin, and Daniel Roggen. 2018. The University of Sussex-Huawei locomotion and transportation dataset for multimodal analytics with mobile devices. IEEE Access (2018).

[13]

Nils Y Hammerla, Shane Halloran, and Thomas Ploetz. 2016. Deep, convolutional, and recurrent models for human activity recognition using wearables. arXiv preprint arXiv:1604.08880 (2016).

Digital Library

[14]

Arash Jahangiri and Hesham Rakha. 2014. Developing a support vector machine (SVM) classifier for transportation mode identification by using mobile phone sensor data. In Transportation Research Board 93rd Annual Meeting.

[15]

Arash Jahangiri and Hesham A Rakha. 2015. Applying Machine Learning Techniques to Transportation Mode Recognition Using Mobile Phone Sensor Data. IEEE Trans. Intelligent Transportation Systems 16, 5 (2015), 2406--2417.

Digital Library

[16]

Jennifer R Kwapisz, Gary M Weiss, and Samuel A Moore. 2011. Activity recognition using cell phone accelerometers. ACM SigKDD Explorations Newsletter 12, 2 (2011), 74--82.

Digital Library

[17]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436.

[18]

Abdulmajid Murad and Jae-Young Pyun. 2017. Deep recurrent neural networks for human activity recognition. Sensors 17, 11 (2017), 2556.

[19]

Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. 2017. Exploring generalization in deep learning. In Advances in Neural Information Processing Systems. 5947--5956.

[20]

Tsuyoshi Okita and Sozo Inoue. 2017. Recognition of multiple overlapping activities using compositional CNN-LSTM model. In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. ACM, 165--168.

Digital Library

[21]

Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.

[22]

Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, and others. 2011. Scikit-learn: Machine learning in Python. Journal of machine learning research 12, Oct (2011), 2825--2830.

Digital Library

[23]

Sasank Reddy, Min Mun, Jeff Burke, Deborah Estrin, Mark Hansen, and Mani Srivastava. 2010. Using mobile phones to determine transportation modes. ACM Transactions on Sensor Networks (TOSN) 6, 2 (2010), 13.

Digital Library

[24]

Chenguang Shen, Bo-Jhang Ho, and Mani Srivastava. 2018. Milift: Efficient smartwatch-based workout tracking using automatic segmentation. IEEE Transactions on Mobile Computing 17, 7 (2018), 1609--1622.

[25]

Xuan Song, Hiroshi Kanasugi, and Ryosuke Shibasaki. 2016. DeepTransport: Prediction and Simulation of Human Mobility and Transportation Mode at a Citywide Level. In IJCAI, Vol. 16. 2618--2624.

Digital Library

[26]

Leon Stenneth, Ouri Wolfson, Philip S Yu, and Bo Xu. 2011. Transportation mode detection using mobile phones and GIS information. In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 54--63.

Digital Library

[27]

Jindong Wang, Yiqiang Chen, Shuji Hao, Xiaohui Peng, and Lisha Hu. 2018a. Deep learning for sensor-based activity recognition: A survey. Pattern Recognition Letters (2018).

[28]

Lin Wang, Hristijan Gjoreski, Kazuya Murao, Tsuyoshi Okita, and Daniel Roggen. 2018b. Summary of the Sussex-Huawei Locomotion-Transportation Recognition Challenge. Proceedings of the 6th International Workshop on Human Activity Sensing Corpus and Applications (HASCA2018). (2018).

Digital Library

[29]

Zhibin Xiao, Yang Wang, Kun Fu, and Fan Wu. 2017. Identifying different transportation modes from trajectory data using tree-based ensemble classifiers. ISPRS International Journal of Geo-Information 6, 2 (2017), 57.

[30]

Jianbo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiaoli Li, and Shonali Krishnaswamy. 2015. Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. In IJCAI. 3995--4001.

Digital Library

[31]

Gong Yanyun, Zhao Fang, Chen Shaomeng, and Luo Haiyong. 2017. A convolutional neural networks based transportation mode identification algorithm. In Indoor Positioning and Indoor Navigation (IPIN), 2017 International Conference on. IEEE, 1--7.

[32]

Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, and Joy Zhang. 2014. Convolutional neural networks for human activity recognition using mobile sensors. In Mobile Computing, Applications and Services (MobiCASE), 2014 6th International Conference on. IEEE, 197--205.

[33]

Faming Zhang, Xinyan Zhu, Wei Guo, Xinyue Ye, Tao Hu, and Liang Huang. 2016. Analyzing urban human mobility patterns through a thematic model at a finer scale. ISPRS International Journal of Geo-Information 5, 6 (2016), 78.

[34]

Yu Zheng, Quannan Li, Yukun Chen, Xing Xie, and Wei-Ying Ma. 2008. Understanding mobility based on GPS data. In Proceedings of the 10th international conference on Ubiquitous computing. ACM, 312--321.

Digital Library

Cited By

Li GXu RShi TDeng XLiu YDi DZhao CLiu G(2024)Fine-Grained Metro-Trip Detection from Cellular Trajectory Data Using Local and Global Spatial–Temporal CharacteristicsISPRS International Journal of Geo-Information10.3390/ijgi1309031413:9(314)Online publication date: 30-Aug-2024
https://doi.org/10.3390/ijgi13090314
Jeyakumar JSarker AGarcia LSrivastava M(2023)X-CHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808047:1(1-28)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580804
Lee SLee JLee K(2023)DeepVehicleSense: An Energy-Efficient Transportation Mode Recognition Leveraging Staged Deep Learning Over Sound SamplesIEEE Transactions on Mobile Computing10.1109/TMC.2022.314139222:6(3270-3286)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TMC.2022.3141392
Show More Cited By

Deep Convolutional Bidirectional LSTM Based Transportation Mode Recognition
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches

Recommendations

Facial expression recognition using bidirectional LSTM - CNN
Abstract
Nowadays, there has been much attention on computer vision regarding human-computer interaction, especially facial expression recognition (FER). Many researchers have explored and suggested systems for this field. In this paper, we propose the ...
A Survey on Deep Learning for Human Activity Recognition
Human activity recognition is a key to a lot of applications such as healthcare and smart home. In this study, we provide a comprehensive survey on recent advances and challenges in human activity recognition (HAR) with deep learning. Although there are ...
Automatic Radar Waveform Recognition Based on Deep Convolutional Denoising Auto-encoders

Aimed at the deficiency of traditional feature extraction techniques in radar emitter recognition, a novel deep feature extraction and recognition architecture is proposed. To fit into the model, the time-domain emitters are transformed into unique time-...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UbiComp '18: Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers

October 2018

1881 pages

ISBN:9781450359665

DOI:10.1145/3267305

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
University of Florida: University of Florida
SIGCHI: ACM Special Interest Group on Computer-Human Interaction

In-Cooperation

SIGSPATIAL: ACM Special Interest Group on Spatial Information

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 October 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

U.S. Army Research Laboratory and the UK Ministry of Defence
King Abdullah University of Science and Technology
National Science Foundation
NIH Center of Excellence for Mobile Sensor Data-to-Knowledge

Conference

UbiComp '18

Sponsor:

SIGMOBILE
University of Florida
SIGCHI

UbiComp '18: The 2018 ACM International Joint Conference on Pervasive and Ubiquitous Computing

October 8 - 12, 2018

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 764 of 2,912 submissions, 26%

Upcoming Conference

UbiComp '24

Sponsor:
sigchi
sigchi

The 2024 ACM International Joint Conference on Pervasive and Ubiquitous Computing

October 5 - 9, 2024

Melbourne , VIC , Australia

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
756
Total Downloads

Downloads (Last 12 months)97
Downloads (Last 6 weeks)6

Reflects downloads up to 30 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li GXu RShi TDeng XLiu YDi DZhao CLiu G(2024)Fine-Grained Metro-Trip Detection from Cellular Trajectory Data Using Local and Global Spatial–Temporal CharacteristicsISPRS International Journal of Geo-Information10.3390/ijgi1309031413:9(314)Online publication date: 30-Aug-2024
https://doi.org/10.3390/ijgi13090314
Jeyakumar JSarker AGarcia LSrivastava M(2023)X-CHARProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35808047:1(1-28)Online publication date: 28-Mar-2023
https://dl.acm.org/doi/10.1145/3580804
Lee SLee JLee K(2023)DeepVehicleSense: An Energy-Efficient Transportation Mode Recognition Leveraging Staged Deep Learning Over Sound SamplesIEEE Transactions on Mobile Computing10.1109/TMC.2022.314139222:6(3270-3286)Online publication date: 1-Jun-2023
https://doi.org/10.1109/TMC.2022.3141392
Pias TEisenberg DFresneda Fernandez J(2022)Accuracy Improvement of Vehicle Recognition by Using Smart Device SensorsSensors10.3390/s2212439722:12(4397)Online publication date: 10-Jun-2022
https://doi.org/10.3390/s22124397
Hussain AZafar KBaig AAlmakki RAlSuwaidan LKhan S(2022)Sensor-Based Gym Physical Exercise Recognition: Data Acquisition and ExperimentsSensors10.3390/s2207248922:7(2489)Online publication date: 24-Mar-2022
https://doi.org/10.3390/s22072489
Zhang SLi YZhang SShahabi FXia SDeng YAlshurafa N(2022)Deep Learning in Human Activity Recognition with Wearable Sensors: A Review on AdvancesSensors10.3390/s2204147622:4(1476)Online publication date: 14-Feb-2022
https://doi.org/10.3390/s22041476
Mishra RGupta AGupta HDutta T(2022)A Sensors Based Deep Learning Model for Unseen Locomotion Mode Identification using Multiple Semantic MatricesIEEE Transactions on Mobile Computing10.1109/TMC.2020.301554621:3(799-810)Online publication date: 1-Mar-2022
https://doi.org/10.1109/TMC.2020.3015546
Zhu YLiu YYu JYuan X(2022)Semi-Supervised Federated Learning for Travel Mode Identification From GPS TrajectoriesIEEE Transactions on Intelligent Transportation Systems10.1109/TITS.2021.309201523:3(2380-2391)Online publication date: Mar-2022
https://doi.org/10.1109/TITS.2021.3092015
Zhang CZhu YMarkos CYu SYu J(2022)Toward Crowdsourced Transportation Mode Identification: A Semisupervised Federated Learning ApproachIEEE Internet of Things Journal10.1109/JIOT.2021.31320569:14(11868-11882)Online publication date: 15-Jul-2022
https://doi.org/10.1109/JIOT.2021.3132056
Matthes PSpringer T(2022)Selecting Resource-Efficient ML Models for Transport Mode Detection on Mobile Devices2022 IEEE International Conference on Internet of Things and Intelligence Systems (IoTaIS)10.1109/IoTaIS56727.2022.9976004(135-141)Online publication date: 24-Nov-2022
https://doi.org/10.1109/IoTaIS56727.2022.9976004
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents