research-article

Automation of feature engineering for IoT analytics

Authors:

Snehasis Banerjee,

Tanushyam Chattopadhyay,

Utpal GarainAuthors Info & Claims

ACM SIGBED Review, Volume 15, Issue 2

Pages 24 - 30

https://doi.org/10.1145/3231535.3231538

Published: 05 June 2018 Publication History

Abstract

This paper presents an approach for automation of interpretable feature selection for Internet Of Things Analytics (IoTA) using machine learning (ML) techniques. Authors have conducted a survey over different people involved in different IoTA based application development tasks. The survey reveals that feature selection is the most time consuming and niche skill demanding part of the entire workflow. This paper shows how feature selection is successfully automated without sacrificing the decision making accuracy and thereby reducing the project completion time and cost of hiring expensive resources. Several pattern recognition principles and state of art (SoA) ML techniques are followed to design the overall approach for the proposed automation. Three data sets are considered to establish the proof-of-concept. Experimental results show that the proposed automation is able to reduce the time for feature selection to 2 days instead of 4 -- 6 months which would have been required in absence of the automation. This reduction in time is achieved without any sacrifice in the accuracy of the decision making process. Proposed method is also compared against Multi Layer Perceptron (MLP) model as most of the state of the art works on IoTA uses MLP based Deep Learning. Moreover the feature selection method is compared against SoA feature reduction technique namely Principal Component Analysis (PCA) and its variants. The results obtained show that the proposed method is effective.

References

[1]

Ian Goodfellow, Yoshua Bengio and Aaron Courville Deep Learning, MIT Press, 2016.

Digital Library

[2]

Banerjee, Snehasis, et al. Windowing mechanisms for web scale stream reasoning, Proceedings of the 4th international workshop on Web-scale knowledge representation retrieval and reasoning, CIKM, 2013.

Digital Library

[3]

Banerjee, Snehasis, et al. Semantic exploration of sensor data, Proceedings of the 5th International Workshop on Web-scale Knowledge Representation Retrieval and Reasoning, CIKM, 2014.

Digital Library

[4]

T. Chattopadhyay et.al. A Smart Framework for IoT Analytic Workflow Development, In Proc. of demo of Sensys, South Korea, Nov 2015

Digital Library

[5]

Wang et. Al. A Feature Subset Selection Algorithm Automatic Recommendation Method, Journal of Artificial Intelligence Research 47 (2013) 1--34, May, 2013.

Digital Library

[6]

Hong Lu et.al. SoundSense: Scalable sound sensing for people-centric applications on mobile phones. In Proceedings of MobiSys'09.

Digital Library

[7]

Hong Lu, Jun Yang, Zhigang Liu, Nicholas D. Lane, Tanzeem Choudhury, and Andrew T. Campbell. The jigsaw continuous sensing engine for mobile phone applications. In Proceedings of SenSys'10.

Digital Library

[8]

Emiliano Miluzzo et.al. Sensing meets mobile social networks: The design, implementation and evaluation of the CenceMe application. In Proceedings of SenSys'08.

Digital Library

[9]

Nicholas D. Lane et.al. BeWell: A smartphone application to monitor, model and promote wellbeing. In Proceedings of Pervasive Health'11.

[10]

Andreas Krause et.al. Context-awaremobile computing: Learning context-dependent personal preferences from a wearable sensor array. IEEE Transactions on Mobile Computing 5, 2, 113--127.

Digital Library

[11]

Roger G. Eston et.al. Validity of heart rate, pedometry, and accelerometry for predicting the energy cost of children's activities. Journal of Applied Physiology 84, 1, 362--371.

[12]

Mashfiqui Rabbi et.al.Passive and in-situ assessment of mental and physical well-being using mobile sensors. In Proceedings of UbiComp'11.

Digital Library

[13]

Saeed Abdullah et.al. Towards population scale activity recognition: A scalable framework for handling data diversity. In Proceedings of AAAI'12.

Digital Library

[14]

Nadav Aharony et.al. The social fMRI: Measuring, understanding, and designing social mechanisms in the real world. In Proceedings of UbiComp'11.

Digital Library

[15]

Celine Coutrix and Nadine Mandran et.al.Identifying emotions expressed by mobile users through 2D surface and 3D motion gestures. In Proceedings of UbiComp'12.

Digital Library

[16]

Shaojiang Dong et.al. Bearing Degradation Process Prediction Based on the Support Vector Machine and Markov Model, Hindawi Publishing Corporation, Shock and Vibration, Volume 2014, Article ID 717465, 15 pages

[17]

Hai Qiu et.al. Robust Performance Degradation Assessment Methods for enhanced rolling element bearings prognostics

[18]

Anand Rajaraman, Jeffrey David Ullman, Mining of Massive Datasets, Cambridge University Press New York, 2010.

Digital Library

[19]

Pradipta Maji et. al Fuzzy-Rough MRMS Method for Relevant and Significant Attribute Selection, 2012, Advances on Computational Intelligence: 14th International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems.

[20]

Pradipta Maji and Sushmita Paul Dependence Issues in Knowledge-Based Systems Rough set based maximum relevance-maximum significance criterion and Gene selection from microarray data, 2011, International Journal of Approximate Reasoning.

[21]

Peng, Hanchuan, Fuhui Long, and Chris Ding. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. Pattern Analysis and Machine Intelligence, IEEE Transactions on27.8 (2005): 1226--1238.

Digital Library

[22]

Voyant, Cyril, et al. Meteorological time series forecasting based on MLP modelling using heterogeneous transfer functions. Journal of Physics: Conference Series. Vol. 574. No. 1. IOP Publishing, 2015.

[23]

W. K. Ngui et al., Wavelet Analysis: Mother Wavelet Selection Methods, Applied Mechanics and Materials, Vol. 393, pp. 953--958, 2013.

[24]

T. Batabyal, T. Chattopadhyay, D. P. Mukherjee, Action Recogntion using joint co-ordinates of 3-D Skeleton Data, ICIP 2015, Quebec, Canadada, Sept, 2015.

[25]

Banerjee, Snehasis, et al. What affects me?: a smart public alert system based on stream reasoning, Proceedings of the 7th International Conference on Ubiquitous Information Management and Communication. ACM, 2013.

Digital Library

[26]

Chattopadhyay, T. et al. Automatic selection of binarization method for robust ocr, ICDAR, pp. 1170--1174, IEEE, 2013.

Digital Library

[27]

Mukherjee, Debnath, et al. "Ad-hoc ride sharing application using continuous sparql queries." Proceedings of the 21st International Conference on World Wide Web. ACM, 2012.

Digital Library

[28]

Chattopadhyay, T., et al. Recognition of channel logos from streamed videos for value added services in connected TV, ICCE, IEEE, 2011.

[29]

Mukherjee, Debnath and Misra, Prateep and Banerjee, Snehasis. System And a Method For Reasoning And Running Continuous Queries Over Data Streams, US Patent 20,150,039,648. 2015.

[30]

S. Dong et. al., Bearing degradation process prediction based on the support vector machine and Markov model, Journal of Shock and Vibration, Hindawi Publishing Corporation, 2014.

[31]

Banerjee, Snehasis, et al. Towards Wide Learning: Experiments in Healthcare. Machine Learning for Health Workshop, NIPS 2016. arXiv preprint arXiv:1612.05730 (2016).

[32]

Banerjee, Snehasis, et al. Towards a universal notification system, In Web Intelligence (WI) and Intelligent Agent Technologies (IaT), 2013 IEEE/WIC/ACM International Joint Conferences on (Vol. 3, pp. 286--287). IEEE, 2013.

Digital Library

[33]

Mukherjee, Debnath, et al. Towards efficient stream reasoning, ODBASE, On the Move to Meaningful Internet Systems. Springer Berlin Heidelberg, 2013.

[34]

Mukherjee, Debnath, et al. System and a method for reasoning and running continuous queries over data streams, U.S. Patent Application No. 14/387,540.

Cited By

Isken MAydas ORoumani Y(2023)Queueing inspired feature engineering to improve and simplify patient flow simulation metamodelsJournal of Simulation10.1080/17477778.2023.2181716(1-18)Online publication date: 26-Feb-2023
https://doi.org/10.1080/17477778.2023.2181716
Singh VSharma KSur S(2023)A survey on preprocessing and classification techniques for acoustic sceneExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120520229:PAOnline publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120520
Choudhury ABanerjee RKimbahune SPal A(2022)Looking at the futureNew Frontiers of Cardiovascular Screening Using Unobtrusive Sensors, AI, and IoT10.1016/B978-0-12-824499-9.00008-8(191-210)Online publication date: 2022
https://doi.org/10.1016/B978-0-12-824499-9.00008-8
Show More Cited By

Index Terms

Automation of feature engineering for IoT analytics
1. Information systems
  1. Information systems applications
    1. Decision support systems
      1. Data analytics
2. Social and professional topics
  1. Professional topics
    1. Computing and business
      1. Automation

Recommendations

A systematic review of automated feature engineering solutions in machine learning problems
SBSI '20: Proceedings of the XVI Brazilian Symposium on Information Systems

During the last decades, machine learning has played an important role in building data-driven experiments. Based on information extracted from a variety of sources, new patterns can be identified, predictions can be made more easily and decisions made ...
Automated feature engineering for HTTP tunnel detection

Generating discriminative input features is a key requirement for achieving highly accurate classifiers. The process of generating features from raw data is known as feature engineering and it can take significant manual effort. In this paper we propose ...
Feature Selection and Interpretable Feature Transformation: A Preliminary Study on Feature Engineering for Classification Algorithms
Intelligent Data Engineering and Automated Learning – IDEAL 2018
Abstract
This paper explores the limitation of consistency-based measures in the context of feature selection. These kinds of filters are not very widespread in large-dimensionality problems. Typically, the number of selected of attributes is very small ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGBED Review

ACM SIGBED Review Volume 15, Issue 2

Special Issue on Advances in IoT Architecture and Systems (AIoTAS'17)

March 2018

36 pages

EISSN:1551-3688

DOI:10.1145/3231535

Editors:
Ansuman Banerjee
Indian Statistical Institute, India
,
Himadri Sekhar Paul
Tata Consultancy Services Limited, India
,
Chayan Sarkar
TCS Research, India

Issue’s Table of Contents

Copyright © 2018 Authors.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 June 2018

Published in SIGBED Volume 15, Issue 2

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

13
Total Citations
View Citations
280
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)2

Reflects downloads up to 12 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Isken MAydas ORoumani Y(2023)Queueing inspired feature engineering to improve and simplify patient flow simulation metamodelsJournal of Simulation10.1080/17477778.2023.2181716(1-18)Online publication date: 26-Feb-2023
https://doi.org/10.1080/17477778.2023.2181716
Singh VSharma KSur S(2023)A survey on preprocessing and classification techniques for acoustic sceneExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120520229:PAOnline publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120520
Choudhury ABanerjee RKimbahune SPal A(2022)Looking at the futureNew Frontiers of Cardiovascular Screening Using Unobtrusive Sensors, AI, and IoT10.1016/B978-0-12-824499-9.00008-8(191-210)Online publication date: 2022
https://doi.org/10.1016/B978-0-12-824499-9.00008-8
Gjoreski MMahesh BKolenik TUwe-Garbas JSeuss DGjoreski HLustrek MGams MPejovic V(2021)Cognitive Load Monitoring With Wearables–Lessons Learned From a Machine Learning ChallengeIEEE Access10.1109/ACCESS.2021.30932169(103325-103336)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3093216
Nazir QShao C(2021)Online tool condition monitoring for ultrasonic metal welding via sensor fusion and machine learningJournal of Manufacturing Processes10.1016/j.jmapro.2020.12.05062(806-816)Online publication date: Feb-2021
https://doi.org/10.1016/j.jmapro.2020.12.050
Pal SMishra DPal ADutta SChakravarty DPal SPal SMishra DPal ADutta SChakravarty DPal S(2021)Artificial Intelligence and Machine Learning in ManufacturingDigital Twin – Fundamental Concepts to Applications in Advanced Manufacturing10.1007/978-3-030-81815-9_6(337-412)Online publication date: 13-Aug-2021
https://doi.org/10.1007/978-3-030-81815-9_6
Jaiswal DMoulick MChatterjee DRanjan RRamakrishnan RPal AGhosh RTran VAshok A(2020)Assessment of cognitive load from bio-potentials measured using wearable endosomatic deviceProceedings of the 6th ACM Workshop on Wearable Systems and Applications10.1145/3396870.3400012(13-18)Online publication date: 19-Jun-2020
https://dl.acm.org/doi/10.1145/3396870.3400012
Pal AUkil ADeb TSahu IMajumdar A(2020)Instant Adaptive Learning: An Adaptive Filter Based Fast Learning Model Construction for Sensor Signal Time Series Classification on Edge DevicesICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP40776.2020.9054418(8339-8343)Online publication date: May-2020
https://doi.org/10.1109/ICASSP40776.2020.9054418
Bordel BAlcarria RRobles TYou I(2020)A Predictor-Corrector Algorithm Based on Laurent Series for Biological Signals in the Internet of Medical ThingsIEEE Access10.1109/ACCESS.2020.30012758(109360-109371)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3001275
Kumar PGupta GTripathi R(2020)An ensemble learning and fog-cloud architecture-driven cyber-attack detection framework for IoMT networksComputer Communications10.1016/j.comcom.2020.12.003Online publication date: Dec-2020
https://doi.org/10.1016/j.comcom.2020.12.003
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents