research-article

Izinto: a pattern-based IoT testing framework

Authors:

Pedro Martins Pontes,

João Pascoal FariaAuthors Info & Claims

ISSTA '18: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops

Pages 125 - 131

https://doi.org/10.1145/3236454.3236511

Published: 16 July 2018 Publication History

Abstract

The emergence of Internet of Things (IoT) technology is expected to offer new promising solutions in various domains and, consequently, impact many aspects of everyday life. However, the development and testing of software applications and services for IoT systems encompasses several challenges that existing solutions have not yet properly addressed. Particularly, the difficulty to test IoT systems - due to their heterogeneous and distributed nature -, and the importance of testing in the development process give rise to the need for an efficient way to implement automated testing in IoT. Although there are already several tools that can be used in the testing of IoT systems, a number of issues can be pointed out, such as focusing on a specific platform, language, or standard, limiting the possibility of improvement or extension, and not providing out-of-the-box functionalities. This paper describes Izinto, a pattern-based test automation framework for integration testing of IoT systems. The framework implements in a generic way a set of test patterns specific to the IoT domain which can be easily instantiated for concrete IoT scenarios. It was validated in a number of test cases, within a concrete application scenario in the domain of Ambient Assisted Living (AAL).

References

[1]

Cedric Adjih, Emmanuel Baccelli, Eric Fleury, Gaetan Harter, Nathalie Mitton, Thomas Noel, Roger Pissard-Gibollet, Frederic Saint-Marcel, Guillaume Schreiner, Julien Vandaele, and Thomas Watteyne. 2015. FIT IoT-LAB: A large scale open experimental IoT testbed. In IEEE World Forum on Internet of Things, WF-IoT 2015 - Proceedings. 459--464.

Digital Library

[2]

Abbas Ahmad, Fabrice Bouquet, Elizabeta Fourneret, Franck Le Gall, and Bruno Legeard. 2016. Model-based testing as a service for IoT platforms. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 9953 LNCS. 727--742.

[3]

Ba Bagula and Zenville Erasmus. 2015. Iot emulation with Cooja. In Workshop on Scientific Applications for the Internet of Things ICTP.

[4]

Don Batory. 2005. Feature models, grammars, and propositional formulas. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 3714 LNCS. 7--20.

Digital Library

[5]

Boris Beizer. 2003. Software Testing Techniques. 550 pages. http://books.google.com/books?id=Ixf97h356zcC

[6]

L Coetzee and J Eksteen. 2011. The Internet of Things - promise for the future? An introduction. IST-Africa Conference Proceedings, 2011 (2011), 1--9. https://doi.org/ISBN:978-1-905824-24-3

[7]

João Pedro Dias, Flávio Couto, Ana C.R. Paiva, and Hugo Sereno Ferreira. 2018. A Brief Overview of Existing Tools for Testing the Internet-of-Things.

[8]

Roy Thomas Fielding. 2000. Architectural Styles and the Design of Network-based Software Architectures. Ph.D. Dissertation. AAI9980887.

[9]

Ralf Gitzel, Subanatarajan Subbiah, and Christopher Ganz. 2018. A Data Quality Dashboard for CMMS Data. ICORES 2018 Icores (2018), 170--177.

[10]

Alexander Gluhak, Srdjan Krco, Michele Nati, Dennis Pfisterer, Nathalie Mitton, and Tahiry Razafindralambo. 2011. A survey on facilities for experimental internet of things research., 58--67 pages. arXiv:1609.07712

[11]

Harshit Gupta, Amir VahidDastjerdi, Soumya K. Ghosh, and Rajkumar Buyya. 2017. iFogSim: A toolkit for modeling and simulation of resource management techniques in the Internet of Things, Edge and Fog computing environments. In Software - Practice and Experience, Vol. 47. 1275--1296. arXiv:1606.02007

[12]

JUnit. {n. d.}. JUnit - About. https://junit.org/junit4/. Accessed: 2018-05-24.

[13]

Dmitry G. Korzun, Sergey I. Balandin, and Andrei V. Gurtov. 2013. Deployment ofSmart Spaces in Internet of Things: Overview of the Design Challenges. In Internet of Things, Smart Spaces, and Next Generation Networking, Sergey Balandin, Sergey Andreev, and Yevgeni Koucheryavy (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 48--59.

[14]

Philip Levis and Nelson Lee. 2003. TOSSIM: a simulator for TinyOS networks. (12 2003).

[15]

Shancang Li, Li Da Xu, and Shanshan Zhao. 2015. The internet of things: a survey. Information Systems Frontiers 17, 2 (2015), 243--259. arXiv:arXiv:1011.1669v3

Digital Library

[16]

Bruno Lima and João Pascoal Faria. 2017. A Survey on Testing Distributed and Heterogeneous Systems: The State of the Practice. In Software Technologies, Enrique Cabello, Jorge Cardoso, André Ludwig, Leszek A. Maciaszek, and Marten van Sinderen (Eds.). Springer International Publishing, Cham, 88--107.

[17]

Vilen Looga, Zhonghong Ou, Yang Deng, and Antti Yla-Jaaski. 2013. MAMMOTH: A massive-scale emulation platform for Internet of Things. In Proceedings - 2012 IEEE 2nd International Conference on Cloud Computing and Intelligence Systems, IEEE CCIS 2012, Vol. 3. 1235--1239.

[18]

Nitinder Mohan and Jussi Kangasharju. 2017. Edge-Fog cloud: A distributed cloud for Internet of Things computations. In 2016 Cloudification of the Internet of Things, CIoT 2016. arXiv:1702.06335

[19]

Matthew Murdoch. 2013. ArduinoUnit. https://github.com/mmurdoch/arduinounit. Accessed: 2018-05-10.

[20]

Amy Nordrum. 2016. Popular internet of things forecast of 50 billion devices by 2020 is outdated. https://spectrum.ieee.org/tech-talk/telecom/internet/popular-internet-of-hings-forecast-of-50-billion-devices-by-2020-is-outdated

[21]

OASIS. 2014. MQTT Version 3.1.1. OASIS Standard October (2014), 81. http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html

[22]

PlatformIO. 2018. An open source ecosystem for IoT development - PlatformIO. https://platformio.org/. Accessed: 2018-05-16.

[23]

Philipp Rosenkranz, Matthias Wählisch, Emmanuel Baccelli, and Ludwig Ortmann. 2015. A Distributed Test System Architecture for Open-source IoT Software. In Proceedings of the 2015 Workshop on IoT challenges in Mobile and Industrial Systems - IoT-Sys '15. 43--48.

Digital Library

[24]

Lu Tan. 2010. Future internet: The Internet of Things. 2010 3<sup>rd</sup> International Conference on Advanced Computer Theory and Engineering(ICACTE) (2010), V5-376--V5-380.

Cited By

Minani JSabir FMoha NGuéhéneuc Y(2024)A Systematic Review of IoT Systems Testing: Objectives, Approaches, Tools, and ChallengesIEEE Transactions on Software Engineering10.1109/TSE.2024.336361150:4(785-815)Online publication date: Apr-2024
https://doi.org/10.1109/TSE.2024.3363611
Minani JSabir FMoha NGuéhéneuc Y(2024)A Multimethod Study of Internet of Things Systems Testing in IndustryIEEE Internet of Things Journal10.1109/JIOT.2023.329123311:1(1662-1684)Online publication date: 1-Jan-2024
https://doi.org/10.1109/JIOT.2023.3291233
Ojo-Gonzalez KBonilla-Morales BVargas-Lombardo M(2023)Software Quality in the IOT in Health Sector and Commerce SectorAdvanced Research in Technologies, Information, Innovation and Sustainability10.1007/978-3-031-48858-0_2(14-25)Online publication date: 20-Dec-2023
https://doi.org/10.1007/978-3-031-48858-0_2
Show More Cited By

Recommendations

Test patterns for IoT
A-TEST 2018: Proceedings of the 9th ACM SIGSOFT International Workshop on Automating TEST Case Design, Selection, and Evaluation

The Internet of Things (IoT) is expected to bring forward new promising solutions in various domains. Consequently, it can impact many aspects of everyday life, and errors can have serious consequences. Despite this, there is a lack of standard testing ...
Adapting ACATS to the Ahven testing framework
Ada-Europe'11: Proceedings of the 16th Ada-Europe international conference on Reliable software technologies

The Ada Conformity Assessment Test Suite (ACATS) includes thousands of individual executable test programs, but no test driver or tools for grading the output. We show how ACATS can be adapted to work with the Ahven testing framework, resulting in a ...
Conformance Testing in Service-Oriented Interconnection Technologies in NoTA
ITNG '11: Proceedings of the 2011 Eighth International Conference on Information Technology: New Generations

This paper introduces a testing framework for conformance testing in service-oriented interconnection technologies. The conformance testing framework is utilized in a case study, which concentrates on testing Network on Terminal Architecture (NoTA) ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA '18: Companion Proceedings for the ISSTA/ECOOP 2018 Workshops

July 2018

143 pages

ISBN:9781450359399

DOI:10.1145/3236454

Conference Chairs:
Julian Dolby
IBM Thomas J. Watson Research Center
,
William G. J. Halfond
University of Southern California
,
Ashish Mishra
Northeastern University, United States.

Copyright © 2018 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 July 2018

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '18

Sponsor:

ISSTA '18: International Symposium on Software Testing and Analysis

July 16 - 21, 2018

Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
304
Total Downloads

Downloads (Last 12 months)21
Downloads (Last 6 weeks)1

Reflects downloads up to 04 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Minani JSabir FMoha NGuéhéneuc Y(2024)A Systematic Review of IoT Systems Testing: Objectives, Approaches, Tools, and ChallengesIEEE Transactions on Software Engineering10.1109/TSE.2024.336361150:4(785-815)Online publication date: Apr-2024
https://doi.org/10.1109/TSE.2024.3363611
Minani JSabir FMoha NGuéhéneuc Y(2024)A Multimethod Study of Internet of Things Systems Testing in IndustryIEEE Internet of Things Journal10.1109/JIOT.2023.329123311:1(1662-1684)Online publication date: 1-Jan-2024
https://doi.org/10.1109/JIOT.2023.3291233
Ojo-Gonzalez KBonilla-Morales BVargas-Lombardo M(2023)Software Quality in the IOT in Health Sector and Commerce SectorAdvanced Research in Technologies, Information, Innovation and Sustainability10.1007/978-3-031-48858-0_2(14-25)Online publication date: 20-Dec-2023
https://doi.org/10.1007/978-3-031-48858-0_2
Almeida PFaria JLima B(2022)A Pattern-Based Test Platform for Families of Smart Health Products2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS57517.2022.00066(595-605)Online publication date: Dec-2022
https://doi.org/10.1109/QRS57517.2022.00066
Weiss KRottleuthner MSchmidt TWählisch M(2021)PHiLIP on the HiL: Automated Multi-Platform OS Testing With External Reference DevicesACM Transactions on Embedded Computing Systems10.1145/347704020:5s(1-26)Online publication date: 22-Sep-2021
https://dl.acm.org/doi/10.1145/3477040
Bures MAhmed BRechtberger VKlima MTrnka MJaros MBellekens XAlmog DHerout P(2021)PatrIoT: IoT Automated Interoperability and Integration Testing Framework2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST49551.2021.00059(454-459)Online publication date: Apr-2021
https://doi.org/10.1109/ICST49551.2021.00059
Hu LWong WKuhn DKacker RLi S(2021)CT-IoT: a combinatorial testing-based path selection framework for effective IoT testingEmpirical Software Engineering10.1007/s10664-021-10017-127:2Online publication date: 14-Dec-2021
https://doi.org/10.1007/s10664-021-10017-1
Araujo Rodriguez LMacêdo Batista DKhurshid SPăsăreanu C(2020)Program-aware fuzzing for MQTT applicationsProceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis10.1145/3395363.3402645(582-586)Online publication date: 18-Jul-2020
https://dl.acm.org/doi/10.1145/3395363.3402645
Varghese NSinha R(2020)Can Commercial Testing Automation Tools Work for IoT? A Case Study of Selenium and Node-RedIECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society10.1109/IECON43393.2020.9254910(4519-4524)Online publication date: 18-Oct-2020
https://doi.org/10.1109/IECON43393.2020.9254910
Bures MKlima MRechtberger VBellekens XTachtatzis CAtkinson RAhmed B(2020)Interoperability and Integration Testing Methods for IoT Systems: A Systematic Mapping StudySoftware Engineering and Formal Methods10.1007/978-3-030-58768-0_6(93-112)Online publication date: 8-Sep-2020
https://doi.org/10.1007/978-3-030-58768-0_6
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents