research-article

A Competition to Push the Dependability of Low-Power Wireless Protocols to the Edge

Authors:

Carlo Alberto Boano,

Kay RömerAuthors Info & Claims

EWSN ’17: Proceedings of the 2017 International Conference on Embedded Wireless Systems and Networks

Pages 54 - 65

Published: 20 February 2017 Publication History

Abstract

A large number of low-power wireless communication protocols has been proposed in the last decade by both academia and industry in an attempt to deliver information in an increasingly reliable, timely, and energy-efficient manner. However, their level of dependability has rarely been benchmarked under the same settings and environmental conditions. In this paper we present the execution and results of a competition aimed to evaluate the dependability of state-of-the-art low-power wireless protocols under the same settings, and push their performance to the limit. We define a scenario emulating the operation of a wireless sensor network in industrial environments rich with radio interference and compare the end-to-end dependability of systems based on protocol strategies ranging from adaptive and time-slotted frequency-hopping to multi-modal routing and flooding. To increase fairness and realism, we allow the developers of the competing protocols to interact with the benchmarking infrastructure and optimize the protocol parameters for the scenario at hand. We achieve this by designing, implementing, and employing D-Cube, a low-cost tool that allows to accurately measure key dependability metrics such as end-to-end delay, reliability, and power consumption, as well as to graphically visualize their evolution in real-time. This interaction with the benchmarking infrastructure and the competitiveness of the event have incited the developers to push the performance of their protocols to the limit and reach impressive results.

References

[1]

NavSpark User Guide, Rev. 0.9, 2016.

[2]

B. Al Nahas and O. Landsiedel. Towards low-latency, low-power wireless networking under interference. In Proc. of the 13th EWSN Conf., competition session, 2016.

Digital Library

[3]

N. Baccour, A. Koubâa, L. Mottola, H. Youssef, M. A. Zúñiga, C. A. Boano, and M. Alves. Radio link quality estimation in wireless sensor networks: a survey. ACM (TOSN), 8(4), 2012.

Digital Library

[4]

C. A. Boano, K. Römer, and N. Tsiftes. Mitigating the adverse effects of temperature on low-power wireless protocols. In Proc. of the 11th IEEE MASS Conf., 2014.

Digital Library

[5]

C. A. Boano, T. Voigt, C. Noda, K. Römer, and M. A. Zúñiga. JamLab: Augmenting sensornet testbeds with realistic and controlled interference generation. In Proc. of the 10th IPSN Conf., 2011.

[6]

C. A. Boano, M. A. Zúñiga, J. Brown, U. Roedig, C. Keppitiyagama, and K. Römer. TempLab: A testbed infrastructure to study the impact of temperature on wireless sensor networks. In Proc. of IPSN, 2014.

Digital Library

[7]

C. A. Boano, M. A. Zúñiga, K. Römer, and T. Voigt. JAG: Reliable and predictable wireless agreement under external radio interference. In Proc. of the 33rd RTSS Conf., 2012.

Digital Library

[8]

S. Bouckaert, W. Vandenberghe, B. Jooris, I. Moerman, and P. Demeester. The w-ilab.t testbed. In Proc. of the 6th TridentCom, 2010.

[9]

C. Adjih et al. FIT IoT-LAB: A large scale open experimental IoT testbed. In Proc. of the 2nd WF-IoT, 2015.

Digital Library

[10]

M. Cattani, A. Loukas, M. Zimmerling, M. Zuniga, and K. Langendoen. Staffetta: Smart duty-cycling for opportunistic data collection. In Proc. of the 14th SenSys Conf., 2016.

Digital Library

[11]

Defense Advanced Research Projects Agency. DARPA Urban Challenge. http://archive.darpa.mil/grandchallenge/. Last visited: 27.06.2016.

[12]

M. Doddavenkatappa, M. Chan, and A. Ananda. Indriya: A low-cost, 3D wireless sensor network testbed. In Proc. of TridentCom, 2011.

[13]

A. Dunkels. The ContikiMAC radio duty cycling protocol. Technical Report T2011:13, Swedish Institute of Computer Science, 2011.

[14]

A. Dunkels, B. Grönvall, and T. Voigt. Contiki - a lightweight and flexible operating system for tiny networked sensors. In Proc. of the 1st EmNetS Workshop, 2004.

Digital Library

[15]

A. Dunkels, F. Österlind, N. Tsiftes, and Z. He. Software-based online energy estimation for sensor nodes. In Proc. of the 4th EmNetS Workshop, 2007.

Digital Library

[16]

S. Duquennoy, B. A. Nahas, O. Landsiedel, and T. Watteyne. Orchestra: Robust mesh networks through autonomously scheduled TSCH. In Proc. of the 13th SenSys Conf., 2015.

Digital Library

[17]

S. Duquennoy, F. Österlind, and A. Dunkels. Lossy links, low power, high throughput. In Proc. of the 9th SenSys Conf., 2011.

Digital Library

[18]

P. Dutta, S. Dawson-Haggerty, Y. Chen, C.-J. M. Liang, and A. Terzis. Design and evaluation of a versatile and efficient receiver-initiated link layer for low-power wireless. In Proc. of the 8th SenSys Conf., 2010.

Digital Library

[19]

E. Ertin, A. Arora, R. Ramnath, M. Sridharan, and V. Kulathumani. Kansei: A testbed for sensing at scale. In Proc. of the 5th IPSN, 2006.

Digital Library

[20]

F. Ferrari, M. Zimmerling, L. Mottola, and L. Thiele. Low-power wireless bus. In Proc. of the 10th SenSys Conf., 2012.

Digital Library

[21]

F. Ferrari, M. Zimmerling, L. Thiele, and O. Saukh. Efficient network flooding and time synchronization with Glossy. In Proc. of the 10th IPSN Conf., 2011.

[22]

K. Genter, T. Laue, and P. Stone. Benchmarking robot cooperation without pre-coordination in the robocup standard platform league drop-in player competition. In Proc. of the IROS Conf., 2015.

[23]

J. V.-V. Gerwen, S. Bouckaert, I. Moerman, and P. Demeester. Exploiting low-cost directional antennas in 2.4 GHz IEEE 802.15.4 wireless sensor networks. In Proc. of the 5th SENSORCOMM, 2011.

[24]

J. V.-V. Gerwen, E. D. Poorter, B. Latré, I. Moerman, and P. Demeester. Real-life performance of protocol combinations for wireless sensor networks. In Proc. of the SUTC Conf., 2010.

Digital Library

[25]

O. Gnawali, R. Fonseca, K. Jamieson, D. Moss, and P. Levis. Collection tree protocol. In Proc. of the 7th SenSys Conf., 2009.

Digital Library

[26]

P. H. Gomes, T. Watteyne, P. Gosh, and B. Krishnamachari. Reliability through timeslotted channel hopping and flooding-based routing. In Proc. of the 13th EWSN Conf., competition session, 2016.

Digital Library

[27]

V. Handziski, A. Köpke, A. Willig, and A. Wolisz. TWIST: a scalable and reconfigurable testbed for wireless indoor experiments with sensor networks. In Proc. of the 2nd REALMAN Workshop, 2006.

Digital Library

[28]

I. Haratcherev, G. Halkes, T. Parker, O. Visser, and K. Langendoen. PowerBench: A scalable testbed infrastructure for benchmarking power consumption. In Proc. of the 1st IWSNE Workshop, 2008.

[29]

M. Hempstead, M. Welsh, and D. Brooks. Tinybench: The case for a standardized benchmark suite for TinyOS based wireless sensor network devices. In Proc. of the 29th LCN Conf., poster session, 2004.

Digital Library

[30]

I. Galpin et al. SensorBench: Benchmarking approaches to processing wireless sensor network data. In Proc. of the 26th SSDBM, 2014.

Digital Library

[31]

T. Kim, J. Kim, S. Lee, I. Ahn, M. Song, and K. Won. An automatic protocol verification framework for the development of wireless sensor networks. In Proc. of the 4th TridentCom Conf., 2008.

Digital Library

[32]

A. King, J. Hadley, and U. Roedig. Contikimac with differentiating clear channel assessment. In Proc. of the 13th EWSN Conf., competition session, 2016.

Digital Library

[33]

L. Nazhandali et al. SenseBench: Toward an accurate evaluation of sensor network processors. In Proc. of the IISWC Symposium, 2005.

[34]

R. Lajara, J. Pelegrı́-Sebastiá, and J. J. P. Solano. Power consumption analysis of operating systems for wireless sensor networks. Sensors, 10(6), 2010.

[35]

O. Landsiedel, F. Ferrari, and M. Zimmerling. Chaos: Versatile and efficient all-to-all data sharing and in-network processing at scale. In Proc. of the 11th SenSys Conf., 2013.

Digital Library

[36]

R. Lim, F. Ferrari, M. Zimmerling, C. Walser, P. Sommer, and J. Beutel. FlockLab: A testbed for distributed, synchronized tracing and profiling of wireless embedded systems. In Proc. of the IPSN, 2013.

Digital Library

[37]

R. Lim, B. Maag, B. Dissler, J. Beutel, and L. Thiele. TraceLab: A testbed for fine-grained tracing of time sensitive behavior in wireless sensor networks. In Proc. of the 10th SenseApp Workshop, 2015.

Digital Library

[38]

Q. Luo, H. Wu, W. Xue, and B. He. Benchmarking in-network sensor query processing. Technical Report HKUST-CS05-09, The Hong Kong University of Science and Technology, 2005.

[39]

D. Lymberopoulos, J. Liu, X. Yang, R. R. Choudhury, S. Sen, and V. Handziski. Microsoft indoor localization competition: Experiences and lessons learned. GetMobile, 18(4), 2014.

Digital Library

[40]

A. Maskooki, V. Toldov, L. Clavier, V. Loscrı̀, and N. Mitton. Channel exploration/exploitation based on a thompson sampling approach in a radio cognitive environment. In Proc. of the 13th EWSN Conf., competition session, 2016.

Digital Library

[41]

M. Mohammad, X. Guo, and M. C. Chan. Oppcast: Exploiting spatial and channel diversity for robust data collection in urban environments. In Proc. of the 15th IPSN Conf., 2016.

Digital Library

[42]

S. Mysore, B. Agrawal, F. T. Chong, and T. Sherwood. Exploring the processor and ISA design for wireless sensor network applications. In Proc. of the 21st International Conference on VLSI Design, 2008.

Digital Library

[43]

S. Nabar, A. Banerjee, S. K. Gupta, and R. Poovendran. Evaluation of body sensor network platforms: A design space and benchmarking analysis. In Proc. of the Wireless Health Conference, 2010.

Digital Library

[44]

B. A. Nahas, S. Duquennoy, V. Iyer, and T. Voigt. Low-Power Listening Goes Multi-Channel. In Proc. of the 10th IEEE DCOSS, 2014.

Digital Library

[45]

F. Österlind, L. Mottola, T. Voigt, N. Tsiftes, and A. Dunkels. Strawman: Resolving collisions in bursty low-power wireless networks. In Proc. of the 11th IPSN Conf., 2012.

Digital Library

[46]

S. Duquennoy et al. A benchmark for low-power wireless networking. In Proc. of the 14th SenSys Conf., poster session, 2016.

Digital Library

[47]

F. Schmidt, M. Ceriotti, N. Hauser, and K. Wehrle. Hotbox: Testing temperature effects in sensor networks. Technical Report AIB-201414, RWTH Aachen, Germany, 2014.

[48]

F. Schmidt, M. Ceriotti, N. Hauser, and K. Wehrle. If you can’t take the heat: Temperature effects on low-power wireless networks and how to mitigate them. In Proc. of the 12th EWSN Conf., 2015.

[49]

P. Sommer and Y.-A. Pignolet. Dependable network flooding using glossy with channel-hopping. In Proc. of the 13th EWSN Conf., competition session, 2016.

Digital Library

[50]

M. K. Watfa and M. Moubarak. A benchmarking tool for wireless sensor network embedded operating systems. Journal of Networks, 9(8), 2014.

[51]

T. Watteyne, S. Lanzisera, A. Mehta, and K. S. Pister. Mitigating multipath fading through channel hopping in wireless sensor networks. In Proc. of the IEEE ICC Conf., 2010.

[52]

G. Werner-Allen, P. Swieskowski, and M. Welsh. MoteLab: a wireless sensor network testbed. In Proc. of the 4th IPSN, 2005.

Digital Library

[53]

D. Yuan and M. Hollick. Sparkle: Energy efficient, reliable, ultra-low latency communication in wireless control networks. In Proc. of the 13th EWSN Conf., competition session, 2016.

Digital Library

[54]

D. Yuan, M. Riecker, and M. Hollick. Making ’glossy’ networks sparkle: Exploiting concurrent transmissions for energy efficient, reliable, ultra-low latency communication in wireless control networks. In Proc. of the 11st EWSN Conf., 2014.

Digital Library

Cited By

Trüb RDa Forno RDaschinger LBiri ABeutel JThiele L(2021)Non-Intrusive Distributed Tracing of Wireless IoT Devices with the FlockLab 2 TestbedACM Transactions on Internet of Things10.1145/34802483:1(1-31)Online publication date: 27-Oct-2021
https://dl.acm.org/doi/10.1145/3480248

Recommendations

From Performance to Dependability Benchmarking: A Mandatory Path
Performance Evaluation and Benchmarking

The work on performance benchmarking has started long ago. Ranging from simple benchmarks that target a very specific system or component to very complex benchmarks for complex infrastructures, performance benchmarks have contributed to improve ...
How to advance TPC benchmarks with dependability aspects
TPCTC'10: Proceedings of the Second TPC technology conference on Performance evaluation, measurement and characterization of complex systems

Transactional systems are the core of the information systems of most organizations. Although there is general acknowledgement that failures in these systems often entail significant impact both on the proceeds and reputation of companies, the ...
Low power downlink MAC protocols for infrastructure wireless sensor networks

This paper addresses low power medium access control (MAC) protocols for the downlink channel of infrastructure wireless sensor networks. Access points are assumed to be energy unconstrained. The trade-off between the power consumption of the sensor ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

EWSN ’17: Proceedings of the 2017 International Conference on Embedded Wireless Systems and Networks

February 2017

336 pages

ISBN:9780994988614

General Chairs:
Per Gunningberg
Uppsala University, Sweden
,
Thiemo Voigt
Uppsala University and SICS, Sweden
,
Program Chairs:
Luca Mottola
Politecnico di Milano and SICS, Sweden
,
Chenyang Lu
Washington University in St. Louis, USA

Sponsors

EWSN: International Conference on Embedded Wireless Systems and Networks

In-Cooperation

SIGBED: ACM Special Interest Group on Embedded Systems

Publisher

Junction Publishing

United States

Publication History

Published: 20 February 2017

Check for updates

Author Tags

Qualifiers

Research-article

Acceptance Rates

EWSN ’17 Paper Acceptance Rate 18 of 49 submissions, 37%;

Overall Acceptance Rate 81 of 195 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Trüb RDa Forno RDaschinger LBiri ABeutel JThiele L(2021)Non-Intrusive Distributed Tracing of Wireless IoT Devices with the FlockLab 2 TestbedACM Transactions on Internet of Things10.1145/34802483:1(1-31)Online publication date: 27-Oct-2021
https://dl.acm.org/doi/10.1145/3480248

View Options

View options

Media

Figures

Other

Tables

View Table of Contents