research-article

Integrated energy-directed test suite optimization

Authors:

William G. J. HalfondAuthors Info & Claims

ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and Analysis

Pages 339 - 350

https://doi.org/10.1145/2610384.2610414

Published: 21 July 2014 Publication History

Abstract

In situ testing techniques have become an important means of ensuring the reliability of embedded systems after they are deployed in the field. However, these techniques do not help testers optimize the energy consumption of their in situ test suites, which can needlessly waste the limited battery power of these systems. In this work, we extend prior techniques for test suite minimization in such a way as to allow testers to generate energy-efficient, minimized test suites with only minimal modifications to their existing work flow. We perform an extensive empirical evaluation of our approach using the test suites provided for real world applications. The results of the evaluation show that our technique is effective at generating, in less than one second, test suites that consume up to 95% less energy while maintaining coverage of the testing requirements.

References

[1]

lpsolve.sourceforge.net.

[2]

lpsolve.sourceforge.net/5.5/lp-format.htm.

[3]

www.reftek.com/.

[4]

N. Amsel and B. Tomlinson. Green tracker: A tool for estimating the energy consumption of software. In Proceedings of the 28th International Conference on Human Factors in Computing Systems: Extended Abstracts, pages 3337–3342, 2010.

Digital Library

[5]

T. Ball and J. R. Larus. Eﬃcient path profiling. In Proceedings of the 29th annual ACM/IEEE International Symposium on Microarchitecture, pages 46–57, 1996.

Digital Library

[6]

G. Barrenetxea, F. Ingelrest, G. Schaefer, and M. Vetterli. The hitchhiker’s guide to successful wireless sensor network deployments. In Proceedings of the 6th ACM Conference on Embedded Network Sensor Systems, pages 43–56, 2008.

Digital Library

[7]

A. Bertolino, G. De Angelis, and A. Sabetta. VCR: Virtual capture and replay for performance testing. In Proceedings of the 23rd IEEE/ACM International Conference on Automated Software Engineering, pages 399–402, 2008.

Digital Library

[8]

J. Black, E. Melachrinoudis, and D. Kaeli. Bi-criteria models for all-uses test suite reduction. In Proceedings of the 26th International Conference on Software Engineering, pages 106–115, 2004.

Digital Library

[9]

D. Brooks, V. Tiwari, and M. Martonosi. Wattch: A framework for architectural-level power analysis and optimizations. In Proceedings of the 27th annual International Symposium on Computer Architecture, pages 83–94, 2000.

Digital Library

[10]

Z. Chen and K. G. Shin. Post-deployment performance debugging in wireless sensor networks. In Proceedings of the 30th IEEE Real-Time Systems Symposium, pages 313–322, 2009.

Digital Library

[11]

T. Chikaraishi, T. Minato, and H. Ishiguro. Development of an android system integrated with sensor networks. In Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on, pages 326–333. IEEE, 2008.

[12]

S. Gurumurthi, A. Sivasubramaniam, M. J. Irwin, N. Vijaykrishnan, M. Kandemir, T. Li, and L. K. John. Using complete machine simulation for software power estimation: The SoftWatt approach. In Proceedings of the 8th International Symposium on High-Performance Computer Architecture, pages 141–151, 2002.

Digital Library

[13]

S. Hao, D. Li, W. G. J. Halfond, and R. Govindan. Estimating mobile application energy consumption using program analysis. In Proceedings of the 35th International Conference on Software Engineering, pages 92–101, 2013.

Digital Library

[14]

H.-Y. Hsu and A. Orso. MINTS: A general framework and tool for supporting test-suite minimization. In Proceedings of the 31st International Conference on Software Engineering, pages 419–429, 2009.

Digital Library

[15]

E. Y. Kan. Energy eﬃciency in testing and regression testing – a comparison of DVFS techniques. In Proceedings of the 13th International Conference on Quality Software, pages 280–283, 2013.

Digital Library

[16]

J. R. Kwapisz, G. M. Weiss, and S. A. Moore. Activity recognition using cell phone accelerometers. ACM SIGKDD Explorations Newsletter, 12(2):74–82, 2011.

Digital Library

[17]

K. Langendoen, A. Baggio, and O. Visser. Murphy loves potatoes: Experiences from a pilot sensor network deployment in precision agriculture. In Proceedings of the 20th International Conference on Parallel and Distributed Processing, pages 174–174, 2006.

Digital Library

[18]

D. Li, S. Hao, W. G. Halfond, and R. Govindan. Calculating source line level energy information for android applications. In Proceedings of the International Symposium on Software Testing and Analysis (ISSTA), July 2013.

Digital Library

[19]

D. Li, C. Sahin, J. Clause, and W. G. Halfond. Energy-directed test suite optimization. In Proceedings of the 2nd International Workshop on Green and Sustainable Software, pages 62–69, 2013.

[20]

B. Liblit, M. Naik, A. X. Zheng, A. Aiken, and M. I. Jordan. Scalable statistical bug isolation. In Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation, pages 15–26, 2005.

Digital Library

[21]

Y. Liu, C. Xu, and S. Cheung. Where has my battery gone? Finding sensor related energy black holes in smartphone applications. In Proceedings of the 2013 IEEE International Conference on Pervasive Computing and Communications, pages 2–10, 2013.

[22]

T. Menzies, A. Butcher, A. Marcus, T. Zimmermann, and D. Cok. Local vs. global models for effort estimation and defect prediction. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering, pages 343–351, 2011.

Digital Library

[23]

R. Mittal, A. Kansal, and R. Chandra. Empowering developers to estimate app energy consumption. In Proceedings of the 18th annual International Conference on Mobile Computing and Networking, pages 317–328, 2012.

Digital Library

[24]

T. Mudge, T. Austin, and D. Grunwald. The reference manual for the Sim-Panalyzer, version 2.0. http://web.eecs.umich.edu/~panalyzer/.

[25]

C. Murphy, G. Kaiser, I. Vo, and M. Chu. Quality assurance of software applications using the in vivo testing approach. In Proceedings of the 2nd International Conference on Software Testing Verification and Validation, pages 111–120, 2009.

Digital Library

[26]

A. Noureddine, A. Bourdon, R. Rouvoy, and L. Seinturier. A preliminary study of the impact of software engineering on GreenIT. In Proceedings of the First International Workshop on Green and Sustainable Software, pages 21–27, 2012.

[27]

J. Paek, J. Kim, and R. Govindan. Energy-eﬃcient rate-adaptive GPS-based positioning for smartphones. In Proceedings of the 8th international Conference on Mobile Systems, Applications, and Services, pages 299–314, 2010.

Digital Library

[28]

A. Pathak, Y. C. Hu, and M. Zhang. Bootstrapping energy debugging on smartphones: A first look at energy bugs in mobile devices. In Proceedings of the 10th ACM Workshop on Hot Topics in Networks, pages 5:1–5:6, 2011.

Digital Library

[29]

A. Pathak, A. Jindal, Y. C. Hu, and S. P. Midkiff. What is keeping my phone awake?: Characterizing and detecting no-sleep energy bugs in smartphone apps. In Proceedings of the 10th International Conference on Mobile Systems, Applications, and Services, pages 267–280, 2012.

Digital Library

[30]

C. Pavlopoulou and M. Young. Residual test coverage monitoring. In Proceedings of the 21st International Conference on Software Engineering, pages 277–284, 1999.

Digital Library

[31]

B. Priyantha, D. Lymberopoulos, and J. Liu. LittleRock: Enabling energy-eﬃcient continuous sensing on mobile phones. IEEE Pervasive Computing, pages 12–15, 2011.

Digital Library

[32]

G. Rothermel and M. J. Harrold. Analyzing regression test selection techniques. IEEE Transactions Software Engineering, 22(8):529–551, 1996.

Digital Library

[33]

G. Rothermel and M. J. Harrold. A safe, eﬃcient regression test selection technique. ACM Transactions on Software Engineering and Methodology, 6(2):173–210, 1997.

Digital Library

[34]

C. Sahin, F. Cayci, I. Gutierrez, J. Clause, F. Kiamilev, L. Pollock, and K. Winbladh. Initial explorations on design pattern energy usage. In Proceedings of the First International Workshop on Green and Sustainable Software, pages 55–61, 2012.

[35]

C. Seeger, A. Buchmann, and K. Van Laerhoven. myhealthassistant: A phone-based body sensor network that captures the wearer’s exercises throughout the day. In Proceedings of the 6th International Conference on Body Area Networks, pages 1–7. ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2011.

Digital Library

[36]

D. Singh, P. A. H. Peterson, P. L. Reiher, and W. J. Kaiser. The Atom LEAP platform for energy-eﬃcient embedded computing: Architecture, operation, and system implementation, 2010. http://lasr.cs.ucla.edu/leap/FrontPage?action= AttachFile&do=get&target=leapwhitepaper.pdf.

[37]

R. Smith, J. Das, H. Heidarsson, A. Pereira, F. Arrichiello, I. Cetnic, L. Darjany, M.-E. Garneau, M. Howard, C. Oberg, M. Ragan, E. Seubert, E. Smith, B. Stauffer, A. Schnetzer, G. Toro-farmer, D. Caron, B. Jones, and G. Sukhatme. USC CINAPS builds bridges. Robotics & Automation Magazine, IEEE, 17(1):20––30, 2010.

[38]

G. Tolle, J. Polastre, R. Szewczyk, D. Culler, N. Turner, K. Tu, S. Burgess, T. Dawson, P. Buonadonna, D. Gay, and W. Hong. A macroscope in the redwoods. In Proceedings of the 3rd International Conference on Embedded Networked Sensor Systems, pages 51–63, 2005.

Digital Library

[39]

watts up. https://www.wattsupmeters.com/secure/index.php.

[40]

G. Werner-Allen, J. Johnson, M. Ruiz, J. Lees, and M. Welsh. Monitoring volcanic eruptions with a wireless sensor network. In Wireless Sensor Networks, 2005. Proceeedings of the Second European Workshop on, pages 108–120. IEEE, 2005.

[41]

G. Werner-Allen, K. Lorincz, J. Johnson, J. Lees, and M. Welsh. Fidelity and yield in a volcano monitoring sensor network. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation, pages 381–396, 2006.

Digital Library

[42]

J. Whipple, W. Arensman, and M. S. Boler. A public safety application of gps-enabled smartphones and the android operating system. In Systems, Man and Cybernetics, 2009. SMC 2009. IEEE International Conference on, pages 2059–2061. IEEE, 2009.

Digital Library

[43]

S. Yoo and M. Harman. Regression testing minimization, selection and prioritization: a survey. Software Testing, Verification and Reliability, 22(2):67–120, 2012.

[44]

T. Zimmermann, N. Nagappan, H. Gall, E. Giger, and B. Murphy. Cross-project defect prediction: A large scale experiment on data vs. domain vs. process. In Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 91–100, 2009.

Digital Library

Cited By

Kim JKim MLee ELo DMcIntosh SNovielli N(2022)ECenchProceedings of the 19th International Conference on Mining Software Repositories10.1145/3524842.3528028(634-638)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1145/3524842.3528028
Ozener OSozer H(2022)Summary of An Effective Formulation of the Multi-Criteria Test Suite Minimization Problem2022 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST53961.2022.00052(459-459)Online publication date: Apr-2022
https://doi.org/10.1109/ICST53961.2022.00052
Couto MFernandes JSaraiva J(2021)Statically Analyzing the Energy Efficiency of Software Product LinesJournal of Low Power Electronics and Applications10.3390/jlpea1101001311:1(13)Online publication date: 23-Mar-2021
https://doi.org/10.3390/jlpea11010013
Show More Cited By

Index Terms

Integrated energy-directed test suite optimization
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Energy-aware test-suite minimization for Android apps
ISSTA 2016: Proceedings of the 25th International Symposium on Software Testing and Analysis

The rising popularity of mobile apps deployed on battery-constrained devices has motivated the need for effective energy-aware testing techniques. Energy testing is generally more labor intensive and expensive than functional testing, as tests need to ...
Energy-directed test suite optimization
GREENS '13: Proceedings of the 2nd International Workshop on Green and Sustainable Software

Post-deployment, in situ testing and validation techniques have become an important means of ensuring the reliability of mobile and embedded systems. However, these techniques do not take into consideration the amount of energy they consume, which is an ...
Directed Test Suite Augmentation
APSEC '09: Proceedings of the 2009 16th Asia-Pacific Software Engineering Conference

As software evolves, engineers use regression testing to evaluate its fitness for release. Such testing typically begins with existing test cases, and many techniques have been proposed for reusing these cost-effectively. After reusing test cases, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and Analysis

July 2014

460 pages

ISBN:9781450326452

DOI:10.1145/2610384

General Chair:
Corina S. Păsăreanu
NASA Ames, USA
,
Program Chair:
Darko Marinov
University of Illinois at Urbana-Champaign, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '14

Sponsor:

SIGSOFT

ISSTA '14: International Symposium on Software Testing and Analysis

July 21 - 25, 2014

CA, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
313
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)2

Reflects downloads up to 09 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kim JKim MLee ELo DMcIntosh SNovielli N(2022)ECenchProceedings of the 19th International Conference on Mining Software Repositories10.1145/3524842.3528028(634-638)Online publication date: 23-May-2022
https://dl.acm.org/doi/10.1145/3524842.3528028
Ozener OSozer H(2022)Summary of An Effective Formulation of the Multi-Criteria Test Suite Minimization Problem2022 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST53961.2022.00052(459-459)Online publication date: Apr-2022
https://doi.org/10.1109/ICST53961.2022.00052
Couto MFernandes JSaraiva J(2021)Statically Analyzing the Energy Efficiency of Software Product LinesJournal of Low Power Electronics and Applications10.3390/jlpea1101001311:1(13)Online publication date: 23-Mar-2021
https://doi.org/10.3390/jlpea11010013
Noureddine AMartinez MKanso H(2021)A Preliminary Study of the Impact of Code Coverage on Software Energy Consumption2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW)10.1109/ASEW52652.2021.00057(263-264)Online publication date: Nov-2021
https://doi.org/10.1109/ASEW52652.2021.00057
Escobar-Velásquez CMazuera-Rozo ABedoya COsorio-Riaño MLinares-Vásquez MBavota G(2021)Studying eventual connectivity issues in Android appsEmpirical Software Engineering10.1007/s10664-021-10020-627:1Online publication date: 27-Nov-2021
https://doi.org/10.1007/s10664-021-10020-6
Maia DCouto MSaraiva JPereira RGrundy JLe Goues CLo D(2020)E-DebitumProceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering10.1145/3417113.3422999(170-177)Online publication date: 21-Sep-2020
https://dl.acm.org/doi/10.1145/3417113.3422999
Couto MSaraiva JFernandes J(2020)Energy Refactorings for Android in the Large and in the Wild2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER48275.2020.9054858(217-228)Online publication date: Feb-2020
https://doi.org/10.1109/SANER48275.2020.9054858
Zhou JLi FDong JZhang HHao D(2020)Cost-Effective Testing of a Deep Learning Model through Input Reduction2020 IEEE 31st International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE5003.2020.00035(289-300)Online publication date: Oct-2020
https://doi.org/10.1109/ISSRE5003.2020.00035
Özener OSözer H(2020)An effective formulation of the multi-criteria test suite minimization problemJournal of Systems and Software10.1016/j.jss.2020.110632168(110632)Online publication date: Oct-2020
https://doi.org/10.1016/j.jss.2020.110632
Mazuera-Rozo ATrubiani CLinares-Vásquez MBavota G(2020)Investigating types and survivability of performance bugs in mobile appsEmpirical Software Engineering10.1007/s10664-019-09795-6Online publication date: 5-Mar-2020
https://doi.org/10.1007/s10664-019-09795-6
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents