research-article

Pairwise feature-interaction testing for SPLs: potentials and limitations

Authors:

Sebastian Oster,

Mark GrechanikAuthors Info & Claims

SPLC '11: Proceedings of the 15th International Software Product Line Conference, Volume 2

Article No.: 6, Pages 1 - 8

https://doi.org/10.1145/2019136.2019143

Published: 21 August 2011 Publication History

Abstract

A fundamental problem of testing Software Product Lines (SPLs) is that variability enables the production of a large number of instances and it is difficult to construct and run test cases even for SPLs with a small number of variable features. Interacting features is a foundation of a fault model for SPLs, where faults are likely to be revealed at execution points where features exchange information with other features or influence one another. Therefore, a test adequacy criterion is to cover as many interactions among different features as possible, thus increasing the probability of finding bugs. Our approach combines a combinatorial designs algorithm for pairwise feature generation with model-based testing to reduce the size of the SPL required for comprehensive coverage of interacting features. We implemented our approach and applied it to an SPL from the automotive domain provided by one of our industrial partners. The results suggest that with our approach higher coverage of feature interactions is achieved at a fraction of cost when compared with a baseline approach of testing all feature interactions.

References

[1]

R. V. Binder. Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1999.

Digital Library

[2]

M. Calder, M. Kolberg, E. H. Magill, and S. Reiff-Marganiec. Feature Interaction: A Critical Review and onsidered Forecast. Computer Networks, 41(1):115--141, 2003.

Digital Library

[3]

M. B. Cohen, P. B. Gibbons, W. B. Mugridge, and C. J. Colbourn. Constructing test suites for interaction testing. In ICSE, pages 38--48, 2003.

Digital Library

[4]

K. Czarnecki. Mapping Features to Models: A Template Approach based on superimposed Variants. In GPCE'05, vol. 3676 of LNCS, pages 422--437, 2005.

Digital Library

[5]

K. Czarnecki and U. W. Eisenecker. Generative programming: methods, tools, and applications. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 2000.

Digital Library

[6]

S. Ferber, J. Haag, and J. Savolainen. Feature Interaction and Dependencies: Modeling Features for Reengineering a Legacy Product Line. In Proc. of the 2nd Int. Conf. on Software Product Lines, pages 235--256, London, UK, 2002. Springer-Verlag.

Digital Library

[7]

M. L. Griss. Implementing product-line features by composing aspects. In SPLC, pages 271--288, Norwell, MA, USA, 2000. Kluwer Academic Publishers.

Digital Library

[8]

D. Harel. Statecharts: A Visual Formalism for Complex Systems. Sci. Comput. Program., 8(3):231--274, 1987.

Digital Library

[9]

D. Harel and H. Kugler. The Rhapsody Semantics of Statecharts (or, On the Executable Core of the UML). In In LNCS 3147, pages 325--354. Springer, 2001.

[10]

J. Hartmann, M. Vieira, and A. Ruder. A UML-based Approach for Validating Product Lines. In B. Geppert, C. Krueger, and J. Li, editors, Proc. of the International Workshop on Software Product Line Testing (SPLiT 2004), pages 58--65, 2004.

[11]

M. Jackson and P. Zave. Distributed feature composition: A virtual architecture for telecommunications services. IEEE Transactions on Software Engineering, 24:831--847, 1998.

Digital Library

[12]

K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. S. Peterson. Feature-Oriented Domain Analysis (FODA) Feasibility Study. Technical report, CMU Software Engineering Institute, November 1990.

[13]

M. Lochau and U. Goltz. Feature Interaction Aware Test Case Generation for Embedded Control Systems. Electronic Notes in Theoretical Computer Science (ENTCS), 264:37--52, 2010.

Digital Library

[14]

P. Masiero, J. Maldonado, and I. Boaventura. A Reachability Tree for Statecharts and Analysis of some Properties. Information and Software Technology, 36(10):615--624, 1994.

[15]

T. Müller, M. Lochau, S. Detering, F. Saust, H. Garbers, L. Märtin, T. Form, and U. Goltz. A comprehensive Description of a Model-based, continuous Development Process for AUTOSAR Systems with integrated Quality Assurance. Technical Report 2009-06, TU Braunschweig, 2009.

[16]

E. M. Olimpiew. Model-Based Testing for Software Product Lines. PhD thesis, George Mason University, 2008.

Digital Library

[17]

S. Oster, F. Markert, and P. Ritter. Automated Incremental Pairwise Testing of Software Product Lines. In Proc. of the 14th International Software Product Line Conference, 2010.

Digital Library

[18]

S. Oster, I. Zorcic, F. Markert, and M. Lochau. MoSo-PoLiTe - Tool Support for Pairwise and Model-Based Software Product Line Testing. In Proc. of the Fifth Int. Workshop on Variability Modelling of Software-intensive System, 2011.

Digital Library

[19]

G. Perrouin, J. Klein, N. Guelfi, and J.-M. Jézéquel. Reconciling automation and flexibility in product derivation. In SPLC, pages 339--348, Limerick, Ireland, 2008. IEEE Computer Society.

Digital Library

[20]

E. Reisner, C. Song, K.-K. Ma, J. S. Foster, and A. Porter. Using symbolic evaluation to understand behavior in configurable software systems. In ICSE' 10), pages 445--454, 2010.

Digital Library

[21]

A. Reuys, E. Kamsties, K. Pohl, and S. Reis. Model-based System Testing of Software Product Families. In CAiSE, pages 519--534, 2005.

Digital Library

[22]

T. K. Satyananda, D. Lee, S. Kang, and S. I. Hashmi. Identifying traceability between feature model and software architecture in software product line using formal concept analysis. In Proc. of Int. Conf. Computational Science and its Applications, pages 380--388, Washington, DC, USA, 2007. IEEE CS.

Digital Library

[23]

K. Scheidemann. Verifying Families of System Configurations. Doctoral Thesis, TU Munich, 2007.

[24]

K. D. Scheidemann. Optimizing the Selection of Representative Configurations in Verification of Evolving Product Lines of Distributed Embedded Systems. In Proc. of the Int. Software Product Line Conf, pages 75--84, 2006.

Digital Library

[25]

P. Sochos, M. Riebisch, and I. Philippow. The Feature-Architecture Mapping (FArM) Method for Feature-Oriented Development of Software Product Lines. In Proc. of the 13th Annual IEEE Int. Symp. and Ws on Eng. of Computer Based Systems, pages 308--318, Washington, DC, USA, 2006. IEEE CS.

Digital Library

[26]

M. Utting and B. Legeard. Practical Model-Based Testing. A Tools Approach. Morgan Kaufmann, 2007.

Digital Library

[27]

A. Wasowski. Automatic Generation of Program Families by Model Restrictions. In SPLC, pages 7389, 2004.

[28]

S. Weißleder, D. Sokenou, and B. Schlinglo. Reusing State Machines for Automatic Test Generation in Product Lines. In Proc. of the 1st Workshop on Model-based Testing in Practice (MoTiP2008), 2008.

Cited By

Knop UHofman PMihatsch MSiegmund M(2023)Balancing Variability and Costs in Software Product LinesProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608992(213-222)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579027.3608992
Korsman DDamasceno CStrüber DFelfernig AFuentes L(2022)A tool for analysing higher-order feature interactions in preprocessor annotations in C and C++ projectsProceedings of the 26th ACM International Systems and Software Product Line Conference - Volume B10.1145/3503229.3547027(70-73)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3503229.3547027
Ferreira FVale GDiniz JFigueiredo E(2021)Evaluating T-wise testing strategies in a community-wide dataset of configurable software systemsJournal of Systems and Software10.1016/j.jss.2021.110990179(110990)Online publication date: Sep-2021
https://doi.org/10.1016/j.jss.2021.110990
Show More Cited By

Index Terms

Pairwise feature-interaction testing for SPLs: potentials and limitations
1. General and reference
  1. Cross-computing tools and techniques
    1. Metrics
2. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Reusability
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

MoSo-PoLiTe: tool support for pairwise and model-based software product line testing
VaMoS '11: Proceedings of the 5th International Workshop on Variability Modeling of Software-Intensive Systems

Testing Software Product Lines is a very challenging task and approaches like combinatorial testing and model-based testing are frequently used to reduce the effort of testing Software Product Lines and to reuse test artifacts. In this contribution we ...
IncLing: efficient product-line testing using incremental pairwise sampling
GPCE '16

A software product line comprises a family of software products that share a common set of features. It enables customers to compose software systems from a managed set of features. Testing every product of a product line individually is often ...
Model-based pairwise testing for feature interaction coverage in software product line engineering

Testing software product lines (SPLs) is very challenging due to a high degree of variability leading to an enormous number of possible products. The vast majority of today's testing approaches for SPLs validate products individually using different ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SPLC '11: Proceedings of the 15th International Software Product Line Conference, Volume 2

August 2011

306 pages

ISBN:9781450307895

DOI:10.1145/2019136

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Pure-Systems: Pure-Systems GmbH
Hitachi
Siemens
Software Eng Inst: Software Engineering Institute
Biglever: BigLever Software, Inc.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SPLC '11

Sponsor:

Pure-Systems
Software Eng Inst
Biglever

SPLC '11: The 2011 Software Product Line Conference

August 21 - 26, 2011

Munich, Germany

Acceptance Rates

Overall Acceptance Rate 167 of 463 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
466
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Knop UHofman PMihatsch MSiegmund M(2023)Balancing Variability and Costs in Software Product LinesProceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A10.1145/3579027.3608992(213-222)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1145/3579027.3608992
Korsman DDamasceno CStrüber DFelfernig AFuentes L(2022)A tool for analysing higher-order feature interactions in preprocessor annotations in C and C++ projectsProceedings of the 26th ACM International Systems and Software Product Line Conference - Volume B10.1145/3503229.3547027(70-73)Online publication date: 12-Sep-2022
https://dl.acm.org/doi/10.1145/3503229.3547027
Ferreira FVale GDiniz JFigueiredo E(2021)Evaluating T-wise testing strategies in a community-wide dataset of configurable software systemsJournal of Systems and Software10.1016/j.jss.2021.110990179(110990)Online publication date: Sep-2021
https://doi.org/10.1016/j.jss.2021.110990
Pett TEichhorn DSchaefer IGuerra EIovino L(2020)Risk-based compatibility analysis in automotive systems engineeringProceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings10.1145/3417990.3421263(1-10)Online publication date: 16-Oct-2020
https://dl.acm.org/doi/10.1145/3417990.3421263
Nieke MMauro JSeidl CThüm TYu IFranzke F(2020)Anomaly analyses for feature-model evolutionACM SIGPLAN Notices10.1145/3393934.327812353:9(188-201)Online publication date: 7-Apr-2020
https://dl.acm.org/doi/10.1145/3393934.3278123
Ferreira FDiniz JSilva CFigueiredo EWeyns DPerrouin GWeyns D(2019)Testing Tools for Configurable Software SystemsProceedings of the 13th International Workshop on Variability Modelling of Software-Intensive Systems10.1145/3302333.3302344(1-10)Online publication date: 6-Feb-2019
https://dl.acm.org/doi/10.1145/3302333.3302344
Nieke MMauro JSeidl CThüm TYu IFranzke FVan Wyk ERompf T(2018)Anomaly analyses for feature-model evolutionProceedings of the 17th ACM SIGPLAN International Conference on Generative Programming: Concepts and Experiences10.1145/3278122.3278123(188-201)Online publication date: 5-Nov-2018
https://dl.acm.org/doi/10.1145/3278122.3278123
Filho HFerreira TVergilio SKulesza UPrikladnicki RGerosa MWerner CAndrade R(2018)Multiple objective test set selection for software product line testingProceedings of the XXXII Brazilian Symposium on Software Engineering10.1145/3266237.3266275(162-171)Online publication date: 17-Sep-2018
https://dl.acm.org/doi/10.1145/3266237.3266275
Abdessalem RPanichella ANejati SBriand LStifter THuchard MKästner CFraser G(2018)Testing autonomous cars for feature interaction failures using many-objective searchProceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering10.1145/3238147.3238192(143-154)Online publication date: 3-Sep-2018
https://dl.acm.org/doi/10.1145/3238147.3238192
Varshosaz MAl-Hajjaji MThüm TRunge TMousavi MSchaefer I(2018)A classification of product sampling for software product linesProceedings of the 22nd International Systems and Software Product Line Conference - Volume 110.1145/3233027.3233035(1-13)Online publication date: 10-Sep-2018
https://dl.acm.org/doi/10.1145/3233027.3233035
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents