Context: Search Based Software Testing refers to the use of meta-heuristics for the optimization ... more Context: Search Based Software Testing refers to the use of meta-heuristics for the optimization of a task in the context of software testing. Meta-heuristics can solve complex problems in which an optimum solution must be found among a large amount of possibilities. The use of meta-heuristics in testing activities is promising because of the high number of inputs that should be tested. Previous studies on search based software testing have focused on the application of meta-heuristics for the optimization of structural and functional criteria. Recently, some researchers have proposed the use of SBST for mutation testing and explored solutions for the cost of application of this testing criterion.
Objective: The objective is to identify how SBST has been explored in the context of mutation testing, how fitness functions are defined and the challenges and research opportunities in the application of meta-heuristic search techniques.
Method: A systematic review involving 263 papers published between 1996 and 2014 examined the studies on the use of meta-heuristic search techniques for the optimization of mutation testing.
Results: The results show meta-heuristic search techniques have been applied for the optimization of test data generation, mutant generation and selection of effective mutation operators. Five meta-heuristic techniques, namely Genetic Algorithm, Ant Colony, Bacteriological Algorithm, Hill Climbing and Simulated Annealing have been used in search based mutation testing. The review addressed different fitness functions used to guide the search.
Conclusion: Search based mutation testing is a field of interest, however, some issues remain unexplored. For instance, the use of meta-heuristics for the selection of effective mutation operators was identified in only one study. The results have pointed a range of possibilities for new studies to be developed, i.e., identification of equivalent mutants, experimental studies and application to different domains, such as concurrent programs.
The testing of concurrent programs is very complex due to the non-determinism present in those pr... more The testing of concurrent programs is very complex due to the non-determinism present in those programs. They must be subjected to a systematic testing process that assists in the identification of defects and guarantees quality. Although testing tools have been proposed to support the concurrent program testing, to the best of our knowledge, no study that concentrates all testing tools to be used as a catalog for testers is available in the literature. This paper proposes a new classification for a set of testing tools for concurrent programs, regarding attributes, such as testing technique supported, programming language, and paradigm of development. The purpose is to provide a useful categorization guide that helps testing practitioners and researchers in the selection of testing tools for concurrent programs. A systematic mapping was conducted so that studies on testing tools for concurrent programs could be identified. As a main result, we provide a catalog with 116 testing tools appropriately selected and classified, among which the following techniques were identified: functional testing, structural testing, mutation testing, model based testing, data race and deadlock detection, deterministic testing and symbolic execution. The programming languages with higher support were Java and C/C++. Although a large number of tools have been categorized, most of them are academic and only few are available on a commercial scale. The classification proposed here can contribute to the state-of-the-art of testing tools for concurrent programs and also provides information for the exchange of knowledge between academy and industry.
O teste de software é uma atividade que busca garantir a qualidade por meio da identificação de f... more O teste de software é uma atividade que busca garantir a qualidade por meio da identificação de falhas no produto. Para a aplicação do teste de software, algumas técnicas são utilizadas como, por exemplo, a técnica estrutural, a funcional e a baseada em defeitos. Cada técnica possui seus critérios para auxiliar na criação dos casos de teste. Um dos critérios da técnica baseada em defeitos é o teste de mutação. Este critério de teste baseia-se nos enganos que podem ser cometidos pelos desenvolvedores de software. O teste de mutação tem se mostrado eficaz para revelar defeitos, porém, o seu alto custo compromete sua utilização, principalmente quando são considerados programas complexos. A Programação Concorrente é um paradigma de desenvolvimento essencial para a construção de aplicações com o intuito de reduzir o tempo computacional em muitos domínios. Estas aplicações têm novas características como comunicação, sincronização e não determinismo que precisam ser considerados durante a atividade de teste. No contexto de programas concorrentes, novos desafios são impostos e precisam ser adequadamente tratados. Isso ocorre devido ao não determinismo e as diferentes sincronizações entre processos. Search-Based Software Testing é uma abordagem que vem sendo empregada para otimizar a atividade de teste, aplicando meta-heurísticas para a solução de problemas complexos, como por exemplo, geração de dados de teste. Este projeto de doutorado está inserido nesse contexto, no qual se pretende investigar o uso de Search-Based Software Testing para redução do custo da aplicação do teste de mutação no contexto de aplicações concorrentes.
Mutation testing has been successfully used due its high effectiveness to reveal faults. However,... more Mutation testing has been successfully used due its high effectiveness to reveal faults. However, when this criterion is applied in context of concurrent programs, new challenges are presented. Due to non-determinism, the execution of the program with one test input may traverses different and correct synchronization sequences. The challenge is to find the set of all possible synchronization sequences and evaluate the mutants behavior with them. In this paper is proposed the use of two approaches to deal with it. The results indicate that both approaches contribute in this direction, supporting the mutant analysis and the generation of feasible synchronization sequences.
Concurrent Programming became an essential paradigm to reduce the computational time in many appl... more Concurrent Programming became an essential paradigm to reduce the computational time in many application domains. Mutation testing is a test criterion which uses mistakes made by software developers to derive test requirements. To apply this criterion in context of concurrent programs is necessary considers the implicit features these programs, such as: communication, synchronization and non-determinism. Due to the non-determinism, special attention must be given during the behavior analysis. This paper presents a master’s project under development whose goal is investigate the definition, implementation and evaluation of the mutation testing for MPI (Message Passing Interface) programs.
Context: Search Based Software Testing refers to the use of meta-heuristics for the optimization ... more Context: Search Based Software Testing refers to the use of meta-heuristics for the optimization of a task in the context of software testing. Meta-heuristics can solve complex problems in which an optimum solution must be found among a large amount of possibilities. The use of meta-heuristics in testing activities is promising because of the high number of inputs that should be tested. Previous studies on search based software testing have focused on the application of meta-heuristics for the optimization of structural and functional criteria. Recently, some researchers have proposed the use of SBST for mutation testing and explored solutions for the cost of application of this testing criterion.
Objective: The objective is to identify how SBST has been explored in the context of mutation testing, how fitness functions are defined and the challenges and research opportunities in the application of meta-heuristic search techniques.
Method: A systematic review involving 263 papers published between 1996 and 2014 examined the studies on the use of meta-heuristic search techniques for the optimization of mutation testing.
Results: The results show meta-heuristic search techniques have been applied for the optimization of test data generation, mutant generation and selection of effective mutation operators. Five meta-heuristic techniques, namely Genetic Algorithm, Ant Colony, Bacteriological Algorithm, Hill Climbing and Simulated Annealing have been used in search based mutation testing. The review addressed different fitness functions used to guide the search.
Conclusion: Search based mutation testing is a field of interest, however, some issues remain unexplored. For instance, the use of meta-heuristics for the selection of effective mutation operators was identified in only one study. The results have pointed a range of possibilities for new studies to be developed, i.e., identification of equivalent mutants, experimental studies and application to different domains, such as concurrent programs.
The testing of concurrent programs is very complex due to the non-determinism present in those pr... more The testing of concurrent programs is very complex due to the non-determinism present in those programs. They must be subjected to a systematic testing process that assists in the identification of defects and guarantees quality. Although testing tools have been proposed to support the concurrent program testing, to the best of our knowledge, no study that concentrates all testing tools to be used as a catalog for testers is available in the literature. This paper proposes a new classification for a set of testing tools for concurrent programs, regarding attributes, such as testing technique supported, programming language, and paradigm of development. The purpose is to provide a useful categorization guide that helps testing practitioners and researchers in the selection of testing tools for concurrent programs. A systematic mapping was conducted so that studies on testing tools for concurrent programs could be identified. As a main result, we provide a catalog with 116 testing tools appropriately selected and classified, among which the following techniques were identified: functional testing, structural testing, mutation testing, model based testing, data race and deadlock detection, deterministic testing and symbolic execution. The programming languages with higher support were Java and C/C++. Although a large number of tools have been categorized, most of them are academic and only few are available on a commercial scale. The classification proposed here can contribute to the state-of-the-art of testing tools for concurrent programs and also provides information for the exchange of knowledge between academy and industry.
O teste de software é uma atividade que busca garantir a qualidade por meio da identificação de f... more O teste de software é uma atividade que busca garantir a qualidade por meio da identificação de falhas no produto. Para a aplicação do teste de software, algumas técnicas são utilizadas como, por exemplo, a técnica estrutural, a funcional e a baseada em defeitos. Cada técnica possui seus critérios para auxiliar na criação dos casos de teste. Um dos critérios da técnica baseada em defeitos é o teste de mutação. Este critério de teste baseia-se nos enganos que podem ser cometidos pelos desenvolvedores de software. O teste de mutação tem se mostrado eficaz para revelar defeitos, porém, o seu alto custo compromete sua utilização, principalmente quando são considerados programas complexos. A Programação Concorrente é um paradigma de desenvolvimento essencial para a construção de aplicações com o intuito de reduzir o tempo computacional em muitos domínios. Estas aplicações têm novas características como comunicação, sincronização e não determinismo que precisam ser considerados durante a atividade de teste. No contexto de programas concorrentes, novos desafios são impostos e precisam ser adequadamente tratados. Isso ocorre devido ao não determinismo e as diferentes sincronizações entre processos. Search-Based Software Testing é uma abordagem que vem sendo empregada para otimizar a atividade de teste, aplicando meta-heurísticas para a solução de problemas complexos, como por exemplo, geração de dados de teste. Este projeto de doutorado está inserido nesse contexto, no qual se pretende investigar o uso de Search-Based Software Testing para redução do custo da aplicação do teste de mutação no contexto de aplicações concorrentes.
Mutation testing has been successfully used due its high effectiveness to reveal faults. However,... more Mutation testing has been successfully used due its high effectiveness to reveal faults. However, when this criterion is applied in context of concurrent programs, new challenges are presented. Due to non-determinism, the execution of the program with one test input may traverses different and correct synchronization sequences. The challenge is to find the set of all possible synchronization sequences and evaluate the mutants behavior with them. In this paper is proposed the use of two approaches to deal with it. The results indicate that both approaches contribute in this direction, supporting the mutant analysis and the generation of feasible synchronization sequences.
Concurrent Programming became an essential paradigm to reduce the computational time in many appl... more Concurrent Programming became an essential paradigm to reduce the computational time in many application domains. Mutation testing is a test criterion which uses mistakes made by software developers to derive test requirements. To apply this criterion in context of concurrent programs is necessary considers the implicit features these programs, such as: communication, synchronization and non-determinism. Due to the non-determinism, special attention must be given during the behavior analysis. This paper presents a master’s project under development whose goal is investigate the definition, implementation and evaluation of the mutation testing for MPI (Message Passing Interface) programs.
Uploads
Papers by Rodolfo Silva
Objective: The objective is to identify how SBST has been explored in the context of mutation testing, how fitness functions are defined and the challenges and research opportunities in the application of meta-heuristic search techniques.
Method: A systematic review involving 263 papers published between 1996 and 2014 examined the studies on the use of meta-heuristic search techniques for the optimization of mutation testing.
Results: The results show meta-heuristic search techniques have been applied for the optimization of test data generation, mutant generation and selection of effective mutation operators. Five meta-heuristic techniques, namely Genetic Algorithm, Ant Colony, Bacteriological Algorithm, Hill Climbing and Simulated Annealing have been used in search based mutation testing. The review addressed different fitness functions used to guide the search.
Conclusion: Search based mutation testing is a field of interest, however, some issues remain unexplored. For instance, the use of meta-heuristics for the selection of effective mutation operators was identified in only one study. The results have pointed a range of possibilities for new studies to be developed, i.e., identification of equivalent mutants, experimental studies and application to different domains, such as concurrent programs.
Objective: The objective is to identify how SBST has been explored in the context of mutation testing, how fitness functions are defined and the challenges and research opportunities in the application of meta-heuristic search techniques.
Method: A systematic review involving 263 papers published between 1996 and 2014 examined the studies on the use of meta-heuristic search techniques for the optimization of mutation testing.
Results: The results show meta-heuristic search techniques have been applied for the optimization of test data generation, mutant generation and selection of effective mutation operators. Five meta-heuristic techniques, namely Genetic Algorithm, Ant Colony, Bacteriological Algorithm, Hill Climbing and Simulated Annealing have been used in search based mutation testing. The review addressed different fitness functions used to guide the search.
Conclusion: Search based mutation testing is a field of interest, however, some issues remain unexplored. For instance, the use of meta-heuristics for the selection of effective mutation operators was identified in only one study. The results have pointed a range of possibilities for new studies to be developed, i.e., identification of equivalent mutants, experimental studies and application to different domains, such as concurrent programs.