I M.

Tech I Semester II Mid Examinations (2010 11) SOFTWARE QUALITY ASSURANCE & TESTING
1. Explain white box and black box testing and specify differences

(Software Engineering)

between them?
Black-box testing is a method of software testing that tests the functionality of an application. Specific knowledge of the application's code/internal structure and programming knowledge in general is not required. The tester is only aware of what the software is supposed to do, but not how i.e. when he enters a certain input, he gets a certain output; without being aware of how the output was produced. Black-box testing, also called behavioral testing, focuses on the
functional requirements of the software. That is, black-box testing enables the software engineer to derive sets of input conditions that will fully exercise all functional requirements for a program. Black-box testing attempts to find errors in the following categories:

1. 2. 3. 4. 5.

Incorrect or missing functions, Interface errors, Errors in data structures or external database access Behavior or performance errors, and Initialization and termination errors.

There are various techniques in developing test cases for black box testing. Equivalence Partitioning, BVA (Boundary Value Analysis) and Cause-Effect are different methods of black box testing. Equivalence partitioning is a black-box testing method that divides the input domain of a program into classes of data from which test cases can be derived. Equivalence classes may be defined according to the following guidelines: 1. If an input condition specifies a range, one valid and two invalid equivalence classes are defined. 2. If an input condition requires a specific value, one valid and two invalid equivalence classes are defined. 3. If an input condition specifies a member of a set, one valid and one in valid equivalence class are defined. 4. If an input condition is Boolean, one valid and one invalid class are defined. Boundary Value analysis: Cause Effect Graphing: Cause Effect Graphing is a software test design technique that involves identifying the cases (input conditions) and effects (output conditions), producing a Cause-Effect Graph, and generating test cases accordingly Advantages of Black Box Testing - Tester can be non-technical.

- Used to verify contradictions in actual system and the specifications. - Test cases can be designed as soon as the functional specifications are complete.

White Box Testing: White-box testing (also known as clear box testing, glass box testing, transparent box testing, and structural testing) is a method of testing software that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing). In white-box testing an internal perspective of the system, as well as programming skills, are used to design test cases. The tester chooses inputs to exercise paths through the code and determine the appropriate outputs. This is analogous to testing nodes in a circuit, e.g. incircuit testing (ICT). While white-box testing can be applied at the unit, integration and system levels of the software testing process, it is usually done at the unit level. It can test paths within a unit, paths between units during integration, and between subsystems during a systemlevel test. Though this method of test design can uncover many errors or problems, it might not detect unimplemented parts of the specification or missing requirements. White-box test design techniques include: Control flow testing Data flow testing Branch testing Path testing Black Box Testing White Box Testing

Criteria

Definition

Black Box Testing is a software White Box Testing is a software testing method in which the internal testing method in which the internal structure/ design/ implementation of structure/ design/ implementation of the item being tested is NOT known the item being tested is known to to the tester the tester. Mainly applicable to higher levels of testing: Acceptance System Testing Mainly applicable to lower levels of testing: Unit Integration Testing Generally, Software Developers

Levels Applicable To

Responsibility Programming Knowledge

Generally, independent Software Testers Not Required

Required

Implementation Knowledge Basis for Test Cases

Not Required

Required

Requirement Specifications

Detail Design

2. Explain Performance testing and Adhoc testing.

Performance Testing:
Performance testing is the process of determining the speed or effectiveness of a computer, network, software program or device. This process can involve quantitative tests done in a lab, such as measuring the response time or the number of MIPS (millions of instructions per second) at which a system functions. Qualitative attributes such as reliability, scalability and interoperability may also be evaluated. Different types of performance Testing: Load testing Load testing is the simplest form of performance testing. A load test is usually conducted to understand the behavior of the system under a specific expected load. This load can be the expected concurrent number of users on the application performing a specific number of transactions within the set duration. This test will give out the response times of all the important business critical transactions. If the database, application server, etc. are also monitored, then this simple test can itself point towards any bottlenecks in the application software. Stress testing Stress testing is normally used to understand the upper limits of capacity within the system. This kind of test is done to determine the system's robustness in terms of extreme load and helps application administrators to determine if the system will perform sufficiently if the current load goes well above the expected maximum. Endurance testing (soak testing) Endurance testing is usually done to determine if the system can sustain the continuous expected load. During endurance tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation. That is, to ensure that the throughput and/or response times after some long period of sustained activity are as good as or better than at the beginning of the test. It essentially involves applying a significant load to a system for an extended, significant period of time. The goal is to discover how the system behaves under sustained use. Spike testing Spike testing is done by suddenly increasing the number of, or load generated by, users by a very large amount and observing the behavior of the system. The goal is to determine

whether performance will suffer, the system will fail, or it will be able to handle dramatic changes in load. Configuration testing Rather than testing for performance from the perspective of load, tests are created to determine the effects of configuration changes to the system's components on the system's performance and behavior. A common example would be experimenting with different methods of load-balancing. Isolation testing Isolation testing is not unique to performance testing but a term used to describe repeating a test execution that resulted in a system problem. Often used to isolate and confirm the fault domain. Adhoc Testing:

This type of testing is done without any formal Test Plan or Test Case creation. Ad-hoc testing helps in deciding the scope and duration of the various other testing and it also helps testers in learning the application prior starting with any other testing. It is the least formal method of testing. Advantages: We check the application randomly so we reduce time consuming. 1. 2. 3. 4. Ad-hoc Testing is very cost effective Ad-hoc Testing can be followed even when documentation is not present for the system Ad-hoc Testing can be adjusted and done as per the budget Ideal for small size companies

Disadvantage: We will check the application randomly so we miss some activities. So there is chance of bugs in that activity.

3.

Write about software quality metrics.

4.

Explain Test management plan. Scope Of the project: Objective of the testing

In scope: What to test Out Scope: what not to test Software &Hardware Requirements: Training Requirements: Types of testing: Functional: Integration, re-testing, regression Non functional: UI, Performance, security, system testing Staffing: Resource planning Contacts: Test lead. Test man Risks & Mitigations: Defect Reporting Mechanism: Exit Criteria: No defect with p1 or p2 while rolling the application to production Test closure activities: what will be delivered, test report test plan Acronyms & Abbreviations:

5. Discuss the role of software testing coverage criteria in testing control. Requirements traceability matrix The term Test Coverage used in the context of programming / software engineering refers to measuring how much a software program has been exercised by tests. Coverage is a means of determining the rigour with which the question underlying the test has been answered. There are many kinds of test coverage: Code coverage, Feature coverage, Scenario coverage, Screen item coverage Model coverage.

Each of these coverage types assumes that some kind of baseline exists which defines the system under test. The number of types of test coverage therefore varies as the number of ways of defining the system. For example, in code coverage: Has a particular statement ever been executed?

How many times has a statement been executed? Have all the statements in a program been executed, at least once? Have all the decision points in the code been exercised such that every decision path has been taken? Has the last optimization reduced the instruction path length significantly?

(b). Write about automated tools for software testing. Functional: QTP, Test partner, Selenium, Test runner. Performance: QA load, load runner, rational performance, J meter

QTP: HP QuickTest Professional is automated testing software designed for testing various software applications and environments. It performs functional and regression testing through a user interface such as a native GUI or web interface. It works by identifying the objects in the application user interface or a web page and performing desired operations (such as mouse clicks or keyboard events); it can also capture object properties like name or handler ID. HP QuickTest Professional uses a VBScript scripting language to specify the test procedure and to manipulate the objects and controls of the application under test.

Drawbacks: HP QuickTest Professional runs only in Windows environments. It relies on largely obsolete Windows-only technologies such as ActiveX.QTP cannot test with all browser types and versions. In particular it does not support Safari, or Opera. High licensing costs often mean that the tool is not widely used in an organization. Exception handling HP QuickTest Professional manages exception handling using recovery scenarios; the goal is to continue running tests if an unexpected failure occurs. Because HP QuickTest Professional hooks into the memory space of the applications being tested, some exceptions may cause HP QuickTest Professional to terminate and be unrecoverable. Data-driven testing HP QuickTest Professional supports data-driven testing. For example, data can be output to a data table for reuse elsewhere. Data-driven testing is implemented as a Microsoft Excel workbook that can be accessed from HP QuickTest Professional. HP QuickTest Professional has two types of data tables: the Global data sheet and Action (local) data sheets. The test steps can read data from these data tables in order to drive variable data into the application under test, and verify the expected result. Automating custom and complex UI objects

HP QuickTest Professional may not recognize customized user interface objects and other complex objects. Users can define these types of objects as virtual objects. HP QuickTest Professional does not support virtual objects for analog recording or recording in low-level mode. 6. Explain project process standards. There are several project process standards followed by different organizations. A few of them were discussed below.

ISO 9000-3: ISO 9000 [ISO91] supplies a set of standards for quality management of production activities. Organizations are supposed to set up a quality system in order to supervise all phases of a production and delivery process. Some of the activities carried out by the quality system are:a) Auditing the projects to ensure that quality controls are respected, b) Improving the quality system itself, and c) Providing input to the development group, such as new notations, procedures and Standards producing reports to the high-level management.

PSS-05: Practices in PSS-05 are mandatory, recommended or guiding. A mandatory practice must be followed without exception in all software projects. Recommended practices may or may not be followed. However, a justification has to provide if a recommended practice is abandoned. Guideline practices are less crucial; no justification is needed if they are not followed.

PSS-05 does not prescribe a particular lifecycle model. However, any lifecycle model adopted for a project must be defined in a software project management plan and must include the following mandatory phases:a) Definition of user requirements, b) Definition of software requirements, c) Definition of architectural design, d) Detailed design and production of code, e) Transfer of the software to operations, and

f) Operations and maintenance

CMMI [Capability Maturity Model]: Capability Maturity Model Integration (CMMI) is a process improvement approach whose goal is to help organizations improve their performance. CMMI can be used to guide process improvement across a project, a division, or an entire organization. Currently supported is CMMI Version 1.3. CMMI in software engineering and organizational development is a process improvement approach that provides organizations with the essential elements for effective process improvement. CMMI is registered in the U.S. Patent and Trademark Office by Carnegie Mellon University.

7. Explain SQA unit. SQA unit structure varies by type and, of course, size of the organization. As it is impossible to describe all the optional arrangements, the chapter presents a model whose structure and task distribution are readily adaptable to the characteristics and procedures characterizing the internal environment of a spate of major organizations.

The head of the SQA unit is responsible for all the quality assurance tasks performed by the SQA unit and its sub-units. In addition, some SQA tasks are assigned only to him: this allocation reflects the managers professional Experience and administrative position. The typical tasks performed by an SQA unit head may be classified into the following categories: Planning Management of the unit Tasks related to contacts with customers and other external bodies as well as with the executive in charge of software quality SQA professional activities