Continental Test Lab

Continental Test Lab
Test types
August 2014
Interior Infotaiment and Connectivity R&D Romania

Agenda
1Non-Functional testing - What is?
2Efficiency testing - performance

3Efficiency testing – load and volume testing
4Efficiency testing - stress and spike
5Usability testing
6Maintainability testing
7Portability testing

2 11-03-26 Autor, © Continental Automotive Romania SRL
Test Types – Non-Functional testing
Non-Functional testing
Is designed specifically to evaluate the readiness of a system according to the various

criteria which are not covered by functional testing.
The term “Non-functional testing” describes the tests required to measure the
characteristics of system and software that can be quantified on a varying scale(response
times for performance testing)
Non-functional testing is the testing of a software application for its non-functional

requirements.
As well as functional testing, and the non-functional can be done at all levels of testing
18 August 2014 , © Continental Automotive Romania SRL

For example considering the case when we buy a new mobile phone. To use the mobile
we have to insert the sim-card and the battery.
Next step is to turn-on the phone -> We see that starting and loading until the phone is in
operable mode takes time. This time is important for measuring performance.
As an end user of the phone, I am delighted that the product that I bought can use to
make or receive calls or send messages but also to listen to music on it or surf the
Internet.
Aspects of the reaction time, ergonomics, stability, ease of navigation through menu and
they are important.

Agenda

5Usability testing

Efficiency Testing:
Factors influencing the efficiency are: response time (speed and power calculation
system), low consumption of resources, the ability to serve a larger number of users
without noticeable degradation parameters.
-time behaviour (performance) -> How fast?

-resource behaviour -> how much?
Efficiency is the ability of a product (software) to provide adequate performance

in relation to the quantity of resources used in different conditions.
Performance, Load, Stress, Spike , Resource Utilisation, etc

Performance testing
Performance testing is in general testing performed to determine how a system performs
in terms of responsiveness and stability under a particular workload.
It can also serve to investigate, measure, validate or verify other quality attributes of
the system, such as scalability, reliability and resource usage.

Performance testing
The performance metrics needed to be measured vary from application to application.
An example of expected performance is:
The response time should be less than 1 millisecond 90% of the time in an application
of the “push-to-talk” type
A transaction in an on-line system requires a response of less than 1 second 90% of

the time.

Performance testing
For performance tests, one needs to be clear about the specific data to be captured in
order to evaluate performance metrics(if the objective is to evaluate the response time):
End to end response time (as seen by external user)
CPU time
Network connection time
Database access time
Network connection time
Waiting time

Performance testing
FIAT_VP2_DAB_2172 PVV V&V The system shall be capable to provide DAB Audio
after startup within 10 seconds plus the max. tuning time from 'Tuning Time' table for
different ensemble and the related DAB standard (see also req.
FIAT_VP2_DAB_1517).
FIAT_VP2_DAB_1517 PVV V&V The HU DAB shall perform tuning according to the
following timings:

Performance testing

Performance testing
How can you do a performance test on
Your cellular phone?
Your TV?
Your oven?

Agenda

5Usability testing

Load testing
Load testing is the process of putting demand on a system or device and measuring its
response.
Load testing is performed to determine a system’s behavior under both normal and
anticipated peak load conditions.
It helps to identify the maximum operating capacity of an application as well as any

bottlenecks and determine which element is causing degradation.
For example, if you are running an e-commerce site you might design something like the
following:
50 users browsing the online store
25 users adding items to cart and checking out
10 users viewing their account and past orders
Your first step would be to create scripts for each of these behaviors.
Then you can create a scenario to run them all at once.

Load testing
How can you perform a load test on:
A crane?
A mattress?

Volume testing
4418141122_RO.PDF –Sony BDV-N790W
The system can recognize the following files or

folders in BDs, DVDs, CDs and devices
USB:
up to 9 levels of folders, including the root folder
Up to 500 files / folders on one level
The system can Recognize the Following files or

Stored in the DLNA server folders:
up to 19 levels of folders
pană la 999 de fişiere/foldere pe un singur nivel

Agenda

5Usability testing

Stress testing
The goal of stress testing is to evaluate and determine the behavior of a software
component while the offered load is in excess of its designed capacity.
The system is deliberately stressed by pushing it to and beyond its specified limits.
It ensures that the system can perform acceptably under worst-case conditions under an
expected peak load.
If the limit is exceeded and the system does fail, then the recovery mechanism should be
invoked.
Stress tests are targeted to bring out the problems associated with one or more of the
following:
Memory leak
Buffer allocation and memory carving

Stress testing
How can you stress:
A radio?
A cabinet?

Spike
There is a test that combines the volume and the stress -> spike test is a sharp transition
between these two.
The purpose is to determine whether the performance of the system are affected;
The system will fail or will cope with sudden and unpredictable variations

Agenda

5Usability testing

Usability testing
Usability testing is a technique used in user-centered interaction design to evaluate a
product by testing it on users.
Usability testing focuses on measuring a human-made product's capacity to meet its

intended purpose.
Examples of products that commonly benefit from usability testing are foods, consumer
products, web sites or web applications, computer interfaces, documents, and devices
Usability testing measures the usability, or ease of use, of a specific object or set of
objects, whereas general human-computer interaction studies attempt to formulate
universal principles.
For example in Games, there is a usability test, when the game enters in the Alpha/Beta
stage(selected users test the application from the user point of view).

Usability testing
As more users are involved in
testing the product, the more
chances to find more usability
problems are bigger (increase
the number of problems found)
As a method of analysis and
report generation of test results,
can be used an application that
can record everything you see
on your test participants but his
voice and facial expressions
during the test. The application
can track and participant
reactions like: keystrokes,
mouse clicks, etc..

Agenda

5Usability testing

Maintainability testing
It basically defines that how easy it is to maintain the system.
This means that how easy it is to analyze, change and test the application or product.
Corrective maintenance - Correcting problems. The maintainability of a system can
be measured in terms of the time taken to diagnose and fix problems identified within
that system.
Perfective maintenance - Enhancements. The maintainability of a system can also
be measured in terms of the effort taken to make required enhancements to that
system. This can be tested by recording the time taken to achieve a new piece of
identifiable functionality such as a change to the database, etc..
Adaptive maintenance - Adapting to changes in environment. The maintainability of
a system can also be measured in terms on the effort required to make required
adaptations to that system.
Preventive maintenance - Actions to reduce future maintenance costs. This refers to
actions to reduce future maintenance costs.

Agenda

5Usability testing

Portability testing
Portability testing is the process of testing an existing software component or application
in a new environment.
Examples:
Software designed to run on Macintosh OS X and Microsoft Windows operating

systems.
Applications developed to be compatible with Google Android and Apple iOS phones.
Video Games or other graphic intensive software intended to work with OpenGL and
DirectX API's
Web pages that should be compatible with IE, Google Chrome and Mozilla Firefox
browsers

Continental Test Lab

Uploaded by

Copyright:

Available Formats

Continental Test Lab

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Continental Test Lab

Uploaded by

Copyright:

Available Formats

Continental Test Lab

Interior Infotaiment and Connectivity R&D Romania

1Non-Functional testing - What is?

2Efficiency testing - performance

Interior Infotaiment and Connectivity R&D Romania

Is designed specifically to evaluate the readiness of a system according to the various

Non-functional testing is the testing of a software application for its non-functional

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

1Non-Functional testing - What is?

2Efficiency testing - performance

Interior Infotaiment and Connectivity R&D Romania

-time behaviour (performance) -> How fast?

Efficiency is the ability of a product (software) to provide adequate performance

Performance, Load, Stress, Spike , Resource Utilisation, etc

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

The performance metrics needed to be measured vary from application to application.

An example of expected performance is:

A transaction in an on-line system requires a response of less than 1 second 90% of

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

End to end response time (as seen by external user)

Network connection time

Database access time

Network connection time

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

Your cellular phone?

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

1Non-Functional testing - What is?

2Efficiency testing - performance

Interior Infotaiment and Connectivity R&D Romania

It helps to identify the maximum operating capacity of an application as well as any

50 users browsing the online store

25 users adding items to cart and checking out

10 users viewing their account and past orders

Then you can create a scenario to run them all at once.

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

The system can recognize the following files or

up to 9 levels of folders, including the root folder

Up to 500 files / folders on one level

The system can Recognize the Following files or

pană la 999 de fişiere/foldere pe un singur nivel

Interior Infotaiment and Connectivity R&D Romania

18 August 2014 , © Continental Automotive Romania SRL

1Non-Functional testing - What is?