Pooja Malhotra

SOFTWARE QUALITY MANAGEMENT

Module 5
1
 Quality as Multidimensional Concept

Pooja Malhotra
The degree to which a product or service possesses
a desired combination of attributes

2
WHAT IS QUALITY?

 “an inherent or distinguishing characteristic; a

property; having a high degree of excellence”
 Features & functionality

Pooja Malhotra
 “fitness for use”
 “conformance to requirements”

3
 Broadening the Concept of Quality
EVOLUTION OF SOFTWARE TESTING
• Product Quality
• Process Quality
Product Quality

Pooja Malhotra
Process

4
FIVE VIEWS OF SOFTWARE QUALITY
 Transcendental view
 User view

Pooja Malhotra
 Manufacturing view

 Product view

 Value-based view

5
FIVE VIEWS OF SOFTWARE QUALITY
 Transcendental view
 Quality is something that can be recognized through experience, but
not defined in some tractable form.

Pooja Malhotra
 A good quality object stands out, and it is easily recognized.
 It is "something toward which we strive as an ideal, but
may never implement completely“
 It's mainly feelings about something.
 User view
 Quality concerns the extent to which a product meets user needs and
expectations.
 Is a product fit for use?
 This view is highly personalized.
 A product is of good quality if it satisfies a large number of users.
 It is useful to identify the product attributes which the users consider to
be important.
 This view may encompass many subject elements, such as usability,
6
reliability, and efficiency.
FIVE VIEWS OF SOFTWARE QUALITY
 Manufacturing view
 This view has its genesis in the manufacturing industry – auto and
electronics.

Pooja Malhotra
 Key idea: Does a product satisfy the requirements?
 Any deviation from the requirements is seen as reducing the quality of
the product.
 The concept of process plays a key role.
 Products are manufactured “right the first time” so that the cost is
reduced
 Development cost
 Maintenance cost
 Conformance to requirements leads to uniformity in products.
 Product quality can be incrementally improved by improving the
process.
 The CMM and ISO 9001 models are based on the manufacturing view. 7
FIVE VIEWS OF SOFTWARE QUALITY
 Product view
 Hypothesis: If a product is manufactured with good internal
properties, then it will have good external properties.

Pooja Malhotra
 One can explore the causal relationship between internal properties
and external qualities.
 Example: Modularity enables testability.

 Value-based view
 This represents the merger of two concepts: excellence and worth.
 Quality is a measure of excellence, and value is a measure of worth.
 Central idea
 How much a customer is willing to pay for a certain level of quality.
 Quality is meaningless if a product does not make economic sense. 8
 The value-based view makes a trade-off between cost and quality.
MCCALL’S QUALITY FACTORS AND CRITERIA
 Quality Factors
 McCall, Richards, and Walters studied the concept of

Pooja Malhotra
software quality in terms of two key concepts as follows:
 quality factors, and
 quality criteria.

 A quality factor represents the behavioral characteristic of a

system.
 Examples: correctness, reliability, efficiency, testability, portability,
…

9
MCCALL’S QUALITY FACTORS AND CRITERIA

Pooja Malhotra
10
Table : McCall’s quality factors
MCCALL’S QUALITY FACTORS

Pooja Malhotra
11
MCCALL’S QUALITY FACTORS AND CRITERIA

Pooja Malhotra
12
MCCALL’S QUALITY FACTORS AND CRITERIA
 Quality Criteria
 A quality criterion is an attribute of a quality factor that is

Pooja Malhotra
related to software development.
 Example:
 Modularity is an attribute of the architecture of a software system.
 A highly modular software allows designers to put cohesive
components in one module, thereby increasing the maintainability of
the system.

 In Table:- quality criteria have been listed.

13
MCCALL’S QUALITY FACTORS AND CRITERIA

Pooja Malhotra
Table 17.3: McCall’s quality criteria [10]. 14
MCCALL’S QUALITY FACTORS AND CRITERIA
 Relationship Between Quality Factors and Quality
Criteria

Pooja Malhotra
 Each quality factor is positively influenced by a set of quality
criteria, and the same quality criterion impacts a number of
quality factors.
 Example: Simplicity impacts reliability, usability, and testability.
 If an effort is made to improve one quality factor, another
quality factor may be degraded.
 Portable code may be less efficient.
 Some quality factors positively impact others.
 An effort to improve the correctness of a system will increase its
reliability.
15
MCCALL’S QUALITY

Pooja Malhotra
FACTORS AND
CRITERIA

16
QUALITY
QualityMANAGEMENT
management is to

•plan suitable Quality control and Quality assurance activities,

•define procedures and standards which should be used during

software development and verifying that these are being followed

Pooja Malhotra
by everyone

•Properly execute and control activities.

•Verifying and improving the software process.

• Quality Management and Project Management

1
 THE QUALITY REVOLUTION - PDCA CYCLE
The Shewhart cycle

Pooja Malhotra
 Deming introduced Shewhart’s PDCA cycle to Japanese researchers
 It illustrate the activity sequence:
 Setting goals
 Assigning them to measurable milestones
 Assessing the progress against the milestones
 Take action to improve the process in the next cycle

18
 QUALITY COST & BENEFITS OF INVESTMENT ON
QUALITY

Cost of quality is decomposed into three major categories:

1. Prevention Costs(Quality Planning, FTRs, Testing , training etc.)
2. Appraisal Costs(SMP, testing, inspection, equipment calibrations and
maintenance)

Pooja Malhotra
3. Failure Costs(analyse and remove failures)
 Internal Failure Costs(developer’s site)
 External Failure Costs(customer’s site)

19
 QUALITY CONTROL AND QUALITY ASSURANCE

Quality Control is basically related to software product such that there

is minimum variation, according to the desired specifications. This

Pooja Malhotra
variation is checked at each step of development. Quality control may
include the following activities: Reviews, Testing using manual
techniques or with automated tools (V & V).

Quality Assurance is largely related to the process. In addition, quality

assurance activities are in management zone. Therefore auditing and
reporting of quality based on quantitative measurements are also
performed.

20
 Methods of quality management
SOFTWARE TESTING MYTHS

Pooja Malhotra
PROCEDURAL APPROACH TO QM
 SQA activities(Product evaluation and process
monitoring)

Pooja Malhotra
 Products- Standards and Process- Procedures
 SQA relationships with other assurance activities
 Configuration Management monitoring
 Verification and validation monitoring
 Formal test monitoring
 SQA during SDLC

22
SOFTWARE QUALITY ASSURANCE BEST
PRACTICE
 Continuous improvement: All the standard process in SQA
must be improved frequently and made official so that the
other can follow. This process should be certified by popular
organization such as ISO, CMMI… etc.
 Documentation: All the QA policies and methods, which are
defined by QA team, should be documented for training and reuse

Pooja Malhotra
for future projects.
 Experience: Choosing the members who are seasoned SQA
auditors is a good way to ensure the quality of management
review
 Tool Usage: Utilizing tool such as the tracking tool,
management tool for SQA process reduces SQA effort and project
cost.
 Metrics: Developing and creating metrics to track the software
quality in its current state, as well as to compare the
improvement with previous versions, will help increase the value
and maturity of the Testing process
 Responsibility: The SQA process is not the SQA member’s task,
but everyone’s task. Everybody in the team is responsible for
23
quality of product, not just the test lead or manager.
QUANTITATIVE APPROACH TO QM
Major Issues
 Setting Quality Goal

Pooja Malhotra
Estimate for defects(P)current Project=
Defects(SP)X effort estimate(P)/Actual effort(SP)
 Managing software development process
quantitatively- Intermediate goals

24
PAUL GOODMAN MODEL FOR SOFTWARE
METRICS PROGRAM

Pooja Malhotra
SOFTWARE QUALITY METRICS

Product Quality Metrics

 Mean time to failure (MTTF)

Pooja Malhotra
MTTF metric is an estimate of the average or mean time
until a product’s first failure occurs.

Defect Density Metrics

 Defect Density = Number of Defects / Size of Product

Customer problem metrics

 problems per user month (PUM) = Total problems
reported by the customer for a time period / Total number
of licensed months of the software during the period.
 Customer Satisfaction Metrics

26
SOFTWARE QUALITY METRICS

In-Process Quality Metrics

Defect Density during testing

Pooja Malhotra


 Defect Arrival pattern during Testing

 Defect Removal Efficiency

DRE = (Defects removed during the month / No.
of problems arrivals during the month) X 100

27
 Software Quality Metrics

Metrics for Software maintenance

Fix backlog and backlog management index

Pooja Malhotra
BMI = (Number of problems closed during the month /
Number of problem arrivals during the month) x 100 %

Fix response time and fix responsiveness

Percent Delinquent fixes

Percent Delinquent fixes = (Number of fixes that exceeded
the response time criteria by severity level / Number of fixes
delivered in a specified time) X 100%

Fix Quality: Defective fix percentage

28
Capability Maturity Model (CMM)

Pooja Malhotra
CAPABILITY MATURITY MODEL (CMM)

Pooja Malhotra
CAPABILITY MATURITY MODEL (CMM)

Pooja Malhotra
31
CAPABILITY MATURITY MODEL (CMM)

Pooja Malhotra
32
CAPABILITY MATURITY MODEL (CMM)

Pooja Malhotra
33
6-SIGMA
 Originated by Motorola in Schaumburg, IL
 Based on competitive pressures in 1980s – “Our

Pooja Malhotra
quality stinks”

Sigma Defects Per Million

1δ 690,000
2δ 308,537
3δ 66,807
4δ 6,210
5δ 233
6δ 3.4
34
6-SIGMA

3δ 6δ

Pooja Malhotra
Five short or long landings at any One short or long landing in 10
major airport years at all airports in the US

Approximately 1,350 poorly One incorrect surgical operation in

performed surgical operations in 20 years
one week
Over 40,500 newborn babies Three newborn babies dropped by
dropped by doctors or nurses each doctors or nurses in 100 years
year
Drinking water unsafe to drink for Water unsafe to drink for one
about 2 hours each month second every six years
35
6-SIGMA D-M-A-I-C CYCLE
 Define
 The first step is to define customer satisfaction goals and subgoals; for
example, reduce cycle time, costs, or defects. These goals then provide a
baseline or benchmark for the process improvement.

Pooja Malhotra
 Measure
 The Six Sigma team is responsible for identifying a set of relevant
metrics.
 Analyze
 With data in hand, the team can analyze the data for trends, patterns,
or relationships. Statistical analysis allows for testing hypotheses,
modeling, or conducting experiments.
 Improve
 Based on solid evidence, improvements can be proposed and
implemented. The Measure-Analyze-Improve steps are generally
iterative to achieve target levels of performance.
 Control
 Once target levels of performance are achieved, control methods and 36
tools are put into place in order to maintain performance.
6-SIGMA ROLES & RESPONSIBILITIES
 Master black belts
 People within the organization who have the highest level of
technical and organizational experience and expertise. Master
black belts train black.

Pooja Malhotra
 Black belts
 Should be technically competent and held in high esteem by their
peers. They are actively involved in the Six Sigma change process.
 Green belts
 Are Six Sigma team leaders or project managers. Black belts
generally help green belts choose their projects, attend training
with them, and then assist them with their projects once the
project begins.
 Champions
 Leaders who are committed to the success of the Six Sigma project
and can ensure that barriers to the Six Sigma project are removed.
Usually a high-level manager who can remove obstacles that may
involve funding, support, bureaucracy, or other issues that black 37
belts are unable to solve on their own.
SOSOFTWARE TOTAL QUALITY MANAGEMENT
(STQM)
TQM is defined as a quality-centered, customer-focused,
fact-based, team-driven, senior-management-led process
to achieve an organization’s strategic imperative through

Pooja Malhotra
continuous process improvement.

T = Total = everyone in the organization

Q = Quality = customer satisfaction
M = Management = people and processes

38
 SOSOFTWARE TOTAL QUALITY MANAGEMENT
(STQM)

The elements of TQM / STQM

Pooja Malhotra
 Customer-focus/Customer-focus in software
development
 Process / Process, technology, and development quality

 Human-side of quality/Human-side of software quality

 Measurement and analysis

39
ISO 9000 STANDARD

The ISO 9000 family of quality management

systems standards is designed to help organizations
ensure that they meet the needs of customers and

Pooja Malhotra
other stakeholders while meeting statutory and
regulatory requirements related to a product or
program. ISO 9000 deals with the fundamentals of
quality management systems, including the seven
quality management principles upon which the
family of standards is based. ISO 9001 deals with
the requirements that organizations wishing to meet
the standard must fulfill.
40
ISO 9000:2015 SOFTWARE QUALITY
STANDARD AND FUNDAMENTALS
This International Standard provides the fundamental
concepts, principles and vocabulary for quality management
systems (QMS) and provides the foundation for other QMS
standards. This International Standard is intended to help

Pooja Malhotra
the user to understand the fundamental concepts, principles
and vocabulary of quality management, in order to be able to
effectively and efficiently implement a QMS and realize value
from other QMS standards.
This International Standard proposes a well-defined QMS,
based on a framework that integrates established
fundamental concepts, principles, processes and resources
related to quality, in order to help organizations realize their
objectives. It is applicable to all organizations, regardless of
size, complexity or business model. Its aim is to increase an
organization’s awareness of its duties and commitment in
fulfilling the needs and expectations of its customers and
interested parties, and in achieving satisfaction with its
products and services. 41
ISO 9000:2015 SOFTWARE QUALITY
STANDARD AND FUNDAMENTALS
This International Standard describes the fundamental concepts
and principles of quality management which are universally
applicable to the following:
— organizations seeking sustained success through the

Pooja Malhotra
implementation of a quality management system;
— customers seeking confidence in an organization’s ability to
consistently provide products and services conforming to their
requirements;
— organizations seeking confidence in their supply chain that
product and service requirements will be met;
— organizations and interested parties seeking to improve
communication through a common understanding of the
vocabulary used in quality management;
— organizations performing conformity assessments against the
requirements of ISO 9001;
— providers of training, assessment or advice in quality
management; 42
— developers of related standards.
ISO 9000:2015 SOFTWARE QUALITY STANDARD AND
FUNDAMENTALS

Essentially the layout of the standard is similar to the previous

ISO standard in that it follows the Plan, Do, Check, Act cycle in
a process based approach, but is now further encouraging this to
have risk based thinking. The purpose of the quality objectives

Pooja Malhotra
is to determine the conformity of the requirements (customers
and organizations), facilitate effective deployment and improve
the quality management system

43
ISO 9000:2015 SOFTWARE QUALITY STANDARD AND
FUNDAMENTALS

 Risk-based thinking is essential for achieving an effective quality

management system. The concept of risk-based thinking has been implicit in
previous editions of this International Standard including, for example,
carrying out preventive action to eliminate potential nonconformities,
analysing any nonconformities that do occur, and taking action to prevent

Pooja Malhotra
recurrence that is appropriate for the effects of the nonconformity.
 To conform to the requirements of this International Standard, an
organization needs to plan and implement actions to address risks and
opportunities. Addressing both risks and opportunities establishes a basis for
increasing the effectiveness of the quality management system, achieving
improved results and preventing negative effects.
 Opportunities can arise as a result of a situation favourable to achieving an
intended result, for example, a set of circumstances that allow the
organization to attract customers, develop new products and services, reduce
waste or improve productivity. Actions to address opportunities can also
include consideration of associated risks. Risk is the effect of uncertainty and
any such uncertainty can have positive or negative effects. A positive
deviation arising from a risk can provide an opportunity, but not all positive
effects of risk result in opportunities. 44
ISO 9000:2015 SOFTWARE QUALITY
STANDARD AND FUNDAMENTALS
The ISO 9000 series are based on seven quality
management principles (QMP)

Pooja Malhotra
QMP 1 – Customer focus
QMP 2 – Leadership
QMP 3 – Engagement of people
QMP 4 – Process approach
QMP 5 – Improvement
QMP 6 – Evidence-based decision making
QMP 7 – Relationship management
45
ISO 9001:2015 REQUIREMENTS
SO 9001:2015 Quality management systems —
Requirements is a document of approximately 30 pages
which is available from the national standards

Pooja Malhotra
organization in each country.
ISO 9001:2015 is the latest revision of the ISO 9001
standard. In it, there are 10 sections (clauses) with
supporting subsections (sub clauses). The requirements
to be applied to your quality management system
(QMS) are covered in sections 4-10. To successfully
implement ISO 9001:2015 within your organization,
you must satisfy the requirements within clauses 4-10.
46
 ISO 9001:2015 REQUIREMENTS
Some of the key changes include:
High Level Structure of 10 clauses is implemented. Now all new standard
released by ISO will have this High level structure.
 Greater emphasis on building a management system suited to each

Pooja Malhotra
organization's particular needs
 A requirement that those at the top of an organization be involved and
accountable, aligning quality with wider business strategy
 Risk-based thinking throughout the standard makes the whole
management system a preventive tool and encourages continuous
improvement
 Less prescriptive requirements for documentation: the organization
can now decide what documented information it needs and what
format it should be in
 Alignment with other key management system standards through the
use of a common structure and core text
47
 Inclusion of Knowledge Management principles
 Quality Manual & Management representative is now not mandatory
requirements.
ISO 9001:2015 REQUIREMENTS
Contents of ISO 9001:2015 are as follows:
Section 1: Scope

Pooja Malhotra
Section 2: Normative references
Section 3: Terms and definitions
Section 4: Context of the organization
Section 5: Leadership
Section 6: Planning
Section 7: Support
Section 8: Operation
Section 9: Performance evaluation
Section 10: Improvement
48
SOFTWARE QUALITY TOOLS
 Ishikawa Diagram
 Check List

Pooja Malhotra
 Control Chart

 Flow Chart

 Pareto Chart

 Histogram

49
CAUSE & EFFECT DIAGRAMS

Kaoru Ishikawa (1915 - 1989)

 Studied under Deming

 Believed quality is a continuous process that relies on all

levels of the organization

Pooja Malhotra
Cause and effect diagrams (Ishikawa Diagram) are used for
understanding organizational or business problem causes.
Organizations face problems everyday and it is required to
understand the causes of these problems in order to solve
them effectively. Cause and effect diagrams exercise is
usually a teamwork.
A brainstorming session is required in order to come up
with an effective cause and effect diagram.
All the main components of a problem area are listed and
possible causes from each area is listed.
Then, most likely causes of the problems are identified to
carry out further analysis.
50
 ISHIKAWA, OR FISHBONE DIAGRAM
BEST DEVELOPED BY BRAINSTORMING OR BY USING A
LEARNING CYCLE APPROACH

Pooja Malhotra
51
CONTROL CHARTS
 Walter A. Shewhart (1891 – 1967)
 Worked for Western Electric Company (Bell
Telephones)

Pooja Malhotra
 Introduced the concept of the control chart as a tool
better to understand variation and to allow
management to shift its focus away from inspection
and more towards the prevention of problems and the
improvement of processes.

52
CONTROL CHARTS
A quality control chart is a graphic that depicts
whether sampled products or processes are meeting
their intended specifications and, if not, the degree by
which they vary from those specifications.

Pooja Malhotra
The control chart is a graph used to study how a
process changes over time. Data are plotted in time
order. A control chart always has a central line for the
average, an upper line for the upper control limit, and
a lower line for the lower control limit. These lines are
determined from historical data. By comparing
current data to these lines, you can draw conclusions
about whether the process variation is consistent (in
control) or is unpredictable (out of control, affected by
special causes of variation). 53
CONTROL CHARTS
Common and special causes are the two distinct
origins of variation in a process, as defined in the
statistical thinking and methods of Walter A.

Pooja Malhotra
Shewhart and W. Edwards Deming. Briefly,
"common causes", also called Natural patterns,
are the usual, historical, quantifiable variation in
a system, while "special causes" are unusual, not
previously observed, non-quantifiable variation.

54
CONTROL CHARTS
Common causes
 Inappropriate procedures
 Poor design

 Poor maintenance of machines

Pooja Malhotra
 Measurement error

 Quality control error

Special causes
 Faulty controllers
 Machine malfunction

 Computer crash

 Poor batch of raw material

55
CONTROL CHARTS

Pooja Malhotra
56
 CONTROL CHARTS
Purpose:
The primary purpose of a control chart is
to predict expected product outcome.

Benefits:
 Predict process out of control and out of
specification limits
 Distinguish between specific, identifiable
causes of variation
 Can be used for statistical process
control
Pooja Malhotra 57
PARETO CHART

Pooja Malhotra
58
PARETO CHART
Pareto charts are used for identifying a set of
priorities. You can chart any number of
issues/variables related to a specific concern and

Pooja Malhotra
record the number of occurrences.
This way you can figure out the parameters that
have the highest impact on the specific concern.
This helps you to work on the propriety issues in
order to get the condition under control.

59
FLOW CHART

Pooja Malhotra
FOR PROJECT
SCOPE
VERIFICATION

60
 Pooja Malhotra
61
CHECK LIST
CHECK LIST/SHEET

A check sheet can be introduced as the most basic

tool for quality.A check sheet is basically used for
gathering and organizing data.
When this is done with the help of software packages

Pooja Malhotra
such as Microsoft Excel, you can derive further
analysis graphs and automate through macros
available.
Therefore, it is always a good idea to use a software
check sheet for information gathering and organizing
needs.
One can always use a paper-based check sheet when
the information gathered is only used for backup or
storing purposes other than further processing.
62
HISTOGRAM
A histogram is a graphical representation that
organizes a group of data points into user-specified
ranges. It is similar in appearance to a bar graph.

Pooja Malhotra
The histogram condenses a data series into an easily
interpreted visual by taking many data points and
grouping them into logical ranges or bins.
 A histogram is a bar graph-like representation of
data that buckets a range of outcomes into
columns along the x-axis.
 The y-axis represents the number count or
percentage of occurrences in the data for each
column and can be used to visualize data
distributions. 63
 Pooja Malhotra
64
HISTOGRAM
 Pooja Malhotra
65
HISTOGRAM