Software Quality Assurance
Software Quality Assurance
Software Quality Assurance
Users, however, often measure more than reliability. They are also
concerned about usability, including ease of installation, learning
and use.
MEASURING QUALITY –
manufacturer’s view
Manufacturer’s vies of quality suggests two characteristics to
measure:
Rework Costs: Defects differ in their effect on the system: some take
a little time to find and fix; others are catastrophic and consume
valuable resources. To monitor the effect of defect detection and
correction, we often measure rework costs-
A quality model is the set of characteristics and
the relationships between them, which
provide the basis for specifying quality
requirements, and evaluating quality.
McCall started with a volume of 55 quality characteristics
which have an important influence on quality, and called them
"factors". For reasons of simplicity, McCall then reduced the
number of characteristics to the following eleven:
External quality integrity Internal quality efficiency
▪Integrity ▪Efficiency
▪Reliability ▪Maintainability
▪Usability ▪Testability
▪Accuracy ▪Flexibility
▪Interface Facility
▪Re-usability
▪Transferability
Efficiency: McCall's view of efficiency or performance is concerned with the efficient
use of computer code to perform processes and the efficient use of storage resources.
There are a number of techniques that can be used to achieve both of these objectives.
Typical of these are:
Programming languages. Selecting the most appropriate programming language for
the problem has a major impact on program efficiency.
Operating systems. Modern operating systems have the ability to perform multi-
tasking thereby improving system performance by facilitating background operations.
Design. Strategies that address cohesion and coupling, normalization techniques to
reduce data redundancy and algorithms that optimize process time should always be
employed.
Access strategies. Algorithms that optimize seek time, rotational delay and data
transfer time must be continuously searched out and implemented to improve
efficiency.
Programming techniques. Typical good programming techniques and practice like:
Integrity: The extent to which illegal access to the programs and data of a product can
be controlled. McCall et al.
Integrity has to concern itself with controls for preventing inaccurate data entering the
system and detecting it if it does. It also has to concern itself with preventing changes to
the software which compromise its original purpose.
French (1986) states that the aims of these controls are:
He also suggests that there are five different types of control. Manual checks which are
applied to documents before computer processing, data preparation controls,
validation checks, batch controls and file controls. Only the last three of these can be
built into the software to ensure integrity.
Reliability: The extent to which a program can be maintained so that it can fulfill its
specific function.
The mean time between failures - under pre-defined conditions, the average time
between consecutive failures over a given period in the life of a system.
The mean time to repair - the average time to repair or maintain equipment.
The mean time to recover - the average time to return a system to operation after a
failure. The time involved should include periods taken to re-instate from previous
checkpoints.
The probability of failure - the use of formal methods to predict the likelihood that a
system will behave in an expected way under certain circumstances. Probability of
failure is most appropriate to safety-critical systems and "continuous running" systems.
Usability: The cost/effort to learn and handle a product.
Software ergonomics is concerned with topics like how suitable is the software for the
intended operations. How easy is it for users to learn and to master it.
Accuracy: The extent to which a program fulfils its specification.
Ghezzi et al. also prefer the term correctness and their definition is "A
program is functionally correct if it behaves according to the specifications of the
functions it should provide".
Maintainability: The cost of localizing and correcting errors.
Ghezzi et al. (1991) divide maintenance into three categories: corrective, adaptive and
perfective and only corrective is concerned with correcting errors as suggested by
McCall.
Corrective maintenance is concerned with removing minor bugs left after development
and testing are completed. This process is also involved after other maintenance
activities.
Recent interpretation of flexibility would be more associated with adaptability, ie. being
able to change or reconfigure the user interface to suit users' preferences. This is a
usability quality issue and is better considered in the usability section.
Interface facility (Interoperability): The cost of connecting two products with one
another.
This is the development strategy that encourages product development in a
manner that it can interact with other products.
Functionality :
The F in the FURPS acronym represents all the system-wide functional requirements
that we would expect to see described.
Usability:
Usability includes looking at, capturing, and stating requirements based around user
interface issues, things such as accessibility, interface aesthetics, and consistency within
the user interface.
Reliability:
Reliability includes aspects such as availability, accuracy, and recoverability, for
example, computations, or recoverability of the system from shut-down failure.
Performance:
Performance involves things such as throughput of information through the system,
system response time (which also relates to usability), recovery time, and startup time.
Supportability :
Finally, we tend to include a section called Supportability, where we specify a number
of other requirements such as testability, adaptability, maintainability, compatibility,
configurability, installability, scalability, localizability, and so on
Software quality management is split
into three main activities:
Quality assurance: Quality planning: Quality control:
The development of The selection of Definition of
a framework of appropriate processes ensuring
organizational procedures and that software
procedures and standards from this development follows
standards that lead framework and adapt the quality
to high quality for a specific procedures and
software. software project. standards.
What is quality control -- the series of inspections, reviews,
and test used throughout the develop cycle of a software
product
Quality control includes a feedback loop to the process.
Objective ---> minimize the produced defects, increase
the product quality
Implementation approaches:
- Fully automated
- Entirely manual
- Combination of automated tools and human
interactions
Key concept of quality control:
--> compare the work products with the
specified and measurable standards
Quality assurance consists of:
- the auditing and reporting function of
management
Goal --> provide management with the necessary
data about product quality.
--> gain the insight and confidence of product
quality
Systematic activities providing evidence of
the fitness for use of the total software
product.
It is achieved through the use of established
guidelines for quality control to ensure
integrity and prolonged life of software.
It is a planned effort to ensure that a software
product fulfills criteria and has additional
attributes specific to the product.
It is the collection of activities and functions
used to monitor and control a software
project so that specific objectives are
achieved with the desired level of confidence.
It is not the sole responsibility of the
software quality assurance group but is
determined by the consensus of the project
manager ,project leader, project personnel,
and the users.
Software Quality Assurance
The first formal quality assurance and control function was introduced
at Bell Labs in 1916 in the manufacturing world.
- During the 1950s and 1960s, the programmers controls their product
quality.
- Audits designated software work products to verify compliance the defined process
- Ensure the deviations in software work and products according to a documented procedure
People involved
• Producer, review leader, 2 or 3 reviewers (one of
in a review them is recorder)
meeting:
Review Guidelines (for FTR)
Statistical
quality Statistical quality assurance implies the following steps:
assurance
reflects a
growing
trend
throughout
industry to
- Using the
become
Pareto
more - Information Once the vital
- An attempt is principle (80
quantitative about few causes
made to trace percent of the
about software have been
each defect to defects can be
quality. defects is identified,
its underlying traced to 20
collected and correct the
cause percent, and
categorized defects.
isolate the 20
percent)
Statistical Quality Assurance
Causes of •
•
- Inconsistent module interface (IMI)
- Error in design logic (EDL)
errors: •
•
- Incomplete or erroneous testing (IET)
- Inaccurate or incomplete documentation (IID)
• - Error in programming language translation of design
(PLT)
• - Ambiguous or inconsistent human-computer
interface (HCI)
• - Miscellaneous (MIS)
Statistical Quality Assurance
In conjunction with the collection of defect information, software developers can calculate an
error index (EI) for each major step in the software engineering process.
After analysis, design, coding, testing, and release, the following data are collected:
Ei = the total no. of errors uncovered during the ith step in the process.
Si = the no. of serious errors
Mi = the no. of moderate errors
Ti = the no. of minor errors
PS = the size of the product at the ith step.
At each step in the software engineering process, a phase index (PI i ) is computed:
3
8
Cost of Software Quality
cost of software quality
– the economic assessment of software quality
development and maintenance
– is just another class of software quality metrics,
where financial values are used as the measuring
tool
3
9
The classic model of cost of software quality
The classic quality cost model, developed in the
early 1950s by Feigenbaum
Provides a methodology for classifying the costs
associated with product quality assurance from an
economic point of view
Developed to suit the quality situations found in
manufacturing organizations
4
0
The model classifies costs related to product quality
into two general classes:
Costs of control include costs that are spent to
prevent and detect soft-ware errors in order to
reduce them to an accepted level
Costs of failure of control include costs of failures
that occurred because of failure to prevent and
detect software errors
4
1
Costs of control are assigned to either the prevention or the appraisal
costs subclass:
Prevention costs include investments in quality infrastructure and
quality activities that are not directed to a specific project or system,
being general to the organization
Appraisal costs include the costs of activities performed for a
specific project or software system for the purpose of detecting
software errors
4
3
Prevention costs
Typical preventive costs include:
(1)Investments in development of new or improved SQA
infrastructure components or, alternatively, regular updating of
those components:
Procedures and work instructions
Support devices: templates, checklists, etc.
(2)Regular implementation of SQA preventive activities:
Instruction of new employees in SQA subjects and procedures relat-
ed to their positions
Instruction of employees in new and updated SQA subjects and
procedures
Certification of employees for positions that require special
certification
Consultations on SQA issues provided to team leaders and others.
(3) Control of the SQA system through performance of:
Internal quality reviews
External quality audits by customers and SQA system certification
organizations
9 Department of IEM, MSRIT
Management quality reviews
Appraisal costs
Appraisal costs are devoted to detection of software errors in specific projects
or software systems. Typical appraisal costs cover:
(1)Reviews:
Formal design reviews (DRs)
Peer reviews (inspections and walkthroughs)
Expert reviews
(2)Costs of software testing:
Unit tests
Integration tests
Software system tests
Acceptance tests (participation in tests carried out by the customer).
(3)Costs of assuring quality of external participants, primarily by means of
design reviews and software testing. These activities are applied to the
activities performed by:
Subcontractors
Suppliers of COTS software systems and reusable software modules
The customer as a participant in performing the project
Internal failure costs
Internal failure costs are those incurred when
correcting errors that have been detected by design
reviews, software tests and acceptance tests
performed before the software has been installed at
customer sites
Typical costs of internal failures are:
Costs of redesign or design corrections subsequent to
design review and test findings
Costs of re-programming or correcting programs in
response to test findings
Costs of repeated design review and re-testing
(regression tests)
External failure costs
External failure costs entail the costs of correcting failures
detected by customers or maintenance teams after the
software system has been installed at customer sites
Typical external failure costs cover:
Resolution of customer complaints during the warranty
period. In most cases, this involves a review of the
complaint and transmission of instructions
Correction of software bugs detected during regular
operation
Correction of software failures after the warranty period is
over even if the correction is not covered by the warranty
Damages paid to customers in case of a severe software
failure detected during regular operation
Insurance against customer’s claims in case of severe
software failure
The greater proportion of external failure costs hidden
costs – reflect the indirect damages suffered by the
software development organization as a result of those
same failures
Typical examples of hidden external failure costs are:
Damages of reduction of sales to customers suffering from
high rates of software failures
Severe reduction of sales motivated by the firm’s
damaged reputation
Increased investment in sales promotion to counter the effects of
past software failures
Reduced prospects to win a tender or, alternatively, the
need to under-price to prevent competitors from winning 13tenders
An extended model for cost of software
quality
Managerial preparation and control costs
Managerial preparation and control costs are associated
with activities per-formed to prevent managerial failures or
reduce prospects of their occurrence
Typical managerial preparation and control costs include:
Costs of carrying out contract reviews (proposal draft and
contract draft reviews)
Costs of preparing project plans, including quality plans
and their review
Costs of periodic updating of project and quality plans.
Costs of performing regular progress control of internal
software development efforts
Costs of performing regular progress control of external
participant’s contributions to the project
Managerial failure costs
Managerial failure costs can be incurred throughout the
entire course of software development, beginning in the
pre-project stage
Typical managerial failure costs include:
Unplanned costs for professional and other resources,
resulting from underestimation of the resources upon
which the submitted proposals are based
Damages paid to customers as compensation for late
completion of the project, a result of the unrealistic
schedule presented in the company’s proposal
Damages paid to customers as compensation for late
completion of the project, a result of management’s
failure to recruit sufficient and appropriate team members
Domino effect: damages to other projects performed by
the same teams involved in the delayed projects
Application of a cost of software quality
system
In order to apply a cost of software quality system in an
organization, the following are required:
Definition of a cost of software quality model and array of
cost items specifically for the organization, department,
team or project
Each of the cost items that constitute the model should
be related to one of the sub-classes of the chosen cost of
software quality model (the classic model or the extended
model)
Definition of the method of data collection.
Application of a cost of software quality system, including
thorough follow-up.
Actions to be taken in response to the findings produced
HUMAN
RESOURCE
PLANNING
The Global IT Workforce
Although there have been ups and downs in the IT labor market, there
will always be a need for good IT workers
By the end of 2014, there were almost 3 billion Internet users and 2.3
billion mobile-broadband subscriptions
PMI estimates demand for 15.7 million project management jobs from
2010 to 2020
What is Project Human Resource
Management?
Making the most effective use of the people involved with a
project
Processes include
Planning human resource management: identifying and documenting project roles,
responsibilities, and reporting relationships
Acquiring the project team: getting the needed personnel assigned to and working
on the project
Developing the project team: building individual and group skills to enhance project
performance
Managing the project team: tracking team member performance, motivating team
members, providing timely feedback, resolving issues and conflicts, and coordinating
changes to help enhance project performance
Project Human Resource Management
Summary
Keys to Managing People
• Motivation theories
• Influence and power
• Effectiveness
• Emotional intelligence
• Leadership
Intrinsic and Extrinsic Motivation
Contents include
R = responsibility
A = accountability, only one A per task
C = consultation
I = informed
Note that some people reverse the definitions of responsible and accountable.
Staffing Management Plans and Resource
Histograms
Staffing plans and good hiring procedures are important, as are incentives for
recruiting and retention
• Some companies give their employees one dollar for every hour a new person they helped hire works
• Some organizations allow people to work from home as an incentive
Enrollment in U.S. Computer science and engineering programs has dropped almost
in half since 2000, and one-third of U.S. Workers were over the age of 50 by 2010.
• Physical challenges
• Psychological preference indicator tools
Reward and Recognition Systems
View Project
Administrative
Communication in Controlling Closure
the context of the Processes
five PM process Performance Closing
groups. Reporting Processes
The project development team (PDT)
develops a communication plan by asking
the following questions:
• Who needs what information?
• When do they need the information?
• Who delivers the information?
• How should the information be delivered?
• WBS product list — a list of potential
The PDT project products, based on the work plan
develops two that includes all the elements of the
inputs for the WBS, and the sub-products of the WBS.
• Project charter — the record of the
project agreement between the sponsor and the
communication project manager on the key elements of
planning a project. The project charter lists the
project manager, the project sponsor,
process: and the PDT
The PDT must identify the stakeholders on a project, determine what
their needs and expectations are, and then manage and influence
those expectations to ensure a successful project.
The functional managers list all the assigned task managers on the
communication matrix and the stakeholder analysis.
The project manager or PDT members incorporate
changes from the project stakeholders into the project
communication plan.
Plan Contracting
Select Sellers
Contract Administration
Contract Closeout
Major Processes:
• Determining what to procure and when and how (make or buy)
Primary Inputs:
• Enterprise Environmental Factors
• Organizational Process Assets
• Project scope statement
• WBS
• WBS Dictionary
• Project Management Plan
Tools & Techniques
• Make or buy analysis
• Expert judgment
• Contract types
Primary Outputs
• Procurement mgmt plan
Contract Statement(s) of Work
• Make or Buy Decisions
• Requested Changes
Major Processes:
• Preparing the documents needed to do contracting. Document requirement and identify sellers
Primary Inputs:
• Procurement management plan
• Contract Statement(s) of work
• Make or Buy Decisions
• Project Management Plan
Primary Outputs
• Procurement documents
• Evaluation criteria
• Contract Statement of work (updates)
Major Processes:
• Obtaining quotations, bids, offers, or proposals
Primary Inputs:
• Organizational Process Assets
• Procurement Management Plan
• Procurement Documents
Primary Outputs
• Qualified sellers list
• Procurement Document Package
• Proposals
Major Tools & Primary
Primary Inputs:
Processes: Techniques Outputs
• Involves the receipt • Organizational Process • Weighting system • Selected Sellers
of bids or proposals Assets • Independent estimates • Contract
• Procurement • Screening system • Contract Management
and the application
Management Plan • Contract Negotiation Plan
of evaluation
• Evaluation Criteria • Seller Rating System • Resource Availability
criteria to select a
• Procurement Document • Expert Judgment • Procurement
seller. Also involves Package • Proposal Evaluation Management Plan
applying evaluation • Proposals (Updates)
Techniques
criteria. • Qualified Sellers List • Requested changes
• Project Management
Plan
Major Tools & Primary
Primary Inputs:
Processes: Techniques Outputs
• Ensuring that the • Contract • Contract change • Contract
seller’s performance • Contract control system Documentation
meets contractual Management Plan • Buyer conducted • Requested Changes
requirements. • Selected Sellers performance • Recommended
• Performance review Corrective actions
Reports • Organizational
• Inspections and
• Approved change Process assets
requests audits • Project
• Work Performance • Performance Management plan
information reporting • Procurement
• Payment system Management
• Claims Plan
administration • Contract
• Records management
plan
management
system
• Information
technology
Major Processes:
• Product verification and administration closeout
Primary Inputs:
• Procurement management Plan
• Contract management plan
• Contract documentation
• Contract closure procedure
Primary Outputs
• Closed Contracts
• Organizational process assets (Updates)
Software
Configuration
Management
The problem:
Multiple people have to work on software that is
changing
More than one version of the software has to be
supported:
▪ Released systems
▪ Custom configured systems (different functionality)
▪ System(s) under development
▪ Software on different machines & operating systems
Definition Software Configuration
Management:
A set of management disciplines within a
software engineering process to develop a
baseline
Software Configuration Management
encompasses the disciplines and techniques of
initiating, evaluating and controlling change to
software products during and after a software
project
Software Configuration Management is a
project function with the goal to make
technical and managerial activities more
effective
Software Configuration Management can
be administered in several ways:
Organization-wide
Project-specific
Distributed among the project members
Mixture of all of the above.
Software Configuration Management
Activities:
Configuration item identification
Promotion management
Release management
Branch management
Variant management
Change management
Configuration item identification
Modeling the system as a set of evolving components
Promotion management
the creation of versions for other developers
Release management
the creation of versions for clients and users
Change management
the handling, approval & tracking of change requests
Branch management
the management of concurrent development
Variant management
the management of coexisting versions
Configuration Manager
Responsible for identifying configuration items
Also often responsible for defining the procedures for creating
promotions and releases
Change Control Board Member
Responsible for approving or rejecting change requests
Developer
Creates promotions triggered by change requests or the
normal activities of development. The developer checks in
changes and resolves conflicts
Auditor
Responsible for the selection and evaluation of promotions for
release and for ensuring the consistency and completeness of
this release.
We will define the following terms
Configuration Item
Baseline
SCM Directories
Version
Revision
Release
Configuration Item: An aggregation of hardware,
software, or both, designated for configuration
management and treated as a single entity in
the configuration management process.
Branching and merging. Having team members work concurrently is a no-brainer, but even
individuals working on their own can benefit from the ability to work on independent streams
of changes. Creating a "branch" in VCS tools keeps multiple streams of work independent from
each other while also providing the facility to merge that work back together, enabling
developers to verify that the changes on each branch do not conflict.
Traceability. Being able to trace each change made to the software and connect it to project
management and bug tracking software, and being able to annotate each change with a
message describing the purpose and intent of the change can help not only with root cause
analysis and other forensics.
Baseline: “A specification or product that has been
formally reviewed and agreed to by responsible
management, that thereafter serves as the basis for
further development, and can be changed only
through formal change control procedures.”
Examples:
Baseline A: The API has been completely been defined; the
bodies of the methods are empty
Baseline B: All data access methods are implemented and
tested
Baseline C: The GUI is implemented.
Effective baselines have the following
characteristics:
• A baseline must be associated with the production
and formal approval of a physical deliverable such as
a document or hardware/software component
• All items associated with a baseline must be placed
under formal change control.
Many naming scheme for baselines exist (1.0,
6.01a, ...)
A 3 digit scheme is quite common:
7.5.5
User
Programmer Master Software
Promotion Directory Release Repository
Change management is the handling of change
requests
The general change management process:
The change is requested
The change request is assessed against requirements
and project constraints
Following the assessment, the change request is
accepted or rejected
If it is accepted, the change is assigned to a developer
and implemented
The implemented change is audited.
No project mandate: Without a mandate (mission and objectives) it’s difficult for an at-risk project to recover. The
mandate is a blueprint for your program. As McKinsey states, “This mandate should include business case, project
justification, high-level requirements and success criteria.”
Unclear expectations: A mandate gets an IT project off on the right foot, but it’s no substitute for gathering
detailed requirements and expectations from all stakeholders. This sounds almost intuitive but project managers
eager to start can overlook this critical step.
Poor communications between IT and the business: Communicating well with your internal client is a must.
Business and IT often speak different language, so the project manager must translate. A big data project is
challenging enough, so don’t let miscommunication derail your efforts.
No user input: It’s one thing to engage line-of-business managers in the project requirements, but don’t forget the
end users who will actually work with your project deliverables. Identify potential gaps between what business
executives want and what their employees will use.
Sufficient software specific expertise: The software selection process can be time-consuming and tedious for
business owners and executives, and when it comes to implementations, finding project management
professionals with the relevant experience can be just as difficult, whether in-house or outsourced.
Quality testing and bug fixes requiring numerous software iterations: It is common to discover issues/bugs
throughout the testing phases that require fixing and retesting until the issues are resolved. It can be extremely
difficult for project teams to isolate issues, requiring escalation to more senior IT staff/developers.