AFROSAI-E

PERFORMANCE AUDIT
TEMPLATE
MANUAL
2010

Copyright clause
Compiled, edited and designed for the AFROSAI-E region
All rights reserved
No part of this manual may be reproduced or photocopied without permission in writing
from AFROSAI-E
AFROSAI-E
C/O THE AUDITOR-GENERAL OF SOUTH AFRICA
PO BOX 446
PRETORIA
SOUTH AFRICA
0001

Foreword
The Auditing Standards of INTOSAI1 stipulate that each SAI should adopt policies and
procedures to prepare manuals and other written guidance and instructions concerning
the conduct of audits. The implementation of a performance audit manual in line with
INTOSAI Auditing Standards is a key to the development of performance auditing at
individual SAIs.
The development of a performance audit template manual for the AFROSAI-E region
forms part of AFROSAI-Es efforts to support the development of performance
auditing.2 The performance audit template manual gives directions for the planning,
execution and finalisation of audits conducted by SAIs. It is important that the manual
be used as a tool to work smartly and that it is applied using common sense.
To learn performance audit is like learning how to drive a car. From reading manuals
and handbooks a driver will understand the theory but to become a good driver the
driving skills need to be applied and practised. For a SAI and an auditor to fully
understand performance auditing they also needs to conduct performance audits.
SAIs should customise and adapt the template manual to their own needs.
Consequently, a SAI that is about to introduce performance auditing can use the
template as its manual. As the SAI gains more experience in performance auditing, it
should adjust the template manual to local needs, particularly by customising relevant
sections.
This template manual is based on INTOSAIs Code of Ethics and Auditing Standards,
the Lima Declaration, the Implementation Guidelines for Performance Auditing, and the
Swedish National Audit Offices Handbook in Performance Auditing.
The template manual can be seen to be as a frozen moment on a journey where
several institutions and individuals have contributed their ideas and gradually modified
and improved the template manual over the years and several will hopefully contribute
on the onward journey. So enjoy the journey so far and please join the onward journey
and come with suggestions for improvement and amendments. The manual will only be
better with the feedback we receive from you.
We hope that the template manual will be a tool for you and your SAI to conduct good
performance audits and create change and better functioning public entities in the
region. Use the manual with the main performance audit tool - common sense - and
apply the manual with rational consideration and sound judgement.
M W Pretorius
Executive Officer
AFROSAI-E Secretariat

International Organization of Supreme Audit Institutions

AFROSAI-E is an organisation that consists of 23 supreme audit institutions (SAIs) in Africa
that are English or Portuguese speaking. AFROSAI-E has declared performance auditing as a
strategic imperative for the development of auditing within the region. AFROSAI-E supports its
members, the SAIs, with training and the production of training material, guidelines, quality
assurance visits and a website, etc.
2

Advice on how to customise the AFROSAI-E template

manual
The structure and standards set in the template derive from principles established by
INTOSAI. However, since the conditions for performance audit vary between countries,
each SAI should adapt the template to its own circumstances when developing a
performance audit manual. The extent to which a SAI will customise the template
depends on that SAIs specific conditions and demands. It is recommended that the
following sections of the template manual be customised. These sections appear
against a grey background in the template manual:
Foreword: Customise and make the foreword more country-specific.
Advice on how to customise the AFROSAI-E template manual into a performance
audit manual for the SAI: Delete when the manual has been customised.
Chapter 1.1 Purpose of the performance audit manual: Customise and, if appropriate,
add more bullets regarding the purpose of the manual.
Chapter 1.2 Organisation of the manual: List annexure relevant to the SAI.
Chapter 2.6 The mandate of the auditor-general: Customise to the mandate of the
Auditor-General in the SAI. Include each one of the issues mentioned if applicable.
Chapter 2.7 The performance auditor: Check whether all INTOSAI Auditing Standards
are compatible with the SAIs mandate or circumstances. Customise and explain the
SAIs recruitment policy for performance auditors.
Chapter 2.8 Organisation of performance auditing: Describe the performance audit
organisation within the SAI.
Chapter 2.9 Key players within the audit office: Describe the SAIs key players,
replacing the example with one relevant to the SAI.
Chapter 2.10 Monitoring and supervision: Add the SAI format for the role of
management during different stages in a performance audit.
Chapter 2.11 Quality control: Describe how quality control is performed within the SAI.
Chapter 2.12 Documentation: Describe the documentation and file standard of the
SAI.
Chapter 2.13 Internal communication: Add the format or routines for communication
within the SAI, replacing the example with a relevant one.
Chapter 2.14 External communication: Add the SAIs policy for external
communication.
Chapter 3.2.1 Strategic planning: Add the format and procedures for the SAIs
strategic planning.
Chapter 3.2.2 Annual planning: Add the format and procedures for the SAIs annual
operational planning.

Chapter 3.3 Area watching and general surveys. Amend to reflect the format and
procedures of the SAI.
Chapter 3.3.4 Presentation of possible audit topics: Add the SAIs format for
presentation of possible audit topics.
Chapter 3.4.5 The selection criteria: Customise and explain the SAIs selection criteria
and how the ranking between different criteria should be done.
Chapter 3.4.26 Pre-study memorandum: Add the SAIs procedures for presenting prestudy memorandums, including a policy on how pre-studies should be treated and
documented in cases where audit teams do not propose to launch a main study.
Chapter 3.5.99 Resource plan: Add the SAIs own format and an example of a
resource plan.
Chapter 4.1 Introduction of the main study to the auditee: Customise and state the
SAIs policy on contacts with the auditee.
Chapter 4.2 Data collection methods: The SAI could add its own structures,
suggestions and examples concerning data collection. For example, standard forms
for questionnaires or supplementary advice on conducting interviews can be added.
Chapter 4.15 Format for the report: The SAI should add its own report structure.
Chapter 4.17.4 Revising: Describe how the SAIs review process is structured.
Chapter 4.17.1 Review of the draft report: Customise to the SAIs standard.
Chapter 4Error! Reference source not found. Sending a draft report to the auditee:
Customise and explain the SAIs policy on releasing the draft report to the auditee.
Chapter 4.17.3 Internal finalisation: Customise and explain the SAIs finalisation
process.
Chapter 4.17.4 Exit meeting: Customise the SAIs exit meeting procedures.
Chapter 5.1 Presentation of the performance audit report: Include the SAIs policy and
procedures for presenting reports, including submission to stakeholders, contacts with
the media, etc.
Chapter 6.1 Follow-up of own work: Describe how the SAI follows up its work.
Chapter 6.2 Follow-up of the recommendations: Describe the SAIs format for the
follow-up and the design of the formal letter requesting information.
Chapter 6.3.2 Reporting on follow-up audits: Add the SAIs policies and procedures for
reporting on follow-up audits.

TABLE OF CONTENTS
Foreword ..................................................................................................................... 1
Advice on how to customise the AFROSAI-E template manual .............................. 2
1.

INTRODUCTION................................................................................................... 6
1.1
1.2

2.

Purpose of the performance audit template manual ...................................... 6

Structure of the manual ................................................................................. 6

THE CONTEXT OF PERFORMANCE AUDITING ................................................ 7

2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
2.10
2.11
2.12

Why performance auditing ............................................................................. 7

The 3Es: Economy, efficiency and effectiveness ........................................... 7
Definition of performance auditing ................................................................. 9
The special features of performance auditing .............................................. 10
Stages in the performance audit process..................................................... 11
The mandate of the Auditor-General ........................................................... 13
The performance auditor ............................................................................. 14
Organisation of performance auditing .......................................................... 14
Key players within the audit office................................................................ 15
Monitoring and supervision .......................................................................... 16
Quality control ............................................................................................. 16
Documentation ............................................................................................ 17

2.12.1
2.12.2
2.12.3

2.13
2.14
3.

Internal communication ............................................................................... 20

External communication .............................................................................. 21

PLANNING ......................................................................................................... 22
3.1
3.2
3.2.1
3.2.2

3.3
3.3.1
3.3.2
3.3.3
3.3.4
3.3.5

3.4
3.4.1
3.4.2
3.4.3
3.4.4
3.4.5
3.4.6

3.5
3.5.1
3.5.2
3.5.3
3.5.4
3.5.5
3.5.6
3.5.7
3.5.8
3.5.9

Documentation and ethics ................................................................................. 17

Guidelines for the preparation of working papers ............................................. 18
Audit files ........................................................................................................... 19

Planning and selection of audit topics.......................................................... 22

Planning at the overall level ......................................................................... 22
Strategic plan ......................................................................................................... 22
Annual planning ..................................................................................................... 23

Area watching and general surveys ............................................................. 23

Selecting potential audit topics .............................................................................. 24
Area watching ........................................................................................................ 24
General survey ...................................................................................................... 24
Presentation of possible audit topics ..................................................................... 26
Differences between area watching and a general survey ................................... 26

Pre-study ..................................................................................................... 27
Planning a pre-study .............................................................................................. 28
Collection of data ................................................................................................... 28
Analysis of data and identification of possible audit problems .............................. 29
Activities in organisations or programmes can be audited .................................... 31
The selection criteria.............................................................................................. 31
Pre-study memorandum ........................................................................................ 32

Work plan .................................................................................................... 33

Audit problem......................................................................................................... 33
Audit objective ....................................................................................................... 33
Audit scope ............................................................................................................ 34
Audit questions ...................................................................................................... 35
Assessment criteria ............................................................................................... 37
Methodology and sources of data ......................................................................... 39
Expected findings, conclusions, effects and risks ................................................. 40
Audit design ........................................................................................................... 40
Resource plan ........................................................................................................ 42

3.5.10

4.

EXECUTION ....................................................................................................... 45
4.1
4.2

Introduction of the main study to the auditee ............................................... 46

Data-collection methods .............................................................................. 47

4.2.1
4.2.2
4.2.3

4.3
4.4
4.5
4.6
4.7
4.8
4.9
4.10
4.11
4.12
4.13

Data collected through documents ........................................................................ 48

Testimonial collected data, asking people ............................................................. 49
Physically collected data ....................................................................................... 52

Surveys and case studies ............................................................................ 53

Prepare for analysis while collecting data .................................................... 54
Compilation and analysis of data ................................................................. 55
Quantitative analysis ................................................................................... 55
Qualitative analysis ..................................................................................... 57
Mixture of quantitative and qualitative data analysis .................................... 57
Audit evidence ............................................................................................. 58
Audit findings ............................................................................................... 59
Causes and effects ...................................................................................... 60
Conclusions ................................................................................................. 61
Recommendations ...................................................................................... 61

4.13.1
4.13.2

4.14
4.15
4.16
4.17

4.18

Synopsis ............................................................................................................ 66
Addressing the readers need ........................................................................... 67
Drafting .............................................................................................................. 67
Revising ............................................................................................................. 69

Clearance of the report ............................................................................... 70

4.18.1
4.18.2
4.18.3
4.18.4

Review of the draft report .................................................................................. 70

Sending a draft report to the auditee ................................................................. 71
Internal finalisation ............................................................................................ 71
Exit meeting ....................................................................................................... 71

REPORTING ....................................................................................................... 72
5.1

6.

Formulating recommendations .......................................................................... 61

Characteristics of useful recommendations ...................................................... 62

Drafting the report ....................................................................................... 63

Reporting standards .................................................................................... 63
Format of the report ..................................................................................... 64
The writing process ..................................................................................... 66

4.17.1
4.17.2
4.17.3
4.17.4

5.

Contacts with the auditee .................................................................................. 43

Presentation of the report ............................................................................ 72

FOLLOW-UP ...................................................................................................... 74
6.1
6.2
6.2.1

6.3
6.3.1
6.3.2

Follow-up by the auditors of their own work ................................................. 74

Follow-up of the recommendations .............................................................. 75
Following up the impact of recommendations ....................................................... 75

Follow-up audit ............................................................................................ 76

Planning follow-up audits ....................................................................................... 76
Reporting on follow-up audits ................................................................................ 76

Annexure 1: RESPONSIBILITIES WITHIN THE AUDIT PROCESS .......................... 77

Annexure 2 INTOSAI AUDITING STANDARDS ........................................................ 78
Annexure 3 Index and abbreviations ....................................................................... 82
Index of words......................................................................................................... 82
List of figures .......................................................................................................... 83
List of tables ............................................................................................................ 84
Abbreviations .......................................................................................................... 84

1. INTRODUCTION
1.1 Purpose of the performance audit template manual
The Auditing Standards of the International Organization of Supreme Audit Institutions
(INTOSAI) stipulate that each supreme audit institution (SAI) should adopt policies and
procedures to prepare manuals and other written guidance and instructions concerning
the conduct of audits. 3 This includes a manual for performance audit (see annexure 2).
The purpose of this performance audit template manual is to:
support the individual SAI to develop methods and practices on how to conduct
performance audits.
enhance the capability and skill level of managers and auditing staff to conduct
performance audits of an acceptable standard and uniform quality.
provide practical guidance for the planning, execution, reporting and follow-up of
audits.
form the basis for decision-making within the performance audit process.
enhance standardisation in performance audit practice.
be used as a training tool in the region
promote the development of performance audit within the region

1.2 Structure of the manual

This manual generally follows the stages in the performance audit process, namely
planning, conducting the main study (execution), reporting and follow-up. However it
also contains a chapter that places performance auditing within a context. The manual
consists of six chapters:
Chapter 1 is an introduction to the purpose and context of the template manual.
Chapter 2 looks at the context within which performance auditing takes place.
Chapter 3 provides guidance for the planning of performance audits.
Chapter 4 describes the execution of performance audits, including the collection,
documentation and analysis of audit evidence as well as the development of audit
findings and provides guidance on drafting a report.
Chapter 5 describes the reporting phase, including the presentation procedures of
performance audit reports.
Chapter 6 deals with the follow-up process for performance audits.
The manual includes three annexure:
Annexure 1: Responsibilities within the audit process.
Annexure 2: Information about INTOSAI Auditing Standards.
Annexure 3: Indexes and abbreviations.
The sections of the template manual that are recommended to be customised appear
against a grey and examples against a yellow background in the template manual.

INTOSAI Auditing Standards, as approved by the 35th meeting of the Governing Board in
October 1991

2. THE CONTEXT OF PERFORMANCE AUDITING

2.1 Why performance auditing
In every society resources are scarce and should be used in the best possible way. The
main reason for carrying out performance audits is that the audits can lead to better use
of resources by public bodies and provide support to democratic government.
Performance audits help ministries, departments and agencies to improve their
operations. Performance audits can be seen as investments that should lead to better
functioning public entities. Performance audits identify important problems, analyse the
causes and effects and present recommendations for using resources better.
Performance audits are not restricted by departmental boundaries. By using
experiences gained from studies of other organisations performance audits can bring
new insights into problems faced by audited entities. Performance audit assists
government in the decision-making process.
Performance audit deals with deficiencies in economy, efficiency and effectiveness.
Performance audits do not target individual corrupt or fraudulent activities. However a
corrupt civil servant needs a public sector that does not function well with complicated
decision processes and/or long waiting times. If corruption is common it means the
citizens and businesses are used to a public sector that does not deliver as it should.
Corruption can be one reason for deficiencies in economy, efficiency and effectiveness.
Where the government functions efficiently and effectively there is less risk of
corruption. Performance auditors present findings and make recommendations that can
create a more efficient and effective public sector and in that sense performance audits
also address the root cause of corruption.
According to economic theory, in a perfect market the customer chooses the product
from the company that gives them the best deal. A successful company will make a
profit and stay in the market. The companies that do not perform will not survive in the
market. In a perfect market situation companies need to improve their production,
product or their price all the time to stay alive. In the public sector there are no market
mechanisms and the public entity often has a monopoly on the service it delivers.
Therefore there is a need for performance auditing as a substitute for market
mechanisms.
Public accountability means that those in charge of government activities are
responsible for their decisions and actions. Performance auditing has a role to play in
making this visible and transparent, by providing parliamentarians, the media and
citizens with professional and independent assessments of government activities.

2.2 The 3Es: Economy, efficiency and effectiveness

INTOSAI defines the three Es as follows:4
Economy
Minimising the cost of resources used for an activity, with regard to
the appropriate quality
Efficiency
The relationship between outputs, in terms of goods, services and
other results, and the resources used to produce them
Effectiveness The extent to which objectives are achieved and the relationship
between the intended impact and the actual impact of an activity
4

ISSAI 3000, Implementation guidelines for performance auditing, page 15 20.

To understand these concepts further the input-output model can be used.5 It helps to
understand the three Es, describes processes and organises thoughts and
observations. The model is based on the assumption that it is possible to arrange
concepts in chains and to identify how they influence each other as causes and effects.
Another assumption is that organisations have established goals that they want to
achieve or that such goals can be deduced. The conversion of input into output takes
place within the organisation (in terms of the model the organisation is sometimes
called the black box) through different activities. In the input-output model, different
types of inputs are transformed by activities into different types of outputs and products,
which will have an effect on the achievement of the goals.
How the three Es fit into the input-output model is illustrated in the following figure:
Figure 1: The input-output model

Goals

Input

Activities at the auditee

Output

Effects

Feedback

Economy

Efficiency

Effectiveness

Economy is concerned with the inputs. Economy deals with the resources procured or
the staff recruited and the cost involved getting the input. Economy is about keeping the
cost of input and the resources that the entity utilises low without compromising on the
quality.
Efficiency is about how these inputs are transformed into outputs. Efficiency deals with
what is produced with the available resources. When assessing efficiency the
usefulness of the inputs for producing the outputs has to be taken into account.
Efficiency is about getting the most or best output (production) from available
resources.

The input-output model can be seen in the concept of system theory where:

a system has one or several goals or a purpose;

a system consists of interacting elements or parts;

resources and information flow among the different elements that compose the system

a system is situated within an environment;

resources and information flow from and to the surrounding environment via boundaries;
Elements or parts of a system can also be defined as systems and are composed of other parts,
just as the system itself is generally a part of a larger system.

Effectiveness relates to the effects generated by the outputs. Effectiveness is about

achieving the stipulated aims or objectives. When assessing the effects it is necessary
to take into consideration the cost of the inputs used to generate the effects.
Effectiveness deals with achieving the goals at the lowest possible cost.6
Figure 2: The input-output model, an example of the performance audit process within a SAI
7

To exemplify how the input-output model can be used practically as a description tool of
the reality we can look at the performance audit process at a SAI.

Input
The SAI
employs
auditors

Economy
Are the
auditors
qualified?
Do auditors
have the right
equipment?

Activities at the SAI

Output
Performance
audit reports

Efficiency
How many
performance audit
reports are produced
within a year?
Do the SAI produce
more reports
compared to
previous years?
What is the ratio
between reports/
auditors?

Effects

Effectiveness
Are the SAIs PA
goals achieved
at a reasonable
cost?
Are the
recommendations
implemented?
Is the
governement
functioning
better?

In the example we can see that different types of questions can be relevant, depending
on which of the three Es you focus on.
The example helps to clarify the difference between economy, efficiency and
effectiveness. A SAI may well be very efficient, for example, while at the same time not
being very effective. A large number of reports may be produced, giving high efficiency.
However if the recommendations are not implemented and the government does not
function better the SAI will not achieve its goal of a high effectiveness,

2.3 Definition of performance auditing

Economy, efficiency and effectiveness constitute the three Es that form the basis of
performance auditing. A performance audit is an audit of the economy, efficiency or
effectiveness with which the audited entity/entities uses its resources to achieve its
6

Please note that the definition of effectiveness in the AFROSAI-E manual is somewhat different
from the one used by INTOSAI. The INTOSAI definition of effectiveness relates to goal
fulfilment, but does not take resources into account. The advantage of adding the cost of the
input to the definition is that it also focuses on the use of resources. Consequently, an effective
organisation gives value for money while a similar judgement according to the INTOSAI requires
additional consideration to take costs into account.
7
Examples are marked with a yellow colour in the template manual.

goals. The three Es are useful concepts that guide the auditor when formulating
problems and outlining the general direction of the audit.
Basic questions in performance auditing are the following:
Are entity/entities doing the right things?
If so, are entity/entities doing things in the right way; and
If not, what are the causes?
According to INTOSAI8, performance auditing embraces the following:
Audit of the economy of administrative activities in accordance with sound
administrative principles and practices, and management policies.
Audit of the efficiency of the utilisation of human, financial and other resources,
including an examination of information systems, performance measures and
monitoring arrangements, and procedures followed by audited entities for remedying
identified deficiencies.
Audit of the effectiveness of performance in relation to the achievement of the
objectives of the audited entity as well as audit of the actual impact of activities
compared with the intended impact.
For the auditor, identifying and quantifying effectiveness can be difficult. The auditor
may find that:
objectives may be non-existent
objectives may be vague, numerous and in conflict with each other
objectives may be irrelevant to the tasks in question
objectives may change over time
Performance audits can be conducted on or in a single entity.9 It may encompass the
whole or part of the operations of an entity. Performance audits can also examine
particular issues or functions, for example procurement or human resource
management, across a number of entities.
Political goals and public commitments are often the starting points for performance
audits. Performance audits do not question these political decisions but can point out
unforeseen consequences regarding the implementation of such decisions.
Performance audits can provide a view on the means of achieving/fulfilling political
goals. Furthermore, performance audits can draw Parliaments attention to conflicting
political goals. Performance audits are aimed at increasing the economy, efficiency and
effectiveness of public entities and programmes.

2.4 The special features of performance auditing

Performance auditing is about analysing and assessing the performance of government
programmes or public services. It is an information-based activity. Unlike regularity
auditing, it focuses on activities rather than the accounts and flow of money, and unlike
to compliance auditing10, it relates mainly to the intention behind government
interventions and to the concepts of economy, efficiency and effectiveness.
Each performance audit has a different focus and different assessment criteria
Within its legal mandate, performance audit is free to examine all government activities
from different perspectives. As stated in the INTOSAI Auditing Standards, performance
audit is not overly subject to specific requirements and expectations. Performance audit
8

ISSAI 3000, Implementation guidelines for performance auditing, page 11.

A public sector organisation of any kind.
10
Audits with the aim of scrutinize the audited entitys compliance with established legislation
and regulations.
9

10

is flexible in its choice of, for example, audit objects and methods. Performance audit is
not based on formalised opinions; it is an independent examination on a non-recurring
basis. Performance auditing deals with many different types of problems and is by
nature wide-ranging and open to judgements and interpretations. Each performance
audit has a different focus and different assessment criteria and performance auditors
must therefore have at their disposal a wide selection of investigative and evaluative
methods, and operate from a knowledge base different from that of regularity auditing
which tends to apply relatively fixed standards.
Table 1 Some differences between performance auditing and regularity auditing

Performance auditing

Regularity auditing

Purpose

Assess whether the performance of

the auditee meets one or several of
the three Es, (economy, efficiency
and effectiveness)

Focus on

Methods

The organisation, programme or

system, its activities, output and
effects
Economics, political science,
sociology, etc.
Varies from project to project

Aims to give an opinion on

whether financial statements
are complete and accurate,
and on whether transactions
are legal and regular (accord
with relevant laws and
regulations).
The financial statements and
management systems

Assessment
criteria

Unique criteria for the individual

audit

Reports

Varying content
Published on ad hoc basis

Academic base

Accounting
Standardised
Standardised criteria suitable
for the relevant regularity
audits
Standardised content
Published on regular basis

The character of the two types of audit must not be taken as an argument for
undermining collaboration between performance and regularity audit. Performance
audit and regularity audit have a number of similarities. Regularity and performance
auditors should have a mandate to carry out the audit and the same obligations to the
auditee. The two types of audits both assess the performance of the auditee. The
performance auditors focus on economy, efficiency or effectiveness while the regularity
auditors focus on the accounts.

2.5 Stages in the performance audit process

The performance audit process covers several phases. The process comprises
planning, carrying out of the main study (execution), reporting and follow-up. Each one
of these phases can be divided into different stages. An overview of the performance
audit process is described in the figure below it is also developed in annexure 1
together with the responsibilities within the different phases.

11

Figure 3: Stages in the performance audit process

PLANNING

Selection
and
planning of
audit topics

Completing
a pre-study
and work
plan

PRE-STUDY

EXECUTION

Collection
and
analysis
of audit
evidence

Drafting
of the
report

REPORTING

Clearance
of the
report

Presentation of the
report

FOLLOW-UP

Follow-up

MAIN STUDY

The purpose of the planning phase is to select suitable areas for audit, identify
possible audit problems, and prepare for the execution of a main study. Before planning
for the individual audits can begin, it is necessary for management to set strategic
priorities that can guide the auditors through the selection process. The first stage is
usually area watching in accordance with managements strategic priorities (see
chapter 3.2). It can be done in a more structured manner in the form of a general
survey, if required for the analysis of potential themes and topics. Once an audit topic
has been selected for performance audit, a pre-study may be undertaken to gather
information in order to decide whether or not a main study should be carried out. During
the pre-study the auditors increase their knowledge of the audit topic and consider
different possible approaches and methods for conducting a main study.
Conducting the main study (the execution phase) involves the collection,
documentation and analysis of audit findings, culminating in the drafting of the audit
report. The report-drafting stage is a continuous analytical process of formulating,
testing and assessing audit findings, conclusions and recommendations. Issues such
as the expected impact and value of the audit should be considered throughout the
audit. The emphasis should be on the production of a final report to be considered by
Parliament and presented to government and/or the audited entities.
The reporting phase involves the clearance of the report through reviews, quality
controls and exit meetings with the auditee, and submission of the report to Parliament,
government and/or the audited entities.
The follow-up phase contains processes that identify and document the audit impact
and the progress the auditee has made in implementing audit recommendations, such
processes providing feedback to the SAI and to Parliament and government.
Another way to describe the stages in a performance audit is to describe the process as
a funnel.

12

Figure 4: The audit funnel

Problem

Area
watching/
General
survey

Audit
problem
Audit
design

Prestudy

Execution
Main
study
Reporting

Follow-up

The "audit funnel" in the figure above illustrates how a performance audit successively
becomes more narrow and focused during the planning process. Indications on a
problem within society initiate a pre-study. When the audit problem has been chosen,
the next step is to design the main-study. This is done during the pre-study and involves
specifying the objective and scope of the audit, formulating audit questions, establishing
audit criteria, developing methods for data collection, outlining possible results of the
audit and administrative planning. During the main-study the plan that was compiled
during the pre-study should be executed and published in the form of a report. The
follow-up is done after the audit report has been published.

2.6 The mandate of the Auditor-General

According to INTOSAI the basic audit powers of a SAI should be embodied in the
Constitution and details entrenched in legislation. 11
Performance audits should be conducted within the mandate of an independent audit
institution whose authority is defined by relevant acts. The mandate ordinarily specifies
the minimum audit and reporting requirements, stipulates what is required of the
11

The Lima Declaration of Guidelines on Auditing, October 1977 at the IX INCOSAI in

Lima (Peru), ISSAI 1.

13

auditor-general, and provides the Auditor-General with authority to carry out the work
and report the results. The mandate should cover the whole state budget, including all
relevant executive bodies and all corresponding government programmes or public
services.
The SAI should not be subject to direction from anyone in relation to whether or not a
particular audit is to be conducted or the timing of that audit. The SAI shall have
complete discretion regarding the audits to be undertaken.
The actual terms of the SAIs audit powers will depend on the conditions and
requirements of each country. Therefore, the scope of the audit mandate and
international standards will determine the scope of the standards to be applied by the
SAI.

2.7 The performance auditor

It is of importance that the SAI is looked upon with trust, confidence and credibility.
Performance auditors should be aware of and adhere to the Basic Principles stated in
INTOSAIs Auditing Standards and Code of Ethics. The standards and codes should
guide the daily work of the auditors. The INTOSAI Auditing Standards as well as the
INTOSAI Code of Ethics are presented in annexure 2.
INTOSAIs General Standards in Government Auditing stipulate that all auditors should
possess adequate professional proficiency to perform their tasks. The SAI should
recruit personnel with suitable qualifications and adopt policies and procedures to
develop and train SAI employees.
The ability to recruit the right staff is a decisive factor in performance auditing. To
become a performance auditor, a performance audit team leader or a performance
audit manager, certain distinctive qualifications have to be met. Personal qualities, such
as an analytical ability, creativity, receptiveness, social skills, integrity, judgement,
endurance as well as good oral and writing skills are vital. It is recommended that the
performance auditors have a university degree. Performance auditors can have
different backgrounds and skills compared to staff doing regularity auditing. A suitable
academic background could include political science, economics, social science, law,
statistics or accounting, etc.
For performance auditors to be able to develop their skills and achieve a good quality of
results in their audits, an environment that is stimulating and conducive to quality
improvements needs to be created. The composition of the audit team should be such
that proper supervision is secured, experiences are shared, and the best possible use
is made of the skills of different auditors.

2.8 Organisation of performance auditing

The SAI should organise its performance audit separately from regularity audit. The SAI
needs to establish a specialised performance audit unit big enough to sustain
performance audit activities on a continuous basis and to develop a performance audit
culture. In the AFROSAI-E institutional capacity building framework, level 3, the critical
mass of performance auditors necessary for coping with staff turnover, while
maintaining a constant level of quality production, is at least 10 performance auditors.
They shall be able to produce at least three performance audit reports per year. A
condition for a SAI to reach level 5 of the AFROSAI-E institutional capacity building
framework is that at least 50% of the auditors within a SAI are performance auditors.

14

2.9 Key players within the audit office

The main person within the audit office is the Auditor-General who makes the final
decisions on what to audit and approves the final report for publication.
The performance auditors are also of course important persons within the audit office. It
is the responsibility of the performance audit staff to administer the audits in a
satisfactory manner. They must be able to plan and carry out the performance audit
project in an economic, efficient and effective way, supported by convincing evidence
and ensuring a high quality of work.
Other key players within the audit office are managers who influence the decisions on
what to audit and the final report for publication. It is also the responsibility of managers
to ensure that workable systems for management reporting, review and quality have
been put in place.
Staff responsible for training play an important role, as it will not be possible to meet the
quality requirements specified in the INTOSAI auditing standards unless the SAI has a
programme to ensure that their staff maintain professional proficiency through
continuous training. Staff providing administrative and IT support are also important for
the success of performance auditing. The same applies to the members of a quality
assurance unit and an advisory committee if the SAI has created such functions.
EXAMPLE: Key players in performance auditing within a SAI (the titles and
organisations can vary between different SAIs)
The Auditor-General (AG) approves annual plans and individual audit plans. He/she
scrutinise draft reports, decides whether or not to publish the final reports and also
signs the final audit reports. The AG further interacts with and provides the public
accounts committee with up-to-date information regarding audit reports and related
matters.
The Deputy Auditor-General (DAG) is a link between the AG and the organisation.
He/she prepares annual plans, etc.
The senior manager compiles departmental annual plans based on the information
generated and audit plans developed by PA departments. The senior manager has
overall responsibility for the management of the entire audit cycle. She/he organises
and assigns work to PA departments. Some of his/her responsibilities are as follows:
- Direct, supervise and monitor the activities of the PA departments
- Training.
The director of a PA department is the operational manager and supervises the
audit teams assigned to him/her during the course of the audit. The operational
manager shall continuously follow the fieldwork of the audits and to point out
deficiencies to the team leaders and, if necessary, to the senior manager. Some of the
responsibilities of the operational manager are as follows:
- Coordinate performance audit plans
- Propose audits
- Report audit findings, conclusions and recommendations
- Review the project files during the audit
- Train and coach performance auditors
- Work with the subordinates in the field to do on the job training
- take initiative to follow-up audits

15

The team leader is responsible for executing the audit work as well as allocating tasks
to her/his team members. In performance auditing, it is not fruitful to use a far-reaching
division of work. All members of the team should have insight into the overall
development of the project. Nevertheless, one person - usually the most experienced
or highest-ranking auditor - will be appointed team leader. The team leader ensures
the high quality of the output and its timely production. She/he ensures that all data
collected is thoroughly analysed and that the findings, the conclusions and
recommendations arrived at are consistent with the data collected. The team leader,
together with the team, prepares and submits a draft report with the open file for
discussion with the operational manager. Some of the responsibilities of the team
leader are as follows:
- Produce performance audit plans
- Allocate tasks to the team members
- Supervise the team during execution of performance audits
- Report audit findings, conclusions and recommendations
- Train the team of performance auditors
- Maintain a good team spirit within the team
The team members participate in the actual audit of the entity under guidance of the
team leader. Some of the responsibilities of team members are as follows:
- Collect data
- Document the collected data
- Analyse data
- File working papers
- Participate in in-house training

2.10 Monitoring and supervision

It is important that the role of different levels of management is clear. A work plan
should be decided on at an appropriate management level as defined by the SAI. The
decision should encompass milestones and the monitoring process for the project.
According to the INTOSAI Auditing Standards, supervision should ensure that:
all members of the audit team have a clear and consistent understanding of the work
plan and how it should be implemented
the audit is conducted in accordance with the standards and practices of the SAI
the work plan and action steps specified in that plan are followed, unless a variation is
authorised by the appropriate manager
the audit team achieves the stated audit objectives.
Supervision, whether within the audit team or at higher levels, is only one aspect of the
management role. To achieve good results, it is imperative that management also plays
a proactive, supportive and visionary role in the audit work.

2.11 Quality control

Quality control shall be built into the whole audit process, from planning and selection of
audit problems to follow-up of completed audits.12 The internal quality control in
performance audits is normally focused on oral presentations and written memos from
12

ISSAI 40 establishes a general framework for quality control (see annexure 2). See
also AFROSAI-E Quality Assurance Handbook who provides guidance to quality
reviewers working with quality assurance at the SAIs.

16

the project group, such as pre-study memos and draft reports. Oral presentations shall
be done on dates specified in a work plan (see chapter 3.5) and also on the audit team
initiative when the team encounter something that have influence on the project or the
work plan,

Quality control activities can be in the form of:

review of the planning document by the team and managers.
review by other team members and by the team leader, in order to improve texts.
Team members are expected to review and comment on the texts written by other
team members,
seminars where colleagues are invited at various stages of the audit, for example
before a draft pre-study memorandum and work plan is submitted to management.
The draft report can be presented during a seminar with the audit team, the manager
of a performance audit unit and other performance audit staff. The draft should be
made available to all participants before the seminar and they should all be invited to
participate in the discussion.
regular meetings with the senior manager for monitoring the progress of the audit.
review by the senior manager: working papers could be reviewed and the draft report
should be reviewed by the senior manager who can also ask independent experts to
review the draft.
regular information to top management.
informal discussions with the auditee. During the main study a contact person can be
informed about the progress of the audit. The team can also use informal contacts to
discuss findings, criteria, conditions, causes and effects, conclusions and
recommendations with representatives of the auditee preferably including the
contact person. In this way, the team can verify their analysis and check on
misunderstandings, conflicting evidence and the need for additional information. It is
also an opportunity to identify possible areas of different opinions between the team
and the auditee.
peer review of the draft audit report. Other performance auditors besides the team
can be appointed to review the draft report.

2.12 Documentation
INTOSAI Auditing Standards state that auditors should adequately document the audit
evidence in working papers, including the basis and extent of the planning, the work
performed and the findings of the audit.13
2.12.1 Documentation and ethics
Before data is collected from individual sources, in the form of for example an interview
or a questionnaire, the auditor should inform the person providing information how that
information is going to be used and whether the source of information is going to be
revealed.
During data collection the auditor may obtain sensitive information for example the
interviewees opinion on management of the auditee. For many people revealing that
they are the source of that information could be disastrous. Consequently, the auditors
should, if possible, guarantee anonymity and not divulge an individuals opinions. In
some situations it might not be possible to use data as audit evidence where it is
impossible to maintain confidentiality of the source. If the auditors disclose the name of
a person as the source of information when they had promised anonymity it will
negatively influence the reputation of the SAI. In such situations the auditor must
13

INTOSAI, Code of Ethics and Auditing Standards, page 57.

17

balance the value for the audit if the source of information is identified against the
damage it may cause the individual by doing so.
The management of the auditee represents the auditee and it is often not necessary to
grant them the same type of anonymity.
2.12.2 Guidelines for the preparation of working papers
Working papers are all relevant documents collected and generated during a
performance audit. Working papers serve as a connecting link between the findings
during the fieldwork and the audit report.14 Working papers also include documents
recording the audit planning and the nature, timing and extent of the audit procedures.
The main working papers in the clearance stage are the various drafts of the audit
report and the managers and the auditees review notes.
Working papers need to be of sufficient quality. They should be:
Clear: Working papers must be clear and understandable. The information they
reveal should be complete yet concise. Anyone using the working papers should be
able to determine their purpose, the nature and scope of work, and the audit teams
observations. Conciseness is important, but clarity and completeness should not be
sacrificed.
Neat: Working papers must be easily readable.
Relevant: The information contained in working papers should be important and
useful with reference to the objectives established for the audit.
Working papers should:
assist in the planning and performance of the audit
record evidence resulting from audit work performed to support the audit conclusions
and recommendations.
facilitate effective management of the audit
assist in the supervision and review of audit work.
The SAI should adopt appropriate procedures to maintain the confidentiality and safe
custody of the working papers.
Some basic questions regarding documents and interview notes that can help you
classify and file your data are as follows:
Documents
The source of the documents (records or books, data bases, particular government
offices etc.)?
The status of the document?
What does the document consist of?
Is the document limited to a particular geographical area or does it cover a particular
period?
Where, or from whom, can you obtain further information?
Interview notes
Who is the interviewee?
Who conducted the interview?
When was the interview conducted?
How did you get hold of the interviewee (address, telephone number, etc.)?
Were any promises made to the interviewee about checking your interview notes,
confidentiality, etc.?
14

According to ISSAI 300 Field Standards in Government auditing, paragraph 5.5 Auditors
should adequately document the audit evidence in working papers, including the basis and
extent of the planning, work performed and the findings of the audit.

18

2.12.3 Audit files

The data collection is a lengthy process, during which the team will interview a large
number of people, read and extract information from numerous documents, make
several observations or inspections etc. The auditors cannot store all the information in
their heads. There may also be an external demand that the SAI should be able to state
the source of their findings. This will become virtually impossible if the documentation is
not stored correctly. The working papers need to be filed properly.
The audit file should contain all evidence accumulated in support of the findings,
conclusions and recommendations in the report. The audit file should enable an
experienced auditor with no previous connection with the audit to ascertain what work
had been performed to provide proper support for findings, conclusions and
recommendations.
Working paper should normally be filed in an open and then after finalisation of the
audit in a closed file.
Open files
An open file should be opened when the decision to start a pre-study or a follow up
audit is taken. Working papers should be added continuously during the audit as they
are created. The following are examples of working papers to be included in the open
file:
Interview notes
Documents produced by the auditee like budgets, plans, etc.
Memos from discussions with representatives of the auditee and other officials
Memos comprising analyses of various documents
Analyses in support of the audit work
Correspondence/communication with managers
Work plan
Materials and papers accumulated in report preparations, including drafts
Reviews of management controls.
Table 2: Index for an open file, example

EXAMPLE: Index for an open file

A Planning
A1 Project
approval
A2 Project
plan
A3 Work plan

B Execution
B1 Minutes of the
introductory meeting
B2 Extracts from
relevant
documentation
B3 Questionnaires

B4 Interview guides
B5 Interview notes
and summaries

B6 Analytical reviews
19

C Reporting
C1 First draft
report
C2 Comments
from staff
C3 Comments
from
management
C4 Second draft
report
C5 Written
comments from
auditee, exit
meeting
C6 Final draft

D Finalisation
D1 Final report
D2 Letter of
submission to
auditee

Closed file
When the audit report has been completed, the auditors should keep the relevant
working papers from the open file and then close the file. Working papers of less
importance that do not substantiate the findings should be discarded.
There are several reasons for using a closed file. If the audit report is questioned by
someone, the SAI will be able to check the sources of the findings. This is particularly
useful if the responsible auditors and managers have left the audit office. The closed file
will also be a source of knowledge when planning future work. The closed file also
needs to be stored for a period sufficient to meet legal and professional requirements of
record retention.

2.13 Internal communication

Depending on the mandate and organisational set-up, the roles and responsibilities of
the respective key players will vary from one SAI to another. Each SAI needs to define
the roles and responsibilities within its organisation and establish a format for
communication between its staff and managers.
EXAMPLE: Format for internal organisation of communication regarding performance
audits
Top management meetings
Frequency: Monthly
Participating: Auditor-General, deputy Auditor-General and senior manager or the
director of the PA department if there is just one PA department and no need to
establish a post as senior manager for performance audits.
Chairing: Auditor-General
Contents: Short-, medium- and long-term issues affecting audit operations, e.g.
staffing, delegation of assignment, fixing deadlines for completing mandatory
assignments, delegations for training, delegations for international conferences and
seminars, budgeting and control of expenditures.
Departmental meetings
Frequency: Monthly
Participating: The director of the PA department and members of the department
Chairing: The director of the PA department
Contents: Audit teams report on progress and obstacles in audits, other departmental
issues
Further reporting: The director of the PA departments reports back to the Senior
manager or directly to the top management if there is one PA department
Audit progress meetings
Frequency: Monthly and sometimes also weekly. Can be done for individual teams or
during a common meeting and then be part of a departmental meeting.
Participating: The director of the PA department and the audit team(s)
Contents: Team leaders report about the teams progress compared to the work plan
and budgeted output compared to actual output and remarks on the differences
Further reporting: The director of the PA department reports back to senior manager.
It may also be necessary to inform or request information from regularity auditors.

20

2.14 External communication

Each SAI needs to establish and implement its own strategy and policy for external
communication with regard to performance auditing. This strategy should be in line with
the overall strategic plan and give guidance on how the SAI should communicate with
stakeholders and others in different situations.
The main external stakeholders are Parliament, parliamentary oversight committees
like the public accounts committee etc., audited entities and ultimately the public. The
media are also important, because they serve as a link between the SAI and the public.
The best marketing tool for performance audit is good reports.
A prerequisite for good external contacts is that the stakeholders have a basic
understanding of the role and purpose of performance auditing. Staff and management
of the SAI should therefore actively inform the stakeholders about performance
auditing. This could be done in several ways, through meetings or written material; for
example leaflets or brochures explaining the role and purpose of performance audits
which can be distributed to audited bodies and key stakeholders.
Each SAI should ensure that its audit staff are aware of the strategies for external
communication and are able to interact in a proper way with the auditees and with any
other party that provides information essential for the performance audit project. It is
important to maintain an open dialogue with the auditee during the audit. Before any
further contact is made with the auditee in the main study, a formal introduction should
take place. This could be in the form of a letter signed by the Auditor-General an
engagement letter and an introductory meeting an engagement meeting. Regular
contact should be maintained throughout the audit, preferably through a contact person
appointed by the audited entity. Before finalisation of the audit, a draft report should be
sent to the auditee and discussed at an exit meeting. There should also be an
established practice (in line with the communication policy) regulating to whom the
reports should be routinely delivered.

21

3. PLANNING
3.1 Planning and selection of audit topics
The purpose of the planning phase is to select suitable areas for audit, identify audit
problems, prioritise and select performance audits and prepare for the execution of a
main study. The figure below gives an overview of the responsibilities for the audit team
and the management during the planning phase. An overview of the responsibilities
within the whole audit process is presented in appendix 1.
Figure 5: Planning and selection of audit topics

Process

Planning and
selection of
audit topics

Pre-study and
work plan

Audit team

- Carry out area watching and general surveys

- Select and present possible audit topics for pre-studies

Management

- Carry out strategic planning and annual planning

- Launch pre-study

Planning is aimed at making the audit work more effective. During the planning process
decisions are made on the nature, extent and timing of the performance audit work to
be undertaken. The planning of performance audits should be based on strategic
choices. It is helpful to adopt a structured approach to performance audit, whilst still
allowing flexibility in the timing and priority of particular audits.
The planning and selection of audit topics take place at several levels:
The overall level (strategic planning and annual planning)
The area level (area watching and general surveys, and later pre-studies)
The scope and methods of the planning for a specific SAI depend on characteristics
such as the size of the SAI and the performance audit unit. Small performance audit
units can use more simplified methods for planning.

3.2 Planning at the overall level

The overall planning deals with strategic and annual planning.
3.2.1 Strategic plan
Strategic planning is a process of defining goals, setting objectives and examining the
current situation, to define future developments and strategies to reach the goals. The
SAIs overall strategies are developed on the basis of the SAIs mandate, national goals
and issues that may have been raised in previous performance or regularity audit
reports.

22

The SAIs overall strategic plan should set out its long term vision and mission as to
what the SAI wants to achieve. The strategic plan provides an opportunity to direct the
performance and regularity audit activities to meet the SAIs responsibilities in terms of
audit legislation and international auditing standards. It should aim to improved
utilisation, accountability and transparency with regard to public funds and other
resources. The strategic plan usually covers a period of three to five years, and is
reviewed periodically to reflect changing needs and circumstances.
The strategic plan normally contains a section about performance audits. Strategic
choices for performance audits can be made in respect of broad focus areas, for
example, health care, education or poverty reduction. The criteria to use when selecting
areas for performance audits deal with broad policy issues for the SAI, including the
definition of audit entities and functions, the determination of overall priorities and
resource allocation. The strategic plan should contain policy decisions that serve as a
starting point for the planning of performance audits.
The purpose of a SAIs strategic performance audit plan is to ensure the following:
Resources are allocated to areas with the greatest potential impact.
Audit priorities are adequately determined.
A comprehensive performance audit focus for the annual plans is in place.
3.2.2 Annual planning
An annual performance audit plan for all performance audit activities to be carried out
during the year is prepared based on the strategic plan. The annual performance audit
plan will normally form part of the SAIs annual operational plan which also covers
regularity audit and human resource plan etc.
The plan should comprise information about the expected nature, timing and extent of
the audits. The annual plan incorporates the performance audit departments work
plans for ongoing audits and the new audits that will be started during the year. It
focuses on strategic priorities, significant matters listed in memos from area watching or
general surveys, pre-study reports, and follow-ups. An annual performance audit plan
would normally entail departmental objectives, intended outputs and outcomes, areas to
be audited, time frames, and resource allocation for each audit.
The aim of the annual performance audit plan is to determine the annual programme of
performance audit work and the staff and other resources needed to deliver on the
programme. Adequate planning of the audit work helps to ensure that appropriate
attention is devoted to important areas, that potential problems are identified, and that
the work is completed expeditiously.
Annual planning also assists in the proper assignment of work to team members and
coordination of work performed by different audit teams.
The development of the annual plan is normally the responsibility of the head of the
senior manager or the director of the PA department depending of the organisational
structure.

3.3 Area watching and general surveys

Area watching and general surveys are part of the planning process. Area watching and
general surveys should be conducted in accordance with the priorities laid down in the
strategic and annual performance audit plan. The purpose of area watching and general
surveys is to determine the potential areas from which topics for pre-studies will be
selected.
23

3.3.1 Selecting potential audit topics

During area watching and general surveys information is collected. A number of
potential generic problems would probably have been identified even before the area
watching and general surveys. To make recommendations about the topics that warrant
further examination, the auditors need some basis for evaluating the information that
has been collected. In this situation there are a number of factors that need to be taken
into account to compare and rank potential audit topics.
Some factors to consider could be the following:
Complaints from clients due to long waiting times, poor quality of service, defective
products, etc.
Environmental problems like polluted drinking or ground water,
Changes in legislation or government policies which can influence auditees
performance.
New technologies introduced or other circumstances that have changed.
Unauthorised over expenditure or rising costs resulting in demands for more
resources.
Performance targets are not being met.
Complexity due to scattered activities.
An entity is involved in activities outside its mandate.
Projects are not completed on time.
Complaints from staff or high staff turnover.
Substantial losses because of theft or waste.
Tender procedures are not adhered to.
3.3.2 Area watching
Area watching entails monitoring key issues in the public sector to keep abreast of
developments. Its purpose is to identify possible audit areas for further scrutiny. It is
carried out by reading relevant publications and previous reports relating to
performance and regularity audits; listening to the experience of regularity auditors;
listening to or reading transcripts of parliamentary debates; attending conferences and
seminars; discussions with colleagues, stakeholders and specialists; listening to radio
and television broadcasts; and reading newspapers and journals.
Area watching should be a continuous process that ensures that the SAI is always in
possession of updated information about what happens in society and areas that may
require further examination. Area watching should be an ongoing process during the
year done by performance auditors, and there should be some time allocated for them
doing area watching. At times as indicated by management (for example, in the
beginning of the annual planning process), results from area watching that indicate
potential audit topics should be presented to management for a decision on whether a
general survey or a pre-study should be carried out. The area watching results could be
presented orally or in writing,
3.3.3 General survey
The general survey it not always necessary and should be used when the SAI enters
into a completely new area of which the audit office has little knowledge. It should be
done in the form of a project, with a goal, time frame and a budget.
The purpose of general surveys is to collect and evaluate relevant information in order
to provide an understanding of a particular sector or programme, identify potential audit
areas, and allow the auditor to direct his/her attention to the most important areas and

24

issues. It is a process aimed at gathering information relatively quickly, without detailed

verification of the activities or operations of an organisation.
The general survey goes a bit deeper than area watching and could include studies of
acts and regulations relevant to the auditee. It can also include a literature search,
annual reports from the auditee, consultation of regularity audit files, reports from
parliament and its oversight committees like the public accounts committee and contact
with professional bodies. When necessary, information should be collected directly from
the relevant organisations through interviews and/or written communication.
The results of the general survey may consist of a number of different ideas for audits
and could be kept as a bank of ideas. These ideas can later be used as a starting point
for a pre-study or sometimes even directly for a main study.
While the results from area watching could be presented orally or in writing, the results
of the general survey should be presented in a memorandum. The memorandum
should provide background information about the area and the role of the auditee, such
as legislation, goals, objectives, structure/organisation, outputs, and costs. It should
also contain an analysis of problems encountered and recommendations on the way
forward. The memorandum should be submitted to management for a decision on
whether to conduct a pre-study, go ahead with a main study, or choose other topics.
EXAMPLE: Possible format for a general survey memo
1.
2.
3.
4.
5.
6.

Background
Description of the surveyed area
Risks
Materiality
Conclusion
Recommendation

1. Background
The reasons why the decision to conduct a general survey was made.
2. Description of the area
A description of the sector and the relevant part of the entity under audit, including the
auditees:
- task, objectives and undertakings
- organisational structure
- division of responsibilities within the auditee/s.
3. Risks
An appraisal of the risks due to the auditees shortcomings within the audit area. The
risks should be divided as follows:
- Inherent risk, i.e. factors that are embodied in the nature of the auditee but fall
outside its control, e.g. new legislation, complex area.
- Identified risk, i.e. factors identified by the audit team or others, which were created
by the way the auditee performs its duties, e.g. the auditees practices or internal
organisation.
4. Materiality
An assessment of the extent to which the auditees shortcomings within the audit area
affect the state and the public, namely:
- Materiality by value, i.e. the amount of money spent or the relation between

25

expenditures and revenues.

- Materiality by nature, i.e. how people are affected by the audited area, e.g. peoples
time, health or rights.
5. Conclusion
It should point out the main results off the general survey and identify potential audit
problems.
6. Recommendation
Based on the overall conclusion, the audit team should recommend whether or not to
launch one or more pre-studies.
3.3.4 Presentation of possible audit topics
Area watching results from are normally not documented in a structured format. At
times, as indicated by management in the planning process, such results should be
presented to management for a decision on whether a general survey or a pre-study
should be carried out. If the area watching gives an indication of problems issues that
could concern regularity auditors they should be informed. If the indicators that point out
problems are broad and need to be specified further, the decision would normally be to
conduct a general survey. If, on the other hand, more specific problems have been
identified, the process could continue into a pre-study directly or sometimes even a
main study.
3.3.5 Differences between area watching and a general survey
Some differences between area watching and a general survey are highlighted in the
following figure:
Table 3: Differences between area watching and a general survey

Purpose

Area watching
Keeping updated
information on key issues
to identify possible audit
Continuous
Broad
Various sources

Time frame
Focus
Sources of
information
Documentation Oral or written

26

General survey
Identify possible problem areas

Ad hoc
Specific target
Various sources and information
from and about the auditees
Written

3.4 Pre-study
Figure 6: Pre-study and work plan

Process

Audit team

Management

Planning
and
selection of
audit topics

Pre-study
and work

plan

Collection and
analysis of audit
evidence

- Plan pre-study
- Carry out pre-study
- Write pre-study memo
- Prepare work plan for main study
- Approve pre-study plan
- Launch pre-study and contact auditee
- Supervise/Monitor pre-study work
- Decide on pre-study memo
- Decide on a work plan

A performance audit represents an investment for the SAI in terms of time and money.
The SAI can undertake only a limited number of audits each year. Therefore, all
projects must be carefully selected and designed to bring maximum benefits to the
stakeholders and to the nation at large.
The final part of the selection process is the pre-study. The pre-study is a process
where it is possible to test different ideas, alternative audit problems and methods.
The pre-study takes the assessment made during area watching and general survey
work further and provides sufficient justification to proceed with a main study or,
alternatively, conclude any further work.
If the pre-study results justify proceeding with a main study then it contains information
about the most important economy, efficiency or effectiveness problems for the entity
that is scrutinised as a guideline for the main audit. In addition, it provides background
knowledge and information needed to understand the entity, programme or function.
The pre-study results in a pre-study memo which normally is not intended for
publication. The pre-study is an internal document that is the source for a decision to
continue with a main study or not. If the management decides to start a main study, a
work plan is prepared and decided upon. The work plan contains the audit objective,
scope and criteria, possible data sources that are available as well as an estimate of
resource requirements for specialist skills. If the main study has been discussed and
agreed on as part of the pre-study process, the work plan can be prepared at the same
time as the pre-study memo. This means that the pre-study memo sometimes includes
a work plan for a main study, while the pre-study memo and the work plan are
sometimes two separate documents.

27

The pre-study and work plan lays the foundation for the audit to be conducted. In the
end of the pre-study phase the decision on what to do is taken, in the main study the
plan is executed. If the pre-study and work plan are not of a high standard, there is a
high risk that problems will be experienced later in the audit.
3.4.1 Planning a pre-study
The pre-study plan is a plan for the execution of the pre-study. A pre-study plan should
contain information regarding the approach, timing and resources required for
conducting the proposed pre-study. The following issues should normally be included in
the pre-study plan:
Motivation: The reasons why it was decided to carry out a pre-study.
Objectives and issues to be investigated: Focus on the chosen audit object and
illustrate what the auditors hope to achieve by means of a pre-study.
Collection of information: Mention the type and sources of information that will be
needed in the pre-study and what methods will be used to gather the information.
Sources of information could be interviews, documents and observations.
Organisation and need for resources: Indicate who will be involved in the pre-study
and what role they will play. All other resources, such as money and time spent on the
pre-study, should be indicated.
When the pre-study should be finalised: A pre-study could take from one month to
maximum three months.
Given the nature of performance audits and the particular pre-study it might be difficult
to standardise all working methods or to identify all sources of information in advance.
For members of the audit team the plan serves as a guideline according to which the
pre-study is to be pursued. The team should decide:
what information is needed for the pre-study
how the information will be collected
how the information will be used to identify the audit problems.
3.4.2 Collection of data
After the SAI has made a decision to start a pre-study the auditees top management
should be informed that a pre-study would be carried out and of the purpose of the prestudy before data collection at the auditee starts.
Collecting data takes place during both the pre-study and main study of an audit. The
primary role of data collection in the pre-study is to gather enough information to be
able to assess alternative audit problems. Data can be collected by a number of
methods in the form of documents, testimonial data and physical observations (see
chapter 4). The audit team should devote time to studying relevant documents and
other literature. The audit team should utilise existing reports, evaluations and statistics
in the area and avoid collecting primary data if other sources are already available. The
audit team may interview people with special knowledge of the audit object and people
who are willing to assist in giving information and discussing possible audit problems.
During the pre-study the audit team should also look for information that can be used as
assessment criteria (see chapter 3.5.5) and try to find out what documents, files,
computerised data and statistics are available that can later be used to collect data
during the main study.
The documentation of data is of great importance in auditing. This is true for the prestudy as well as the main study. During the pre-study, it is important to bear in mind that
properly documented data might later be used as audit evidence in the main study.
28

Therefore, all data should be carefully documented and the data collected should be
filed in accordance with the filing format established by the SAI (see chapter 2.12).
3.4.3 Analysis of data and identification of possible audit problems
All data collected should be analysed and converted into meaningful information. The
analysis of data will be further discussed in chapter 4 of this manual.
Meaningful information in the pre-study is data relating to possible audit problems.
Selecting the audit problem is the most important step in the audit because it lays the
foundation for all the activities that take place later during the audit.
The findings of the pre-study should be presented in such a way that related problems
are presented together so that the links between them become clear. The most
important audit problem can be determined by using the SAIs selection criteria in
conjunction with some type of structured technique. The following steps can be applied:
Define the activities that an entity perform to transform input into output (see chapter
2.2).
Define good performance and/or good economy, efficiency and effectiveness.
Use the collected data and analyse whether there are indications of poor
performance.
Identify possible audit problems by grouping the indications of poor performance
according to the three Es.
The black box can also be broken down by studying what activities that take place in
the black box using process studies or flowchart techniques. Brain storming techniques
or the problem-tree analysis technique and/or the input/output model or other
techniques can be used (for example taken from social science). The techniques can
also be combined, for example the brainstorming technique could be used to identify
possible problems and the problem-tree technique could be used to understand the
relationship between those different problems.
Brainstorming
Selection of the audit problem can be done by using the brainstorming technique.
Brainstorming is a group creativity technique designed to generate a large number of
ideas. Firstly, new and innovative ideas are generated; and secondly, the ideas put
forward are critically assessed. The team should not try to be creative and critical at the
same time.
:
Brainstorming entails a session where ideas about the problems and problem
indicators as seen from the data gathered in the pre-study is listed. There are four
basic rules during the brainstorming session; focus on quantity, withhold criticism,
welcome unusual ideas, combine and improve ideas.
Group the problem found during the brainstorming session into possible problem
areas.
Rank the possible problem areas that can be scrutinised.
Identify the key factors most likely to influence economy, efficiency and effectiveness.
Indicate why the factors did not meet the three Es.
Formulate the audit problem
The problem-tree analysis technique
The problem-tree technique is a tool to decide what problems are relevant and how to
relate and structure those problems. The problem-tree analysis is a work tool used to
relate and link different problems to each other hierarchically according to how they
influence each other. The problem tree list the problems that came up during the pre-

29

study and their relationships (which problems are causes and which are
consequences). Any box in the tree can be defined as a problem. Causes of that
problem will then be found as you move downwards in the tree and the consequences
as you move upwards. An example of a problem-tree is shown in the figure below:
Figure 7: Example of a problem-tree
Many students leave
school without
adequate
qualifications

High student
dropout
levels

Long
distances to
schools

High failure
rates

Unaffordable
high school
fees

Poor
teaching
methods

Outdated
curriculum

Shortage of
learning
materials

Shortage of
teachers

Substandard
learning
materials

Low output from

teachers training
colleges

Inadequate
infrastructural
capacity

Inefficient
deployment
of teachers

Inadequate
needs
assessment

Absence of a
monitoring
system

Lack of
coordination by
ministry

High staff
turnover

Inappropriate
transfer
procedures

Poor record
keeping

Before arriving at the final version of the problem tree, the audit team would have
reformulated many problems and eliminated others. Interrelated problems that have the
same causes and consequences would have been merged. In this example, problems
such as inadequate facilities, poor attitudes or poverty have been left out, since they lie
outside what can be audited in this example. Other problems may have been eliminated
simply because the auditors considered them to be too insignificant.
The problem tree technique help the team to identify different audit problems. Different
selection criterias (see chapter 3.4.5) are used to choose an audit problem. If, in the
example above, the audit problem is chosen at a very high level (for example high
failure rates), it will probably be of great interest to the stakeholders, but it might not be
possible to deal comprehensively with the problem since there is a risk that the audit
will either be shallow and trivial or demand unreasonably large resources to cover all
important aspects of the audit problem. If, on the other hand, the audit problem is
chosen at a very low level (for example poor record keeping), it might be possible to
investigate it very well, but it may be of little interest to the stakeholders.
.

30

When in doubt, it is generally better to choose an audit problem at a lower rather than at
higher level. Problems are usually more complex and complicated than initially
envisaged. With good arguments you can explain to stakeholders how a problem at a
lower level relates to problems at higher levels. It is also possible to launch several
audits that deal with related problems at a lower level. The audits can be conducted
parallel or in a sequence. The results of these separate (but coordinated) audits could
then be used in a synthesis report, addressing problems at a higher level.
Organising thoughts in the form of a problem tree takes quite some time, as the tree will
probably have to be revised several times before the team is satisfied with the result.
Staff members from outside the project team could be invited to give feedback on the
problem tree. This will most likely provide new insights and generate a better
understanding of the problems at hand.
3.4.4 Activities in organisations or programmes can be audited
A situation in society that is not satisfactory may be defined as a problem from an
auditees point of view, but from an auditors perspective it is not an audit problem that
can be scrutinised in a performance audit. Activities undertaken by auditee/s where
performance is not satisfactory may be defined as audit problems.
The audit problem should be linked to one or more of the three concepts of economy,
efficiency and effectiveness. A performance audit can focus on a range of different
problems. However, performance audits give priority to problems that are relevant for
groups of people and not only to one individual, and which are of a long-term and
structural nature. Problems may also be looked at from different perspectives. What is
defined as a problem by the headmaster of a primary school may not be regarded as a
problem by the teachers or the pupils, parents, administrators, cleaners or politicians.
Therefore, the auditor should make an independent and impartial assessment,
approaching the problem from different perspectives.
Possible audit problems emanate from the findings of the pre-study. During the prestudy work, the auditor should identify and analyse the characteristics of the problems
found. When several problems have been identified, the team has to limit the scope by
choosing the audit problem judged as the most important to audit. This is done because
it is usually impossible to audit all the problems identified during the pre-study.
Some advice when formulating audit problem:
Concentrate on one problem and do not try to mix several different topics
Formulate the audit problem so it is evident that it is a problem
Do not include explanations or causes of the problem
3.4.5 The selection criteria
Selecting an appropriate audit problem is a prerequisite for a successful performance
audit. The assessment of the selection criteria can be done in several ways. It can be
rhetorical with arguments or by ranking how each possible audit problem scores
according to a scale for each selection criterion. The individual selection criteria can
also be ranked according to a scale. Irrespective of which method is used, it is
important that the motivation for the selection of the audit problem that is chosen is
stated clearly. Below are examples of selection criteria that can be used.
Mandate
- Does the problem fall within the mandate of the SAI?
- It is clear that it is an audit problem and not a political problem?
Materiality

31

- To what extent does the problem affect the citizens, the economy and government
finance?
- Is it a general problem throughout the organisation or confined to a small part of the
organisation?
- Is it a long-term or a short-term problem?
Risk
- To what extent is there a risk that a lot of money may be lost or used wastefully?
- To what extent is there a risk that shortfalls in the audited entitys performance leads
to poor output?
- To what extent is there a risk that important objectives are not being achieved?
Auditability
- Are relevant and accepted audit methods applicable?
- Are sufficient resources available, e.g. budget, transport and human resources?
Potential for change
- What is the possibility that the conclusions and recommendations of the audit will be
accepted?
- Are there political, financial, ethical or other factors that limit the possibilities for
change? Can the problem be solved without an increase in resources?
- Have there been any major changes in responsibilities, laws or organisational set-up
in the area recently which indicates that the audit should not be conducted?
- Are there other bodies already considering how the problem can be solved?
After finalising the problem-tree analysis and considering the selection criteria the
audit team in the example above suggests inefficient deployment of teachers as the
audit problem. This audit problem appears to be neither too broad nor too narrow
and probably meet each selection criteria.

3.4.6 Pre-study memorandum

At the end of a pre-study, the audit team presents a written report (a pre-study
memorandum), summarising the findings of the study and recommendation for further
action. This pre-study memo is normally an internal document. The pre-study memo
should be written as precisely and unambiguously as possible. The information
included and the structure should help the reader to understand why the team
selected the specific audit problem
The following should be covered in a pre-study memorandum:
Background to the pre-study, gives a brief history of the problem and the reasons
or explanation for conducting the pre-study.
Design of the pre-study, gives a brief outline as to how the study was carried out
in terms of scope and what methods and sources of information have been used.
Description of the audit problem environment, which should provide sufficient
background information to understand the context in which the audit problem that
has been reviewed appears in.
The findings from the data collection during the pre-study should be described.
Results of the pre-study in terms of the problems identified and the reasons why
they are indentified to be problems.
Selection analysis between the alternative audit problems
A description of the selected audit problem if the team recommends that a main
study should be carried out.

32

 Recommendation to management on whether or not to carry out a main study.

3.5 Work plan

When top management decides to audit the audit problem suggested by the team, a
work plan for a main study has to be prepared by the team. The work plan should also
be approved by management. The purpose of the work plan is to plan how the audit of
the specific problem will be carried out during the main study. The work plan will help
the team to structure their thoughts and guide them through the audit process. It also
provides a basis for management to monitor the main study throughout the process.
The work plan of the main study is divided into two interrelated parts. The
methodological plan of the main study gives answers to questions on what and how.
The resource plan determines whom, when, and at what costs. Basically, this entails
making an activity plan and a budget for the project.
The work plan should include the following:
Audit problem
Audit objective
Audit scope
Audit questions
Assessment criteria
Methodology and sources of data
Expected findings
Expected result
Risks for carrying out the project and possible ways to deal with these risks
Resource planning, including budget, human resources and activity plan
This work plan should be regarded as a contract between the project team and
management; if management provides the required resources (time and money) then
the team should be able to deliver a report according to the work plan.
We can see the project as the black box in the input output model (see chapter 2.2),
time and money are the inputs and the output will be the report which hopefully will
create changes in the society or effects.
3.5.1 Audit problem
The audit problem as defined in the pre-study memo provides the starting point for the
planning of the main study.
In our example the audit problem was inefficient deployment of teachers
3.5.2 Audit objective
Based on the audit problem that the audit will address the auditor should formulate an
audit objective. The audit objective relates to the reason for conducting the audit. It is
more neutrally formulated than the audit problem. An audit objective is a precise
statement of what the audit intends to accomplish.
In our example if the audit problem is inefficient deployment of teachers the audit
objective would be to the audit will assess whether the government is taking steps to
improve the deployment of teachers. 15
15

Some countries work with a general audit question the audit question for the audit could be Is
the government taking steps to improve the deployment of teachers?

33

3.5.3 Audit scope

When the audit problem and the audit objective have been defined, the audit team
needs to specify the scope of the audit. The audit objective and scope are interrelated
and should be considered together.16 The formulation of the scope can influence the
formulation of the audit objective. The scope deals with how the audit shall be limited to
certain time periods, geographical areas, population groups, seasons, etc. The audit
scope defines the focus and/or boundaries of the audit. The scope is determined by
answering the following questions17:
What? Audit object
What is the audit object, i.e. the activities, programmes or processes that will be
audited?
The audit object is an ongoing or terminated activity carried out by one or several
auditee. It could cover the whole process, from input of resources to output and effects,
or merely a part of that process.
The audit object in our example is the system for deployment of teachers.
Who? Auditee
Who is the auditee, i.e. the ministry/entity responsible for the audit object? Is there more
than one entity that will be scrutinised?
After having defined the audit object, it is necessary to indicate clearly who is
responsible for carrying out the activity under audit.
The auditee in our example is the Ministry of Education, including its regional
directorates and relevant specialised functions.
When? Time coverage
Are there limits on the time frame to be covered, e.g. a specific year or period of time?
The purpose of a performance audit is usually to form an opinion on ongoing activities.
The aim is often to capture the current situation. It is often not possible to study the
whole period that is relevant to the audit problem and the audit object. In those cases
the period to be covered by the audit has to be limited. If the auditor wants to measure
development over time it is often advisable to cover a longer period than just one or two
years. It might, for example, be of interest to illustrate developments in a certain field
over a number of years. The availability of data and changes such as major reforms
launched by government should also be taken into account when defining the time
coverage for the audit.
We assume that the system for deployment of teachers has been in operation for five
years.
The team, in our example, chooses not to cover all events that have taken place during
each one of the five years that the system has been in operation. The audit team
therefore decides to analyse available documents and statistics for the whole period,
but to limit interviews and other data collection to the current situation and to
developments during the last year.
Where? Geographical coverage

16
17

According to ISSAI 3000 Implementation Guidelines for Performance Auditing, page 48.
According to ISSAI 3000 Implementation Guidelines for Performance Auditing, page 50.

34

Are there geographical limits concerning the area to be covered e.g. to be concluded
on, Is the geographic coverage covering the whole country or one or more regions?
It is important to understand the difference between having an audit with a scope of for
example three regions out of ten regions, and an audit where an important method of
data collection is case studies in three regions. In the latter case the intention is to
generalise the results, e.g. to make conclusions that are valid for the whole country. To
do that, a combination of case studies with other sources of information is often used,
for example statistical data from all regions and interviews at the central level.
In our example, the problem is general to the whole country so the geographical
coverage will be the whole country.
(Would there be problems with the deployment of teachers only in the eastern part of
the country the geographic scope could also be more narrow and the conclusions from
the audit could be applicable only to that part of the country.)
When defining the scope of an audit, it can be useful to specify any associated matters
that are not to be audited and the reasons why.
3.5.4 Audit questions
After determining the audit problem the audit objective and the audit scope, the team
should break the audit down into smaller pieces by define specific audit questions. By
answering the questions the team should be able to reach the audit objective within the
audit scope.18
Good audit questions are SMART:
Specific
Measurable
Actionable (relevant due to the input-output model)
Realistic (possible to answer properly)
Time bound
Good audit questions are also open (not assuming any result) and organised logically
(not overlapping).
In formulating the audit questions, performance auditors must rely on the information
collected and their own skills and experience. Techniques similar to the problem-tree
analysis, brainstorming, etc. are sometimes used at this stage. When formulating audit
questions, the auditor is advised to ensure that each issue to be raised in the audit is
handled separately and, where issues are linked, there should be a clear indication of
how they are linked. The audit questions will deal with issues such as what is or why
is it. It will also be an advantage if the questions indicate which entity is responsible.
When the question is answered, they should provide sufficient information about the
audit problem and its causes within the given scope. The number of audit questions that
should be formulated will depend on the scope of the audit problem defined. It is
advisable to use between three and five specific audit questions. In general, a larger

18

Audit hypotheses can sometimes be used instead of, audit questions. (Hypotheses are
statements that are relevant to the audit problem; possible to test if they are true or false and
formulated in a negative way.) It can be argued, however, that formulating questions is better
than hypotheses as it helps the auditors to employ an open and inquisitive mind throughout the
audit. Using questions instead of hypotheses, which are formulated in a negative way, can also
make it easier when the audit is introduced to the auditee. If hypotheses are used by the
auditors, it is advisable to rephrase them to audit questions when the audit is presented to the
auditee and other stakeholders.

35

number of questions will increase the risk of the auditor losing his/her focus on the key
issues of the audit.
Figure 8: Example of link between the audit objective and audit questions

Possible audit questions in our example can be found by moving one step down, from
the audit objective to its main causes.
Question 1: To what extent do the regional directorates carry out proper needs
assessments at the schools?
Question 2: Does the Ministry of Education implement adequate routines to
coordinate the deployment of teachers over the country?
Question 3: Is the Ministry of Education securing a swift transfer of teachers in
accordance with the needs?
It is also possible to formulate hypotheses instead of questions, for example The
Ministry of Education do not transfer teachers within a reasonable time.
The link between the audit objective and the audit questions in our example can be
illustrated with the following figure:
Audit objective: To assess whether the government is
taking steps to improve the deployment of teachers

Q1: Do the regional

directorates carry out
proper needs
assessments at the
schools?

Q2: Does the Ministry of

Education implement adequate
routines to coordinate the
deployment of teachers over the
country?

Q3: Is the ministrys of

Education securing a
swift transfer of
teachers in accordance
with the needs?

The audit questions can be divided further into sub-questions. It is not necessary to
present those sub-questions in the work plan, but the team can do it as a tool for
themselves.
This can be done by issue analysis technique were the audit questions are broken
down in a hierarchy of sub-questions until the subsidiary questions are simple and
manageable enough to be answered by conducting a particular piece (or pieces) of
audit work. The sub-questions should be; mutually exclusive they do not overlap with
each other; and collectively exhaustive between them, they should cover the whole
topic.

36

Figure 9: Audit objective and audit questions

Audit objective

Audit question
1.1

Audit question
1.1.1

Audit question
1

Audit question
2

Audit question
1.2

Audit question
1.3

Audit question
3

Audit question
1.1.2

3.5.5 Assessment criteria

Performance auditing is concerned with comparing an actual situation with what is
expected (i.e. what is versus what should be), and having an opinion about the
difference between the two states of affairs. To facilitate such a comparison,
Assessment criteria need to be formulated as explicitly as possible. Assessment is the
same as appraisal or evaluation; criterion is standard for comparison and judgement.
Assessment criteria = evaluation standards = what we need to compare with in order to
make an appraisal.
Assessment criteria are the standards against which the performance of the audited
entity will be measured in a particular audit. The assessment criterias help us get from
findings to conclusions. We cant say whether the findings are good or bad if we dont
have assessment criteria to assess them with.
The assessment criteria determine whether or not the entity/ies, operations or the
programme meet or exceed expectations. The audit criteria are intended to give
direction to the assessment (helping the auditor to answer questions such as On what
grounds is it possible to assess actual behaviour? What is required or expected?,
What results are to be achieved and how?19
The assessment criteria will also help the auditor to create a common understanding
with the auditee. It is important to inform the auditee about the assessment criteria early
in the audit. The reason for that is that the auditee will be informed about what the audit
assesses, the auditee can also have good arguments about the choice of the
assessment criteria which can influence the audit. It will also be more difficult for the
auditee to question the assessment criteria at the end of the audit if they can comment
on them in the beginning of the audit. Even if the assessment criteria are discussed with
the auditee in the beginning of the audit it is important to underline that the auditors
have the final choice in the selection of the assessment criteria.
Characteristics of useful assessment criteria
For the criteria to be useful, they should meet certain standards:
Objective criteria are free from any bias on the part of the auditor or the audited entity.
19

According to ISSAI 3000 Implementation guidelines for performance auditing, page 52.

37

 Understandable criteria are clearly stated and not subject to significantly different
interpretations.
Comparable criteria are consistent with those used in performance audits of other
similar entities or activities and with those used in previous performance audits.
Complete criteria refer to the development of all significant criteria appropriate to
assessing the performance.
Acceptable criteria are those that independent experts in the field, audited entities, the
legislature, media and general public are agreeable to.
Sources of Assessment criteria
When developing assessment criteria, it can be useful to consider the following
sources:
Laws governing the operations of the audited entity
Parliamentary statements
Declarations of principle of the government or a ministry
Planning documents, contracts and budgets
Standards and measures set by the auditee20
Organisational policies and procedures
Comparisons with corresponding activities in the private sector
Comparisons with international benchmarks
Criteria used by professional organisations and standard-setting bodies
Standards set by the auditors in previous audits
Criteria used by other SAIs in similar audits
The expectations or experiences of the citizens affected
Common sense, what system in place or service delivery is reasonable to expect from
the auditee
The process of identifying assessment criteria requires rational consideration and
sound judgement. Goals and objectives set by an act of parliament, by the government
or by the auditee are sometimes vague or very broad. Under such conditions, the audit
team might have to interpret the goals to make them more specific or measurable.
When the goals are in general terms, it is difficult to get a clear idea of the audited entity
is effective or not. The auditor may choose to carry out the performance audit without
any goals having been defined focusing on economy and efficiency. When no goals
can be found it may also be possible to identify various types of effects, irrespective of
the fact that no goals exist. The auditors themselves may define goals which seem
reasonable. The auditor may ask the decision-makers to define the goals of the audited
entity. The auditor may also state or discuss in the audit report what should/could be
the goals for the entity.
In our example, suitable sources of assessment criteria for the respective questions
could be fetched from:
Q1 - Standards defined by an act of parliament or by the government regarding
staffing at schools
- Criteria laid down by the ministry
- Established procedures and routines to apply when assessing needs
Q2 - National goals regarding equal treatment of different regions
- Standards defined by an act of parliament or by the government regarding
staffing at schools
20

Standards set by the auditee itself must be used with caution. They should be analyzed with
extra care and judgment should be made if they are objective and free from any bias from the
audited and can be used.

38

Q3 - There may be a period set for when transfers are allowed

- The auditors could develop a norm that transfers should be made in such a way
that all teaching staff are available at the beginning of the school year/term.
3.5.6 Methodology and sources of data
The character of data collection will usually change over time as the project proceeds.
At the start of the project, during the pre-study, the auditor will be interested in general,
broader types of information where the auditor tries to understand the way the audited
organisation functions. As the project progresses, the data needed would be
increasingly specific with regard to sources, types of questions asked, etc.
Performance audits can draw upon a large variety of data-gathering techniques, the
same techniques that are commonly used in the social sciences. Data can be
categorised by its source as follows:
Documentary data consists of all sorts of written documents, for example, reports,
research papers, records, and statistical data.
Testimonial data is obtained through interviews or questionnaires.
Physical data is obtained by the direct observation of people, buildings, events, etc.
Data collection is one of the most time consuming part of an audit and often involves
extensive fieldwork at different levels.
Because of financial and staff constraints, the audit team cannot collect data from all
available sources within the scope of the audit. It needs to choose representative data
sources (sampling). How this selection is done is crucial for the team so that it will be
able to generalise the conclusions and recommendations at the end of the audit to
cover the whole audit objective and scope. Selecting a few representative regions could
be one way of limiting the audit. Applying sampling methods to select a limited number
of entities in each region and a number of interviewees at each entity could be another.
In our example the audit objective is: To assess whether the government is taking
steps to improve the deployment of teachers. The three audit questions are;
Question 1: Do the regional directorates carry out proper needs assessments at the
schools?
Question 2: Does the Ministry of Education implement adequate routines to coordinate
the deployment of teachers over the country?
Question 3: Is the Ministry of Educations human resources division securing a swift
transfer of teachers in accordance with the needs?
The scope defined for the geographical coverage is the whole country.
- The team will interview people at the Ministry of Educations head office.
- A representative sample of regions is selected so that the team decides to go to three
specific regions out of 12 regions. The ministry of education agrees that the three
picked regions are representative for the whole country. In the regions the team talks
with the regional heads and administrative staff.
- In each region two districts are chosen where staff who compile information are
interviewed. The team lets the head of the region influence the choice of districts.
- In each district the team chooses two schools where the team talk to the headmaster
and five teachers who had recently been employed. The team lets the district heads
influence the choice of schools.
From the sample the team will draw conclusions of how the system of deployment of
teachers works in the whole country and not just in the visited regions, districts and
schools.

39

For each individual audit question, the audit team should indicate sources of information
and methods for data collection during the main study. The team is expected to assess
the practicability of the proposed audit methods, the availability of data in the required
forms, and the teams ability to collect, analyse and interpret the data.
To be able to corroborate the findings at a later stage, it is usually preferable to
combine data from different sources and methods. The selection of sources and
methods to collect data in the work plan should be as complete as possible, yet
allowing additional ones to be included, if necessary, during the main study. The
unavailability of data may, however, restrict the choice of methods. As a general rule, it
is advisable to be open and pragmatic in the choice of methods. Nevertheless, the
planning in the work plan has to be detailed enough to allow for proper resource
planning. Data collection and analytical methods are described more extensive in
chapter 4.
3.5.7 Expected findings, conclusions, effects and risks
Expected findings are to speculate about what deviation will be seen between the
assessment criterias and what actually take place in reality. What conclusion will
eventually be drawn from the expected deviation. How can the audit create change so
that what actually take place in the reality will reach or come closer to the assessment
criterias.
Any foreseeable problems in executing the main study and getting the expected
findings should also be identified. Strategies for overcoming the foreseeable problems
should be developed. Any foreseeable risk should be mentioned.
3.5.8 Audit design
The audit design aims at making it possible to investigate the audit problem and
consequently it must be clearly linked to the audit problem. It is necessary that all the
different components of the audit design presented above are closely linked to each
other. This means that the audit questions the audit scope and methods must be
congruent with each other.
One approach to planning the audit is to make a list of the audit question. For each
audit question state the assessment criteria, the data collection methods and expected
findings. To facilitate this, auditors can use a verification form.

40

Table 4: Example of verification form for audit questions, assessment criteria, data collection methods and expected findings

To illustrate, from our example were the audit objective is: To assess whether the government is taking steps to improve the deployment of
teachers:
Data collection
Expected findings
Audit
Assessment
criteria
questions
What?
How to obtain?
How to compile and
analyse?
The ratio between the
Compare the submitted
- In an act of
Audit
Number Read the act and criteria and the
reports with the ministrys number of student and
ministrys specification
parliament the
question 1:
of
teacher differs a lot within
specification
Do the
students, Collect the submissions of the
maximum ratio
the country and is often
Table showing average
regions, districts and schools.
regional
number
between student to
number of student per
higher than the maximum.
Interviews with:
directorates
teacher for staffing
of
teacher in different
carry out
at schools is 35
teachers staff at the ministry who compile
proper needs students to 1
by
the regional submissions, and staff subjects and forms for
different regions
Many regions submit their
assessments teacher
subject,
who uses the compiled reports
Table showing when the
staff needs to late.
at the
- Criteria laid down
per
with staff at NGOs and the
different regions submit
schools?
by the ministry says school
university who work with need
their needs, when they
that each region
and
assessments of teachers
get the requests from the
shall submit their
form.
staff from 3 regional directorates
districts and when the
staff need 6 months
staff from 2 district headquarters
districts get them from the All regions do not follow the
before the school
who compile information for the
schools.
ministrys form for
year starts
region in each of the chosen
calculating staff needs.
- The ministry has
region.
specified that the
headmaster from 2 schools in each
regions shall assess
of the chosen districts
the needs by
Observe the mathematics classes in
calculating expected
form 4 in each visited school
students, including
new and leaving
student, in different
form and subject
compared to
expected teachers
that will quit or retire
Audit
question 241

3.5.9 Resource plan

When the methodological plan of the main study is done the resource plan shall be
done who give answer to questions such as whom, when, and at what costs the main
study shall be executed. It should include an activity plan and a budget for the project.
The resource plan should specify how the resources will be used. Both human and
financial resources, including the time required to complete the main study, need to be
specified.
The resource plan should be closely linked to the methodological plan. The activities
planned in the methodological plan take time and need to be realistically scheduled. In
reality the methodological and recourse plans are linked together, the recourses
available will influence the methods that can be used. As already mentioned financial
and staff constraints will influence the data collection..
Human resources
It is usually difficult to conduct a performance audit project alone, but too many auditors
in the team may lead to problems of internal communication, coordination, etc. The size
and complexity of the audit will determine the number of auditors needed.
When composing audit teams it is important to ensure that the members collectively
possess the necessary skills, expertise and experience the audit requires. The names
of the auditors who will conduct the audit and the supervisor responsible should be
mentioned in the resource plan (see also chapter 2.9).
Responsibility within the team should be clarified and work should be allocated
appropriately. All members of the team should have insight into the overall development
of the project. Nevertheless, the team leader has specific responsibilities when it comes
to planning the project, reporting to management, contacts with the auditee, etc.
None of the audit team members should have any ties to the auditee that could
question his/her objectivity. No member of the audit team should have any affiliation, for
example, kinship or other relationship, with staff at the auditee which could affect
his/her objectivity and/or independence (see also annexure 2).21
There may be cases when the audit team needs specialised knowledge acquired
outside of the SAI. In these cases the possibility of using the expertise of consultants,
doctors, engineers, etc. should be considered. If so, the assignment and the role of the
expert should be clearly specified.
Time schedule and activity plan
The resource plan should include a time schedule and an activity plan for all main
activities, and should be as realistic as possible. The time schedule should highlight
important activities and decision points or milestones when the management needs to
be involved. It is important with regular meetings with the manager for monitoring the
progress of the audit. Presentations shall be done on dates specified in the work plan
and also on the audit team initiative when the team encounters something that influence
the project or the work plan,

21

A code of ethics declaration could be used by the SAI to be filled in by performance audit team
members and managers responsible for the audit in the same way as it is used by financial
auditors by SAIs who is members of AFROSAI-E (see AFROSAI-Es Regularity audit manual
2010).

42

The time schedule makes it easier to complete the project within a reasonable time and
with a specific number of days spent by the auditors. The time schedule and activity
plan will make it possible to link different activities within the project to each other.
Some of the activities will be based on the results of other activities and must therefore
be carried out in the right sequence. It is also necessary to allow time for analysing and
interpreting data after the fieldwork and to write the report.
If it has been decided to conduct field visits to three regions in two districts within each
region and at four schools within each district, the times of such visits must be planned.
Time should be allocated for travelling and difficulties in getting hold of the right people.
There is also a need to allocate time to conduct interviews at the central level, with
university and NGOs and to retrieve and analyse data from the reports from regions,
districts and schools.
The team should keep in mind that all calendar time is not available for project work.
The team members will perhaps be required to work on other tasks, or participate in a
training programme or take annual leave. To make the activity plan realistic, these and
other considerations should be incorporated.
Tips when compiling an activity plan:
Do not be over-optimistic when compiling the plan. Auditors might have to spend time
on other tasks, such as area watching, seminars and training programmes.
The plan should consider public holidays, training and vacations.
Take into consideration the logistics for travelling, weather conditions and the working
hours of the client.
Compare activity plans from previous audits in order to estimate the amount of time
that will be needed to complete a specific activity.
Add some time, maybe 10-20%, for unforeseen events, particularly if there is a new
area, complicated methods, etc.
Put in sufficient time to analyse data and to write the report.
Take into account that the managers need time to read drafts, memos etc. and plan
what you can do before you get a decision.
The decision points or milestones in the activity plan can also be used as agreement
with management so that the reading time for management doesnt becomes too
long.
As stated earlier a pre-study could take from one month to maximum three months and
the total time for a performance audit (pre-study and main study) should normally be
shorter than a year.22
Financial resources
Budgeting involves deciding on the amount of money needed for the project. Aspects
such as salaries, accommodation, transport, stationery and printing as well as various
allowances for staff conducting the audit must be determined in planning the projects
budget. Unrealistic budgeting can be a serious problem for the audit. It is therefore
important that the plan focuses on what is reasonable and realistic and not on what is
desirable. The use of a budget for a performance audit project should be linked to the
way budgets are normally used at the SAI. For example, does the internal budget
system in the SAI distribute salary cost for staff and/or overhead cost on audit work?
3.5.10 Contacts with the auditee
22

To be considered for the SNAO prize for the best performance audit report the production time
should be less than 12 months.

43

The planning process also includes reflections on the relationship with the auditee.
Contacts with the auditee have already been established during the pre-study. The
audit team should specify the need for a contact person at the auditee(s) and the role of
that person.

44

4. EXECUTION
During the execution phase the plan that was drafted during the pre-study is
implemented.
Figure 10: Collection, analysis and documentation of audit evidence

Pre-study
and work
plan

Process

Collection,
analysis and
documentation
of audit
evidence

Drafting of
the report

- Contacts with the auditee

- Collect and document evidence
- Analyse evidence
- Develop findings, conclusions and
recommendations

Audit team

Management

- Introduce the main study to the auditee

- Supervise and monitor work processes

In performance auditing it is not enough to show that problems exist. There is also a
need to explain why they exist and to recommend what should be done about them.
The purpose of data collection in the main study is to be able to answer the audit
questions and to develop audit findings, conclusions and recommendations that will be
presented in the performance audit report.
Figure 11: The relationship between assessment criteria, audit evidence, audit findings,
conclusions and recommendations

Assessment criteria (what should be)

Audit evidence (what is condition)
What is compared with what should be
Determine the causes and effects
Develop conclusions

Develop recommendations
Estimate likely effects of the recommendations wherever possible

45

The figure is a general description of the relationships between the assessment criteria,
audit finding, causes and effects, conclusions, recommendations and effects of
recommendations.
In actual audit work the steps do not follow one after the other. Instead they overlap
each other. Some of them will go on during the whole audit, but with different emphasis
over time. The structure of a report can also be different from the figure depending on
what is suitable in the individual audit. For example, the assessment criteria and the
audit evidence can be described for each finding in a findings chapter. It is more
common to develop conclusions and recommendations in separate chapters than to
include them in one or more findings chapters,

4.1 Introduction of the main study to the auditee

Contacts with the auditee have already been established during the pre-study.
However, the SAI has now decided to continue with the audit and chosen the topic for
the main study. Therefore, an introduction to the main study is also necessary which
can be done in the form of a letter an engagement letter and an introductory meeting
an engagement meeting.
The letter, signed by the Auditor-General or someone with delegated authority, should
inform the auditee about the SAIs decision to launch a main study and explain its
purpose.
The letter should provide a statement of the objective and scope of the audit together
with relevant contact details. The letter can contain information on the Auditor-Generals
mandate and other relevant information; for example if the report will be tabled in
Parliament. The letter can state how and when the audit report is expected to be
finalised and the possibility of the auditee providing commentary during the reporting
phase. It is recommended that it also contain an invitation to an introductory meeting. In
the case of an audit that covers more than one entity, separate letters should be sent to
each entity subject to audit.
The purpose of the introductory meeting is to present the forthcoming audit. At the
meeting the SAI would normally be represented by the audit team the performance
audit manager and the top manager responsible for performance audit. The audit team
should inform the auditee about the following:
1. Purpose of the main study
2. Time schedule for the main study
3. Members of the audit team
4. The audit objective, audit question and assessment criteria.
5. Methods that will be used to collect data
6. The information and support that will be requested from the auditee
7. The purpose of the auditee appointing a contact person
8. The purpose of the exit meeting.
In countries were performance audit is a new and unknown concept, a general
information about performance audit should also be given.
The SAI can gather the auditees opinion about the chosen assessment criteria. The SAI
can also let the auditee bring to their attention any contextual matters that might
influence the audit and give the auditee the opportunity to ask questions about the
audit.

46

The performance auditor will need assistance from the auditee when it comes to finding
persons to interview, developing questionnaires, looking for statistics, etc. To deal with
these practical arrangements, the auditors may ask the auditee to appoint a special
contact person. The auditees contact person should assist the audit team throughout
the audit with required information and he/she should be consulted if the audit team
needs to visit the auditees regional or local branches. The contact person is also a link
between the auditors and the auditees management. The introductory meeting is
usually followed by a meeting between the audit team and the appointed contact
person.
Throughout the audit it is important to maintain a productive relationship with the
auditee. The SAI should seek to create an understanding of its role and function among
audited entities with a view to maintaining amicable relationships with them. Good
relationships can help the SAI to obtain information freely and frankly and to conduct
discussions in an atmosphere of mutual respect and understanding.
The SAI shall remain independent from audited entities. The SAI has to discharge its
mandate freely and impartially. The SAI should take the auditees views into
consideration when formulating audit findings, conclusions and recommendations, but
with no responsibility towards the audited entity for the scope or nature of the audit
undertaken. The final decision on what to take into consideration or what not shall rest
with the SAI. It is important that the auditee understands that the SAI will always have
the final word on what to audit and what to write in the report.

4.2 Data-collection methods

SAIs shall have access to all records and documents relating to financial management
and shall be empowered to request, orally or in writing, any information deemed
necessary by the SAI.23
To answer the audit question and meet the audit objective there is a need to collect
different kinds of data in an audit. The purpose of the data collection is to increase the
auditors knowledge about the audit area and its problems; and to collect sufficient data
in relation to the audit questions and assessment criteria that have been formulated in
the work plan.
Different methods and techniques for collecting and compiling data can be used. Data
collection methods need to be carefully chosen. These methods/techniques are also
commonly used in social science. Practical reasons such as the availability of data may
restrict the choice of methods, so the audit team may have to settle for second best. As
a general rule, the audit team should be pragmatic in the choice of methods.
Data collection is one of the more time-consuming activities of a performance audit. It
often involves visits at different levels within the auditee which can include travelling to
remote places. It can also often involve visits at the lowest level of an organisation
where staff execute the main tasks of the organisation and meet the citizens or
customers.
Audit teams should seek to combine data from different sources. Audit teams should
ensure that different findings, based on different kinds of data, are consistent. When
data from one data source appears inconsistent with data obtained from another
source, the reliability of each data remains in doubt until further work has been done to
23

The Lima Declaration of Guidelines on Auditing, ISSAI 1.

47

resolve or explain the inconsistency. When data from different sources and findings in
related parts of the audit is consistent, it gives assurance that is higher than that
attached to the individual source.
It is vital that the team adopt a critical approach and professional scepticism.
Independent judgements need to be made about the relevance and validity of data
obtained during the audit. The team need to see things from different perspectives and
maintain an objective distance from the data put forward. At the same time, they must
consider the views and arguments of the auditee and other stakeholders.
It is seldom possible to collect information from the whole population that the team
wants to assess. With almost all methods of data collection, different forms of sampling
techniques are applied. From the sample, the auditor wants to draw general
conclusions. The SAI has to decide, for example, which part of the country the audit will
collect information from to make a statement that is valid for the whole country, or which
staff members should be interviewed to be able to make statements valid for the whole
entity or category of staff. Different methods can be applied to select the sample.
Data collection, and to a certain extent analysis, can be performed by others besides
the SAI. Consultants can provide useful expertise in a number of areas. However, the
accuracy of the evidence remains the auditors responsibility. All work produced by
others needs to be critically assessed.
4.2.1 Data collected through documents
Table 5: Data collected through documents

Method
Review of
written
documents

Purpose
To understand
the auditee and
how it should
operate and how
it operates

Advantages
Give comprehensive
and historical information
Information already
exists
Do not burden the
auditee unnecessarily

Challenges
Often time-consuming
Info may be
incomplete
Needs to be clear
about what to look for
Not flexible; data is
restricted to what
already exists

Policy statements and legislation

The audit team should gather policy documents, operating guidelines, manuals,
ministerial directives, decisions on delegation, etc that are relevant for the audit. The
auditors should also consider changes that have been made to legislation and the
document trail leading to the need for change, such as submissions, press clips,
complaints, case histories and speeches.
Auditee budget statements
Auditee budget statements include an auditee overview providing financial information
as well as information about the objectives and performance of the auditee.
Management reports, reviews and minutes
Entities usually generate a number of internal documents for senior management that
summarise current issues and/or propose courses of action. The auditors should locate
and analyse such documents. Ways of identifying reports include interviews and
examination of minutes from management meetings.
File examination
48

One source of documentary evidence is examination of files. It can be summarised and

recorded in the auditors own words in working papers. In exceptional cases, important
documents and those conveying significant or potentially controversial matters could be
photocopied and the original files identity be recorded on the copy.
It is important that the audit team realises that all relevant papers may not be contained
on files that are registered. There may be other relevant documents of which the team
is unaware. The team should, however, seek to ensure that the evidence obtained is
complete enough to answer the audit questions.
Databases
Many organisations have compiled databases, both manual and computerised. In many
countries there is a government bureau of statistics and there may be commercial
databases that are run, for example, by banks, which may be used to collect data.
These databases may enable the auditor to use data that has already been collected
and compiled. This will save time and money. It might also be possible to order special
computer compilations.
Statistics
Auditors can also use already compiled statistics. Statistics is data from databases
already collected and compiled by others for their own purposes. The advantage is that
it is faster and cheaper to use rather than to compile own statistics. The disadvantage is
that the statistics compiled by others are not always collected and compiled according
to the audits needs. The statistics could also have flaws.
External evaluations, reports and literature
When conducting an audit, the auditors should read previous evaluations and reports
written in the area. Sometimes these have data that can be utilised in the audit. Such
secondary data must be used with caution. It is also wise to consult academic research
and other literature concerning the audit area. Alone this may not provide audit
evidence, but it could provide the theoretical framework or points of reference
necessary to elaborate on other audit evidence.
4.2.2 Testimonial collected data, asking people
One source of data is to ask people about facts or opinions.
Table 6: Testimonial collected data

Method
Interviews

Purpose
To
understand
someones
views or
experiences

Advantages
Obtain full range and
depth of information
Possible to ask for
clarifications and followup questions
- Obtain document and
information about other
data sources

Focus groups,
seminars and
reference
groups

To explore a
topic indepth by
combining

Quickly and reliably

obtain common views
Efficient way to get
range and depth of

49

Challenges
Each interview can
take a long time
Interviewer could
bias the interviewees
responses
Could be difficult to
summarise and
compare to each
other
A lot of interviews
could be costly
Difficult to schedule
several people
together
Need a skilled

views from
different
persons
Questionnaires To get
information
from a large
number of
people

information in short time

Can be answered
anonymously
Each reply is
inexpensive to
administer
Easy to compare and
analyse
Get the reply from
many people
Obtain large amounts
of data

facilitator for good

discussions
Could be difficult to
analyse responses
People might not
answer
Require knowledge
to construct
- Is impersonal and
wording could bias
clients responses
- People maybe
interpret the question
differently compared
to the auditors
intention
Require some skills
in statistics

Interviews
The structure of an interview depends on its objective. In the pre-study, the questions
will typically be broader, the objective being to formulate possible audit problems. In the
main study, the focus will shift to verify the audit problem, the reasons behind it and
possible solutions.
As far as possible, the interviewer should be familiar with the subject to be discussed
and the areas to be covered. Interviews should be planned. A good auditor must be a
skilled interviewer, and this demands preparation, an effective questioning technique
and the ability to listen carefully. The word auditor is derived from the Latin verb
audire to hear or to listen.
When asking for opinions on different issues, the auditors will usually benefit from not
giving the questions to the interviewee in advance. The same applies if for any reason
the audit team needs to interview several people within the same organisation. On the
other hand, if the purpose of the interview is to collect specific facts about the audited
entity, it is advisable to provide the questions in advance to enable the interviewee to
collect relevant statistics and other documentation. Whichever the case, questions
should be prepared in advance in the form of an interview guide.
The typical interview is held in the context of a meeting. Sometimes telephone
interviews provide an alternative that can save both time and costs, especially if the
questions are relatively simple and standardised. A standardised interview can be done
when information is collected with a questionnaire.
A memorandum containing the information gained during the interview should be
recorded as soon as possible after the interview. In some cases it can be advisable to
give the interviewee a chance to comment and sign to confirm that it is a true record of
what he/she has said.
Interviews some practical advice
Careful preparations
Determine the purpose and objectives of the interview.
Study relevant documents before the interview.
Prepare an interview guide that allows you to be flexible during the interview.

50

 Consider practical arrangements (time, venue, information to the auditee).

Assign roles to each person before the interview (if you are more than one person
making the interview; which is recommended) and rotate roles between interviews.
Flexible implementation
Try to make the interviewed confident and willing to share information particularly when
interviewing lower level staff.
An interview may be structured in an introduction, placing the questions and concluding
remarks.
Introduce the purpose of the interview; make it clear how long you expect the
interview to take.
Place your question, but be flexible to where the interview leads, you can come back
to questions that you want to address later in the interview if answers to later
questions comes up earlier than expected.
Be brave enough to ask stupid questions.
Start each topic with an open and broad question; lead your interview object by
paying attention and using follow up questions. Probing by using questions like, Could
you give me an example? Could you develop this further?
You can repeat the answer of the interviewee in your own words to make sure that
you havent misunderstood her/him.
Think critical about responses
Ask an open question at the end for example Is there anything you want to add?
Sum up and explain what will happen next. Finally, thank the interviewee for the
assistance given.
The continuous learning process
Give each other in the interview team constructive feedback directly after the
interview (what was good; what can be improved?).
Secure information
Have a discussion right after the interview, to compare impressions and
interpretations. Have you understood what the interviewee had said and that your
questions have been answered? Did the interviewee bring in new information that
need to be acted on, document to be collected, new question to be asked in future
interviews?
Put down detailed unstructured raw notes the same day as the interview.
Structure notes under suitable headings later on (if there is not time to do it straight
away).
Get comments and clarifications from other participating team members; add and
subtract, and clarify outstanding questions.
From the notes, the working paper, it should be possible to see: who was
interviewed; by whom; when the interview took place; and for what purpose. The
main content of the interview should be structured under suitable headings; in such
a way that information will be easy to retrieve at a later stage. Add conclusion that
you draw from the interview. Furthermore, the notes should be written with details
that capture the interview as much as possible. It may turn out that certain aspects
will be of importance later on, even if the team did not realise that at the time of the
interview.
File the notes as a working paper.
Focus groups, seminars and reference groups
Focus groups (a selection of a few individuals brought together to discuss specific
topics) are primarily used to collect qualitative data that can provide insights into the
values and opinions of individuals involved in the process or activity under audit.
Seminars have the advantage of bringing together a large number of people
representing a wide range of knowledge and perspectives.
51

Reference groups may be composed of people drawn from within or outside the SAI
and are usually made up of experts, such as academic researchers. The group usually
meets on a few occasions during the audit. The reference group may be used both for
collecting data and for quality-control purposes.
Focus groups, seminars and reference groups might involve both cost of time and
money for the auditee and the participants. It should be used with a clear purpose and
be well prepared.
Questionnaire
Questionnaires are used to gather detailed and specific information from a population
consisting of a group of people or organisations. A questionnaire provides a systematic
collection of information from a defined population, usually administered to a sample of
units in the population.
Questionnaires are mainly used to gather facts that are not available in any documents
and that are important as a reference to substantiate a finding. Questionnaires are used
when comprehensive knowledge is needed. The information is usually collected in the
form of very structured interviews or questionnaires that the respondents complete
themselves, Questionnaires that the respondents complete themselves can be
distributed by using regular post, the Internet or be given directly.
It is important that the questions are correctly formulated and easy to understand. The
interpretation of the response will be easier. The respondents will probably not be able
to respond if questions are incorrectly constructed or difficult to understand. It will also
be harder to interpret the responses from the individual that respond if the questionnaire
is difficult to understand for the respondents. The auditors should therefore put a lot of
effort into the construction of the questionnaires.
For analysing the responses to the questionnaire, computer-processing skills are often
necessary.
When formulating a questionnaire, the auditor should consider:
the purpose of the different questions.
how the questions can be formulated so they will not be misunderstood
how the answers will be compiled
The structure of a questionnaire is usually built up with:
-information to the respondent about the questionnaire and its purpose.
- background questions, which facilitate the grouping of answers into different
categories like age, organisation, sex, etc.
- central questions, which may be formulated in a number of ways depending on what
the auditors want to know and how the data will be compiled (Questions can be placed
with a number of alternative answers or open ended question, or a combination of both)
- concluding questions which are open questions where the respondent may give some
general comments
Have a system for checking questionnaires received so that it will be possible to remind
people who havent answered to return the questionnaire.
4.2.3 Physically collected data
Table 7: Physically collected data

Method
Observation/Ins
52

Purpose
To gather

Advantages
View operations as

Challenges
May be difficult

pecting
physical objects

information on
how a
programme
actually
operates,
particularly with
regard to
processes and
to what extent
objective exist
and their
status.

they occur
Method can be
adapted to events as
they occur
- Give reliable
information on easily
observed
objects/phenomena

to interpret
observed
behaviour
May be complex
to categorise
observations
The auditors
presence may
influence
behaviour of
observed people
May be time
consuming

Observation
Observations give the audit team an opportunity to understand the reality behind
inspection reports and other official documents. It may provide a clearer bottom-up view
of essential problems, which can be compared to the picture given in official reports or
by the executive level of the organisation. It also offers an opportunity to study
relationships and processes between staff and management. There are different ways
of using this method. The auditor may choose to visit workplaces, schools or hospitals.
Another alternative is to accompany inspectors or other officials in their day-to-day
work. A further possibility is to sit in the reception hall of, for example, the local clinic to
observe how patients are treated. Sitting in (without participating) on meetings at the
auditee may also provide useful information. An inherent risk of observing people is that
the auditors presence may alter the peoples behaviour, and as a consequence the
evidence collected will be less valid.
Inspecting physical objects
A wide range of information can be collected by inspecting physical objects, such as
buildings and other facilities. . A camera can be used to take photos and in that way
document the inspections. Inspection of physical objects is mainly used to study a few
objects in order to provide detailed examples of what has been observed. Physical data
could also be collected from a large number of objects and compiled and analysed for
further use

4.3 Surveys and case studies

There is a need to collect different kinds of data in an audit, quantitative as well as
qualitative. Another way to categorise data collection methods may be described in
terms of what type of conclusion you can draw when you measure the results.
One type of conclusion that the audit team wants to give could be an overall picture of
the situation in the whole country or for the whole population. This type of study is
sometimes referred to as a survey and usually uses quantitative data compiled from
questionnaires, databases or files. Through surveys the auditors can get an overall
picture of the situation. In general, it is not necessary to collect data from the whole
population, such as all hospitals; instead, a sample of hospitals is taken. If the audit
team wants to apply the description of the studied sample to the whole population, the
sample should be large enough, and randomly and independently selected. Sound
judgements have to be made on for example, which part of the country the audit shall
collect information from to make a statement that is valid for the whole country; or which
staff members shall be interviewed to be able to make statements for the whole entity
or category of staff. Various sampling methods can be applied.

53

On the other hand, there is the case study that aims at thoroughly exploring a small
number of cases and then draw conclusion about the population. Case studies provide
the opportunity to thoroughly explore a small number of cases in order to have an indepth knowledge of organisations, systems, processes and activities relevant for the
audit problem. It enables the auditor to concentrate on details and on understanding
organisational processes. The cases can be examples of a situation that may be
prevalent throughout a population. Generalising from case studies is a question of good
arguments, not absolute proof or statistical certainties. It is essential for the
argumentation to use a clear and specified logic in the selection of cases, a logic that
supports the intended use of the information. It is wise to choose a case (or a few
cases) that are the most or least likely to have certain characteristics. Another option is
to choose one or a few cases that are considered to be representative of the whole
population. Depending on the purpose, there are several possible ways to select cases,
e.g. the best cases, the worst cases, extreme cases, or typical cases for the whole
country or for a group of possible cases.
Several different methods can be used to collect data about the particular studied case,
for example, interviews, observations and statistics from for example a district, a
hospital, an admission to the hospital or a file in the hospital record, etc.
It is often useful to combine surveys with case studies when possible so that the audit
both can give an overall picture and in dept knowledge of the situation that is under
audit.

4.4 Prepare for analysis while collecting data

The different data-collection methods have so far been treated separately. There are,
however, several common traits, especially when it comes to the steps that should be
taken to facilitate the future analysis of the collected data.
The more data, the more need for structure
The more data that is to be collected and analysed, the more structured the data
collection should be. This will facilitate the analysis by providing the auditors with data
that can be compared and it will be easier to categorise the data further. This is
especially relevant for interviews, physical observation and some documents, such as
worksheets. If a large number of observations/interviews are to be carried out or
regions/districts to be visited it is particularly important that the auditors start by testing
the outline of the data collection so that it can be revised and perfected.
In our previous example with the audit of the deployment of teachers, if the team
interviewed one minister and 30 teachers, the team need to structure the data from the
interviews with the teachers more than the interview with the minister.
Scrutinise data sources critically
There may be reasons why the sources of data (whether an interviewee or the author of
a written document) is not objective. It is therefore important to carefully consider the
possibility of bias in the collected data and to try to adjust for it through double-checking
and corroboration of data.
Document directly
If data is not properly documented soon after it is collected the auditors risk missing
important aspects. This is especially important in cases where the auditor is creating
new primary data through interviews or physical observations. Documenting, in this
sense, is not about simply recording an interview by taking notes. The auditors notes
should as soon as possible be typed and structured in relation to the audit questions.
54

This facilitates later analysis of the data so that it can be used as audit evidence. If the
interviewee is a manager it could also sometimes be wise to send the interview notes to
the interviewee for confirmation to ensure that the information is not contested at a later
stage of the audit.

4.5 Compilation and analysis of data

Analysing data is an important step in all performance audits. When analysing data, the
audit team should start by revisiting the audit objective and the audit questions. This will
help them to organise their data and focus their analysis in line with the audit questions.
To analyse and interpret information effectively will require time, communication,
creativity and a systematic utilisation of the extracted and summarised data.
It is sometimes useful to distinguish between the compilation of data and the analysis of
data. For example, when working on a questionnaire, the answers given to each
question can be compiled for example by an average representing all the answers of
the respondents. After this, the compiled data may be analysed from different
perspectives or together with other data. The distinction between compilation and
analysis is not always clear-cut and in the case of interviews it is sometimes not
possible to maintain the distinction. In the following sections we therefore use the term
analysis to cover both the analysis and the compilation of data.
It is important to study the data gathered both in-depth and extensively. Analysing and
interpreting data is a process that requires the audit team to constantly move between
the different stages as the analysis gives rise to new knowledge and new ideas. This
process should go on until the auditors are satisfied with the result.
Data, information and knowledge are similar concepts linked together in a chain. Data is
the primary observation. Data which has been compiled is thereby transformed into
information. Information which is analysed has become knowledge. The transformation
of data to knowledge can be described in the following picture.
Figure 12: Transformation of data to knowledge

Data
Information
(Interpreted data)

Analysing

Knowledge

4.6 Quantitative analysis

Quantitative analysis is often equated with statistical analysis. Whether or not it can be
applied, partly depends on the kind of data that has been collected. The difference
between quantitative and qualitative analysis is often described as the difference
between analysing numbers (quantitative analysis) and analysing texts (qualitative
analysis). Information that has been collected as text can, nevertheless, often be
transformed into numbers. One can, for example, go through documents to see how
many of them include a positive statement about a certain issue. A calculation can then
be made, expressing the percentage of investigated documents that include this type of
positive statement.

55

The kind of statistical analysis most frequently used in performance auditing is

descriptive statistics. More advanced forms of statistical analysis, such as regression
analysis are not described in this manual, but can be valuable tools for auditors who are
acquainted with statistical theory. The auditor can use descriptive statistics to describe
the audit object or to present findings from the audit. Descriptive statistical information
can be used to present the size of the audit objects personnel, resources, operations,
finances, etc. If the sample on which the auditors have gathered is chosen with
statistical methods numerical conclusion about the whole population can be drawn.
Some basic statistical concepts are presented in the table below.
Table 8: Basic statistical concepts

Concepts
Centrality
Mean

Median

Mode

Dispersion
Range

Variance

Standard
deviation

Explanation

Usefulness

The sum of the values of all

observations divided by the
number of all observations
First the observations should be
arranged progressively. The
median is the observation in the
middle of all observations.
The most frequent observation

When scores are more or less

symmetrically distributed, e.g.
height or weight
When extreme scores distort
the mean, e.g. income

The difference between the

highest and the lowest observation

Average of the squared distance

between a single
observation and the mean value
The square-root of the variance

Percentage

(The part of the population/the

total population) multiplied with
100

Index

Relates the development of a variable

to a base level, often 100 a particular
year.

To describe dichotomous
values and proportions, e.g.
men and women, or to point out
the typical value, e.g. the
typical number of children in a
family
To complement the
mean/median as a
measurement on how scores
are distributed
To complement the mean as a
measurement on how scores
are distributed
To complement the mean as a
measurement on how scores
are distributed
To see the size of part of the
population. For example the
number of yes in relationship to
the total number of responses
Makes it easy to understand and
compare the development for
different variables over several
years. It also makes it possible to
compare different years with
regards to inflation.

When generalising from a sample to the whole population, attention has to be given to
the level of certainty with which this can be done. This depends on the construction and
the size of the sample and the size of the population. In the case of means, the degree
of certainty can be calculated with statistical methods by establishing the confidence

56

interval, i.e. the interval within which the true mean of the population falls within
reasonable doubt.

4.7 Qualitative analysis

Qualitative analysis is a broad term used to describe a wide range of methods for
structuring, comparing and describing data. It is usually used when combining different
kinds of data. Unlike quantitative analysis, qualitative analysis does not use statistical
methods. Instead it is based on logical reasoning and arguments. In general qualitative
analysis means creating own ways of systemising data that convince the reader that the
evidence is true. Common components of this systematisation are comparing, sorting
after differences and sorting after similarities.
Generalising through qualitative analysis beyond the cases that have been investigated
is a complex issue that needs careful consideration each time. If there is reason to
believe that the auditee, or other stakeholders, will dispute the generalisation, the
auditor should exercise particular caution.
Qualitative analysis is commonly used when deriving analytical evidence from certain
sources of data, such as interviews and documents.
An example of how to analyse interviews is given below:
1. Choose a method for structuring the data from the interviews, using audit questions
as the first choice; and problems, actors, regions, etc. as the next choice if it is not
meaningful to structure the data only in line with the audit questions.
2. Read the interview notes again and focus on the structure. If interviews are to be
organised according to audit questions, make a note in the margin when something is
said that is relevant for question number one, two, etc.
3. Go through all the notes regarding audit question number one. If there are many
relevant remarks, make a written summary. If necessary, choose a new factor to
structure the remarks. Key players could be used as such a structuring factor.
4. Compile and analyse the opinions of each type of key player, one at a time.
5. Compile and analyse the opinions of all types of key players together.
6. Look for similarities and differences between the opinions of different categories of
key players.
7. Summarise the information (the interpreted data) in a few sentences and judge
whether or not the audit question can be answered. Continue with the next audit
question.

4.8 Mixture of quantitative and qualitative data analysis

Very often data is analysed using different sources, different methods of data collection
as well as different types of data analysis. Consequently, the final step in data analysis
consists of combining information from different types of data sources to gain
information and knowledge. This means that information from interviews may be
combined with analysis of statistical records; information from case studies may be
combined with information from surveys; some information may emanate from field
studies in one province while other information refers to another province. Combining
information from different sources is a process that may be compared to doing a jigsaw
puzzle, where the pieces are the results of different data-collection activities. There is
no general solution on how to handle these situations. It is, however, of central
importance that the auditor works systematically and carefully in interpreting the data
collected. By combining multiple data sources, methods, analyses, auditors seek to
overcome the bias that can come from using a single source of information. When three
or more sources are used to verify and substantiate a finding is called triangulation.

57

4.9 Audit evidence

Audit evidence is gathered in order to support a description of the activity under review
in an organisation or program. Audit evidence is information retrieved from the data
collected that forms the foundation that supports the SAIs findings, conclusions and
recommendations. It consists of specific information collected and used to test the
assessment criteria, and subsequently to support the audit findings. All work should be
planned from the perspective of acquiring the evidence needed to identify findings that
satisfy the audit objective. The selection, examination and evaluation of audit evidence
are the backbone of any audit. It is of great importance that the auditors, as well as SAI
management, critically assess the quality of the audit evidence that has been collected
during the main study.
When analysing the evidence that has been collected, the concepts below should be
used to assess the quality of the audit evidence.24
Relevant evidence is information that can be used to answer the audit questions.
Relevant evidence bears a clear and logical relationship to audit objectives, audit
questions and assessment criteria. An important aspect of relevance, if the audit object
is an ongoing activity, is that the evidence used to reach a conclusion should not be
outdated. If recent evidence is not available, the auditor should be able to justify why
historical evidence is still relevant.
Reasonable evidence should have been collected and compiled in the most efficient
way and yet allows the audit objective to be achieved.
Competent evidence is usually explained as evidence that is both sufficient and
appropriate.
Sufficient evidence relates to the quantity of audit evidence that is needed to support an
audit finding. The decision as to whether evidence is sufficient in quantity will be
influenced by its quality. Audit teams should gather enough evidence to persuade a
reasonable person that their findings, conclusions and recommendations are well
founded. The judgement on what can be considered as sufficient evidence will be
influenced by a wide variety of matters, including:
the materiality of the matter in hand
the risk that insufficient evidence will lead to the wrong conclusion
the persuasiveness of the evidence
the likelihood that findings will be challenged by the auditee.
Be aware that sometimes the evidence is not sufficient and more information needs to
be collected but it also often happens that too much information is collected.
Appropriate evidence means that the evidence is reliable and valid.
Reliable evidence is based on data that would be the same if the same study is carried
out repeatedly in the same environment by different auditors using the same methods.
Therefore, the auditors should make sure that their methods are clearly described and
generally accepted.

24

INTOSAI Auditing Standards stipulate that relevant, reasonable and competent evidence
should be obtained to support the auditors conclusions regarding the audit object. Already when
planning the collection of evidence, it is essential that the audit team is aware that the audit
should be able to live up to INTOSAI Auditing Standards.

58

Valid evidence is evidence that describes what it is intended to describe.

It can be illustrated graphical:
Figure 13: Competent audit evidence

Competent

Sufficient

Appropriate

Reliable

Valid

An example that illustrates the two concepts, reliability and validity, is if you put a
thermometer in boiling water and it consistently shows 91 degrees Celsius (at sea
level), then the thermometer is reliable since it shows the same temperature for boiling
water every time. However, it is not valid since it shows 9 degrees too low. On the other
hand, if you have a thermometer that gives different readings every time but on average
indicates 100 degrees then you have a valid but unreliable instrument.
The following guidelines will help the auditors to assess the quality of audit evidence:
Evidence corroborated by several different sources is more sufficient than from a
single source.
Evidence collected by several auditors applying the same methodology is more
reliable.
Evidence that is old and does not reflect changes that has occurred since data was
collected might not be relevant.
Evidence that is too expensive to obtain may not be reasonable.
Evidence collected from sources that have a vested interest in the outcome might be
less valid.
Evidence collected from non-representative samples is not valid for the whole
population and therefore insufficient.
Evidence from first-hand informers is often more reliable and valid than the same
information obtained indirectly.
Evidence obtained from knowledgeable independent sources is less biased and
therefore usually more valid than that evidence obtained from within the audited
organisation.
Evidence from statements made by officials of the audited entity is more reliable when
confirmed in writing.

4.10 Audit findings

The INTOSAI Auditing Standards define findings as the specific evidence gathered by
the auditor to satisfy the audit objectives. The findings are gathered to be able to
answer the audit questions. Audit findings contain assessment criteria (what should be),
condition (what is), and effect (what are the consequences observed as well as
reasonable and logical future effect), plus cause (why is there a deviation from norms or
criteria), when problems are found.
59

The use of criteria has been explained in previous chapters. The term condition
encompasses the evidence, relevant to the audit question, which has been collected
and analysed. In other words, the findings compare what should be (criteria) to
what is actually happening (condition).

4.11 Causes and effects

Once the deviation has been identified, the audit team should determine why there is
deviation from the criteria (causes) and what the consequences (effects) of these
deviations are.
Causes are the reasons why the condition deviates from the criteria. Causes must be
presented with caution unless they are supported by evidence. The auditors should
access causes that are stated by the auditee, the explanations to why the criteria are
not met, and make a judgement if they are relevant or not. The audit team needs to
identify possible causes and then determine the ones that could have been prevented
(actionable causes). The validity of actionable causes should be confirmed.
Below are some examples of how to pursue the analysis once a cause has been
identified:
Determine whether the cause is isolated or systematic and inbuilt in the system.
Systematic causes generally require more significant corrective action and may be
creating other adverse effects outside the reviewed activity. However, the team
should be careful because in some cases an isolated cause can also create
significant problems. There might be a need to assess the materiality of the cause.
Determine whether the cause was an error or an omission, or whether it was
intentional. Intentional causes generally suggest disregard of procedures and
regulations. Errors or omissions generally suggest a lack of knowledge of what is
expected.
Determine whether the cause was internal to the audited body or external for example
unclear legislation. External causes not within the control of the agency may have a
significant bearing on the problem. Recommendations for corrective actions must
consider these factors and how their effects might be mitigated.
Determine the internal controls and routines that should have prevented the cause
from occurring.
The team should identify possible effects of the criteria not having been met. If possible,
in identifying the effects, the actual situation (where the criteria are not met) should be
compared with the ideal situation where the criteria would have been met. To a certain
extent these possible effects would have been considered at an earlier stage as a
motivation for carrying out the audit of this particular problem.
The effects could be noted either as what has already occurred or, based on logical
reasoning, as the likely future impact. The nature of the findings determines whether the
audit team can present actual or potential effects. Actual effects from past or current
conditions help to demonstrate the harmful consequences and generally provide
evidence that corrective action is needed. Potential effects are generally described as
the logical consequences that could follow should the condition not meet the criterion.
Potential effects are to some degree speculative, so teams should use them with care,
especially in the absence of any related evidence of observed past effects.

60

4.12 Conclusions
Once the auditors have documented the condition, compared it against the criteria,
determined that the condition does not meet the criteria, determined why the criteria are
not being met (causes), and possible consequences (effects) they should draw
conclusions. Performance audits should point out deficiencies in economy, efficiency
and/or effectiveness. The conclusion should specify the reasons behind the lack of
economy, efficiency or effectiveness.
Audit conclusions reflect the view of the auditors deduced from the findings. A
conclusion is the auditors view or opinion formed after considering the findings. Audit
conclusions should clarify and add meaning to specific findings in the report. It is not
always easy to make a clear distinction between the findings and the conclusions. One
reason for this is that conclusions are based on findings and can include summaries of
the findings. Conclusions, however, go beyond merely restating the findings. Whereas
the audit findings are identified by comparing what should be (assessment criteria) to
what was actually happening (audit observations) when data was collected, the
conclusions reflect the audit teams explanations and views based on these findings.
This is why findings are usually expressed in the past tense and conclusions in the
present tense. Conclusions might include identifying a general topic or a certain pattern
in the findings. An underlying problem that explains the findings may also be identified.
The conclusions should flow logically from the findings, their causes and their effects.
All analytical steps taken beyond the findings should be clearly explained and justified.

4.13 Recommendations
From the conclusions, the auditors can develop recommendations to guide corrective
actions. Recommendations are not required for each audit finding. Recommendations
are the auditors proposals for improvement in operations and/or performance of either
the audited entity as a whole or a particular section or area audited within the entity or
to the decision makers above the entity. Recommendations can target changes in
policies, procedures, practices or organisational structure. Recommendations cannot
suggest changes of political decisions but the implementation of political decisions.
4.13.1 Formulating recommendations
Audit recommendations emerge from identifying the causes of the audit findings which
ought to be addressed by the audited entity.
Recommendations should be neither too detailed nor too general. Recommendations
should be sufficiently detailed so that they can be understood and implemented by the
audited entity and followed up by the SAI. The recommendations should focus on what
should be changed and leave the questions of how to make the changes to the auditee.
The audit team should not prescribe detailed steps to be taken by the auditee. This will
be for the auditee to decide at a later stage.
The audit team needs to consider the following questions in order to develop good
recommendations:
What needs to be done?
Why does it need to be done?
Where does it need to be done?
Who is responsible for doing it?
What are the expected effects of the recommendations?
Could the implementation have negative effects elsewhere?

61

If the audit covers more than one audited entity, it should be clear which entity should
implement which recommendation.
The audit team must develop audit recommendations that can be implemented and that
will not necessarily require additional resources. The recommendations should lead to
changes in the auditees organisation and administrative systems, etc. rather than
increased allocation of resources.
4.13.2 Characteristics of useful recommendations
Some characteristics of good recommendations are that they:
Flow logically from the findings and conclusions, referring only to matters addressed
in the report.
Be directed at those who have authority and responsibility to act.
Be stated positively and constructively.
State what actions need to be taken to improve economy, efficiency and/or
effectiveness.
Indicate alternative solutions when more than one course of action could correct the
problem.
Address significant deficiencies and demonstrate that action will improve operations,
safeguard assets, reduce costs or bring the condition into compliance with the criteria.
Identify and weigh costs and risks against potential benefits.
Consider the practical constraints of implementation in the light of limitations, such as
financial constraints.
Avoid recommending additional measures if efforts are already underway to remedy
the problem. In such circumstances, reference to such efforts may be sufficient.
Table 9: Example on how to support the formulation of recommendations

To support formulation of recommendations a table can be used.

Findings

Conclusions

Recommendations

To whom

- the ratio between student to teacher

for staffing differ between 20 and 50
students to 1 teacher, in some cases
far above the maximum ratio of 35
students to 1 teacher
- only two region submit their staff
need 6 months before the school year
starts
- No regions assessing the needs of
teachers according to the ministrys
specification
- The regional directorates do not carry
out needs assessments for teachers at
different schools as they should
-Schools in urban areas have more
teachers per students than in rural
areas
- The ministry do not act if they dont
get the regional staff needs as
requested

Because the
system of regional
submission of
teacher needs do
not function there is
no functioning
system in place for
distribution of
teachers which has
led to an unequal
distribution of
teachers to
different schools.
The teachers
desire to be in
urban areas has a
higher influence on
the distribution of
teachers than the
students needs.

- The distribution of
teachers to schools
should be based on
the number of pupils
- Action should be
taken directly when
regions dont comply
with the requirement

Ministry of
Education

62

4.14 Drafting the report

At the end of each audit, the SAI should publish a written report communicating the
results of the audit to parliament, government and other stakeholders. The published
performance audit report is the product on which the SAIs performance audit function is
judged by its stakeholders. Errors in the report could be potentially damaging to the
credibility of the particular report and to the SAI as such. It is therefore crucial that a
great deal of attention be given to the accuracy, logic and clarity of the report.
Figure 14: Drafting of the report

Process

Audit team

The SAIs
Management

Collection and
analysis of audit
evidence

Drafting of
the report

Clearance of
the report

- Compose synopsis
- Write and revise draft
- Review working papers
- Agree on synopsis
- Review draft chapters

4.15 Reporting standards

The audit report is the most important product of the audit. The audit report is not a
compiled version of internal working papers. The working papers made throughout the
audit project must be transformed into the audit report. In this process there will be
information, which may be interesting, but not fit into the scope and must therefore be
omitted. At the same time, during the writing process the auditors might find that the
collected data might not be completely sufficient, which may mean that the auditors will
have to collect some more data.
The performance auditor is not expected to provide an overall opinion on the
achievement of economy, efficiency and effectiveness by an audited entity in the same
way as the opinion on financial statements. A report is normally addressing specific areas
of an entity's activities, the auditor should provide a report which describes the
circumstances and arrives at a specific conclusion rather than a standardised statement.
According to INTOSAI Auditing Standards in exercising its independence the SAI
should be able to include whatever it sees fit.25 The reporting standards to be met
include the following:
Objectives and scope: The performance audit report should state clearly the
objectives and scope of the audit. This information establishes the purpose and
boundaries of the audit.

25

INTOSAI Auditing Standards stipulate that an audit report should set out the findings in an
appropriate form; its content should be easy to understand and free from vagueness or
ambiguity, include only information which is supported by competent and relevant audit
evidence, and be independent, objective, fair and constructive.

63

Completeness: The audit report should contain all information and arguments needed
to fulfil the audit objective and provide answers to the audit questions. The relationship
between audit objectives, findings and conclusions needs to be complete and clearly
stated.
Accuracy: Evidence presented in the audit report should be true and comprehensive
and all findings correctly and logically portrayed (see also chapter 4.9). The readers
need to know that what is reported is reliable. A high standard of accuracy requires an
effective system of quality assurance.
Objective and logical: The presentation of the report should be balanced in content
and tone. All evidence should be presented in an unbiased manner. Auditors should be
aware of the risk of exaggeration and overemphasis of deficient performance. The audit
report should only present arguments that are logically valid.
Clarity: The audit report should be clear and easily understandable as well as written to
suit the capabilities, interests and time constraints of the readers. The language should
be as simple as the subject matter allows. Technical terms and unfamiliar abbreviations
must be defined. Tables, charts and photographs should be used where appropriate to
present and summarise complex information. Clarity is improved when the report is
concise.
Timeliness: The audit report should be prepared and issued in a timely fashion in order
to be of greatest use to readers and stakeholders, particularly government and the
audited entity that have to take the necessary corrective actions. If the report is delayed
the implementation of the audit recommendation or other changes caused by the audit
will also be delayed. The findings can also become obsolete.

4.16 Format of the report

Some points need to be emphasised with regard to performance audit reports:
The value of concise and focused reports that highlight significant issues of public
administration.
The report should be focused on meeting the audit objective and answering the audit
questions and not be focused on the description of the auditee or what the auditor
have done.
The need for well-developed audit findings based on relevant, reasonable and
competent evidence.
The importance of having a few, comprehensive audit conclusions and
recommendations.
Below is an example of a report format within a SAI (the headings in the example are
descriptive, but in the audit reports the headings could give information on the
contents).
Title
The title should make it clear what the audit object is and indicate the auditee
responsible for the activities audited.
For example, if the audit problem is insufficient supply of potable water in the rural
area, the title could be Provision of water in the rural area by the Ministry of Natural
Resources. It can also be formulated judgemental The Ministry of Natural Resources
do not provision water in the rural area as it should. The formulation depends on what
the SAI want to achieve, if it wants big headlines or a good future relationship with the
auditee. It is a judgment of what is of most benefit for the SAI and what can create most
64

changes in the reality in the long run.

Covering letter
The AG presents the report and acknowledges the cooperation received from the
auditee by the auditors who undertook the audit. The covering letter is written on the
official letterhead of the AG, including the AGs signature and the date. It can also
mention the names of the performance auditor and managers that has produced the
report. In the letter, reference is made to the AGs mandate according to the constitution
and the mandate to undertake special audits in accordance with relevant legislation.
The covering letter should indicate who the report is addressed to. In the covering letter
the SAI can indicate how it intends to follow up the report and the implementation of the
reports recommendations. The SAI can also request that the auditee shall report back
on a later date the action it has taken in line with the recommendations presented in the
audit report.
Table of contents
This illustrates the structure of the report.
Executive summary
The summary summarises the background, major findings, conclusions and
recommendations. The summary does not introduce new things that are not mentioned
in the main body of the report. It does not have to follow the structure as the main body
of the report. The summary is designed for those who have little time to read the full
report. The summary should normally be just a few pages - as a rule of thumb not more
than 10% of the pages in the main report (e.g. 4 pages for a report of 40 pages).
Chapter 1: Introduction
The objective is to give background information about the audit. It could include a short
summary of the auditees legislative framework, goals and objectives, resources and
organisational structure. Information that has no relevance to the understanding of the
findings should preferable be omitted. This chapter also presents the audit motivation,
the audit objective and scope. Data collection methods should be presented briefly. If
necessary, a more detailed presentation of the data collection carried out (list of
documents reviewed and number of interview objects, etc.) may be included in an
annexure.
Chapter 2: Description of the audited activity
A brief description of the system or programme being audited. From this description it
should be clear how the system should work, with details that make it possible to
understand the following part of the report. Information that has no relevance to the
understanding of the findings should preferable be omitted. Part of this work will
normally have been carried out during the pre-study. The title of this chapter should be
the name of the system or programme being audited.
It can be illustrated by using some type of model. A model is a picture or a simplification
of reality. The model can be used as a tool for describe and explain how the system
works, for example we have used the input-output model to describe the 3E;s, but other
types of models like flow-charts can be used.
Chapter 3: Findings
This chapter clearly describes the assessment criteria and relates them to what has
been observed in the reality, i.e. the condition. The findings should be presented in a
correct way without judgements or comments by the auditor. It presents the findings
generated from the comparison between criteria and condition.
Chapter 4: Conclusions
65

The chapter contains the SAIs conclusions.

Chapter 5: Recommendations
The chapter contains the SAI recommendations.
Annexure
Annexure can be used to present detailed descriptions and findings. Annexure may
also be used for comprehensive descriptions of the auditee, statistical tables, publishing
reports from consultants, detailed explanations of methods used, etc. These can be
placed in annexure so the report does not become too long and to facilitate the reading
of the audit report.
Glossary of terms sorted alphabetically.
List of abbreviations sorted alphabetically.

The findings deal with what has been observed by the auditors during the collection of
data and they should therefore be stated in the past tense. The auditee may already
have taken action after the data collection, making the findings obsolete. If the findings
are evidently up-to-date, it might be natural to use the present tense. The conclusions
should state prevailing conditions and consequently the present tense will be suitable.
Recommendations should be written either as a request using the conditional mood or
as a demand using the imperative mood.
Table 10: Usage of tense for findings, conclusions and recommendations

Findings
Conclusions
Recommendations

Past tense
Yesterday
X

Present tense
Today
(X)
X

Conditional or imperative
Tomorrow

4.17 The writing process

The process of writing the report can be broken down into four stages, namely:
1.
Synopsis
2.
Audience analysis
3.
Drafting
4.
Revising
Effective writers keep these stages separated. Audit teams are advised to adhere to
these stages when writing the report. If the audit team fails to keep these stages
separated, they are likely to experience difficulties in writing the report. This might result
in wasted time plus a more disorganised and poorly written report.
4.17.1 Synopsis
Before starting to write the report, the team should organise its work by preparing a
synopsis. A synopsis is an outline or skeleton of the report. The synopsis sets out the
main structure and lists in the form of keywords or short sentences the intended
content under each heading. The aim of the synopsis is to sketch the structure and
broad content of the report. It is a tool for defining the structure of the report. The
synopsis encourages a professional approach to writing, clarifies the team's organising
principles, and facilitates the actual writing of the report. Often a draft synopsis can be
produced at a very early stage of the audit. It can be a useful tool for the audit team,

66

influencing the collection of data and making it possible for the auditors to start writing
early.
In the synopsis, the audit team should:
sketch a logical structure of the report
indicate the findings, together with supporting evidence, and conclusions to be
included in the different sections
identify necessary annexures.
A logically organised synopsis helps the team to decide what to keep, what to develop
and what to reject before writing any pages. In short, the team should know what
message it wants to deliver to the stakeholders. When compiling the synopsis, the
problem-tree technique as well as the input-output model could be tools for structuring
the analytical process, particularly if these techniques had already been used when
identifying the audit problem.
The completion of the synopsis should coincide with a meeting with management.
Management then has an opportunity to see what the report is really about, and to
provide input, comment on the general direction, conclusions, express their opinions
and give directions. This will save time by reducing the need for revision and
amendment at later stages of the drafting process.
4.17.2 Addressing the readers need
The audit report should be written in a way which is relevant for the specific target
groups. Identifying and knowing the target groups will help the audit team to decide on
the structure and tone of the report. As mentioned earlier, in performance audits the
stakeholders are parliament, the government, the audited entities and the general
public. The media are also important because they serve as a link between the SAI and
the general public. The audit team may have valuable findings, conclusions and
recommendations, but unless organised and presented properly, the readers will have
difficulty to understand what the team has found and which conclusion they make. No
report can achieve its purpose if the auditors do not addressing the readers need.
Some questions that may help the audit team to write the report are listed below:
What kind of information are the readers interested in?
What are the readers likely attitudes towards the message?
Will the decision makers implement the recommendations?
What is the likely effect of the report on the readers?
Are the readers familiar with the subject matter?
How do the readers view the SAI's credibility, expertise and authority?
4.17.3 Drafting
Once a synopsis has been produced and the readers have been identified, the audit
team should start drafting the report in accordance with the agreed structure. However,
it should be remembered that the structure may be changed during the drafting stage if
necessary.
During the drafting the auditors understanding of what they have observed will grow,
the understanding of the issues in the report will develop and improve. Writing as such
is an analytical part of the work. Some of the analytical problems faced by the
auditors may not be clarified until they start to write. This means that several drafts will
be written, commented upon and revised. Therefore the auditors need to let this
process take its time until they are clear about what they want to say; i.e. what the main
message is in the report as a whole as well as in different parts.
67

The auditor should write the report so it conveys an understanding of the issues
relevant to the audit. The draft report should reflect an objective audit where issues are
examined in their proper context. Weaknesses in systems and controls and examples
of poor performance should be disclosed in the report, but it should be balanced and
effective management and good performance should not be left out.
The team leader is responsible for coordinating the drafting process. He/she may
decide to split the drafting of different chapters/sections amongst the team members.
After completion of the chapters/sections, the team will come together and agree on the
content.
To shorten the production time it is wise to start writing as soon as possible. Examples
of parts that may be written at an early stage of the audit are the introduction, the
methods, and the description of the audit object. Those parts can then be inserted into
the report according to the structure decided upon in the synopsis.
Writing the report some practical advice
Start writing at an early stage.
Be aware that it is common to write a too long summary and a too long descriptive
chapter but a too short findings chapter.
Lead the reader through the whole text by linking chapters, sections and paragraphs
to a coherent story. Outline the content of each chapter in an introduction.
Start the findings chapter, or different sections in the findings chapter, by summarising
the main messages in a box.
Use tables and diagrams to present important findings, when it makes it easier for the
reader to grasp the message. Highlight the main points that you want the reader to
conclude from the table after the presentation of the table, but be restrictive in giving
other comments.
Some practical advice using tables
- Make tables and diagrams as simple as possible. Exclude superfluous information, so
that the main message is not lost. More detailed information can clouds the view and
give a false impression of exactness. More comprehensive tables can be presented in
appendices that you can refer to.
- Only use tables and diagrams when it is an effective way to communicate the
message. Dont use tables/diagrams just because you have the information.
- Put a number and an informative title on figures and tables.
- State the source of the information under the diagram/table.
- Consider using deflated figures in presenting monetary values. If possible relate
figures to something, e.g. output (unit cost).
- Percentages and index series can often be more effective measures to communicate
the message than absolute figures.
Make clear distinctions between observations and facts and the auditors' conclusions.
Use an objective tone and let facts speak for themselves (dont reinforce or use
emotional language). Avoid jargon.
Keep it simple.
Be concise make every word count. Write accurate, brief and clear. Eliminate
digression and irrelevant sentences.
Write specific and informative headings.
Write in short paragraphs that start with a topic sentence. Develop paragraphs by
using facts, details, examples, definitions etc.
Use acronyms and abbreviations with moderation. Acronyms should be explained
when used the first time in the text. A list of acronyms can be provided in the report.
68

 Use footnotes when you refer to your sources of information.

4.17.4 Revising
After writing a first draft, the team should revise the document. When revising, the team
should aim for clarity and conciseness. Consequently, the auditors should ensure the
following:
The overall report is logically organised and structured.
The style of writing is adapted to the intended readers.
The headings of chapters and sections are appropriate.
The sentences are not so long as to be incomprehensible and not too short as to be
monotonous.
All irrelevant words and information not based on evidence are eliminated.
The words are concrete, professional and commonly known.
The spelling is correct.
It is advised to have somebody proofreading the draft to look out for spelling mistakes,
typographical mistakes, inconsistencies of type of style, and numbering mistakes
(references to footnotes, pages, tables and graphs).
The audit team can make use of peers to help revise the draft. It is often useful to have
someone with no knowledge of the audit read the draft report at an early stage. This
might highlight areas where the presentation is weak. The team shall also crossreference the report its findings, conclusions and recommendations, supporting
evidence, tables and charts, etc. with the relevant working papers to check that
everything therein can be justified. After revision of the draft report, the team leader
should present the draft report to management for review. The management can in their
turn also let other persons or functions review the draft report and the audit file.

69

4.18 Clearance of the report

After receiving the draft report, performance audit management should review the draft
and working papers. To meet the INTOSAIs standard of timeliness it is important to
secure a swift review process. The purpose of this review is to ensure that the audit
meets the SAIs auditing standards. Once the report has been cleared by the auditorgeneral, it should be submitted according to its mandate which can be to parliament,
auditees and other stakeholders.
Figure 15: Clearance of the report

Process

Drafting of the
report

Audit team

SAIs top
management and
performance
audit
management

Clearance of
the report

Presentation
of the report

- Organise contacts with the auditee

- Finalise draft report after management review

- Review draft report and working papers

- Send draft report to the auditee
- Be represented at exit meeting

4.18.1 Review of the draft report

During this review, different level of management should ensure the following:
Legitimacy of the report. The report should not stray outside the SAIs legal
mandate. There should be no suggestion that the SAI is questioning the political
objectives.
The findings, conclusions and recommendations are addressing the audit
objectives. Management should ensure that the report provides complete and
conclusive coverage of the audit problem.
Balance of the various sections. The report should concentrate on the important
aspects and avoid minor issues.
Quality of presentation. The draft should be well written, appropriate tables, charts,
etc. should be used to help the reader understand the findings, conclusions and
recommendations.
Meet the INTOSAI auditing standard described in chapter 4.15 and the SAIs own
standards.
The draft report could be submitted to an internal team of experts for quality review,
advice and comments; sometimes it can also be good to let regularity auditors review
the draft report. This could be done before or after clearance by the performance audit
management. Once the final adjustments have been made, the draft should be
submitted to the Auditor-General, or to a manager with delegated authority, for a
decision on whether the draft report can be sent to the auditee for comments.

70

Top management will not normally review the working papers, as this has already been
done by the performance audit unit manager. However, where findings, conclusions or
recommendations are particularly sensitive, top management may find it necessary to
call for the relevant working papers.
4.18.2 Sending a draft report to the auditee
When the auditor-general, or a manager with delegated authority, is satisfied that the
draft report meets the SAIs auditing standards, he/she sends it to the auditee together
with an invitation to submit written comments.
It is important that the descriptive part and the findings are a correct interpretation of the
reality. From that interpretation of reality the SAI formulates its conclusions and
recommendations. The SAI will have to decide on whether the draft report to be sent to
the auditee should include conclusions and recommendations, or only descriptive parts
about the auditee and the findings.
The purpose is to; get comments on the descriptive chapters, audit findings and
conclusions (if they are sent over to the auditee) outlined in the draft report; give the
auditee an opportunity to correct factual errors; test the feasibility of possible
recommendations (if they are sent over to the auditee); give an opportunity for the entity
to provide additional information and assess whether any audit findings should be
modified in the light of the additional information provided.
In the invitation it is advisable to state the latest date for submission of the comments.
4.18.3 Internal finalisation
The audit team should carefully consider reservations expressed by the auditee. The
team should verify data that has been contested and, if necessary, collect additional
data. It might also be necessary to ensure, once again, that all findings and conclusions
are based on competent evidence. Where differences of opinion cannot be satisfactorily
resolved, the SAI should consider whether this should be highlighted in the report.
After the team amended the report it should be reviewed by the management. The final
decision about the report should be taken.
4.18.4 Exit meeting
After the final decision about the report has been taken an exit meeting will be
arranged. The audit team or performance audit manager will arrange an exit meeting
with the auditee where the report, and the response to the written comments from the
auditee, will be presented. It is polite to supply a copy of the report to the auditee at the
exit meeting so that the auditee is acquainted with the report at the time of publishing.
The purpose of the exit meeting is to ensure that the auditee has been given the
opportunity to see the report before it is published. It also informs the auditee about the
amendments that has been made and the final report that will be tabled. At the exit
meeting it is particularly important that the critical findings of the audit are highlighted.
The meeting should ensure that:
it is clear that the auditee understands the findings, conclusions and
recommendations.
the auditee is informed about tabling and publication procedures.

71

5. REPORTING
5.1 Presentation of the report
Once the performance audit report has been internally finalised and the exit meeting
has taken place, the report should be presented to the stakeholders. Comprehensive
reports and wide distribution of the reports are key to the credibility of the performance
audit function.
Figure 16: Presentation of the report

Process

Clearance of
the report

Presentation
of the report

Follow-up

Audit team

- Prepare press release

- Assist management in contacts with stakeholders and media

Management

- Approve press release

- Present the report to stakeholders

The SAI should have an established policy to the effect that, within the limits of
legislation, performance audit reports are publicly available and widely distributed.
Generally, audit reports will become public documents only after they have been tabled
in parliament, but as mentioned above it is customary for the SAI to provide the auditee
with a copy of the report shortly before it is released.
The report should, when possible, be distributed to the auditee, the government, the
media and other interested parties. Also, people who have contributed to the audit
could be supplied with a copy. Once the report has been published, the SAI should post
it on its website.26
To spread the audit reports message and avoid misinterpretations, it is advisable that
the SAI provides the media with adequate and well-balanced information in the form of
press releases or even press conferences when the report is released.
A press release is a way for a SAI to supply information to journalists and a tool for the
SAI to leading journalists to focus on relevant issues in the report. A press release is
often written when an audit report is about to be published. The content, form and style
of a press release are essential to make the media interested in covering the report.
Below is some advice on how to write a press release:
The press release should have a headline. The headline should be brief, correct and
to the point, and should state the main message in the report.
Start with the most important facts or news items. Start with the main point and
conclusions, and then move on to further details and background information.
26

INTOSAI says in its Performance Audit Guidelines, ISSAI 3000 3100, page 69, that Written
reports should communicate the results of audits to all levels of government, make the results
less susceptible to misunderstanding, make the results available for public inspections, and
facilitate follow-up to determine whether corrective actions have been taken.

72

 The tone of the press release should be factual. Keep sentences short and the
language comprehensible. Avoid technical language.
Try to use phrases and quotes from the report which the journalists can use in their
reports or articles.
The more accessible you make the text in the press release; the more likely it is that
the journalist will use your own words in the article or report.
Keep the press release brief.
Date the press release and include the name of persons that can be contacted for
further information.
Do not forget that the main purpose with performance audits is that the audit should
create change/improvements; the main point is not to have a big press cover.
SAIs are encouraged to send a copy of the report to AFROSAI-E for posting on its
website, for the purpose of regional networking, capacity building and constructing a
database for future research.

73

6. FOLLOW-UP
The priority that can be given to follow-up tasks should be considered in the context of
the SAIs overall audit strategy as determined in the planning process.
There are three common types of follow-up activities that a SAI can carry out. The
timing of such activities and the type of follow-up activities undertaken by the SAI will
depend on its policies and the prevailing circumstances. The follow-up activities provide
a basis for assessing the SAIs performance and an opportunity to validate possible
benefits generated by the audit. Follow-up activities may also provide the legislature
with valuable information.
Figure 17: Follow-up

Process

Audit team

Presentati
on of the
report

Followup

- Compare the work plan with the actual work

done

- Carry out follow-up audits

Management
- Decide on follow-up policy
- Launch follow-up audits
- Report results from follow-ups

6.1 Follow-up by the auditors of their own work

To promote learning within the organisation and provide valuable input towards the
planning of training activities the audit team should make a follow-up of its own work.
Immediately after finalising the report, the audit team should identify weaknesses as
well as strengths that occurred during the audit. This can include comparing the
planning documents with the actual work done or the time spent compared with the
budgeted time in the work plan. If the work was delayed, the auditors should try to find
reasons for the delay from for example documents in the working paper file. Lastly, the
audit team should check that documentation has been compiled in accordance with
standards. This can be done shortly after the report has been tabled.
Experiences gained by the audit team should be available and communicated to the
rest of the performance audit department. This could be done in the form of seminars,
internal memos, postings on the website, etc.
In addition to the internal follow-up by the audit team, the SAI could also try to include
the views of the auditee in follow-up.
It is also recommended that the audit team share the lessons learned with other SAIs
and communicate these to AFROSAI-E for publication on its website.
74

6.2 Follow-up of the recommendations

To promote learning within the SAI and to increase the possibility for the
recommendations to be implemented the SAI should follow up the extent to which the
auditee has taken action in line with the recommendations presented in the audit report.
The timing and format of this follow-up constitute a key management decision to be
taken by each individual SAI in accordance with its policies.
Recommendations can be followed up in several different ways, for example, through
informal contacts with the auditee, information can be gathered by regularity auditors
during regularity audits, a formal request for information can be sent to the auditee, or a
follow-up audit can be carried out.
Depending on the results of the follow-up of recommendations, the SAI will consider
what further action is required.
The PACs recommendations and the debate and decisions that can take place in the
parliament because of an audit should also be followed up.
6.2.1 Following up the impact of recommendations
Assessing the impact of the actions the auditee has taken on the recommendations will
help to measure how well the SAIs performance audits promote economy, efficiency
and effectiveness in the public sector. Follow-up is also necessary to assess the
performance of the SAIs own work.
Impacts may be qualitative or quantitative. In identifying impacts, the costs that are
associated with achieving the impacts should be compared against the benefits.
Significant impacts should be validated with the entity or other relevant bodies where
possible.
It can be difficult to separate the effect caused by the auditees action by implementing
the recommendations and other external effects.
For example, if the SAI has audited the Ministry of Healths campaign to reduce the
number of malaria cases and suggested recommendation that has been implemented
by the ministry.
After a year we can measure a reduction of malaria cases.
We know that during the year an NGO made a big effort to prevent malaria and there
had been a drought, which meant there were fewer mosquitoes that spread malaria.
Therefore it could be difficult to quantify the effect in the reduction in malaria cases:
How much of the reduction was caused by the implementation of the SAIs
recommendations, the NGOs efforts and the drought?
A decision to follow up the impact should be based on an analysis of the results of the
follow-up of recommendations. If, for example, there are indications that the auditee has
been slow in implementing the recommendations, lax in coordinating its actions, or has
difficulties in fully understanding what actions are required, there is a strong case for
following up the impact of the audit.
Following up the impact of the audit requires a lot of effort and analysis what the
auditee has done regarding the recommendations It should be carried out in the form of
a follow-up audit. Furthermore, a follow-up audit of the impact needs to cover the
manner in which recommendations have been implemented and how other issues may
have influenced the impact. The assessment of the impact will nevertheless form the
core of, and basis for, the follow-up audit.

75

6.3 Follow-up audit

In many cases following up on recommendations without carrying out a follow-up audit
is sufficient. This is usually the case when the audit referred to a one-off event or to a
specific programme now abolished. Even in these cases, however, there may
sometimes be important for follow-up audits in order to examine whether
recommendations have been implemented.
Specific follow-up audits can be done by many reasons for example when the audit
revealed significant issues for further review by the legislature or when the audits
recommendations were likely to lead to significant benefits.
If the SAI decides to conduct a follow-up audit of a performance audit report, it is
advised that a new team carry out the follow-up audit. New auditors will probably be
more objective.
6.3.1 Planning follow-up audits
Planning is important for the follow-up audit, and takes a similar form as a main study
work plan with the purpose of follow up what has happened after the audit and if the
recommendation has been implemented or not and the reasons for that
There is a need to define and plan the relevant aspects of the audit that will be followed
up. The team that does the follow-up should indicate the recommendations, projected
impacts and other relevant issues that will be examined. The extent of the proposed
follow-up should be described.
The scope of the follow-up audit should be determined based on an assessment of the
following:
Whether the original conclusions are still relevant.
Corrective actions taken by the auditee.
Work by parties other than the auditee that could influence the impact.
Relevant external factors and other issues.
6.3.2 Reporting on follow-up audits
Reporting on follow-up audits should be done in line with the general reporting
principles of the SAI. Whether or not it is suitable to table the follow-up audit report in
parliament will depend on how the SAI assesses the significance of the findings and
conclusions from the follow-up.
Deficiencies and improvements identified in the follow-up audit should be
communicated to the auditee. Positive action in implementing the audit
recommendations should also be highlighted, as this is to the credit of both the auditee
and the SAI.

76

Annexure 1: RESPONSIBILITIES WITHIN THE AUDIT PROCESS

77

Annexure 2 INTOSAI AUDITING STANDARDS

The INTOSAI Auditing Standards consist of four parts:
Basic principles
General standards
Field standards
Reporting standards
The purpose of auditing standards is to provide the criteria against which the quality of
the audit results can be evaluated.27 The INTOSAI Auditing Standards do not have
mandatory application, but they reflect a best practices consensus among SAIs. Each
SAI should determine whether the INTOSAI Auditing Standards are compatible with the
achievement of its mandate. National standards, taking into consideration the
constitutional, legal and other circumstances under which the SAI operates, should be
defined.
The basic principles are assumptions, principles and requirements, which help in
developing auditing standards and serve the auditors in forming their opinions and
reports, particularly in cases where no specific standards apply. Auditing standards
should be consistent with the principles of auditing and provide minimum guidance for
the auditor to help determine the steps and procedures that should be applied in the
audit.
The general standards describe the qualifications and competence, the necessary
independence and objectivity, and the exercise of due care, which will be required of
the auditor to carry out the tasks related to the field and reporting standards in a
competent, efficient and effective manner.
The field standards establish the criteria or overall framework for the purposeful,
systematic and balanced steps that the auditor has to follow. These steps represent the
research that the auditor, as a seeker of audit evidence, carries out to achieve a
specific result. The standards establish the framework for planning, conducting and
managing audit work.
The reporting standards set the framework for the auditor to report the results of the
audit, including guidance on the form and content of the auditors report.

Standards regarding Performance Audits

INTOSAI issues the International Standards of Supreme Audit Institutions (ISSAIs)
concerning the responsibilities of SAIs. There are ISSAIs that are general and there are
also ISSAIs that are specific for different types of audits. Some of the ISSAIs that
concern performance audits are described below.
Excerpt from INTOSAIs code of ethics, ISSAI 30

27

Issued by the Auditing Standards Committee at the 14th Congress of INTOSAI in 1992 in
Washington, D.C., United States as amended by the 15th Congress of INTOSAI 1995 in Cairo,
Egypt.

78

The INTOSAI Code of Ethics is intended to serve as a foundation for the national
codes of ethics. 28 Each SAI has the responsibility to ensure that all its auditors
acquaint themselves with the values and principles contained in the national code of
ethics and act accordingly.
A code of ethics is a comprehensive statement of the values and principles that should
guide the daily work of auditors. The independence, powers and responsibilities of the
public sector auditor place high ethical demands on SAIs and the staff they employ or
engage for audit work. A code of ethics for auditors in the public sector should consider
the ethical requirements relating to civil servants in general and the particular
requirements relating to auditors.
According to INTOSAI, a code of ethics involves the following:
Trust, confidence and credibility
The legislative and executive authorities, the general public and the audited entities
are entitled to expect the SAIs conduct to be above suspicion and reproach and
worthy of respect and trust.
Integrity
Integrity is the core value of a code of ethics. It requires auditors to observe both the
form and the spirit of auditing and ethical standards. Auditors have a duty to adhere
to high standards of behavior (e.g. honesty and candidness) in their work and in
their relationships with the staff of audited entities. The conduct of auditors should
be above suspicion and reproach.
Independence
Independence from the audited entity and other outside interest groups is
indispensable for auditors. This implies that auditors should behave in a way that
increases, or in no way diminishes, their independence. Auditors should strive to be
independent of audited entities and other interested groups, but also to be objective
in dealing with the issues and topics under review. It is essential that auditors be
independent and impartial, not only in fact but also in appearance.
Political neutrality
It is important to maintain both the actual and the perceived political neutrality of the
SAI. Therefore, it is important that auditors maintain their independence from
political influence and discharge their audit responsibilities in an impartial way. This
is relevant for auditors since SAIs work closely with the legislature, the executive
and government entities required by law to consider the SAIs reports.
Conflict of interest
When auditors are permitted to provide advice or services other than audit to an
audited entity, care should be taken that these services do not lead to a conflict of
interest. Auditors should protect their independence and avoid any possible conflict
of interest by refusing gifts or gratuities, which could influence or be perceived as
influencing their independence and integrity.
Professional secrecy
Auditors should not disclose information obtained in the auditing process to third
parties, either orally or in writing, except for the purposes of meeting the auditing

28

INTOSAI Code of Ethics, Issued by the Auditing Standards Committee at the 15Ith Congress
of INTOSAI in 1998 in Montevideo, Uruguay, ISSAI 30.

79

bodys statutory or other identified responsibilities as part of the auditing bodys

normal procedures or in accordance with relevant laws.
Competence
Auditors have a duty to conduct themselves in a professional manner at all times
and to apply high professional standards in carrying out their work in order to
perform their duties competently and with impartiality. Auditors must not undertake
work they are not competent to perform.
Professional development
Auditors should exercise due professional care in conducting and supervising the
audit and in preparing their reports. They should use methods and practices of the
highest possible quality in their audits and have a continuous obligation to update
and improve the skills required for meeting their professional responsibilities.
Quality Control for SAIs, ISSAI 40
ISSAI 40 establishes a general framework for quality control. ISSAI 40 is based 6
domains:
a) Leadership is responsible for quality within the SAI
The head of the SAI (e.g. the Auditor-General) have the overall responsibility for the
SAIs system of quality control. The head of the SAI may delegate operational
responsibility. SAIs should strive to achieve a culture that recognises and rewards high
quality work.
b) Relevant ethical requirements
Each SAI shall establish policies and procedures designed to provide it with reasonable
assurance that the SAI and its personnel comply with relevant ethical requirements.
c) Acceptance and continuance of client relationships and specific engagements
Each SAI shall establish policies and procedures for the acceptance and continuance
of client relationships and specific engagements designed to provide the SAI with
reasonable assurance that it will only undertake or continue relationships and
engagements where the SAI:
- is competent to perform the engagement and has the capabilities, including time and
resources, to do so;
- can comply with relevant ethical requirements and
- has considered the integrity of the client and does not have information that would
lead it to conclude that the client lacks integrity .
d) Human resources
Each SAI shall establish policies and procedures designed to provide it with reasonable
assurance that it has sufficient personnel with the competence, capabilities and
commitment to ethical principles necessary to:
- perform engagements in accordance with professional standards and regulatory and
legal requirements and
- enable the SAI or engagement partners to issue reports that are appropriate in the
circumstances.
e) Engagement performance
SAI shall establish policies and procedures designed to provide it with reasonable
assurance that engagements are performed in accordance with professional standards
and regulatory and legal requirements, and that the SAI or the engagement partner
issue reports that are appropriate in the circumstances.
f) Monitoring
SAI shall establish a monitoring process designed to provide it with reasonable
assurance that the policies and procedures relating to the system of quality control are
relevant, adequate and operating effectively.

80

Basic Principles in Government Auditing, ISSAI 100

ISSAI 100 states that the full scope of government auditing includes regularity and
performance audit. Performance audit is concerned with the audit of economy,
efficiency and effectiveness.
Standards and guidelines for performance auditing based on INTOSAIs Auditing
Standards and practical experience, ISSAI 3000
ISSAI 3000 is based on generally accepted principles of performance auditing. The
guideline sets out the general framework for performance auditing, defines application
of auditing principles for performance auditing, provides standards and guidance for
planning and conducting performance audits, and for presenting the audit results. It
also includes information on performance auditing in relation to information technology
and on conducting performance audits with an environmental perspective. Further, a
framework of system-oriented approaches in performance auditing is presented.
Performance Audit Guidelines Key Principles, ISSAI 3100
ISSAI 3100 outlines a common understanding of what defines high quality work in
performance auditing; How to formulate objective for performance audits, selecting
audit topics; how to conduct a performance audits and follow-ups and how to perform
quality control. It also contains information about key issues to consider in introducing
and maintaining performance auditing in an SAI.

81

Annexure 3 Index and abbreviations

Index of words

accountability ............................................................ 7
activity plan ............................................................. 42
analys ...................................................................... 55
annual plan ............................................................. 23
area watching.......................................................... 24
assessment criteria ........................................... 37, 38
audit area ................................................................ 24
audit evidence......................................................... 58
audit object ............................................................. 34
audit objective ........................................................ 33
audit of economy .................................................... 10
audit of effectiveness .............................................. 10
audit of efficiency.................................................... 10
audit problem ......................................................... 31
audit progress report .............................................. 20
audit question ......................................................... 35
auditability .............................................................. 32
auditee .................................................................... 34
Auditing Standard ..................................................... 6

economy ................................................................... 7
effect ......................................................................... 9
effectiveness ............................................................. 7
efficiency ................................................................... 7
engagement letter ............................................ 21, 46
engagement meeting ........................................ 21, 46
evidence .................................................................. 58
exit meeting ............................................................ 71
external communication ......................................... 21
external stakeholder ............................................... 21

F
file ........................................................................... 19
finding ..................................................................... 59
focus group ............................................................. 51
follow-up ................................................................. 74
follow-up audit........................................................ 76

G
B
budget ..................................................................... 43

C
case study ............................................................... 54
cause ....................................................................... 60
clearance ................................................................. 70
code of ethics .......................................................... 14
communication ....................................................... 21
competent............................................................... 58
conclusion ............................................................... 61
condition ................................................................. 60
consultant ............................................................... 48
contact person ........................................................ 47
criteria ............................................................... 37, 38
critical mass............................................................. 14

D
data analyse ............................................................ 55
data analyse ............................................................ 54
data compilation ..................................................... 55
data-collection ........................................................ 47
departmental meeting ............................................ 20
document ................................................................ 48
documentation........................................................ 17
draft ........................................................................ 67
draft report ............................................................. 70

82

general survey ......................................................... 24

generalising ............................................................. 57
Geographic cover .................................................... 34
goal ........................................................................... 9

I
identified risk .......................................................... 25
impact ..................................................................... 12
indicator .................................................................. 26
inherent risk ............................................................ 25
input .......................................................................... 8
input-output model .................................................. 8
inspecting objects ................................................... 53
Internal communication.......................................... 20
interview ................................................................. 50
introduction of main study ..................................... 46
introductory meeting .............................................. 46
issue analysis ........................................................... 36

M
main study .............................................................. 45
management meeting............................................. 20
manual ...................................................................... 1
materiality ......................................................... 25, 31
mean ....................................................................... 56
median .................................................................... 56
methodological plan ............................................... 33

mode ....................................................................... 56

objective.................................................................... 7
observation ............................................................. 53
output ....................................................................... 8

reliable .................................................................... 58
report format .......................................................... 64
report writing .......................................................... 66
resource plan .................................................... 33, 42
review ..................................................................... 70
revise....................................................................... 69
risk .................................................................... 25, 32

perfect market .......................................................... 7

performance ........................................................... 10
performance audit template manual ........................ 1
performance audit unit ........................................... 14
performance auditor ............................................... 14
permanent file ........................................................ 20
physically collected data ......................................... 52
plan ......................................................................... 22
potential for change ................................................ 32
press release ........................................................... 72
pre-study memorandum ......................................... 32
pre-study plan ......................................................... 28
problem................................................................... 31
problem indicator ................................................... 26
problem-tree ........................................................... 29

sample..................................................................... 53
sampling ...................................................... 39, 42, 48
selection criteria ..................................................... 31
stakeholder ............................................................. 21
strategic plan .......................................................... 23
sufficient ................................................................. 58
survey...................................................................... 53
synopsis................................................................... 66

Q
qualitative analysis .................................................. 57
quantitative analysis ............................................... 55
questionnaire .......................................................... 52

T
team ........................................................................ 14
team leader ............................................................. 14
testimonial data ...................................................... 49
three E....................................................................... 9
time cover ............................................................... 34
time schedule.......................................................... 42
transparancy ............................................................. 7
triangulation ........................................................... 57

validity..................................................................... 59

range ....................................................................... 56
reasonable .............................................................. 58
recommendation .................................................... 61
regularity audit........................................................ 11
relevant ................................................................... 58

W
work plan ................................................................ 33
working paper ......................................................... 18

List of figures
Figure 1: The input-output model .................................................................................. 8
Figure 2: The input-output model, an example of the performance audit process within
a SAI............................................................................................................................. 9
Figure 3: Stages in the performance audit process ..................................................... 12
Figure 4: The audit funnel ........................................................................................... 13
Figure 5: Planning and selection of audit topics .......................................................... 22
Figure 6: Pre-study and work plan .............................................................................. 27
Figure 7: Example of a problem-tree ........................................................................... 30
Figure 8: Example of link between the audit objective and audit questions ................. 36
Figure 9: Audit objective and audit questions .............................................................. 37
Figure 10: Collection, analysis and documentation of audit evidence.......................... 45
Figure 11: The relationship between assessment criteria, audit evidence, audit findings,
conclusions and recommendations ............................................................................. 45
Figure 12: Transformation of data to knowledge ........................................................ 55
Figure 13: Competent audit evidence ......................................................................... 59

83

Figure 14: Drafting of the report .................................................................................. 63

Figure 15: Clearance of the report .............................................................................. 70
Figure 16: Presentation of the report.......................................................................... 72
Figure 17: Follow-up ................................................................................................... 74

List of tables
Table 1 Some differences between performance auditing and regularity auditing ....... 11
Table 2: Index for an open file, example ..................................................................... 19
Table 3: Differences between area watching and a general survey............................. 26
Table 4: Example of verification form for audit questions, assessment criteria, data
collection methods and expected findings ................................................................... 41
Table 5: Data collected through documents ................................................................ 48
Table 6: Testimonial collected data ............................................................................. 49
Table 7: Physically collected data ............................................................................... 52
Table 8: Basic statistical concepts .............................................................................. 56
Table 9: Example on how to support the formulation of recommendations.................. 62
Table 10: Usage of tense for findings, conclusions and recommendations ................. 66

Abbreviations
AFROSAI-E
AG
INCOSAI
INTOSAI
ISSAI
NGOs
PA
PAC
SAI
SMART
3Es

84

African Organisation of English-speaking Supreme Audit Institutions

Auditor general
It is a congress that is held every third year by INTOSAIs members.
International Organization of Supreme Audit Institutions
Professional standards and best practice guidelines for public sector
auditors, officially authorised and endorsed by the International
Organisation of Supreme Audit Institutions (INTOSAI)
Non governmental organisation
Performance audit
Public account committee
Supreme audit institution, i.e. Auditor-Generals office
Specific, measurable, actionable, realistic and time bound
Economy, Efficiency and Effectiveness