Using the Balanced Scorecard

Planning and Conducting Performance Audits

Southeastern and Southwest Intergovernmental Audit Forum
Joint Conference
August 30 September 1, 2006

Using BSC for Performance Audits

Shared understanding of scorecard

Why bother?

Knowing-doing gap: why you arent

more effective

How to use a scorecard to structure

audit

How to use scorecard to manage and

evaluate audit

Every picture tells a story, dont it*

* apologies to Rod Stewart

Mission story

Customer Focus Story

Internal Process Story

Learning and Knowledge Story

Financial Story

Organizational story

Truth is truth
To the end of reckoning.
William Shakespeare

Measure for Measure, V.i.45

SAO Organizational Scorecard

Balanced Scorecard - Organizational
Actively provide government leaders with useful information that improves accountability

Perspective

Objective
- be a world leader in professional services - competitive with any private firm in quality of
services and efficiency
- identify fiscal impact that equals SAO budget, that is, deliver recommendations that help
the state use its limited resources more effectively and efficiently
- produce results that add value to the operations of the legislature and the client agency
- provide legislators and their staff with information that they find useful in policy
deliverations and decision making

Mission

Measure

Target

OPI - Useful Information Rating

Rating of 2.50: scale from 1 to 3

Org-Fiscal Impact

10 x Annual Budget = $160,000,000 per year

OPI-Customer Feedback Rating

Rating of 4.0: scale from 1 to 5

OPI-Service Delivery Status

85% of budget

Org-Employee Feedback Rating

Rating of 3.75; scale from 1 to 5

- spend taxpayer money wisely to acquire the knowledge, equipment, and other
resources needed to achieve the SAO mission and deliver high quality services

Customer Focus

- build respectful relationships based on trust, cooperation, and open communication with
our clients
- create deliverables that are easy to read, understand, and to use
- maintain effective working relationships with those we audit
- provide customers with information that helps them improve their operations and their
accountability

Internal Processes

- continuously identify better approaches to our work

- deliver high quality products and services within the fastest time frame
- focus resources on service delivery

Learning & Knowledge

- provide information and services when we say we will

- create a positive work environment that encourages creativity, innovation, high quality
results, and low turnover
- give employees the opportunity to fulfill their potential

Org-Workforce Capacity

85% of needs identified

Org- AAS Productive Workforce

98% to 102% of budget

- promote growth of knowledge and learning, risk-taking, innovation, and change

Financial

- deliver support services that help the SAO strategies to achieve their mission
- maintain the right number of people with the right knowledge at the right time to deliver
SAO services

Org-Audit Support Workforce

98% to 102% of budget

Org-Budget Compliance - Dollars

98% to 100% of budget

Org-EES-Cost Recovery

100%

Org-MAS-Cost Recovery

100%

Washington State Scorecard

North Carolina DOT

Safety

Goal 1: Develop and deliver services, products and projects

that ensure a safe and secure environment for employees,
stakeholders /partners, and customers

Objective A: Reduce employee accident and injury/illness rates

1. Reduce number of DOT vehicle and equipment accidents by 5%

from 2004 benchmark (Sanderson)
2. Achieve an overall DOT incident rate of 5.5 or less (Sanderson)
3. No work place fatalities (Sanderson)

Objective B: Reduce accidents, injuries, and fatalities among

stakeholders/partners
1. No work place fatalities (Sanderson)
2. Provide real-time information to employers regarding their
employees driving records (Tatum)
3. Increase the number of employers with access to the
Transportation (Tatum)

BSC Causal Relationships

Mission achievement
Satisfied customers
Efficient, high quality internal
processes
Skilled employees
Allocated resources

Mission
Customer
Focus
Internal
Processes
Learning &
Knowledge
Financial

Research on Scorecard Use in

Public Sector

Comprehensive range of measures covering

operations
Improved strategic planning
Helped translate strategic objectives into
improvement in operations at service level.
European Quality Award Model (similar to TQM)
compared to BSC
BSC linked better to strategy
BSC easier to adapt to customer measures
BSC easier to involve people at all levels and
to relate to specific job
BSC good link between process measures
and business strategy

The Art of Getting Things

Done

About 10% of all managers take

decisive, purposeful action.

Managers know what to do.

Managers have the knowledge.

Managers are very busy.

Managers permanently face requests.

Baruch and Ghoshal

Do not mistake activity

for achievement.
Mabel Newcomber
20th century academic and writer

Manager Actions

High
High

Detached
Detached
Managers
Managers

Purposeful
Purposeful
Managers
Managers

20%
20%

10%
10%

Procrastinators
Procrastinators

Busy
Busy
Managers
Managers

Focus

Low
Low

30%
30%

Low
Low

40%
40%

Energy

High
High

Making choices

Willpower, not motivation, drives

purposeful action

intention formation

engaging willpower

intention protection

Scorecards and Focus

Scorecards tell a story

Scorecards focus attention on critical

activity and results

Goal setting and management lead to

focus

Reticular Formation

Reticular Activating
System - RAS

Brains attention center

Filter between conscious

and subconscious

Real events and thoughts

are the same

Characteristics of a
scorecard

Strategies align with mission

Relationships among limited number of measures

are clear, provide relevant information

Scorecards tell a story about what is important and

about accountability to everyone in the organization

Scorecards balance

financial / non-financial information

short term / long term targets

lead / lag measures

Why don't you write books

people can read?
Nora Joyce to her husband James
1882-1941

Scorecards and Auditing

Scorecard as business model
Key activities in scorecard
Measures reflect key activities
Audit verifies outputs / outcomes

Using BSC to Structure an Audit

Balanced Scorecard: Audit Measures Worksheet
Perspective
Mission

Objective/Goal/Outcome

Measure

Target

Why do we exist?
Why are we doing this?

Customer Focus
To achieve our mission, what
customer needs must we meet
What must we do to satisfy,
retain and attract new

Internal Processes
To satisfy customer needs and
to achieve our mission, in what
internal business processes

Learning & Knowledge

To achieve our goals, what
must the organization know
and learn, and how must we
What must we do to develop
employee skills and technology
to continue adding value and

Financial
To satisfy our stakeholders and
to meet our mission, how will
we allocate our appropriated
How will we acquire
resources?

@work: Auditing Scorecards

City of Bedrock Balanced Scorecard
Provide responsible leadership and the services and infrastructure necessary for a safe environment and a high quality of life for our citizens

Perspective
Mission

Objective

Measure

Provide services that result in overall citizen satisfaction with

community safety
Citizens believe they have a high quality of life
City leaders are responsive to citizen needs and desires

Customer Focus

Telephone courtesy of employee - overall citizen satisfaction

Provide police services perceived as satisfactory by citizens
Phone calls directed accurately

Average score 85%

Average score 85%

Response to items on citizen survey about satisfaction with

responsiveness of leaders

Average score 80%

Response to items on citizen survey about courtesy of call

center employees
Response to items on citizen survey about satisfaction with
police services

Satisfaction with ambulance service

Response to items on citizen survey about satisfaction with

ambulance services
Number of misdirected calls per month
Internal Processes

Staff members are courteous while providing services

Phone call monitoring by supervisors

Quick response to service calls

Effectiveness in maintenance of city buildings

Target

Response to items on citizen survey about feelings of safety

Response to items on citizen survey about satisfaction with
quality of life

Average time to respond to service calls

Accuracy of response to requests

Average score 80%

Average score 80%

Average score 80%
Fewer than 5 instances of misdirected
calls in 1 month
Fewer than 5 instances of
discourteous responses by agents in 1
month
Target to be determined after baseline
established

Acquisition excellence

Learning & Knowledge

Employees received adequate training to perform efficiently

Hours of job-related training per employee annually

Employees are treated as valued customers

12 hours of job related training / yr

Employee engagement indicates overal satisfaction with the

city as an employer

Financial

Operates within budget

Budget variance measured monthly

Variance from budget is 2%

Finance reports are prepared and delivered to users timely

Number of late reports per month

No. reports delivered after due date

Finance reports are accurate and reliable

Number of errors in standard finance reports / month

No. of errors monthly

Payroll is processed accurately

Number of paychecks with errors per pay period

Payables are processed timely

There are enough employees on staff to perform services

Number of vendor payments made more than 30 days after

invoice is received

2 paycheck errors per month

% of late payments per month

@work: Auditing Scorecards

City of Bedrock Balanced Scorecard
Provide responsible leadership and the services and infrastructure necessary for a safe environment and a high quality of life for our citizens

Perspective

Audit Notes

Measure

Target

Mission
Response to items on citizen survey about feelings of safety

Average score 85%

Response to items on citizen survey about satisfaction with quality of life

Average score 85%

Response to items on citizen survey about satisfaction with responsiveness of leaders

Average score 80%

Average score 80%

Customer Focus
Response to items on citizen survey about courtesy of call center employees
Response to items on citizen survey about satisfaction with police services

Internal Processes

Response to items on citizen survey about satisfaction with ambulance services

Number of misdirected calls per month
Phone call monitoring by supervisors
Average time to respond to service calls

Learning & Knowledge

Average score 80%

Average score 80%
Fewer than 5 instances of misdirected
calls in 1 month
Fewer than 5 instances of
discourteous responses by agents in 1
Target to be determined after baseline
established

Hours of job-related training per employee annually

12 hours of job related training / yr

Employee engagement indicates overal satisfaction with the city as an employer

Financial

Budget variance measured monthly

Number of late reports per month
Number of errors in standard finance reports / month
Number of paychecks with errors per pay period

Variance from budget is 2%

No. reports delivered after due date
No. of errors monthly
2 paycheck errors per month
% of late payments per month

Number of vendor payments made more than 30 days after invoice is received

Auditing Scorecards
Review architecture and security of
e-systems
Review relevant policies and plans
Review goals and measures
Identify unintended consequences
Share information with management
Result: ensure that scorecard
processes function as intended,
emphasizing the quality and strength
of process controls and identifying
ineffective metrics

Evaluating Audit Processes &

Results

Mission

Customer Focus

Internal Processes

Learning & Knowledge

Financial

Did client/auditee receive

useful information?
Did client find the
information easy to
understand? Timely?
Project surprise free?
Was the audit work and
project management high
quality and effective?
Did audit team members
learn and grow on this
audit? Was employee
climate positive?
Did we adhere to our time
and dollar budgets?

Evaluating Audit Processes

& Results
Balanced Scorecard Measures - Audit & Review
The State Auditor's Office provides services that improve and assure the quality and accuracy of information for business decision making.

Perspective
Mission

Customer Focus

Objective
- Provide various levels of assurance

Measure
A&R-Audit Plan Completion Status

Target
85% of budget

about that: compliance, information

OPI-Useful Information Rating

Office Wide Target = 2.50, scale from 1 to 3; Project Level Target = 3.00, scale from 1 to 3

accuracy, performance achieves

Org-Fiscal Impact

Audit & Review Target = $160,000,000.00

the expected results, and systems

PL-% of Recommendations Action by Client

90% of budget

effectiveness

PL-Reports with Reviews of IT Systems

60% of budget

effectiveness

PL-Useful Information Rating

Office Wide Target = 2.50, scale from 1 to 3; Project Level Target = 3.00, scale from 1 to 3

- Balanced reporting

A&R-Reports Delivered On-Time

90% of budget

- Credible

OPI-Customer Feedback Rating

Rating of 4.0: scale from 1 to 5

- Independence

PL-Customer Feedback Rating

Average Rating of 4.00; scale from 1 to 5

- Objectivity
- Recognized expertise
- Timely service

Internal Processes

- Audit tools and technology

- Auditing

A&R-Hours Audit Staff-Assurance Services

A&R-% Projects with Complete BSC Data

- Packaging information/deliverables

OPI_A&R-Audit Duration

85% of budget

PL-Projects Addressing at least 2 of 3 SAO IT Objectives

60% of budget

PL-Projects Meeting Fieldwork Checkpoint Meeting Date

80% of budget

(i.e., production)
- Project management

74% of the budget

90% of budget

- Quality control and compliance with

standards
- Risk assessment (statewide and
project level)

Learning & Knowledge

- Positive employee climate

A&R-% of Highly Qual PMs

30% of senior staff

- Technical expertise

A&R-Attendance at Core Courses

93% of budget

- Training

A&R-Compliance w/ Training Standards - CPE Standards

100% of eligible A&R staff for the quarter

A&R-Workforce Capacity

85% of needs identified

Org-Employee Feedback Rating

Financial

Rating of 3.75; scale from 1 to 5

- Budget allocation (size of efforts)

A&R-% Audit Staff with > 100 Hours Comp Time

5% or less

- Recover cost for billable services

A&R-Audit Resources

< 3000 hours

# of productive employees (for the month) established by Direct of Audit at the beginning of each
Fiscal Year
85.00% to 100.00% of budget

A&R-Productive Workforce
F_Budget Summary

10

References Used

McAdam, Rodney and Edel ONeill. 1999. Taking a critical perspective

to the European Business Excellence Model using a balanced
scorecard approach: a case study in the service sector. Managing
Service Quality 9 (3): 191-197.
McAdam, Rodney and Timothy Walker. 2003. An Inquiry into
Balanced Scorecards Within Best Value Implementation in UK Local
Government. Public Administration 81 (4): 873892.
Nohria, Nitin, William Joyce and Bruce Roberson. 2003. What Really
Works. Harvard Business Review 81 (7).
Phillips, Jason Keith. 2004. An Application of the Balanced Scorecard
to Public Transit System Performance Assessment. Transportation
Journal 43 (1): 26-55.
Schmidt, Chris. 2005. The Drivers View. Internal Auditing 62(3): 2932.
Talbot, C. 1999. Public performance towards a new model, in
Special Issue: Quality issues in the public service. Public Policy and
Administration 14 (3): 1535.
Wong, Jeff. 2000. The Role of the Balanced Scorecard in Operational
Auditing. Internal Auditing 15 (4): 33-36.

Contact

Deborah L. Kerr, Ph.D.

deborah.kerr@heart.org
512.433.7106

11