Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                

Spanos Metrics

Download as pdf or txt
Download as pdf or txt
You are on page 1of 10

Computer Aid, Inc.

100 IT Performance Metrics


Produced by:

Nicholas Spanos
Principal, Solutions Consulting
Nick_spanos@compaid.com

Overview
The mission of an IT organization is to provide an information processing capability that benefits the
business. In order to fulfill this mission IT must provide the following services while managing costs and
prioritizing requests to optimize value:

Operate and support the infrastructure required to process, store, secure, and communicate

information.
Operate and support the business applications that process information
Provide technology consulting, training, and planning services
Employ, train, and deploy staff required to provide these services
Plan, develop/purchase, test, and implement new infrastructure or software to fix problems or

provide enhanced information processing capabilities to the business

This document identifies data elements for measuring IT performance by related categories. Some of the
metrics represent averages while others are reported in the form of a graph. By reporting these metrics on a
regular basis (monthly is the minimum recommended reporting period), trends can be observed across the
reporting periods. In many cases the trends are more important than the actual value.
Averages can hide significant problems. Some of the data elements are designed to identify significant
problems that may go unnoticed by simply reporting averages. Example: If a critical application was down
for 24 hours, this is a significant event that should be reported even though the overall average availability
for all applications was within an acceptable range.
One hundred data elements is more than a typical CIO would review. The CIOs management team should be
tracking and managing to these metrics. The highlighted metrics should be reviewed by a CIO on a regular
basis.
The following general objectives have been identified for all TI organizations. The recommended metrics are
organized according to the supported objectives. Some of the metrics are described in general terms and
must be interpreted by each organization. The intent of the metrics is more important then the specific
terminology used to describe the metric.

Ensure the availability of existing Processing Capabilities

Utilize Available Staff Resources Efficiently

Timely Response to Business Requests for New Features or Services

Ensure the successful implementation of system changes

Manage the cost of delivering IT services and optimize value

Top IT Performance Metrics

Computer Aid, Inc.

Recommended Metrics
Goal: Ensure the availability of existing Processing Capabilities
The first priority of an IT organization is to ensure the business has access to the existing processing
capability. The CIO should have access to performance and cost information for the operation and
support of existing Infrastructure and Applications.
Infrastructure Operations and Support
Infrastructure Incidents

Evaluate the number of non-desktop problems that affect more than a single user and
resolution time. Includes hardware and system software. Requires a consistent
definition for severity.

1.

Total Infrastructure Incidents (by


infrastructure type excluding
desktop incidents)

Graph infrastructure incidents by infrastructure type (network, server, etc.). Exclude


desktop problems that only affect a single user.

2.

Infrastructure Incident Resolution


Index (SLA)

Incidents resolved with SLA / Total Incidents. Graph by severity.

3.

Mean time to resolve (by severity)

Graph the mean time to resolve by incident severity

4.

Average Nbr Affected Users by


type of incident

Scope of problems Graph by Type

5.

Max Nbr Affected Users by type

Identifies worst types problems. Graph by type

6.

Maximum time to resolve (by


type)

Identifies worst types of problems.

Graph by Type

Evaluate the availability of non-desktop infrastructure during peak business hours


Infrastructure Availability
7.

% Downtime

% Infrastructure downtime (by type) during peak and non-peak business hours?
Exclude scheduled downtime.

8.

Max Downtime (Peak Hours)

What is the maximum down-time during peak hours

Infrastructure Utilization &

Slow response time and high utilization percentages indicate insufficient capacity. Low
utilization indicates too much capacity.

Performance
9.

Average System Login Time

In most systems, the login function consumes significant resources. An increase in the
average login time indicates possible system capacity issues across the system.

10. Average Response Time

Average response time has little value except as a trend. A gradual increase may
indicate under-capacity while a gradual decrease may indicate too much capacity.

11. Average Response Time (Peak


Hours)

System utilization is highest during peak hours and poor response time

12. Peak Infrastructure Utilization %

% of infrastructure utilized during peak business hours

13. Disk Utilization

Trend Graph showing total space available vs. utilized. Rapid increases in utilization
should be investigated.

14. Disk Utilization by Application


Priority

Graph the disk space utilized by the application priority. Investigate low priority
applications with high disk utilization.

Top IT Performance Metrics

Computer Aid, Inc.

Goal: Ensure the availability of existing Processing Capabilities (cont.)


Application Support
Application Problems

Evaluate trends related to application problems.


application should be defined for reporting purposes.

The business priority of each

15. Application Incidents (by


application priority)

Graph Application incidents by application priority. Lower priority applications with


large numbers of incidents are candidates for retirement.

16. Application Incidents (by incident


severity)

Graph Application incidents by incident severity. Increasing trends of severe problems


identify areas where attention is required.

17. Mean time to resolve (by incident


severity)

Decreasing trend indicates increasing knowledge and greater efficiency. Increasing


trend with decreasing incidents indicates simple problems are being fixed while the
more complex problems remain unaddressed.

18. Maximum time to resolve (by


type)

Identifies severe/complex incidents that should receive extra attention to make sure
they do not re-occur.

19. Average Nbr Affected Users (by


incident severity)

Increasing average trend indicates greater frequency of problems affecting multiple


users. Can also be beneficial for setting SLAs.

20. Max Nbr Affected Users for a


single incident

Increasing trend shows the average severity of problems is increasing. If lower severity
problems are being eliminated, this may be an acceptable trend.

21. Application Incident Resolution


Index

Total Incidents resolved within SLA / Total Incidents (by Severity).

Evaluate trends affecting application availability.


Application Availability
22. % Downtime (by application
priority)

Graph the % of downtime for peak and non-peak hours by application priority.

23. Max Peak Downtime (by


Application Priority)

Evaluate trends affecting application performance


Application Utilization &
Performance
24. Average Application Response
Time

Graph average response time by peak and non-peak by application priority

25. Average Number of Users/Day

Graph average users by internal and external users. Apps with a low number of users
should be investigated and possibly retired.

Top IT Performance Metrics

Computer Aid, Inc.

Goal: Utilize Available Staff Resources Efficiently


The first priority of an IT organization is to ensure the business has access to the existing processing
capability. The CIO should have access to performance and cost information for the operation and
support of existing Infrastructure and Applications.
Staff Resource Management
Report the following in a chart by type of FTE
Total Staff
26. Total Staff
27. Turnover by Position

Graph the turnover for the year by the type of position. Increased turnover is an
indication of management problems

28. Open Positions

The number of open positions can impact scheduled projects or support.

29. Average Duration of Open


Positions

The group is over-staffed if SLAs are being met by the smaller staff for an extended
duration.

30. Total Contractors

The number of contractors and the duration should be a strategic decision.

31. Average Length of Contractor


Assignment

Extended contractor assignments indicate the need for increased staff.

32. Ratio of Management to Staff


33. Average Staff Cost
34. Average Contractor Cost

Report the following in a chart by type of FTE


Staff Utilization
35. Available Man-Days
36. Scheduled Man-Days
37. Actual Hours by Type
38. Actual
Priority

Hours

by

Graph hours by type of work (Admin, Ops, Business Goals)


Application

Graph work hours by Application Priority

39. Actual Hours by Request Priority

Graph work hours by Request Priority

40. % hours for non-Working hours

Total Vacation, Holidays, Sick, Admin/Available Hours

Top IT Performance Metrics

Computer Aid, Inc.

Goal: Timely Response to Business Requests for New Features or


Services
Enhancements or Projects
41. Total Active Requests by Type

Graph by type

42. Total Backlog Requests by Type

Graph by type

43. Total Backlog Requests by Size

(< 100 hrs, < 500 hours, or > 501 hours)

44. Total Backlog Request Hours by


Type

Graph by type

45. Total Completed Requests

Graph by type for the reporting period

46. % Late

% of requests that are late for the reporting period

47. % Over Budget

% of active projects that are over budget

48. Cost Performance Index

Average by type (EV/AC) < 1 means cost overrun

49. Schedule Performance Index

Total On-time divided by total complete < 1 means late projects

50. Total Scope Changes

Graph by size of request

51. Average
request

Scope

Changes

per

52. Enhancement Completion Index

Top IT Performance Metrics

Graph by size of request

Total Enhancements Completed in the period / Last Year Period total

Computer Aid, Inc.

Goal: Ensure the successful implementation of system changes


The first priority of an IT organization is to ensure the business has access to the existing processing
capability. The CIO should have access to performance and cost information for the operation and
support of existing Infrastructure and Applications.
Change Management

Problem Fixes
53. Total Infrastructure Changes

Graph total infrastructure changes and total hours by type of change

54. Infrastructure Change Success


Rate

Total Successful Changes / Total Changes

55. Planned Application Changes

Planned changes to repair a problem

56. Emergency Application Changes

Total Emergency Changes

57. Application Change Success Rate

Total Successful Changes / Total Changes

Enhancements
58. Total Enhancement Changes

Graph total enhancement changes and total hours by type of change

59. Enhancement Change Success


Rate

Total Successful Changes / Total Changes

Projects
60. Total Project Changes

Graph total Project changes and total hours by type of change

61. Project Change Success Rate

Total Successful Changes / Total Changes

Change Quality
62. Total Defects Introduced from
Changes

Graph by type of change

63. Total Issues from Missed


Requirements
64. Total User Generated Tickets for
Defects

User generated tickets means the problem was undetected by IT

65. Total Work Hours to Correct

Commonly referred to as Re-work

66. % Re-Work

Total re-work hours / total hours

67. Total Downtime

Total system downtime caused by the re-work problems

68. Quality Index

(Total Defects/Total Components Changed) Separate calculations for Problem Fixes,


Enhancements, and Projects Graph

Top IT Performance Metrics

Computer Aid, Inc.

Goal: Manage the cost of delivering IT services to ensure that


maximum value and customer satisfaction are delivered from the
available resources
IT organizations have fixed budgets and capabilities and conflicting demands. IT must work with the
business units to ensure that resources are deployed to support initiatives that provide the most value to
the enterprise. The terminology used in this section is based on the Project Management Institutes
definitions of Earned Value.
Financial Management

Operations
69. Annual Budget

Planned Value

70. Earned Value

Estimated Costs Consumed to Date

71. Actual Costs to-date


72. % Variance

Actual Cost / Planned Value

73. Largest Variance Line Item

Identify the budget area that has the largest $ variance from the budgeted amount

74. Largest Variance Amount

Identify the amount of the largest variance

75. Cost Performance Index

(EV/AC) < 1 means cost overrun

Application Support
76. Annual Budget

Planned Value by Application Priority

77. Earned Value

Estimated Budget Consumed to date by Applic Priority

78. Actual Costs-to-date by


Application Priority

Graph actual costs-to-date by application priority

79. % Variance

Actual Cost / Planned Value by Application Priority

80. Cost to-date by request type

Graph costs by type of support problems, enhancements, etc.

81. Graph the Budget by Application


Priority
82. % Costs to resolve problems by
Applic Priority

Graph the % costs to resolve problems vs. total support costs by application priority.
High priority applications should have low problem costs and high enhancement costs.

Projects
83. Planned Value

Top IT Performance Metrics

Estimated Cost

Computer Aid, Inc.

84. Earned Value

Estimate Consumed to-date

85. Actual Cost to-date


86. % Variance

Actual Cost / Planned Value

87. Schedule Variance

Earned Value minus Planned Value

88. Cost Performance Index

(EV/AC) < 1 means cost overrun

89. Schedule Performance Index

(EV/PV) < 1 means late

Delivering Value
The following metrics are based on customer surveys
Customer Satisfaction
90. Infrastructure
Availability/Reliability

Business perception of the availability and reliability of the infrastructure.

91. Operations Support

Operations staff provides support to the business and to other IT organizations.


Measure the satisfaction levels relating to Operations Support.

92. Application Support

Applications support staff provides a wide variety of services including problem


resolution, consultation, and work requests such as enhancements. Measure the
satisfaction levels relating to the support provided by the applications areas.

93. Projects

Business Alignment
94. % of Hours by Business Priority

Graph by high, med, low priority

95. % Hours by Business Initiative

Graph hours by business initiative

96. Cost by Business Initiative

Graph cost by business initiative

Continuous Improvement
97. Anticipating future benefits from
technical investments

Graph the hours applied to projects that anticipate new requirements. Sample benefits
of technical innovations include: Identification of new customers, implementing
technology that attracts customers, improving external customer satisfaction, managing
costs to fund innovation, etc.

98. % Costs for Value Added Services

Total cost of value added services (enhancements vs. cost of other types of support

99. Cost of efficiency improvements

Graph total cost of efficiency improvements (by area)

100. Cost savings from efficiency


improvements

Top IT Performance Metrics

Graph Cost savings from efficiency improvements by area

Computer Aid, Inc.

Top IT Performance Metrics

Computer Aid, Inc.

10

You might also like