Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures

BSS Nexus Global © 2013April 2013
Why Meaning Matters: CSFs, KPIs,
Metrics, Outcomes and Benefits
Larry Cooper
Senior Partner, BSSNexus

Why meaning matters...
“There’s a difference between...
knowing you’re sh*t
and...
knowing your sh*t”

Meaning
• Noun 1. word meaning - the
accepted meaning of a word
 the meaning of a word or
expression; the way in which a
word or expression or situation can
be interpreted; "the dictionary gave
several senses for the word"; "in the
best sense charity is really a duty";
"the signifier is linked to the
signified"

Agenda
Performance Reference Model
Outputs and Outcomes
Value versus Benefits
Critical Success Factors
Key Performance Indicators
Metrics and Measures

Performance Reference Model (PRM)
Effectiveness
Modified version of - http://agict.gov.au/policy-guides-procurement/australian-government-architecture-aga/aga-rm/2-reference-model-overviewnce
Fixed Assets
Efficiency
Technology
Money
People
Data and
Information
Projects
Ad-hoc Tasks
Processes
Products
Services
Service
Outcomes
Business
Outcomes
INPUTS WORK OUTPUTS OUTCOMES
Facilities
Where I am Where I want to go

Measurement Domains
• The Inputs domain contains fixed assets, technology, people,
data and information, and finances (the types of input)
• The Work domain contains ad hoc tasks, projects and
processes and operations (business as usual) — the types of
work
• The Outputs domain contains Products and Services (the types
of output)
• The Outcomes domain contains Service Outcomes and Business
Outcomes (the types of outcome)
Modified version of - http://agict.gov.au/policy-guides-procurement/australian-government-architecture-aga/aga-rm/2-reference-model-overviewnce

Outputs vs. Outcomes
• Outputs are important products, services, profits, and revenues: the What.
• Outcomes create meanings, relationships, and differences: the Why.
• Outputs, such as revenue and profit, enable us to fund outcomes; but
without outcomes, there is no need for outputs.
See more at: http://www.innovationexcellence.com/blog/2013/04/02/its-not-just-semantics-managing-outcomes-vs-
outputs/#sthash.ZB18O0YY.dpuf

What is an Outcome?
• Outcome:
 "something that follows as a result or consequence“ of some action(s)
 An outcome involves an intentional change being imposed on the system (people,
processes, technology), with a resulting end state that can be measured
• Outcome Management:
 the set of activities for the planning, managing, and realizing of the desired
outcomes from initiatives
• Synonyms for Outcome:
 Aftermath, Consequence, Results
International Organization for Collaborative Outcome Management
http://www.iocomsa.org/node/6
• Answers the question “what difference does it make?”

Origin of Outcomes (Management)
• The use of the term to describe the planned consequences from activity originated with
Florence Nightingale who introduced outcome indicators to healthcare
• Later used in “outcomes-based education” and in program management “logic models”
• Fundamentally, it is a quality management approach focused on the “customer”
 Ian Clayton’s USMBOK® calls this an “outside-in” approach
Patient Care Process Outcomes
Care Team
http://books.google.ca/books?id=a8RftMjbxcQC&pg=PA20&lpg=PA20&dq=%22history+of+outcome+management%22&source=
bl&ots=brzKFAXusw&sig=KG6MMhW3krR4LcHB6sYaFNhAaV8&hl=en&sa=X&ei=TVIjUq26FIa5igKe5oCoBQ&ved=0CDsQ6AEwAQ
#v=onepage&q=%22history%20of%20outcome%20management%22&f=false

PRM to Logic Model
Inputs
Activities
Outputs
Outcomes
Goals
Certain resources and people are needed to do your work
IF you have adequate resources and people,
THEN you can use them to accomplish your planned
activities
IF you accomplish your planned activities,
THEN you will deliver a certain amount of product
and/or services
IF you accomplish your planned activities to the extent you
intended,
THEN your customers will benefit in certain ways
IF these benefits to customers are achieved,
THEN certain changes in people,
organizations, or systems would be expected to occur
Benefits

Why Outcomes Management?
Not how many worms the bird feeds its young, but how well the fledgling flies
If we start with the desired ends (i.e. outcomes) in mind
Then we can work backwards towards the means to achieve them
Inputs
Activities
Outputs
Outcomes
Goals
Certain resources and people are needed to do your
work
IF you have adequate resources and people,
THEN you can use them to accomplish your
planned activities
IF you accomplish your planned activities,
THEN you will deliver a certain amount of product
and/or services
IF you accomplish your planned activities to the
extent you intended,
THEN your customers will benefit in certain ways
IF these benefits to customers are achieved,
THEN certain changes in people,
organizations, or systems would be expected
to occur
Benefits

Outcomes Questions
1. What is the current situation that we want to affect?
2. What will it look like when we achieve the desired solution or
outcome?
3. What behaviours need to change for that outcome to be
achieved?
4. What knowledge or skills do people need before the behaviour
will change?
5. What activities need to be performed to cause the necessary
learning?
6. What resources and people will be required to achieve the
desired outcome?

Incident Management (IM) Example
• Goal Statement:
 Minimize the effect of outages by restoring service as quickly as
possible so that we can maintain our customers
• Outcome(s) Statements:
 Business Outcome: Minimize Incident Outage Times…by improving the
accuracy and completeness of Incident resolution details to create an
Incident Knowledge Base thereby increasing the first point of contact
(FPOC) resolution rates
 Learning Outcome: Improve Service Desk and Support Staff knowledge
of how to accurately and completely record Incident Resolution details
for use in resolving future Incidents of this type
 To achieve business outcomes we always start with people…

POro
Process
Model/Tools
Measures Metrics
Outcomes
KPIs
Goal
CSFs Business Outcome:
Minimize Incident
Outage Times
Minimize the effect of
outages by restoring
service as quickly as
possible so that we
can maintain our
customers
Learning Outcome:
Improve Service
Desk and Support
Staff knowledge
Benefits

Agenda
Performance Reference Model
Outputs and Outcomes
Value versus Benefits
Processes and Tooling

Value and Benefits
• Benefits are measurable improvements perceived to be of value by one or
more of the stakeholders (Rajegopal, McGuin, & Waller, 2007, p. 209;
Venning, 2007)
• Bannister (2001) made a distinction between value and benefit in the
following way:
 Value is what we perceive; benefit is what we receive
 Benefits can be thought of as an operationalization of the value construct
• Benefits can be tangible or intangible
 Tangible benefits are often classified as financial or non-financial
• The benefits realizaiton approach: ‘The Information Paradox’,
John Thorpe, 1997 – precursor to ValIT from ISSACA

Benefits Statement Qualifiers
• Increased
• Enhanced
• Strengthened
• Improved
• Reduced
• Lowered
• Eliminated
• Maximized
• Minimized
• Avoided
• ….
Implies a directional change

IM Example
• Outcome:
 Minimize Incident Outage Times
• Perceived customer/client value:
 IT Support Staff are more effective in providing a stable operating
environment
• Benefits:
 Increased Customer Satisfaction (Non-tangible)
 Reduced costs of Incidents (Tangible – Financial)
 Improved Availability of Business Systems and Applications (Tangible –
Financial)

POro
Process
Model/Tools
Measures Metrics
Outcomes
KPIs
Goal
Minimize Incident
Outage Times
possible so that we
can maintain our
customers
Learning Outcome:
Improve Service
Desk and Support
Staff knowledge
Benefits
Increased Customer
Satisfaction
Reduced costs of
Incidents
Improved Availability
of Business Systems
and Applications

• Earliest reference: 10 problems that worry presidents. By:
Spencer, Lyle M.. Harvard Business Review, Nov/Dec55, Vol. 33
Issue 6, p75-83, 9p
 “What are the essential factors that produce success in my company?”
• RH Daniel, 1961, HBR article identified need for “success factors”
that enable the success of the business – he said there should be
at most 3 to 6 such factors to focus on
• Expanded by Rockhart, 1979 to “the limited number of areas in
which satisfactory results will ensure successful competitive
performance for the individual, department or organization. CSFs
are the few key areas where ‘things must go right' for the
business to flourish and for a manager's goals to be attained”

CSF Definition (Rockhart 81)
• key areas of activity in which favorable results are absolutely
necessary to reach goals
• key areas where things must go right for the business to flourish
• “factors” that are “critical” to the “success” of the organization
• key areas of activities that should receive constant and careful
attention from management
• a relatively small number of truly important matters on which a
manager should focus attention

CSF Hierarchy
Industry CSFs
Organizational
CSFs
Division-level
CSFs
Operational
Unit-level CSFs
Individual CSFs
Influence Drive
Drive
Are
supported
by

Other CSF Influencers
CSFs
Temporal
Environmental
The Industry
Managerial
Position
Competitive
Strategy

Some Sample CSFs?
• ITIL® book(s) examples of CSFs:
 A good service desk
 Integrated support tools to drive and control the process
 Manage availability and reliability of IT service
• Recent TSO book on Metrics – CSF examples:
 Incident management plans, policies
 Maintaining customer satisfaction
 Resolving incidents within defined timelines
 Documentation of suggestions to other processes
• In both cases, they did not establish clear linkages to “the Why”
question

IM Example
• Outcome:
 Minimize Incident Outage Times
• Perceived customer/client value:
 IT Support Staff effectiveness in providing a stable operating environment
• Benefits:
 Increased Customer Satisfaction (Non-tangible)
 Reduced costs of Incidents (Tangible – Financial)
 Improved Availability of Business Systems and Applications (Tangible –
Financial)
• CSFs:
 IT Service Desk and Support staff trained and knowledgeable in how to
accurately and completely record Incident Resolution details for use in
resolving future Incidents of this type
 Service Desk tool satisfies defined technical and process capabilities for
Incident Management and integrated Knowledge Management and is
accessible to both Service Desk and other levels of support as needed

POro
Process
Model/Tools
Measures Metrics
Outcomes
KPIs
Goal
Minimize Incident
Outage Times
possible so that we
can maintain our
customers
IT Service Desk
and Support staff
trained and
knowledgeable
Learning Outcome:
Improve Service
Desk and Support
Staff knowledge
Benefits
Increased Customer
Satisfaction
Reduced costs of
Incidents
of Business Systems
and Applications
Service Desk Tool
satisfies business
and
technical
capabilities
Knowledge base
For Incident
Management in
place

• First known reference: BusinessWeek article “Corporate ‘War Rooms’ Plug
Into the Computer” 1976
• Rockart, 1979 described a key indicator system based on three concepts:
1. The selection of key indicators to represent the health of the organization
2. Exception reporting or, in other words, the ability to present only those indicators
where performance was considerably different from expected results
3. The visual display of that information
• A KPI has a lifetime and requires continuous updating. Sometimes, its
replacement is also needed (Ghalayini & Noble, 1996)
• A KPI is a key part of a measurable objective, which is made up of a
direction, the KPI statement, benchmark or target and a timeframe
KPIs may need to change over time

KPI Types
• Lagging Indicator
 Measures focusing on results at the end of a time period, normally characterizing historical
performance. Also referred to as Key Results Indicator (KRI). The importance of a lagging indicator is
its ability to confirm that a pattern is occurring or about to occur.
• Leading Indicators:
 Measures that “drive” or lead to the performance of lag measures; normally measuring intermediate
processes and activities. These types of indicators signal future events. Think of how the amber
traffic light indicates the coming of the red light.
• Leading indicators are activities that should be trended as they predict the outcomes
(i.e. lagging indicators). Quotas or goals should only be placed on lagging indicators and
never on leading indicators. Placing a goal on a leading indicator will result in gaming
and generate the wrong results.

Leading versus Lagging Indicators
Leading Lagging
Advantages • Predictive nature allowing
adjustments based on results so far
• Normally easy to identify and capture
Challenges • May prove difficult to identify and
capture
• Historical in nature and not reflective
of current activities
• Lacks predictive power
IM Examples • % of Incidents solved at FPOC
• % of Severity 1 incidents not
worked on in last four 2 hour
blocks
• % of incidents dispatched more
than 2 times over each of previous
five business days
• Average backlog of incidents per
agent over each of previous five
business days
• % of Incidents at Severity 1 in last 30
days
• % of calls that are Incidents in last
quarter
• % of open incidents older than 1 day
by Severity level
http://www.icmi.com/files/ICMILeading_LaggingIndicatorsExplained.pdf

IM Example
• CSFs:
1. IT Service Desk and Support staff trained and knowledgeable in how to
accurately and completely record Incident Resolution details for use in
resolving future Incidents of this type
2. Service Desk tool satisfies defined technical and process capabilities
for Incident Management and integrated Knowledge Management and is
accessible to both Service Desk and other levels of support as needed
• KPIs:
1. % of Incidents resolved at First Point of Contact in each of previous 5
reporting periods
2. % of Incidents where prior Incident Resolution details were used to
resolve Incident by Level 2 support
3. >70% of Incidents solved at FPOC by end of first year

POro
Process
Model/Tools
Measures Metrics
Outcomes
% of Severity 1
incidents not
worked on in
last four 2 hour
blocks
KPIs
Goal
CSFs
>70% of Incidents
solved at FPOC
Business Outcome:
Minimize Incident
Outage Times
possible so that we
can maintain our
customers
>98% of Severity 1
Incidents meet
SLA Targets
IT Service Desk
and Support staff
trained and
knowledgeable
Learning Outcome:
Improve Service
Desk and Support
Staff knowledge
Benefits
Increased Customer
Satisfaction
Reduced costs of
Incidents
of Business Systems
and Applications
100% of SD staff
trained
on Incident
Management
Processes
Service Desk Tool
satisfies business
and
technical
capabilities
% of Incidents
resolved at First
Point of Contact
in each of
previous 5
reporting periods
Knowledge base
For Incident
Management in
place

http://www.metrum.org/measures/metrics.htm
First references to measurement are from writings dating to 2900 BC Mesopotamia

Danger of “Industry Standards”

Metrics versus Measures
• Measurement:
• Process of experimentally obtaining one or more quantity values that
can reasonably be attributed (e.g. # of Incidents)
• Metric:
 Describe a quality and require a measurement baseline (% of Incidents
at Severity 1 as a result of a Change)
http://samate.nist.gov/index.php/Metrics_and_Measures.html
https://cio.gov/performance-metrics-and-measures/

House of Quality Metrics
http://www.mit.edu/~hauser/Papers/Hauser-Katz%20Measure%2004-98.pdf

Good Metrics
• Reinforce desired behaviour
• Measure results (i.e. outcomes) – not just output or compliance
• Measure trends
• Support both leading and lagging indicators
• Focus on the vital few
• Need to be collectable
• Fuel meaningful conversations with customers and employees
• May measure value to the customer in the form of Benefits that
signal Outcomes achievement or it can be related to the quality
of a process
• Are of good enough quality
• May need to change as we get better

POro
Process
Model/Tools
Measures Metrics
Outcomes
% of Severity 1
incidents not
worked on in
last four 2 hour
blocks
# of Level
2 support staff
trained
KPIs
# of Severity 1
Incidents by day
Goal
% of Incidents at
Severity 1
CSFs
>70% of Incidents
solved at FPOC
Business Outcome:
Minimize Incident
Outage Times
possible so that we
can maintain our
customers
>98% of Severity 1
Incidents meet
SLA Targets
IT Service Desk
and Support staff
trained and
knowledgeable
Learning Outcome:
Improve Service
Desk and Support
Staff knowledge
Benefits
Increased Customer
Satisfaction
Reduced costs of
Incidents
of Business Systems
and Applications
100% of SD staff
trained
on Incident
Management
Processes
Service Desk Tool
satisfies business
and
technical
capabilities
% of Incidents
resolved at First
Point of Contact
in each of
previous 5
reporting periods
% of open Incidents
by date
% of Service Desk
Staff Trained
# of Severity 1
Incidents
solved at FPOC
Knowledge base
For Incident
Management in
place

Processes and Tooling
• Our processes (the What) need to enable what we are trying to
achieve (the Why)
• Our tools need to enable our processes (the How)
• IF we don’t the know the Why, THEN developing the What and
the How are likely must less effective, and may in fact be moot

POro
Process
Model/Tools
Measures Metrics
Outcomes
% of Severity 1
incidents not
worked on in
last four 2 hour
blocks
# of Level
2 support staff
trained
KPIs
Knowledge Management
Incident Management
Escalation Model
Service Desk Tool
# of Severity 1
Incidents by day
Goal
% of Incidents at
Severity 1
CSFs
>70% of Incidents
solved at FPOC
Business Outcome:
Minimize Incident
Outage Times
possible so that we
can maintain our
customers
>98% of Severity 1
Incidents meet
SLA Targets
IT Service Desk
and Support staff
trained and
knowledgeable
Learning Outcome:
Improve Service
Desk and Support
Staff knowledge
Benefits
Increased Customer
Satisfaction
Reduced costs of
Incidents
of Business Systems
and Applications
100% of SD staff
trained
on Incident
Management
Processes
Service Desk Tool
satisfies business
and
technical
capabilities
% of Incidents
resolved at First
Point of Contact
in each of
previous 5
reporting periods
% of open Incidents
by date
% of Service Desk
Staff Trained
# of Severity 1
Incidents
solved at FPOC
Knowledge base
For Incident
Management in
place

From Measures to Outcomes?
Measures
Metrics
KPIs
CSFs
Outcomes/ Benefits
Data
Collection
Metrics
Computation
Metrics
Evaluation
Indicator
Evaluation
Outcomes
Evaluation/
Benefits
Realization
Project view
In-operations view

Outcomes

To get to good Outcomes
1. Listen to the voice of the customer (use “outside-in” thinking)
2. Understand the contributors to outcomes and what drives them
(voice of the employee in the processes and work)
3. Understand the relationships (from Outcomes to Measures – and
back) as well as the linkages between the customer and the
employee (1 and 2)
4. Measure actual outcomes and benefits – not just compliance
Adapted from http://www.mit.edu/~hauser/Papers/Hauser-Katz%20Measure%2004-98.pdf

BSSNexus Model

To summarize
 We need to think “outside-in”
 Outcomes are what we are trying to achieve as indicators of our
goal
 Value is what our customers perceive; benefit is what they receive
 CSFs are the few key areas where ‘things must go right’
 KPIs are simply metrics that are tied to a target
 A KPI is a key part of a measurable objective, which is made up of
a direction, the KPI statement, benchmark or target and a
timeframe
 Outcomes, CSFs, KPIs, Metrics and Measures may need to change
over time!

The final word

For more information
Larry.Cooper@BSSNexus.com
Larry.Cooper@AthenaNexus.com
ca.linkedin.com/lawrencekcooper
Twitter: @coopelk99
Skype Id: Larry.Cooper
+1 (613) 868-0982 (cell)
+1 (888) 316-2745
BSSNexus Global Inc.
AthenaNexus Inc.

Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures (20)

More from AdaptiveOrg Inc. (10)

Recently uploaded (20)

Why meaning matters - Outcomes, Benefits, CSFs, KPIs, Metrics and Measures