INT 3 Schedule Risk Analysis
David T. Hulett, Ph.D.
Hulett & Associates, LLC
ICEAA Professional Development
and Training Workshop
San Diego, CA
June 9 - 12, 2015
1
Agenda
Add uncertainty to the schedule, correlation
Demonstrate the Merge Bias
Install reference ranges of Uncertainty
Adding discrete risk events as Risk Drivers
Probabilistic Branching
Probabilistic Calendars
Inflation
Risk Prioritization
2
Introduction
USAF Approach to Schedule Risk
A Most Probable Schedule (MPS) will be
prepared by assessing the durations
presented in the offerors MIPS (this means
estimating the longest, the shortest, and the
most likely duration for each task, activity,
event, and milestone) and preparing a
network-based Monte Carlo simulation in
order to determine a schedule that has a 90%
probable completion date.
Integrated Risk Management Guide,
Aeronautical Systems Center (ASC), draft, 9 April 1994
Purpose of a Risk Analysis
Promote the language of probability and use of
its mathematics in risk analysis
Why do schedules overrun? Things do not go
according to plan
Examine elements of a project in detail,
determining relationships and formulating a
model
Most people are less able to comprehend the
whole of the problem than risk of the elements
individually
Purpose of a Risk Analysis (continued)
Risk analysis strategy
Describe the risks at the level of the activity
Use the schedule and Monte Carlo simulation to
find the overall project schedule risk
The essence is a statement of the probability
of program outcomes
Source: Risk Assessment Techniques,
Defense Systems Management College 1983
Overrun Risk is Not a New Issue
Initial cost and schedule estimates for major
projects have invariably been over-optimistic.
The risk that cost and schedule constraints will
not be met cannot be determined if cost and
schedule estimates are given in terms of single
points rather than distributions
Overrun Risk is Not a New Issue
(continued)
A formal risk analysis is putting on the table
those problems and fears which heretofore
were recognized but intentionally hidden.
Source: Final Report,
US Air Force Academy
Risk Analysis Study Team 1973
Some Reasons for Schedule Risk
Fundamental uncertainty in the work
Unrealistic baseline schedule
Natural, geological causes
Project complexity
Scheduling abuses
Relying on participants outside the organization
Subcontractor late
Some Reasons for Schedule Risk
(continued)
Design changes
Staffing Manufacturing problems
Contracting problems
Customer (government) not supportive
Cannot get subcontractor under contract
William Cashman, Why Schedules Slip
Air Force Institute of Technology (AFIT) Masters Thesis, 1995
Pitfalls in Relying on CPM
CPM network scheduling is static, not dynamic
Single-point activity durations known with certainty
OK only if everything goes according to plan
CPM durations are really probabilistic assessments
There are no facts about the future
Lincoln Moses, Statistician and Administrator of Energy Information in the US DOE
1977 Annual Report to Congress
Risk Analysis Answers
Many Questions that CPM cannot
Since the inputs are uncertain, the results are
uncertain and we need to make statistical
statements
Can address questions CPM cannot
The 3 promises
1. What is the likelihood of meeting schedule?
2. How much schedule contingency do we need to
provide?
3. Where is there risk to the project schedule?
Why Conduct Monte Carlo Simulation
It would help estimate our project if we had
data on 5,000 projects exactly like our, just
with different risks occurring
We never have such databases
Monte Carlo simulation creates this database
Uses our schedule
Uses our risks
Risks occur in different combinations, 5,000 times,
recording the results
Simplified Progression of Capabilities
Initially, just applied probability distributions to
the activity durations (e.g., PERT) to represent all
duration risk
Within the last 10 years have integrated discrete
risk events commonly found in Risk Registers
Relegated 3-point estimates to represent only
inherent uncertainty, estimating error / bias
Added capabilities such as probabilistic
branching, probabilistic calendars and
prioritization
13
Risk of an Individual Activity
Simple activity duration estimates are risky
100d
Design Unit 1
Uniform and Triangular Distributions
Uncertainty Range .9, 1.0, 1.2
Uniform
Mean 105, Sigma 8.67
Triangular
Mean 103, Sigma 6.25
BetaPERT and Normal Distributions
Uncertainty Range .9, 1.0, 1.2
BetaPERT
Mean 102, Sigma 5.54
Normal
Specify Mean and Sigma
16
Risk Along a Contiguous Schedule Path
Path risk is the combination of the risks of its activities
Start
Design
Unit
Build
Unit
Test
Test
Unit
Unit
Finish
Really Simple Schedule
This schedule finishes on September 3
7-day weeks, like a model changeover, refinery turnaround
If we can get into trouble with this simple schedule, we can
get into trouble with real project schedules
Add Duration Risk to the Schedule
using Triangular Distributions
Duration Ranges Used
Task
Scheduled
Min
Most Likely
Max
Design 1
100
90
100
120
Build 1
200
170
200
250
Test 1
50
40
50
90
This section features Polaris by Booz Allen Hamilton
What is a Simulation?
How do you find total project results?
Cannot add distributions
Must combine distributions
Combining distributions using Monte Carlo
simulation
Almost all possible combinations of durations
Perform the project many times with activity
durations varying randomly each iteration
Each iteration is a possible project, but we do not
know which is ours
Risk Analysis Tools
Monte Carlo Simulation of the schedule - 70-year old
method
A simulation is made up of many (thousands)
iterations represent a synthetic database of projects
Each iteration uses a new set of activity durations
chosen at random from the probability distributions
Iteration is just a CPM analysis using those durations
Iterate many times to reflect uncertainty in duration
estimates
Collect the data, make probability distribution of
results
CPM is not even the most likely completion date and
may not be very likely, given risks
Monte Carlo Simulation Results for
Really Simple Schedule
CPM date is not even the most likely Thats about 6/2
80% Target is
6/23
implying a
schedule
contingency
of 38 days
CPM date is about 18% Likely to be met
Offsetting Underestimation of
Duration Risk the Trigen Function
Often risk workshops or even risk interviews will
result in Minimum and Maximum ranges that are
too narrow
People tend to anchor on the schedule duration and
adjust the extremes insufficiently
Some are just inexperienced and / or timid
Use their ranges but adjust them to the
percentage of total probability they really
represent
Turn the Triangular distribution into a Trigen
distribution
23
Compare the Raw Input Distributions with
Distributions Corrected for Underestimation
The ranges on Build 2 are the same as before, but the Trigen specification is that the
range from 170d to 250d covers only 80% of the total. To cover 100% of the
probability the Min and Max have to be wider
24
Using the Trigen Function Spreads the
Tails of the Triangular Distribution
Triangular
Distribution
Trigen
Distribution
25
Presence of Estimating Bias
Often the project team members do not believe
the scheduled durations
These were once called magic numbers or
everything has to go right for the schedule to work
The Most Likely duration does not need to be
the duration in the schedule
Often happens when the finish date is specified in
advance of knowing project data
Management, the customer or competition often
impose to optimistic duration estimates
Schedule risk analysis can compensate, bring reality
26
Effect of Correcting for
Optimistically-Biased Durations
Assuming no bias
in the durations
Task
Design 1
Build 1
Test 1
Design 2
Build 2
Test 2
Recognizing bias
in the durations
Duration Ranges Used
Scheduled
Min
Most Likely
Path 1 Unbiased Estimates
100
90
100
200
170
200
50
40
50
Path 2 Correcting for Optimistic Estimates
100
90
110
200
170
215
50
40
60
Max
120
250
90
130
270
100
27
Uncorrelated Uncertainty
Independent duration ranges applied to the
activities on the same path will exhibit some
cancelling out some will be long in their
distributions in the same iterations while
others may be short or middling duration
If someone says that the range for the path is,
say, + 10% and 5%, applying these ranges
to the activities will result in a much narrower
total path uncertainty
28
Correlated Uncertainty
To cause the + 10% and = 5% range for the
entire path we need to specify a correlation
between the activities durations of 100%
(perfect positive correlation) between each
pair of durations
29
Comparison with and without
Correlation = 1.0
Schedule Risk Results Correlation
Schedule
20%
80%
NO Correlation
18-May-15 23-Jun-15
100% Correlation
13-May-15 27-Jun-15
30
Risk at Merge Points:
The Merge Bias
Many parallel paths merge in a real schedule
Finish driven by the latest converging path
Merge Bias has been understood for 40
years
Design Unit 1
Build Unit 1
Test Unit 1
Start
Finish
Design Unit 2
Build Unit 2
Test Unit 2
Inserting Uncertainty in the Schedule
Task
Design 1
Build 1
Test 1
Design 2
Build 2
Test 2
Duration Ranges Used
Scheduled
Min
Most Likely
100
90
100
200
170
200
50
40
50
100
90
100
200
170
200
50
40
50
Max
120
250
90
120
250
90
32
Results for One Schedule with
Two Paths The Merge Bias
The Most Likely is now 13 June, not 2 June
The P-80 is
now 2 July
not 23 June
The likelihood of meeting 16 May is now
3.3%, was 18.3%
33
Modeling The Merge Bias
One Path No Merge
Two Paths Merging
Schedule Risk Results
Schedule
Scheduled date Pr (date)
5%
50%
80%
One Path
16-May-15
18% 3-May-15 4-Jun-15 23-Jun-15
Two Path Merge
16-May-15
3% 20-May-15 16-Jun-15 2-Jul-15
95%
11-Jul-25
17-Jul-15
34
Introducing the Gas Production
Platform Schedule
35
Applying Different Uncertainty to
Categories of Tasks as Reference Ranges
Each category of activity may have different levels of uncertainty, called
reference ranges
36
Risk on the Offshore Gas Production
Platform - Reference Range Uncertainties
With Uncertainty by category of
task representing:
Inherent variability
Estimating error
Estimating bias
The CPM date is 20 March 2017
The P-80 date is 7 August 2017
for a contingency just with
Uncertainty of 4 months
This is very likely irreducible.
37
Introducing the Risk Driver Method for Causing
Additional Variation in the Simulation
Using the simple 2-path schedule. Four risks are specified. The first is a general risk
about engineering productivity, which may be under- or over-estimated, with 100%
probability. It is applied to the two Design activities
38
100% Likely Risk Drivers
Effect on Design Duration
With a 100% likely risk the
probability distribution of
the activitys duration looks
like a triangle. Not any
different from placing a
triangle directly on the
activity
39
Risk Driver with
Risk at < 100% likelihood
With this risk, the Construction Contractor may or may not be familiar with the
technology, the probability is 40% and the risk impact if it happens is .9, 1.1 and 1.4. It is
applied to the two Build activities
40
With a 40% Likelihood, the Spike in the
Distribution Contains 60% of the Probability
Here is where the Risk
Driver method gets
interesting. It can create
distributions that reflect:
Probability of occurring
Impact if it does occur
Cannot represent these
two factors with simple
triangular distributions
applied to the durations
directly
41
Risk Drivers Models
how Correlation Occurs
Correlation can be caused by identifiable risks
that are assigned to two different activities
If the risk occurs it occurs for each activity
If the risk impact multiplier is X% it is X% for each
activity
We are not very good at estimating correlation
coefficients, so generating them within the
simulation is a better approach
There still may be correlations among uncertainty
(3-point estimates)
42
Risk Drivers Generate Correlation
between Activities (1)
Risk 1: Probability 100%
Impact .9, 1.05, 1.3
Activity 1
Activity 1
Correlation (Activity 1, Activity 2) = 100%
43
Risk Drivers Generate Correlation
between Activities (2)
But there is no such thing as 100% correlation
Risk 1: Probability 100%
Impact .9, 1.05, 1.3
Activity 1
Activity 1
Uncertainty Not
Correlated: .85, 1, 1.2
Adding uncorrelated uncertainty reduces
correlation (Activity 1, Activity 2) to 86%
44
Risk Drivers Generate Correlation
between Activities (3)
Risk 1: Probability 100%
Impact .9, 1.05, 1.3
Risk 2: Probability 40%
Impact .9, 1.1, 1.4
Activity 1
Risk 2: Probability 65%
Impact .9, 1.15, 1.5
Activity 1
Correlation (Activity 1, Activity 2) = 64%
(without uncertainty)
45
Activities Can be Influenced
by More than One Risk Driver
An Organizational Risk has been added to the mix, assigned to all
activities in the Offshore Gas Production Platform schedule
46
Adding Organizational Risk
to Every Activity
With all risk Drivers
including the
Organizational Risk the
P-80 result is 11 February
2018
With Uncertainty the P80 was 7 August, 2017
The scheduled date is 20
March 2017
47
Risk Drivers can be Applied
In Series or In Parallel
Two or more risks can be applied to the same
activities. If they occur together in an
iteration they may be in parallel or in series
We are talking about the impacts of a risk that
has occurred
Risk takes most resources or is so important to be
addressed that recovering from others must wait
are entered in series (we have been assuming this)
Risks can be recovered from simultaneously can
be entered in parallel
48
Parallel and Series Risks
Multiplicative with Risk Drivers
If these two risks cannot be recovered from
simultaneously, they are entered in series
Risk 1 1.2 factor
Risk 2 1.05 factor
Use (1.2 x 1.05 = 1.26)
Factor, multiply the two
If recovery from two risks can be accomplished
simultaneously, they are entered in parallel
Risk 1 1.2 factor
Risk 2 1.05 factor
Use 1.2 Factor, the largest
factor, only
Changing the Risks to In Parallel
Reduces the Schedule Risk
Putting the Risk Drivers in
Parallel results in an earlier P80 of 28 December 2017
Had been 11 February 2018
with Risk Drivers in series
In this model it puts the
Organizational risk in parallel
with all the others when it
occurs (65% likely)
This capability is more
important if more risks are
assigned to the same activities
50
Failing the Test may lead to Multiple
Activities that are Not In the Schedule
If the test fails we may need to do:
Examine the Root Cause of the failure
Determine what to do next
Do what is needed to be done to recover
Re-test the article
All of these activities need to be done, or
none is needed
These 4 activities constitute a probabilistic branch,
since the possibility of doing them is probabilistic
(C) 2014 Hulett & Associates, LLC
51
Set up the Probabilistic Branch
We create a 4-activity probabilistic branch, adding 4 activities: Root Cause Analysis, Plan
the recovery, Execute the Plan and Retest
Notice that they all have a remaining duration of 0 working days they will not affect the
schedule unless they occur
(C) 2014 Hulett & Associates, LLC
52
Give the New Activities Ranges of
Impact, if they Happen
Highlight the new activities in turn an give them uncertainties:
Root Cause Analysis 20d 40d 60 d
Plan the Recovery 10d 20d 30d
Execute the Plan 10d- 30d- 50d
Retest 20d 30d 50d (probably less time than the first test)
(C) 2014 Hulett & Associates, LLC
53
With the Probabilistic Branch in Place,
Results may show Bi-modal Distribution
Probabilistic branch develops a
shoulder at 60%
There can be more than one
probabilistic outcome from a node. The
probabilities need to sum to 100%.
Probabilistic branch can represent
more planning than just a single
probabilistic activity
(C) 2014 Hulett & Associates, LLC
54
Probabilistic Calendars (1)
In many applications a weather-related event can
impede progress and affect the schedule without
regard to the dates permitted by predecessors
Monsoon weather can impede installation of jackets and
topsides for offshore platforms
Freezing weather can stop supplies getting in to a jobsite
Thawing ground may make it difficult to move equipment
Other calendar-related events can be important for the
schedule
Activities such as moving into a building can be
determined by calendar of events
(C) 2014 Hulett & Associates, LLC
55
Probabilistic Calendars (2)
The static schedule might show that the
weather-sensitive activity will take place
outside of the weather window
However, with schedule risk the date of the
weather-sensitive activity is uncertain
Risk on predecessors might push an otherwise
safe activity into the weather window, causing
it to be unlikely to occur
(C) 2014 Hulett & Associates, LLC
56
Simplify Probabilistic Calendar
by using Two Path Schedule
Using the 2-Path schedule. There are no risks on this schedule
(C) 2014 Hulett & Associates, LLC
57
Set Probability of
Winter Weather Event to 50%
(C) 2014 Hulett & Associates, LLC
58
Winter Weather Event only 50% Likely
Build 1 may finish in March 2015
but may be delayed by a month
because of the 50% likely January
weather event
(C) 2014 Hulett & Associates, LLC
59
Monsoon Calendar
Prohibits Offshore Installation (1)
(C) 2014 Hulett & Associates, LLC
60
Monsoon Calendar
Prohibits Offshore Installation (2)
Notice that the installation activities are, mostly,
nominally before the monsoon season that
occurs November - February
Exception to this is that the last Installation activity,
CPP Topsides, is already scheduled to complete in
November
With schedule risks on predecessors the other
installation activities will occur during monsoon
This calendar might also affect some pipe laying
activities
(C) 2014 Hulett & Associates, LLC
61
Fixed Window of Non-Work
> Add Monsoon Calendar > Apply to Installation tasks > Window > add the impact and dates >
Enable this calendar
(C) 2014 Hulett & Associates, LLC
62
Effect of Probabilistic Calendar on
Install Drilling Jacket
Notice that the period from
November to the end of February
represents no completion because of
the calendar. Predecessor activities
have usual histograms
(C) 2014 Hulett & Associates, LLC
63
Start and Finish of
Monsoon Window is Uncertain (1)
(C) 2014 Hulett & Associates, LLC
64
Start and Finish of
Monsoon Window is Uncertain (2)
Notice that the window of finish
dates is now narrower because the
start and finish of the Monsoon
season is uncertain
(C) 2014 Hulett & Associates, LLC
65
Risk Prioritization Method
Risks should be prioritized through the project
schedule and the Monte Carlo simulation
method to inform the risk mitigation exercise
For management we need to identify those
risks by days saved if they were fully
mitigated so management can do benefit/cost
For management we should identify days
saved at the target level of certainty, say P-80
(C) 2014 Hulett & Associates, LLC
66
Two Approaches to Risk Prioritization
using Quantitative Methods
Typical Tornado Diagram with Risks (not
activities or paths) as the arguments help to
prioritize the risks
However, with the structure of the schedule
the Tornado Diagram is instructive but not
definitive
The order of the risks importance can change
when one is removed, since that exposes other
paths that were risk slack paths before
(C) 2014 Hulett & Associates, LLC
67
Use the Offshore Gas Production
Platform Project
This project has reference ranges by category and 8 Risk Drivers
68
Standard Sensitivity Tornado
This standard Duration
Tornado chart shows
activities in order of
their correlation with
finish date.
These are not risks, and
risks cannot be teased
out of these results,
Also, these bars are
sorted by correlations
rather than days, and
correlations are hard to
understand
(C) 2014 Hulett & Associates, LLC
69
Criticality Tornado Diagram based on
Percent of Iterations on the Critical Path
Using the Criticality
Index.
These are also not risks
but activities sorted by
their percentage of
iterations on the critical
path
Shows which paths are
the most likely to delay
the project, so new
information, but not
risks
(C) 2014 Hulett & Associates, LLC
70
Risk Tornado Highlights Risks, Not
Activities or Paths
This special risk-based
tornado diagram focuses
on the entire impact of the
risks, including their
probability, impact range
and the activities to which
they are assigned
Still, based on correlation
concepts
(C) 2014 Hulett & Associates, LLC
71
Risk Prioritization Approach
Identify the level of uncertainty desired (P-80)
Simulate the schedule as many times as there are
risks (For the first risk this is = single pass
method)
Then identify the risk that saves the most days
when it is eliminated
Eliminating the risk (probability = 0 or disabled)
represents complete mitigation, an ideal but
impractical goal
Once the most impactful risk is identified and
eliminated, look for the second most-important risk,
disable it, then look for the third risk.. This is the
Iterative method
(C) 2014 Hulett & Associates, LLC
72
Run Prioritization
View the Risk List in Priority Order
Risks are prioritized by days saved if
they were completely disabled in the
right order
Do not spend much time mitigating
those lower on the list before handling
those at the to p
(C) 2014 Hulett & Associates, LLC
73
Prioritized Risks in a Table
Gas Platform-1 - Risk Prioritization (80%)
UID Name
Days Saved
8 Organizational Risk
4 Fabrication Risk Driver
2 Engineering Risk Driver
7 HUC Risk Driver
3 Procurement Risk Driver
6 Installation Risk Driver
1 Approval Risk Driver
5 Drilling Risk Driver
Total Days Saved from Risk Drivers
Uncertainty Days Saved
Total Schedule Contingency
106
37
16
16
11
4
0
-2
188
140
328
The Grid view shows the risks in priority order, determined by Monte Carlo
Simulation of the project schedule, with their Days Saved.
(C) 2014 Hulett & Associates, LLC
74
INT 3 Schedule Risk Analysis
David T. Hulett, Ph.D.
Hulett & Associates, LLC
ICEAA Professional Development
and Training Workshop
San Diego, CA
June 9 - 12, 2015
75
