CHAPTER EIGHT

EVALUATING AND PROJECT

CLOSURE
 Cont’d...
 The term “ evaluate ” means to set the value of or
appraise.
 A project evaluation appraises the progress and
performance relative to the project ’ s initial or
revised plan.
 The evaluation also appraises the project against
the goals and objectives set for it during the
selection process amended, of course, by any
changes in the goals and objectives made during
the project ’ s life.
Evaluation criteria

 There are many different measures that

may be applied in a project evaluation. As
indicated, senior management may have
particular areas they want evaluated for
future planning and decisions, and these
should be indicated in the charge to the
evaluation committee.
Cont’d...
 A project evaluation is an appraisal for use by
top management. Its criteria should include the
needs of management; the organization ’ s
stated and unstated goals; the original selection
basis for the project; and its success to date in
terms of its efficiency, customer
impact/satisfaction, business success, and future
potential.
Cont’d…

 Measuring the project ’ s success on

budget, schedule, and performance is
easier than measuring revenues or
qualitative, subjective factors.
 Establishing the measures at project
formation is helpful, as well as using
carefully standardized measurement
techniques for the subjective factors.
Purposes of Evaluation - Goals of the System

 Four independent dimensions of success:

 The most straightforward dimension is the project’s
efficiency in meeting both the budget and schedule
 Another dimension, and the most complex, is that of
customer impact/satisfaction
 A third dimension, again somewhat straightforward
and expected, is business/direct success
 The last dimension, somewhat more difficult and
nebulous to ascertain, is future potential
Cont’d…
 Another primary purpose of evaluation is to help
translate the achievement of the project’s goals
into a contribution to the parent organization’s
goals
 To do this, all facets of the project are studied in
order to identify and understand the project’s
strengths and weaknesses
 The result is a set of recommendations that can
help both ongoing and future projects
Cont’d…
 A successful project evaluation can help
an organization:
 Identify problems earlier
 Clarify performance, cost, and time relationships
 Improve project performance
 Locate opportunities for future technological
advances
 Evaluate the quality of project management
 Reduce costs
Cont’d…
 Speed the achievement of results
 Identify mistakes, remedy them, and avoid them in
the future
 Provide information to the client
 Reconfirm the organization’s interest in, and
commitment to, the project
The Project Audit

 The project audit is a thorough examination of

the management of a project, its methodology
and procedures, its records, its properties, its
budgets and expenditures and its degree of
completion
 The formal report may be presented in various
formats, but should, at a minimum contain
comments on some specific points.
Cont’d…
 Six parts of a project audit:
 1. Current status of the project
 2. Future status
 3. Status of crucial tasks
 4. Risk assessment
 5. Information pertinent to other projects
 6. Limitations of the audit
 It is far broader in scope than a financial audit
and may deal with the project as a whole or any
component or set of components of the project
Depth of the Audit
 Time and money are two of the most common
limits on depth of investigation and level of
detail presented in the audit report
 Accumulation, storage, and maintenance of
auditable data are important cost elements
 Two often overlooked costs are the self
protective activity of team members during an
audit, and the potential for project morale to
suffer as a result of a negative audit.
Cont’d…
 There are three distinct and easily recognized
levels of project auditing:
 General audit - normally most constrained by time
and resources and is usually a brief review of the
project touching lightly on the six parts of an audit
 Detailed audit - usually conducted when a follow-
up to the general audit is required
 Technical audit - generally carried out by a
qualified technician under the direct guidance of the
project auditor
Timing of the Audit
 The first audits are usually done early in the
project’s life
 Early audits are often focused on the technical
issues in order to make sure that key technical
problems have been solved
 Audits done later in the life cycle of a project
are of less immediate value to the project, but
are more valuable to the parent organization.
Cont’d…
 As the project develops, technical issues are less
likely to be matters of concern
 Conformity to the schedule and budget become
the primary interests
 Management issues are major matters of interest
for audits made late in the project’s life
 Post-project audits are often a legal necessity
because the client specified such an audit in the
contract.
Construction and Use of the Audit
Report

 The information should be arranged so as to

facilitate the comparison of predicted versus
actual results
 Significant deviations of actual from predicted
results should be highlighted and explained in
a set of footnotes or comments
 Negative comments about individuals or
groups associated with the project should be
avoided
Cont’d…
 Information that should be contained in the
audit report:
 1. Introduction
 2. Current status
 3. Future project status
 4. Critical Management issues
 5. Risk Analysis
 6. Caveats, Limitations, and Assumptions
 Responsibilities of the Project Auditor/Evaluator

 First and foremost, the auditor should “tell the

truth”
 The auditor must approach the audit in an
objective and ethical manner
 Must assume responsibility for what is included
and excluded from consideration in the report
 The auditor/evaluator must maintain political
and technical independence during the audit
and treat all materials as confidential
Cont’d….
 Steps to carry out an audit:
 Assemble a small team of experienced experts
 Familiarize the team with the requirements of the
project
 Audit the project on site
 After the completion, debrief the project’s
management
Cont’d…

 Produce a written report according to a pre-

specified format
 Distribute the report to the project manager
and project team for their response
 Follow up to see if the recommendations
have been implemented
 The Project Audit Life Cycle

 Like the project itself, the audit has a life cycle

composed of an orderly progression of well-
defined events:
 Project audit initiation
 Project baseline definition
 Establishing an audit database
 Preliminary analysis of the project
 Audit report preparation
 Project audit termination
Essentials of an Audit/ Evaluation

 For an audit/evaluation to be conducted with

skill and precision, and to be generally accepted
by senior management, the client and the
project team, several conditions must be met:
 The audit team must be properly selected
 All records and files must be accessible
 Free contact with project members must be
preserved
The Audit/Evaluation Team

 The choice of the audit/evaluation team is

critical to the success of the entire process
 The size of the team will generally be a function
of the size and complexity of the project
 For a small project, one person can often
handle all the tasks of an audit, but for a large
project, the team may require representatives
from several areas
Cont’d…
 Typical areas that may furnish audit team
members are:
 The project itself
 The accounting/controlling department
 Technical specialty areas
 The customer
 The marketing department
 Purchasing/asset management
 Human resources
 Legal/contract administration department
Cont’d…
 The main role of the audit/evaluation team is to
conduct a thorough and complete examination
of the project or some pre-specified aspect of
the project
 The team must determine which items should
be brought to management’s attention
 The team is responsible for constructive
observations and advice based on the training
and experience of its members.
Access to Records
 In order for the audit/evaluation team to be
effective, it must have free access to all
information relevant to the project
 Most of the information needed will come from
the project team’s records or from various
departments such as accounting, personnel, and
purchasing
 Some of the most valuable information comes
from documents that predate the project
 Cont’d…
 Examples of documents that predate the project:
 Correspondence with the customer that led to RFP
 Minutes of the project selection committee
 Minutes of senior management committees that
decided to pursue a specific area of technical interest
 Priorities must be set to ensure that important
analyses are undertaken before those of lesser
importance
Access to Project Personnel and Others

 There are several rules that should be followed

when contacting project personnel
 Care must be taken to avoid misunderstandings
between the audit/evaluation team and project
team members
 Project personnel should always be made aware
of an in- progress audit
 Critical comments should be avoided
Cont’d…
 At times, information may be given to audit
evaluation team members in confidence
 Discreet attempts should be made to confirm
such information through non-confidential
sources
 If it cannot be confirmed, it should not be used
 The auditor/evaluator must protect the sources
of confidential information
Measurement
 Measurement is an integral part of the
audit/evaluation process
 Performance against planned budget and
schedule usually poses no major measurement
problems
 Measuring the actual expenditure against the
planned budget is harder and depends on an
in-depth understanding of the procedures used
by the accounting department
Cont’d…
 It is a very difficult task to determine what
revenues should be assigned to a project
 All cost/revenue allocation decisions must be
made when the various projects are initiated
 The battles are fought “up front” and the equity
of cost/revenue allocations ceases to be so
serious an issue
 As long as allocations are made by a formula,
major conflict is avoided-or at least, mitigated
The Auditor/Evaluator

 Above all else, the auditor/evaluator needs

“permission to enter the system”
 If the auditor maintains a calm, relaxed
attitude, the project team generally begins
to extend limited trust
 The first step is to allow the auditor
qualified access to information about the
project
Cont’d…
 The auditor/evaluator should deal gently with
information gathered, neither ignoring nor
stressing the project’s shortcomings
 Recognition and appreciation should be given
to the project’s strengths
 If this is done, trust will be extended and
permission to enter the system will be granted
 Trust-building is a slow and delicate process
that is easily thwarted
Project Closure
The Varieties of Project
Termination

 A project can be said to be terminated when

work on the substance of the project has
ceased or slowed to the point that further
progress is no longer possible
 There are four fundamentally different ways
to close out a project: extinction, addition,
integration, and starvation
Termination by Extinction

 The project may end because it has been successful

and achieved its goals
 The project may also be stopped because it is
unsuccessful or has been superseded
 A special case of termination by extinction is
“termination by murder” which can range from
political assassination to accidental projecticide
Cont’d…

 Two important characteristics of termination

by murder are the suddenness of project
demise and the lack of obvious signals that
death is imminent
 When a decision is made to terminate a
project by extinction, the most noticeable
event is that all activity on the substance of
the project ceases
Termination by Addition
 If a project is a major success, it may be
terminated by institutionalizing it as a formal
part of the parent organization
 Project personnel, property, and equipment are
often simply transferred from the dying project
to the newly born division
 The transition from project to division demands
a superior level of political sensitivity for
successful accomplishment
Termination by Integration
 This method of terminating projects is the
most common way of dealing with
successful projects, and the most
complex
 The property, equipment, material,
personnel, and functions of the project
are distributed among the existing
elements of the parent organization
Cont’d…
 In general, the problems of integration are
inversely related to the level of experience
that the parent or client has had with:
 the technology being integrated
 the successful integration of other projects,
regardless of technology
Cont’d…
 A few of the more important aspects of the transition
from project to integrated operation that must be
considered:
 Personnel - where will the team go?
 Manufacturing - is the training complete?
 Accounting/Finance - have the project’s account been
closed and audited?
 Engineering - are all drawings complete and on file?
 Information Systems/Software - has the new system
been thoroughly tested?
 Marketing - is the sales department aware of the change?
Termination by Starvation
 This type of project termination is a “slow
starvation by budget decrement”
 There are many reasons why senior
management does not wish to terminate an
unsuccessful or obsolete project:
 Politically dangerous to admit that one has
championed a failure
 Terminating a project that has not accomplished its
goals is an admission of failure
When to Terminate a Project
 Some questions to ask when considering
termination:
 Has the project been obviated by technical
advances?
 Is the output of the project still cost-effective?
 Is it time to integrate or add the project as a part of
regular operations?
 Are there better alternative uses for the funds, time
and personnel devoted to the project?
 Has a change in the environment altered the need
for the project’s output?
Cont’d…
 Fundamental reasons why some projects fail
to produce satisfactory answers to
termination questions:
 A project organization is not required
 Insufficient support from senior management
 Naming the wrong person as project manager
 Poor planning
 These and a few other reasons, are the base
cause of most project failures
 The specific causes derive from these
fundamental issues
The Termination Process
 The termination process has two
distinct parts
 First is the decision whether or not to
terminate
 Second, if the decision is to terminate
the project, the decision must be
carried out
 The Decision Process
 Decision-aiding models for the termination decision
fall into two generic categories:
 1. Models that base the decision on the degree to which
the project qualifies against a set of factors generally held
to be associated with successful projects
 2. Models that base the decision on the degree to which
the project meets the goals and objectives set
for it
 Just as the decision criteria, constraints, weights,
and environmental data are unique to each
organization, so are the specifics of using any
decision model
The Implementation Process
 The actual termination can be planned and
orderly, or a simple hatchet job
 Special termination managers are sometimes
useful in completing the long and involved
process of shutting down a project
 The primary duties of the manager in charge
of termination can be encompassed in nine
general tasks
Cont’d…
 Duties of the termination manager:
 Ensure completion of the work, including tasks
performed by subcontractors
 Notify the client of project completion and ensure
that delivery is accomplished
 Ensure that documentation is complete including a
terminal evaluation of the project deliverables and
preparation of the project’s Final Report
 Clear for final billings and oversee preparation of the
final invoices sent to the client
Cont’d…
 Redistribute personnel, materials equipment,
and any other resources to the appropriate
places
 Clear project with legal counsel or consultant
 Determine what records to keep
 Ascertain any product support requirements,
decide how each support will be delivered,
and assign responsibility
 Oversee the closing of the project’s books
 Cont’d…
 Most project managers delay the personnel
reassignment/release issue as long as possible for
three main reasons:
1. A strong reluctance to face the interpersonal
conflicts that might arise when new assignments
and layoffs are announced
2. Worry that people will lose interest and stop
work on the project as soon as it becomes
known that termination is being considered
3. Concern that team members will try to avoid death by
stretching out the work as far as possible
The Final Report - A Project
History

 The final report is the history of the project

 It is a chronicle of the life and times of the
project, a compendium of what went right and
what went wrong
 The required information is contained in the
master plan, all project audits, and evaluations
 The precise organization of the report is not of
great concern; the content is
The Final Report
 Several Subjects should be addressed in the
final report:
 Project performance
 Administrative performance
 Organizational structure
 Project and administrative teams
 Techniques of project management
Cont’d..
 For each element covered in the final report,
recommendations for changing current
practice should be made and defended
 Equally important are comments and
recommendations about those aspects of the
project that worked unusually well
 The fundamental purpose of the final report
is to improve future projects
END of THE COURSE

????