Health and Safety

Executive

Application of performance indicators

in the explosives sectors

Prepared by the Health and Safety Laboratory

for the Health and Safety Executive 2009

RR759
Research Report
 Health and Safety
Executive

Application of performance indicators

in the explosives sectors

Iain Ferguson CMIOSH

Kate Nash

Health and Safety Laboratory

Harpur Hill
Buxton
Derbyshire
SK17 9JN

Performance Indicators are increasingly being used in a range of environments as an aid to identifying and
resolving safety and operational issues. Recent incident investigations and guidance promote enhancements
to the indicator regimes to ensure that a balanced approach is adopted that enables organisations to monitor
both the sustained use of management controls and the effectiveness of those controls in maintaining safety.

The Heath and Safety Executive (HSE) commissioned this report to:

n identify the extent to which the concept of dual assurance is understood and applied within a sample
industry;

n identify industry representatives’ opinion in relation to potential indicators that would correlate with the
Major Hazard Performance Indicator Framework proposed by HSL; and

n identify a process that can be used to exemplify HSE guidance HSG254 in the derivation of key
performance indicators in the explosives industry.

This report and the work it describes were funded by the Health and Safety Executive (HSE). Its contents,
including any opinions and/or conclusions expressed, are those of the authors alone and do not necessarily
reflect HSE policy.

HSE Books
© Crown copyright 2009

First published 2009

All rights reserved. No part of this publication may be

reproduced, stored in a retrieval system, or transmitted

in any form or by any means (electronic, mechanical,

photocopying, recording or otherwise) without the prior

written permission of the copyright owner.

Applications for reproduction should be made in writing to:

Licensing Division, Her Majesty’s Stationery Office,

St Clements House, 2-16 Colegate, Norwich NR3 1BQ

or by e-mail to [email protected]

ACKNOWLEDGEMENTS

HSL would like to thank the following organisations for their

participation in the project:

n AWE, Aldermaston
n BAE Systems, Glascoed Site
n Pains Wessex/Chemring, High Post Site
n QinetiQ, Eskmeals Site

The views expressed in their responses are the opinions of

their employees in relation to specific questions. They do
not necessarily reflect the policy or performance of their
organisations.

ii
 CONTENTS

1 INTRODUCTION......................................................................................... 1

2 METHODOLOGY........................................................................................ 2

3 RESULTS ................................................................................................... 6

3.1 Risk controls and current performance measures ................................... 6

3.2 Review of Potential Performance Indicators .......................................... 14

4 CONCLUSIONS........................................................................................ 19

5 RECOMMENDATIONS............................................................................. 20

6 REFERENCES.......................................................................................... 21

APPENDIX 1 BLANK QUESTIONNAIRE........................................................ 22

APPENDIX 2 - INDICATOR EVALUATION TABLES ..................................... 27

iii
iv

 EXECUTIVE SUMMARY
Performance Indicators are increasingly being used in a range of environments as an aid to
identifying and resolving safety and operational issues. Recent incident investigations and
guidance promote enhancements to the indicator regimes to ensure that a balanced approach is
adopted that enables organisations to monitor both the sustained use of management controls
and the effectiveness of those controls in maintaining safety. The Heath and Safety Executive
(HSE) commissioned this report to:

• Identify the extent to which the concept of dual assurance is understood and applied

within a sample industry;

• Identify industry representatives’ opinion in relation to potential indicators that would

correlate with the Major Hazard Performance Indicator Framework proposed by HSL;

• Identify a process that can be used to exemplify HSE guidance HSG254 in the

derivation of key performance indicators in the explosives industry.

Main Findings
In order to establish the current status of industry application and opinion, a questionnaire-based
approach was developed and completed by the participating organisations during site-specific
workshops attended by HSE and Health and Safety Laboratory (HSL) personnel.

The approach taken by the participating organisations supports the concept highlighted in
HSG254, that key performance indicators should promote activity to attain success. All
organisations had considered performance indicators and presented systems for the capture and
monitoring of performance with varying degrees of sophistication, depth and spread of cover.

All organisations reported that as much of their activity was based on discrete batch processes
the most significant areas of vulnerability revolved around the competence and attitude of those
undertaking safety critical activity.

The explosives industry, as a licensed industry, has well defined requirements and a reasonable
level of regulatory attention. The study team observed that considerable difficulty was
encountered in the determination of targets for leading indicator elements: for most of the
activities undertaken, anything less than 100% was considered unacceptable and was generally
managed to resolution within very short timescales.

Recommendations
Explosives industry organisations should make use of HSG254 in order to ensure that they have
suitable indicators in place. The indicators postulated in this report may provide additional
assistance to other, similar organisations.

Work to develop a sector specific example should be considered by HSE. This should build on
the examples and indicators identified in this study and use a scenario that encapsulates the
types of activities common to a large number of the organisations (e.g. transportation and
storage of explosives).

A number of areas of the questionnaire need minor refinement to enable its use within other
organisations and sectors as a means to assist in the development of indicators. It is
recommended that this be undertaken to aid a common understanding of the question intent and
remove any ambiguity before any wider distribution is undertaken.

v
vi

 1 INTRODUCTION

Performance Indicators are increasingly being used in a range of environments as an aid to

identifying and resolving safety and operational issues. Recent incident investigations and
guidance promote enhancements to the indicator regimes to ensure that a balanced approach is
adopted that enables organisations to monitor both the sustained use of management controls
and the effectiveness of those controls in maintaining safety. This is known as dual assurance.
Further information on this concept is provided in HSE’s guidance on process indicators
HSG254.

This project was commissioned jointly between the major hazards strategic programme board
and the Explosives Inspectorate of HSE. The purpose of this small study was to:

• Identify the extent to which the concept of dual assurance is understood and applied
within a sample industry;
• Identify industry representatives’ opinion in relation to potential indicators that would
correlate with the Major Hazard Performance Indicator Framework proposed by HSL;
• Identify a process that can be used to exemplify HSE guidance HSG254 in the
derivation of key performance indicators in the explosives industry.

The work takes forward elements of previous HSL research (Birkbeck and Ferguson 2006,
Sugden et al 2006 and Keeley 2005) and compliments existing work being undertaken by the
Explosives Inspectorate and the Explosives Industry Forum to improve the use of performance
indicators across the sector.

 2 METHODOLOGY

In order to establish the type and extent of indicators currently in place within industry a range
of issues were explored through a questionnaire based approach. The guidance document
HSG254 was used to guide the development of the project. The stages the guidance
recommends in relation to the derivation of performance indicators are as follows:

Select the organisational level

Identify the scope of the measurement system:
Decide the scope of the measurement system. Identify incident scenarios – what can go
Consider what can go wrong and where wrong?
Identify the immediate causes of hazard
scenarios
Review performance and non-conformances
What risk control systems are in place?
Identify the risk control systems in place to Describe the outcome
prevent a major accident. Decide on the outcomes
for each and set a lagging indicator Set a lagging indicator
Follow up derivations from the outcome
What are the most important parts of the risk
Identify the critical elements of each risk control control system?
system (i.e. those actions or processes which must Set leading indicators
function correctly to deliver the outcomes) and set Set tolerances
leading indicators
Follow up derivations from tolerances

Collect information – ensure information/unit of

Establish the data collection and reporting system measurement is available or can be established

Decide on presentation format

Review performance of the process management
system
Review Review the scope of the indicators
Review the tolerances

The questionnaire is therefore structured around the early stages of the process:

• Determining the scope of the indicators;

• Identifying the important control systems/ areas;
• Providing an indication of the lagging indicators that are in place, based on the desired
outcome (this is often compliance with relevant regulation); and
• Evaluating potential leading and lagging indicators based on the degree of usefulness to
the organisation.

Having established a framework for the derivation of major hazard performance indicators in an
earlier project (Birkbeck and Ferguson 2006) a suite of potential measures was developed.
These indicators were grouped according to the framework components under the “low risk”
and “operational efficiency” categories. Figure 1 details this framework.

 Regulatory Indice
Outcome Activity

Attributes Operational Efficiency Risk Control Safety Culture Societal Concern Regulatory Performance
Achieved Target Achieved Target Achieved Target Achieved Target Achieved Target

Top Level

Operational Efficiency and Risk Control Attribute Indicators

Operational Efficiency Risk Control

Attributes Achieved Target Achieved Target

Operating Performance Operational Structure Challenges to Safety System Emergency Preparedness Risk Management

Overall Outcome Activity Outcome Activity Outcome Activity Outcome Activity Outcome Activity

External Cooperation Change control

Productivity Indicator Competence Major Hazard System Integrity

Strategic Degraded/Abnormal operational Internal Planning Societal Risk

Unplanned Interuptions Resourcing modes
Risk Control
Corporate Knowledge Safety System Failures
Measures
Alternatively the Cranfield External Liaison
Accountability Operational Procedures
Model can be used
Management
Leadership Arrangements/Supervision

Learning from Incidents

Figure 1 Performance Indicator Framework as proposed by Birkbeck and Ferguson (2006)

These areas were studied and a number of indicators were elucidated using the literature from
the previous study, Birkbeck and Ferguson (2006). In this study the potential indicators were
incorporated into a Microsoft Excel based question set together with a rating scale based on two
factors.

These factors were:

• The ease of collecting the data that would underpin the indicator; and
• The degree of usefulness the indicator would have in relation to the management of
safety performance within the organisation at a site level.

It was decided that, by deriving indicators and then asking organisations to evaluate them
provided a means to establish whether it is feasible to utilise common indicators within the
major hazard performance indicator framework. If this was possible then it would facilitate
benchmarking within the industry sector.

In addition to the indicator evaluation, a number of generic questions were used to identify and
rank the risk control areas in terms of the vulnerability and criticality of the system. This
correlates with the approach recommended by HSE (2006) for devising indicators. The
questions were intended to explore the risk management controls from a ‘Swiss cheese model’
(defence in depth) perspective as detailed in Figure 2.

Figure 2 Reason’s model of accident causation

A copy of the questionnaire developed is provided in Appendix 1.

It did not prove practicable to pilot or ‘test’ the questionnaire prior to distribution to the
volunteer organisations. Because of this, arrangements were made to run through the
questionnaire on site with each organisation to ensure that the questionnaire was completed
consistently and so that any issues with the construction of the questionnaire could be identified.
A member of the HSL project team visited each of the volunteer sites to facilitate completion of
the questionnaire. For three of the visits, the responsible HSE site inspector also attended the
meetings. For two visits the Operational Policy Inspector for explosives was also in attendance.

Whilst the volunteer organisations selected are part of the explosives sector and operate under a
licensing system, they all undertake a different range of activities. These activities broadly
include:

• Transporting explosives to and from the site;

• Filling of explosives articles;
• Mixing of explosive substances;
• Research and development of explosives; and
• Storage of completed explosive products.

An evaluation of the completed questionnaires in relation to the performance indicators is

provided in Appendix 2. The responses are deliberately not attributable to the individual
companies.

 3 RESULTS

3.1 RISK CONTROLS AND CURRENT PERFORMANCE MEASURES

The first part of the questionnaire provided an overview of the type of control systems in place
together with an indication of the degree of importance they have in relation to directly
preventing a multiple fatality accident event.

It should be noted that the results are the opinion of the parties completing the questionnaire.
As such they do not necessarily reflect the views of the organisations and have not been subject
to any inspection to validate the opinions put forward.

As the questionnaire is designed for use in more than one industry sector and the focus of this
trial has been the explosives sector, the initial questions provided a similar response from all
parties.

3.1.1 Questions 1 and 2 – What best describes your industry?

Most respondents reported explosive manipulation, storage and transportation as an aspect of

their activity. Production of very limited amounts was reported for one organisation. All of
these activities are regulated by HSE as part of a licensing arrangement. This results in a range
of control measures that are mandated on the organisation including:

• Where explosives can be stored;

• Control of the amount stored in any specific location;
• The emergency arrangements; and
• The position of the operation in relation to residential areas.

All of these act to reduce the risk presented by the site and its activities.

3.1.2 Question 3 – What are the main hazards controlled?

All respondents, as expected, reported explosion. One respondent suggested fire as an

additional hazard although this would often form a precursor or consequential element to a
major accident scenario at the site. No further analysis of this issue was considered necessary.

3.1.3 Question 4 – Rank the importance of the following controls in relation

to major hazard risk

The wording of this question seemed to cause some confusion among respondents. The key
concern being that all of the controls can be perceived to be of critical importance because they
all form part of the risk control system. The intent of the question was to define how critical the
control was in relation to the immediate control of a hazard (i.e. how close to the harm the
barrier is using Reason’s model) and the vulnerability of the management system to
deterioration. As the questionnaire was discussed with each organisation, it was possible to
revise the relative importance.

The questionnaire will be redesigned to convey this message and to encourage a ranking in
order from 1 – 10 to allow the relative importance of each control to be identified more easily.
6

The responses produced a fairly consistent trend. All respondents considered that operational
procedures were of critical importance, again reflecting the dependence of the activities on the
competence and complicity of operators. The operational procedures were complimented by a
number of other areas: risk assessment, employee attitude and leadership. All of these areas
were recorded by the respondents as either critical or highly important and were often qualified
with statements such as “[there is an emphasis on] the importance of doing things right” and-
the “attitude towards procedure/terms of reference” were recorded in relation to employee
attitude. “Identification of departures and need to stop”, “[leadership is] important for culture”
and “[there is] vulnerability around delegation, ownership and conflicting interests” were
comments recorded that reflected the need for effective leadership in influencing employee
attitude, especially in relation to compliance with operational procedures.

Communication was considered to be critical/highly important for two organisations but less
so for the others, although one of these organisations suggested that leadership communication
was highly important.

Control of Contractors is often reported as an area of concern in other major hazard areas with
significant vulnerability and potential for major hazard incidents (e.g. Hatfield in the railway
sector). Most of the organisations used contractors in non-critical activities and one
organisation reported that their own employees cleared areas of any explosives before
contractors were permitted to commence work. One organisation reported vulnerabilities
around the use of agency workers, especially in relation to perception of risk and compliance
with operational procedures.

Effective emergency arrangements are a Control of Major Accident Hazards (COMAH)

regulation requirement, reflecting the importance attached to them. However, because of this
recognition and the associated regulatory oversight, the risk control arrangements were not
considered by any organisation to be especially vulnerable. This is an observation that can be
argued for several of the control areas but was especially relevant here.

The adequacy of arrangements and the level of compliance was undertaken using audits and
reviews of varying nature in each of the organisations. Some organisations reported its
importance in relation to ensuring “supervisory resilience”. This was considered important in
maintaining a positive safety culture and reinforcing leadership. A number of organisations
reported that senior managers undertake a number of audits thus providing both visibility and a
review of systems in place at all levels in the organisation.

Identification of potential vulnerabilities in the safety management system

Difficulty was reported by the participants in relation to the degree of specificity associated with
each of the organisation’s processes. The organisations involved reported that they often
operated bespoke batch processes on an infrequent basis. Discussions held during the onsite
workshops explored the implications of this in order to attempt to identify vulnerabilities in the
system. As mentioned previously, all organisations reported that their high degree of
dependence on operational procedures meant that competence and compliance was a key area.
This was especially important due to the variety of processes used. It would therefore be
reasonable to infer that, using the HSE guidance (HSE, 2006) this area should focus on the
identification of the efficacy and efficiency of these arrangements.

Potential areas for performance indicator development

The completion of the questionnaire initiated a wider discussion on a number of occasions.

From this it was possible to gain some insight into the safety management system and any
potential vulnerability that was perceived. The following diagram summarises the main
findings of this question and reflects where the risk controls were considered to be most critical,
and potentially most vulnerable. The diagram is based on Reason’s model of resilience from a
2006 presentation. The concept of resilience correlates well with the approach in HSG254, i.e.
the sustained management of the vulnerabilities of the critical major hazard controls allows the
interface between the management system and the operating environment to be effective in
maintaining the safety of the organisation and mitigate the effects of any incident i.e. providing
resilience.

Figure 3 Diagrammatic representation of the criticality of risk controls systems

Based on the responses to question 4, it is considered that the key areas that merit consideration
in relation to the development of performance indicators in the explosives industry would be:

• Design, operational and other procedures (processes);

• Maintenance and inspection ;
• Leadership;
• Employee attitude;
• Change control;
• Control of contractors; and
• Audit

The principal areas for consideration in developing key performance indicators would be
operational procedures and employee attitude as they were considered to be critical to sustained
safety performance and offer the greatest potential for unidentified degradation over a period of
time (i.e. are most vulnerable). These are discussed in further detail later in the report.

What is clear from the visits and the surveys is the fact that the competency of the individuals is
key, in terms of complying with operational and inspection/maintenance procedures, in relation
to the development of improved controls and in the identification of scenarios where operational
procedures had the potential to lead to undesirable consequences.

3.1.4 Questions 5 and 6, Leading and Lagging Safety Performance

Indicators that are in place

The purpose of these questions was to explore what information is collected by organisations in
relation to performance indicators and how often the data is collected/reviewed in order to
identify trends.

General Observations:

One of the organisations used a hierarchical system, similar in structure to that described in
Birkbeck and Ferguson (2006). Each area of risk control was related to an accountable person
and a procedure. Performance criteria were set for each area and reporting requirements put in
place. A traffic light system enabled issues to be quickly identified from a summary screen and
traced through to the relevant owner and procedure. It was considered that this approach
offered an effective means to identify trends and to quickly establish where the causes of any
potential problems are occurring. Other organisations use a balanced scorecard approach to
report performance information to directors.

The approach of one organisation is worth noting as it offers a potentially effective approach to
the ‘sampling’ of the control areas to enable a regular, ongoing review of the efficacy and
efficiency of control measures. This organisation uses an audit type approach, using a
predetermined plan to identify the area to be covered and with what level of frequency/depth but
does not specify when the audit will be undertaken. It was our understanding that the plan was
not widely distributed and therefore an area of the organisation could not predict when they
would be reviewed. Participation by all in the area being reviewed is encouraged and is
designed to identify where deficiencies in the control area are arising. Complementing this
process are a number of unannounced monitoring and inspection regimes including: compliance
verification, housekeeping and operational safety. These utilise plans to sample control areas
from a more formal perspective. This approach can potentially provide data on the
arrangements that are in place: however, to gain the greatest benefit it is important to ensure that
the areas targeted are correctly identified as critical and vulnerable before they were utilised in
key performance indicators for the organisation.

Operational Procedures:

All but one organisation reported using both leading and lagging indicators to monitor
compliance with operational procedures. Reviews of leading indicators were mainly undertaken
against a predetermined plan to evaluate compliance and also review the suitability of the
procedures to ensure they provided the optimal method of work. This was considered to present
good practice and reflected an understanding of where the system was potentially most
vulnerable. The methods of obtaining leading indicator data usually involved an
audit/inspection against a predefined plan that determined what areas would be looked at to
ensure adequate coverage over a six or twelve month cycle. Types of leading indicators used
were: compliance verification plans completed; safety, health and environmental inspections
undertaken against plan; and unannounced inspections/monitoring completed against a
predetermined plan. The outcomes of many of the activities were reviewed on a monthly basis

using a hierarchical approach. A number of the organisations reported that they were in the
process of, or already had, implemented a competence management framework.

In relation to lagging indicators, a number of approaches were used: the most common being the
review of accidents, incidents and near misses which presented a mechanism to establish root
causes and any underlying trends. One organisation reported that it assessed the adequacy of
investigations to ensure that investigations were completed to sufficient quality, thus ensuring
that the causation data was sufficiently robust to enable adverse trends in relation to operational
procedures to be identified. Some organisations reported that abnormal/unusual occurrences
were also reviewed to provide more data and to act as an early identification of any reduction in
the adequacy of a control area. The use of lagging indicators relying on incidents/unusual
occurrences was observed as the means of establishing trends in the efficacy of a number of
control areas. Most organisations considered that they had a positive reporting culture that
ensured the majority of incidents/occurrences were reported. Review of lagging indicators for
operational procedure efficacy was usually undertaken on a monthly basis.

The approach observed by the organisations seems to support the concept highlighted in
HSG254 that key performance indicators should promote activity to attain success. Success in
relation to this area would be compliance with procedures but with sufficient competence
exhibited to recognise when the procedure is creating a hazard and stop activities in a safe
manner until the situation is resolved. This success criterion also incorporates the employee
attitude and leadership control areas.

Instrumentation and Alarms:

It was reported by most organisations that inspections were undertaken on a weekly basis
against a defined plan and that these were monitored and reviewed on a regular
(weekly/monthly) basis. Efficacy of the instrumentation and alarm systems was undertaken
using the abnormal occurrence/near miss reporting systems. The other organisations did not
regularly review this aspect because of the nature of equipment used. No organisation reported
separately reviewing the number of faults identified on inspection although this would not be
surprising given the relatively low importance reported for this area in question 4 due to the
batch process operations used and the high degree of inherent manual intervention.

Change control:

All organisations reported the use of a change control process to assess the implications of
operational and engineering change. A number of approaches were used to monitor this area
from a leading indicator perspective. Two organisations used specific audits to determine where
any changes had been undertaken without the change being assessed using a change control
process. Another organisation reported that corrective action requests were reviewed to
determine whether a change control procedure had been followed: they stated that a measure of
the number of changes planned and assessed prior to change was monitored. This was
considered to represent good practice as it encouraged behaviours towards the achievement of
success (changes formally assessed) and permitted any potential degradation (or increasing
demand for change control) to be identified, investigated and appropriately resolved.

Considerable difficulty was reported in monitoring the efficacy of controls in this area, the main
issue being how to efficiently monitor whether any changes have occurred without following a
change control process (as this would require audits/inspections of all processes on a regular
basis). One organisation reported using their abnormal event/unusual occurrence system to
identify where change management was a contributory element to any incident. This was
considered to compliment the leading indicators without requiring additional data collection.
10

Maintenance and Inspection:

These control areas received a large variety of approaches to measurement. Leading indicators
were drawn out from ad hoc audits, monitoring preventative maintenance and inspection
undertaken against a predefined plan and number of maintenance activities completed within
agreed timescales. Lagging indicators generally identified where failures had occurred either
due to an incident or where corrective maintenance was required as a consequence of inspection
(although the number of unresolved corrective maintenance issues could be considered to be a
leading indicator). One organisation reported monitoring the numbers of unresolved
maintenance issues.

Analysis of reliability data was used to determine both operational performance and assurance
of safety within one organisation although they did not use the data to create key performance
indicators at the time of completing the questionnaire.

Emergency Arrangements:

Testing of emergency arrangements is a requirement for Top Tier sites as part of the COMAH
regime. Most leading indicators focused on ensuring that exercises were completed on time.
Testing of on-site emergency arrangements is only a requirement for top tier COMAH sites,
although some other sites do have emergency plans that are tested. Monitoring and review was
between monthly and quarterly.

The adequacy of the arrangements was reviewed post exercise in all organisations but one
organisation reported the use of pass criteria. This involved identifying the number and nature
of deficiencies and subjectively evaluating them. Based on this an exercise would either be
classed as having passed or failed. In the event of the latter corrective action is undertaken to
resolve the issue. As with other good performance indicators, this approach provided an
impetus to meet a defined objective (i.e. strive to a predetermined successful outcome) and also
allowed any degradation in performance to be easily identified.

Communication:

Responses to this question for this area revealed that the term ‘communication’ was ambiguous.
One organisation took it to mean the onsite communication arrangements whereas others
considered it to be an element of a positive safety culture through either two-way
communication between employees through the organisation or provision of information from
the top down. The question was intended to explore the arrangements for monitoring
communication between management and employees to facilitate sharing of knowledge and
providing clear guidance on what is expected of those undertaking safety critical activities.
Given the reliance of the participating organisations on compliance with (and hence competency
to follow) operational procedures it would be considered as an important area for monitoring.

This was reflected in the fact that most organisations reported using leading indicators to
monitor this area. These indicators included: the percentage attending safety meetings, the
percentage response to urgent safety communications or the number of toolbox talks undertaken
against a predefined plan.

All respondents identified difficulties in determining the quality of communication and consider
this to be an important area, worth exploring in more detail.

Lagging indicators were considered by a number of the respondents to be covered in the

incident reporting process.
11

Permit to Work:

A number of organisations reported that they had the potential to capture indicator data in
relation to the permit to work system (for example using data to re-accredit contractors). Most
organisations undertook spot checks/unannounced monitoring for competence management.
The data was not used as a performance indicator. This area could be regarded as being
potentially vulnerable, given the observation that most organisations utilise a batch process it
was argued that there is little likelihood of ‘live’ maintenance being undertaken and hence
attracting major hazard risk scenarios. It is noted that in other high hazard industries there have
been reported degradations in the control of contractors in permit to work scenarios often as a
result of cost reduction exercises resulting in an increased incentive to ‘cut corners’. These
“violations” are likely to be detected using a reporting system. Further work in this area may be
desirable.

Control of Contractors:

Two organisations considered this to be an area of some importance but that it was very
unlikely that contractors would be permitted to undertake activities that had major hazard
potential. Contractors’ performance was monitored against contractual requirements that
included safety. Another organisation reported that a quarterly review of contractors
performance against a range of criteria was undertaken as part of the contracts management
process. These included reviews of safety related issues such as breaches of permit to work
systems and near miss events. No organisation reported using specific performance indicators,
reflecting the finding that this area was not considered as being particularly vulnerable because
of the nature of work undertaken and the level of monitoring in place to assess contractual
compliance. The other organisations reported that contractors were subject to the same
arrangements as for direct employees. One organisation reported that testing work is
undertaken jointly on occasion. This activity is managed on a case-by-case basis and is subject
to individual risk assessments and controls.

It may be desirable to review the arrangements for monitoring contractor performance over a
period of time to identify whether this control area does reflect organisations opinion.

Leadership:

All organisations reported that they considered leadership an important part of their
organisations’ safety culture. One organisation utilised specific audits by senior management as
a means to demonstrate positive leadership. Indicators were used to ensure that the audits were
completed against a predefined plan on a monthly basis. One organisation undertook senior
management ‘tool box’ talks on a monthly basis; the activity was monitored and performance
indicators were used to ensure that they were undertaken. Both organisations reported difficulty
over the issue of how the quality of such briefing was best measured.

One organisation reported including leadership elements within the safety training profile;
completion of the training was monitored.

There was no overall agreement over an indicator that would be considered useful in this area
although further indicators may be identified from current HSL research into safety culture
indicators.

12

Employee Attitude:

As reported earlier in this report many organisations viewed this area as being important in
supporting compliance with operational procedures in what is a compliance dependent activity;
it is potentially one of the most vulnerable. It was surprising to note that only one organisation
collected and reviewed data on a regular basis. All other respondents collected lagging
indicators on six monthly to annual basis, often as part of a wider opinion survey.

It may be possible to use performance indicators to monitor activities undertaken to enhance

employee attitude as well as review the nature of employees’ attitude. Further work will be
required to identify how this may be undertaken and to provide a proposed approach.

Risk Assessment:

Leading indicators tended to monitor the number of risk assessments reviewed/revised within
prescribed time limits. A number of the organisations also monitored the competence levels of
those undertaking risk assessments. Most organisations reported monitoring the implementation
of any changes that the risk assessment identified although the data was not used as a
performance indicator. One organisation reported monitoring the numbers of
incidents/irregularities where the risk assessment was subsequently found to be deficient.

Audit:

Most organisations reported monitoring audits undertaken against a predefined plan. A number
subsequently monitored the implementation of corrective actions. No organisation reported that
any trend analysis was undertaken to provide lagging indicators or any indication of the
effectiveness of the audit regime.

3.1.5 Question 7: Target Setting

The explosives industry, as a licensed industry, has well defined requirements and a reasonable
level of regulatory attention. The study team observed that considerable difficulty was
encountered in the determination of targets for leading indicator elements: for most of the
activities undertaken, anything less than 100% was considered unacceptable and was generally
managed to resolution within very short timescales. The study team postulated that an indicator
could be devised that monitored the number of corrective actions (from investigations, risk
assessment and audits) closed out in an agreed timescale.

13

3.2 REVIEW OF POTENTIAL PERFORMANCE INDICATORS

The general results of the combined questionnaires are discussed below, in relation to the
framework attribute level areas.

• Low Risk / Risk Control;

• Positive Safety Culture; and
• Operational Efficiency.

The responses of each organisation were collated and used to compare which indicators were
considered to offer the most value and/or the least amount of effort. Where possible, suites of
indicators were identified that provide dual assurance (i.e. that the measures are in place and are
operating as would be desirable to meet an overall goal, such as low risk, efficient operation or
positive safety culture).

The results are presented in the appendix.

3.2.1 Low Risk / Risk Control

This attribute defines a goal of operation with low risk. Success in relation to this attribute
would be achieved through, for example: the implementation of an effective (and efficient)
safety management process that minimises abnormal, degraded and emergency situations but
also responds effectively to mitigate the risk presented if a hazard is realised (emergency
planning). Most safety performance indicators currently fall into this category as they measure
the realisation of a hazard (lagging indicators) or events that had the potential to give rise to a
major hazard.

Indicators were identified in a number of areas of the low risk attribute: system integrity,
operational procedures, change control, emergency preparedness, degraded/abnormal modes of
operation, control of contractors and safety system failures.

System Integrity:

Reliability achieved against that expected was identified as a useful indicator and could act as a
useful lagging indicator. Some organisations reported that data would be difficult to identify. It
may compliment indicators identifying the level of inspection / preventative maintenance that is
being achieved against a plan. Some organisations stated that the nature of the operation and
the age of the equipment make it difficult to establish the expected reliability. Other
complimentary indicators included the “proportion of plant replaced/overhauled within the
manufacturer’s recommended timeframe” and the proportion of time spent undertaking
unplanned repairs. Whilst the majority of organisations thought these useful indicators the
degree of difficulty in obtaining the data was reported as not being readily available.

Operational Procedures:

The following indicators were identified by most respondents as being relatively easily
collected and useful:

• Proportion of those personnel receiving formal training on time;

• Proportion of safety critical controls (identified from the risk assessment) without
formal procedures in place; and

14

 • Proportion of safety critical control measures reviewed within designated timescales.

The following indicator was identified by all respondents as being useful but a number
considered that it was not easily collected:

• Number of identified violations of procedures.

These responses correlate well with the importance reported to be placed on these areas by the
participating organisations. The three indicators (mentioned above) complement each other and
provide a means to monitor that the control measures and procedures can be reviewed before the
realisation of any latent failure in the management system and that persons applying the
procedures are in receipt of formal, competence based training within prescribed time periods.
The continued efficacy of these two facets (of ensuring the low risk attribute, right control
measures, applied properly) is evaluated by the violations of procedures indicator (the third
measure identified).

Degraded/Abnormal Modes of Operation:

The majority of organisations reported that planning and training for degraded/abnormal
operating modes was important. The majority of organisations considered that the following
indicators were easily obtained and useful indicators:

Planned degraded/abnormal training activity (completed formal training against training

activity planned); and
Number of reviews of operating procedures (completed against planned).

Both of these indicators are leading indicators. The indicator “proportion of time spent in
degraded/abnormal operating modes” is useful but does not correlate as an indicator of the
efficacy of the previous indicators. A better lagging indicator may be “number of near
misses/procedural errors occurring during degraded modes of operation”.

Safety System Failures:

Given the nature of the activity undertaken in the explosives sector (discrete and batch
processes) this area was not considered to be very vulnerable by a number of the organisations.
However all but one respondent considered that the following indicators could be relatively
easily implemented and that they would be of benefit to the organisation:

• Safety critical system maintenance undertaken against those planned;

• Safety critical system inspections undertaken against those planned; and
• Number of inspections identifying a safety critical fault against number of inspections
undertaken.

Again these three indicators are mutually complimentary, offering dual assurance that necessary
activities are being undertaken and that any deficiencies are identified through the reporting of
safety critical faults.

Change Control:

One indicator was identified in relation to this area:

• Proportion of incidents occurring in areas where change has occurred.

15
This was considered to be a sensible approach considering that most organisations audited
whether changes had been adequately managed. This indicator provides an interim monitor to
identify if the change process is performing adequately. Given the compliance with operational
procedures and the potential for these to be revised without the change being assessed it may be
desirable to use a quarterly audit to identify where unevaluated changes have been made. This
indicator was identified in relation to communication of risk control measures and was
considered to be relatively easy to monitor, and of benefit.

Emergency Preparedness:

One indicator was identified by all respondents to be relatively easy to capture data and be
beneficial to the management of major hazard risk:

• Proportion of recommendations implemented following internal review.

This potential indicator reflects the fact that there are a number of formal requirements (such as
for COMAH Top Tier Sites) that influence the periodicity of exercises. One area considered to
be potentially vulnerable to degradation by the respondents was the implementation of any
corrective actions identified by exercise or by audit. This indicator supports the monitoring of
such actions and would, therefore, act to promote timely implementation of remedial actions.

Control of Contractors:

This area was not considered to be a major area of concern, as the level of subcontracting of
safety critical activities was not considered to be a significant cause for concern. The following
indicators were considered useful and capable of being captured:

• Proportion of risk review meetings undertaken with external organisations (completed

against planned); and
• Proportion of incidents attributable to external organisation.

The following measure was considered to be useful, but not all organisations considered it easy
to identify:

• Proportion of safety critical tasks undertaken by an external organisation.

It may be a useful measure in that any trend analysis of the data would identify where a shift in
the level of safety critical outsourcing has occurred and aid decisions on any revisions required
to the management system.

3.2.2 Positive Safety Culture

This area is still being considered in separate work being undertaken by HSL. At the time of
this study a detailed suite of indicators was not available. However a number of examples were
raised in response to part 1 of the questionnaire and will be considered in the parallel project.
Two areas were identified where there was some scope for developing indicators: learning from
accidents and communication of risk control measures.

16

Learning from incidents:

Both of these indicators were considered to provide useful information and were relatively easy
to collect data in relation to:

• Proportion of incidents for which a management review has been undertaken; and
• Proportion of incident inquiry recommendations implemented within agreed timescales.

Risk Control Measure Communication:

Both of these indicators were considered to provide useful information and were relatively easy
to collect data in relation to:

• Proportion of procedures communicated to staff within designated timescales;

• Attendance at toolbox talks; and
• Number of non-compliances with risk controls identified during audit.

These indicators are considered to drive positive behaviour as it promotes communication

within the line management structure and also the identification and resolution of non-
compliances, especially if an unannounced audit regime is used. One area where these
indicators are weak is in establishing the quality of the communication. It was considered that
this and issues such as an understanding of the required process may be correlated to the level of
non-compliances identified, especially if reviews of those non-compliances were appropriately
analysed and trends identified.

3.2.3 Operational Efficiency

The proportion of staff in safety critical posts that are considered competent was considered to
be a useful indicator by all organisations but was not easily captured. This is a good indicator as
it permits an analysis of the level of safety critical activity being undertaken and the competence
of those undertaking those activities. It also drives a positive behaviour of:

“having competent staff in safety critical positions”.

A number of organisations reported that the indicator “proportion of roles for which a safety
responsibility statement has been provided” was also considered to be useful to establish a clear
understanding of what is expected at all levels of safety related roles within an organisation but
is not easily collected. Its use as a process indicator is not considered to be ideal as it is likely to
form part of the audit process and therefore subject to annual review.

Whilst the proportion of delivered output against planned output was considered by the majority
of organisations to be very easy to collect there was no consensus on the value of collecting
such information for safety purposes. Similar opinions were observed for staff turnover and
succession planning. Both of these areas were considered to be relatively easy to collect but
only the succession-planning indicator was considered to offer useful information to the
majority of organisations.

The majority of the organisations reported that the “proportion of safety tours/tool box talks
conducted by senior management (against planned number)” was both useful and easily
collected. This category overlaps with potential indicators for the safety culture attribute.

17

On a similar theme, three of the organisations reported that the frequency of safety related
communication was a useful measure but no consensus was apparent on the ease of collection.
Two organisations reported that this would be easy to collect and two indicated that it was more
difficult. One issue raised in relation to the ease of collection was that the quality of any
communication was difficult to elucidate. The use of a complimentary indicator (such as
changes in the number of near misses) may assist in determining whether the communication is
being effective. This approach could, as a minimum, enable any problem areas to be identified
for appropriate management attention.

18

 4 CONCLUSIONS

The conclusions are presented in accordance with the relevant steps considered useful in the
establishment and implementation of risk control measures. Overall it is considered that the
HSE guidance in HSG254 can be applied to the Explosives Industry. There are however
perceived limitations in relation to the setting of targets. This is because of the perception that
absolute compliance is required within the Explosives Sector, i.e. anything less than 100%
would not be tolerated.

Identifying Risk Control Systems in place

All of the organisations had a good appreciation of the risk controls in place and an appreciation
of the relative importance of the measures based on their criticality and their vulnerability to
decline over time.

Defining what success looks like

Based on the established understanding of the risk controls, it was possible to elucidate the key
elements of the management system that were critical to the success of the activities being
undertaken. For the organisations taking part in this study, success centred around having
competent staff who complied to carefully considered operational procedures. These procedures
must be appropriate and reviewed in a timely manner to ensure the continued assurance of
safety.

Other areas such as emergency preparedness and maintenance and inspection were considered
as important areas but were often subject to detailed statutory requirements and were therefore
considered to be less vulnerable to decline.

Identifying/Establishing Appropriate Indicators

A variety of monitoring systems and a large number of indicators were observed. The concept
of a dual assurance approach was observed in a number of the organisations although this was
not the consequence of any explicit requirement. A number of indicators were identified from
the pre-derived lists and were considered to be useful and the data relatively easily collected.
Many of these correlated with the potential vulnerabilities that had been identified and promoted
activities that would contribute to a successful attribute such as positive safety culture, low risk
or operational efficiency.

Target Setting

This is an area that potentially gives rise to the greatest difficulty in implementing the HSG254
guidance. All organisations noted that for most circumstances, anything less than 100%
compliance is not acceptable. Further work may identify potential target setting arrangements.
However in this study there was insufficient scope to adequately research this area. Further
work with one or more of the volunteer organisations may be desirable to establish targets for
indicators of performance that are separate from license or regulatory conditions.

Construct of the Questionnaire

A number of areas of the questionnaire need minor refinement to enable its use within other
organisations and sectors as a means to assist in the development of indicators. These were
noted within the relevant questionnaires and amendments will be made accordingly. The key
revisions relate to ensuring that the intent of the question is consistently understood.
19

 5 RECOMMENDATIONS

Explosives industry organisations should utilise HSG254 in order to assure themselves that they
have a suitable range of indicators in place. The indicators postulated in this report may provide
additional assistance. The range of indicators chosen should ensure that:

• Successful management outcomes are defined;

• Complimentary leading and lagging indicators are used to monitor the delivery of those
outcomes;
• The indicators are capable of identifying degradation in areas that present significant
major hazard risk and are vulnerable; and
• The indicators are meaningful for those using them.

Work to develop a sector specific example should be considered by HSE. This should build on
the examples and indicators identified in this study and use a hypothetical scenario that
encapsulates the types of activities common to a large number of the organisations (e.g.
transportation and storage of explosives).

A number of areas of the questionnaire need minor refinement. It is recommended that this be
undertaken to aid a common understanding of the question intent and remove any ambiguity
before any wider study is undertaken.

20

 6 REFERENCES

Bell, J. and Healey, N. (2006) The Causes of Major Hazard Incidents and How to Improve Risk
Control and Health and Safety Management: A Review of the Existing Literature, RMS/06/11,
HSL

Birkbeck, D. and Ferguson. I (2006) Development of a Major Hazard Risk Indicator

Framework, RAS/06/11, HSL

HSE (2001) Performance Indicators for the assessment of major hazard accident
hazards, Contract Research Report CRR345/2001

HSE (2006) Step-by-Step guide to developing Process Safety Performance Indicators,

HSE/Chemical Industries Association. HSG254

Keeley. D (2005) Development of Criteria for the Breadth and Depth of Major Hazard
Inspection. RAS/05/08, HSL

OECD (2005) Guidance on Safety Performance Indicators, Organization for Economic

Cooperation and Development

Sugden, C., Birkbeck, D., and Gadd, S. (2006), Major Hazards Industry Performance
Indicators Scoping Study, RMS/06/07, HSL

21

APPENDIX 1 BLANK QUESTIONNAIRE

22

 Preliminary Questions

Q1

Q2

Q3

Q4
Please rank the importance that you attatch to the following
controls in relation to major hazard risk?
Critical Highly Important Important Not Important

Operational Procedures

Instrumentation and Alarms

Change Control

Maintenance

Inspection

Emergency Arrangements

Communication

Design of safety critical equipment (ie equipment whose failure

will lead directly to a multiple fatality accident)

Permit to Work / Control of Contractors

Age of Safety Critical Components

Leadership

Employee Attitude

Risk Assessment

Audit

Commisioning

23
Q5
Do you have safety performance indicators that measure
whether the controls are in place in relation to the following
factors (do you have leading indicators). If so how often is
the data collected? No Data Collected Weekly Monthly Quarterly Every 6-12 months Examples if available

Operational Procedures

Instrumentation and Alarms

Change Control

Maintenance

Inspection

Emergency Arrangements

Communication

Design of safety critical equipment (ie equipment whose

failure will lead directly to a multiple fatality accident)

Permit to Work / Control of Contractors

Age of Safety Critical Components

Leadership

Employee Attitude

Risk Assessment

Audit

Commissioning

24
Q6
Do you have safety performance indicators that measure
whether the controls are effective. If so how often is the
data collected? No Data Collected Weekly Monthly Quarterly Every 6-12 months Examples if available

Operational Procedures

Instrumentation and Alarms

Change Control

Maintenance

Inspection

Emergency Arrangements

Communication

Design of safety critical equipment (ie equipment whose

failure will lead directly to a multiple fatality accident)

Permit to Work / Control of Contractors

Age of Safety Critical Components

Leadership

Employee Attitude

Risk Assessment

Audit

Commissioning

25
Q7
Do you set targets for the following indicators? If
so, who sets them, reviews them and how often
Leading Indicators (See Question Q5) Lagging Indicators (See Question Q6)
Are targets
Reviewed by Reviewed by Frequency Are targets set Reviewed by Reviewed by Frequency
set
Operational Procedures

Instrumentation and Alarms

0 0 0 0 0 0
Change Control

Maintenance 0 0 0 0 0 0

Inspection

0 0 0 0 0 0
Emergency Arrangements

0 0 0 0 0 0
Communication

Design of safety critical equipment (ie equipment

whose failure will lead directly to a multiple fatality
accident) 0 0 0 0 0 0
Permit to Work / Control of Contractors

0 0 0 0 0 0
Age of Safety Critical Components

0 0 0 0 0 0
Leadership

0 0 0 0 0 0
Employee Attitude

0 0 0 0 0 0
Risk Assessment

0 0 0 0 0 0
Audit

0 0 0 0 0 0
Commissioning

0 0 0 0 0 0

26
 APPENDIX 2 - INDICATOR EVALUATION TABLES

Key to Table

Shading Meaning

Concurrence by all respondents that indicator

data is easy to capture/useful

Concurrence by two or three respondents –

analysed on an individual basis in the report

Concurrence that data is not useful

27
 The ease of application to your organisation (on a scale of 1 to 5, one being very How useful are, or would the indicators be to your organisation (on a scale of 1 to 5,
For each of the following indicators, please rate:
easy, 5 being difficult) 1 being not very useful, 5 being very useful)

Attribute 1 - Efficient Operation

It has long been accepted that good safety underpins good business and vice versa; this concept forms a part of the Health and Safety Commission’s strategy. The operational efficiency attribute embraces this philosophy. If an
organisation is operating efficiently then the associated pressures of degraded working, procedural violations and fatigue are reduced. It offers, therefore, opportunity to identify latent failures in the management system at a point anterior
to any challenge on the risk control system and also the potential for conflicting objectives to be identified. This attribute takes into account a range of issues such as organisational stability, competence, succession planning, leadership
and communication. All of these factors have an influence on the efficiency with which a business operates but also identify the potential for departures from normal operation thus creating a challenge to the safety management system.

Potential Overall and Strategic Indicators would include:

Operating Performance
Operational Structure

Operating Performance - to monitor that the

organisation is effectively controlling its operation not
experiencing excessive operational disruption that may
impact on major hazard risk.
1 for safety, 5 for
Proportion/Percentage of delivered output against
3 1 1 1 1 manufacturing 3 5
planned output
(depending on cause)
Number of unplanned interruptions per period (e.g. per
3 4 1 2 2 1 - unreliable 3 5
hour) of operation

Operational Structure - to monitor the stability of the

organisations structure
Proportion of staff in safety critical posts who are
complying with the competence management system 5 2 1 3 5 5 5 4
requirements
1, 5 if competent staff
Proportion of staff in safety critical posts 3 2 1 4 1 5 4
in post

Staff turnover rate 1 2 2 1 3 3 5 5

Proportion of staff within 5 years of retirement without
4 1 1 1 4 3 with key staff 5 5
succession plans in place
4 (not centrally
Proportion of roles for which a safety responsibility
collected, but working 1 1 3 4 5 5 5
statement has been provided
on this)
Proportion of safety critical roles covered by the
5 1 1 4 5 4 5 5
competence management system
Frequency of safety related communication 1 1 4 3 2 5 5 5
Proportion of safety tours/tool box talks conducted by
1 1 1 1 3 5 5 5
senior management (against planned number)

28

 The ease of application to your organisation (on a scale of 1 to 5, one being very How useful are, or would the indicators be to your organisation (on a scale of 1 to 5,
For each of the following indicators, please rate:
easy, 5 being difficult) 1 being not very useful, 5 being very useful)

Attribute 2 Low risk

This attribute defines a goal of operation with low risk. Success in relation to this attribute would be achieved through the implementation of an effective (and efficient) safety management process that minimises abnormal, degraded and
emergency situations but also responds effectively to mitigate the risk presented if a hazard is realised (emergency planning). Most safety performance indicators currently fall into this category as they measure either the realisation of a
hazard (outcome indicators) or the implementation of a risk control system (activity measures).

Major Hazard System Integrity – to monitor that the

appropriate arrangements are in place to maintain the
integrity of safety critical plant and premises
Reliability data for aspects of safety critical plant and
infrastructure - achieved reliability against expected 2 5 1 5 5 1 5 5
reliability

1 - for headcount,
5 - for headcount in
more difficult for
Proportion of resource allocated to safety critical terms of managing
4 4 proportion of time 4 1 2 4
activities (can be headcount or financial allocation) resource, 2 for
spent undertaking SC
measure of safety
activities (4)
Proportion of plant replaced/overhauled within
2 4 2 3 5 1 4 5
manufacturer’s recommended timeframe
Percentage of time spent undertaking unplanned
2 2 3 3 4 4 3 5
repairs
Proportion of non-consumable plant safety critical
1 - because not many
components considered to require replacement within 2 2 3 1 (as proportion) 5 2 5
areas to cover
12 months

Operational Procedures - to monitor the number,

complexity and interdependence of the procedures
considered necessary to adequately control the risk
presented by the major hazards associated with the
industry.
Proportion of personnel (including contractors)
3 4 1 3 1 5 4
undertaking major hazard safety critical activities
Proportion of those personnel receiving formal training
2 2 2 2 5 4 5 5
on time
Proportion of safety critical controls (identified from the
2 1 1 4 5 2 5 5
risk assessment) without formal procedure
Proportion of safety critical control measures reviewed
2 2 1 4 5 4 5 5
within designated timescales
Number of identified violations of procedures (egg
1 5 1 3 5 5 5 5
permit to work systems)

29

 The ease of application to your organisation (on a scale of 1 to 5, one being very How useful are, or would the indicators be to your organisation (on a scale of 1 to 5,
For each of the following indicators, please rate:
easy, 5 being difficult) 1 being not very useful, 5 being very useful)

Degraded / Abnormal Operation - To identify

circumstances when the safety system has been
utilised to deal with degraded and abnormal operating
environments and the level of organisational
competence available to respond to these challenges.
This measure is partially related to operational
efficiency and is important as the nature of the
operation is different to that normally encountered and
hence creates greater potential for a system failure.
Examples of events that would be reported would
include: flammable liquid transfer outside of a bund,
unplanned use of a reduced number of storage tanks or
loss of one of a pair of ventilation fans.
Planned degraded/abnormal training activity (completed
2 1 1 4 5 1 5 5
formal training against training activity planned)
Number of reviews of operating procedures (completed
1 1 1 2 4 3 5
against planned)
Not considered
5 (work deconfliction
Number of planning meetings held/planned 1 1 relevant due to low 2 1 4
etc)
level of degrade ops
Not considered
Percentage of organisations affected that attend
1 relevant due to low 2 5 4
planning meetings
level of degrade ops
1 (if we wanted to Not considered
Proportion of time spent in degraded/abnormal
measure it, no relevant due to low 3 5 5
operating modes
history) level of degrade ops

Safety System Failures - To identify the frequency and

duration of a major hazard related safety critical system
becoming unexpectedly unavailable; and to provide an
indication of the systems in place to minimise the failure
of a safety critical system on demand.
Safety critical system maintenance undertaken/planned 2 1 1 5 5 5 5 5
Safety critical system inspections undertaken/planned 2 1 1 5 5 5 5 5
Number of inspections identifying a safety critical fault
2 1 2 3 5 5 5 4
against number of inspections undertaken

Number of demands placed on safety critical system

per period measured (e.g. number of Train Protection 3 1 4 5 5 1 5 3
and Warning System Activations)
Number of fail to danger incidents recorded (e.g. false
2 1 5 4 5 5
readings)

30

 The ease of application to your organisation (on a scale of 1 to 5, one being very How useful are, or would the indicators be to your organisation (on a scale of 1 to 5,
For each of the following indicators, please rate:
easy, 5 being difficult) 1 being not very useful, 5 being very useful)

Management Arrangements/Supervision - to identify

that suitable management/supervisory arrangements
are in place to adequately control contractors and in-
house personnel who are undertaking safety critical
activities.
Proportion of contractor hours/total hours spent on
4 4 5 2 2 1 2 5
safety critical activities
Number of review meetings per contractor per period 2 2 1 2 3 1 5 5
Number of safety critical supervisory posts
4 2 1 3 4 4 3 5
filled/planned
Number of permit to work violations per contractor hour 2 5 1 3 3 3 4 5
Proportion of accidents/near misses attributable to lack
1 1 3 3 5 5 5 5
of adequate supervision (especially violations)

Learning from Incidents - Good practice and safety

legislation require risks that are reasonably foreseeable
to be controlled. An effective means of achieving this is
by reviewing previous incidents to identify areas for
improvement and then implementing those in order to
prevent a similar accident occurring.
Proportion of risk assessment reviewed against planned 1 1 1 3 3 5 5 5
Proportion of incidents for which a management review
1 1 1 2 4 5 5 4
has been undertaken
1 public; 5 not public
Proportion of major industry events which are reviewed 5 4 3 4 5 2 4
domain / sub judice
Proportion of accident/incidents identifying root causes 1 1 1 4 3 5 5 5
Proportion of incident inquiry recommendations
implemented within agreed timescales, three monthly 1 1 2 3 4 5 4 5
update; and
Number of accidents/incidents with major hazard
potential normalised by an activity derived output figure 2 - Done at Corporate
(e.g. per 10,000 man-hours of operation) or by number 4 1 level (near misses of 3 4 5 5 5
of hours of major hazard process operation, monthly note)
update.

Emergency Preparedness

Internal Planning - To evaluate the preparedness of an

organisation to mitigate the effects of a major accident
through the creation, implementation and review of
emergency plans.
Proportion of plans reviewed within designated
1 1 1 3 3 5 5 5
timescales
Proportion of recommendations implemented following
1 1 1 2 4 5 5 5
internal review

31

 The ease of application to your organisation (on a scale of 1 to 5, one being very How useful are, or would the indicators be to your organisation (on a scale of 1 to 5,
For each of the following indicators, please rate:
easy, 5 being difficult) 1 being not very useful, 5 being very useful)

Proportion of staff hours devoted to emergency

2 2 2 2 2 4 4 5
response training
Number of ‘table top’ exercises completed against those
1 1 1 2 3 5 5 5
planned
Proportion of internal exercises completed against
1 1 1 2 3 5 5 5
those planned

External Cooperation - to ensure that adequate

arrangements are in place with external parties to
mitigate the implications of a major accident.
Number of planning meetings completed against
1 1 1 2 1 5 5 4
planned
Average number of identified external stakeholders in
1 1 1 1 2 5 5 5
attendance against those invited per meeting planned
Number of emergency exercise review meetings
1 1 1 1 3 5 5 5
undertaken against emergency exercises planned
Number of emergency exercises completed against
1 1 1 1 3 5 5 5
planned
Number of deficiencies identified during exercises 1 1 1 2 4 5 3 5

Change Control - To monitor the arrangements in

place to safely manage changes to organisational
arrangements, premises and plant.
Proportion of changes in organisational structure made
without documented evaluation of risk and identification 5 3 4 4 3 5 5 5
of controls
Proportion of changes in plant configuration made
without documented evaluation of risk and identification 5 3 3 5 4 5 4 5
of controls
Proportion of incidents occurring in areas where change
2 2 2 2 4 5 5 5
has occurred

Societal Risk - To ensure that the risk presented by the

location of the plant is mitigated in the event of an
accident through liaison with local planning authorities.

Proportion of planning application reviews completed

2 2 1 2 3 4 2 4
against submitted
Proportion of liaison meetings undertaken against
1 2 1 2 2 4 3 4
planned

2 (as forced to apply

Assessed risk to local population using a recognised consequence
2 2 1 4 Not applicable 5
societal risk technique mitigation for
explosives)

32

 The ease of application to your organisation (on a scale of 1 to 5, one being very How useful are, or would the indicators be to your organisation (on a scale of 1 to 5,
For each of the following indicators, please rate:
easy, 5 being difficult) 1 being not very useful, 5 being very useful)

Risk Control Measures - to demonstrate that risk

control measures are in place and reviewed within
designated timescales
Proportion of major hazard risk controls reviewed for
adequacy within allocated timescales against those 2 1 1 4 3 5 5 5
planned
Proportion of audits completed against planned 1 1 1 3 3 5 5 5
Proportion of procedures communicated to staff within
2 1 1 3 4 5 5 5
designated timescales
Proportion of staff competent in major risk evaluation
2 2 1 3 1 5 5 5
and control procedures
Number of non-compliances with risk controls identified
2 2 1 3 4 5 5 5
during audit

External Liaison - To ensure that interfaces with other

organisations involved in the prevention of major
accidents are appropriately managed

Proportion of safety critical tasks that are undertaken by

2 1 3 3 2 1 2 5
an external organisation
Proportion of risk review meetings undertaken with
1 1 2 2 2 1 2 5
external organisations (completed against planned)
Proportion of incidents attributable to external
1 1 1 2 3 3 3 5
organisation

33

Published by the Health and Safety Executive 11/09

 Health and Safety
Executive

Application of performance indicators

in the explosives sectors

Performance Indicators are increasingly being used

in a range of environments as an aid to identifying
and resolving safety and operational issues. Recent
incident investigations and guidance promote
enhancements to the indicator regimes to ensure
that a balanced approach is adopted that enables
organisations to monitor both the sustained use of
management controls and the effectiveness of those
controls in maintaining safety.

The Heath and Safety Executive (HSE)

commissioned this report to:

n identify the extent to which the concept of dual

assurance is understood and applied within a
sample industry;

n identify industry representatives’ opinion in

relation to potential indicators that would
correlate with the Major Hazard Performance
Indicator Framework proposed by HSL; and

n identify a process that can be used to exemplify

HSE guidance HSG254 in the derivation of
key performance indicators in the explosives
industry.

This report and the work it describes were funded

by the Health and Safety Executive (HSE). Its
contents, including any opinions and/or conclusions
expressed, are those of the authors alone and do
not necessarily reflect HSE policy.

RR759

www.hse.gov.uk