Implementing Structured Judgement Reviews

for Improvement
Introduction

Acknowledgements Mortality Review Steering Group

This toolkit is based on the Royal College of We would like to thank the members of the mortality review steering groups who have engaged with the regional
Physicians’ National Mortality Case Record programmes and enabled us to put this toolkit together.
Review Programme and the regional work
carried out by the Academic Health Science Yorkshire & Humber AHSN:
Networks (AHSN) in Yorkshire and Humber Acute Trusts: Airedale NHS Foundation Trust, Barnsley Hospital NHS Foundation Trust, Bradford Teaching Hospitals
and in the West of England.
NHS Foundation Trust, Calderdale and Huddersfield NHS Foundation Trust, Doncaster and Bassetlaw Teaching
Hospitals NHS Foundation Trust, Harrogate and District NHS Foundation Trust (national pilot site), Hull and East
Yorkshire Hospitals NHS Foundation Trust, Leeds Teaching Hospitals NHS Trust, Mid-Yorkshire Hospitals NHS Trust,
North Lincolnshire and Goole NHS Foundation Trust, Rotherham NHS Foundation Trust, Sheffield Teaching Hospitals
NHS Foundation Trust, York Teaching Hospital NHS Foundation Trust (national pilot site).
Mental Health Trusts: Bradford District Care NHS Foundation Trust, Humber NHS Foundation Trust, Leeds and
York Partnership NHS Foundation Trust, South West Yorkshire Partnership NHS Foundation Trust

West of England AHSN:

Acute Trusts: University Hospitals Bristol NHS Foundation Trust, North Bristol NHS Trust, Royal United Hospitals
Bath NHS Foundation Trust, Great Western Hospitals NHS Foundation Trust, Gloucestershire Hospitals NHS
Foundation Trust, Weston Area Health NHS Trust, Taunton & Somerset NHS Foundation Trust, Salisbury NHS
Foundation Trust.
Mental Health Trusts: 2Gether NHS Foundation Trust, Avon and Wiltshire Mental Health Partnership NHS Trust
(from November 2017).

1
Aim Licence
This toolkit aims to support the This toolkit was created in collaboration with the Royal College of Physicians, Yorkshire & Humber AHSN
implementation of the Structured Judgement Improvement Academy, and the West of England AHSN.
Review (SJR) process to effectively review the
The toolkit template is © West of England AHSN 2018, and provided under licence for use under the following
care received by patients who have died. This
terms:
will in turn allow learning and support the
development of quality improvement initiatives • Attribution — You must give appropriate credit, provide a link to the licence, and indicate if changes
when problems in care are identified. were made. You may do so in any reasonable manner, but not in any way that suggests the licensor
endorses you or your use.
This toolkit also provides information and links
to resources on change management and • Non Commercial — You may not use the material for commercial purposes.
quality improvement methodologies.
• Share Alike — If you remix, transform, or build upon the material, you must distribute your contributions
under the same licence as the original.
Who will benefit from this
• No additional restrictions — You may not apply legal terms or technological measures that legally
document?
restrict others from doing anything the licence permits.
This document is for those wishing to
implement the SJR process at a regional or
local level, with specific reference to clinicians,
managers, commissioners and trainers in
secondary and tertiary care. It should also This document is version 1.3 June 2018
be useful as a reference for community and
primary care providers.

2
Glossary Contents
AHSN Academic Health Science Network Introduction.................................................................................................................................................. 1

CQC Care Quality Commission About Academic Health Science Networks.................................................................................................... 4

DNACPR Do not attempt resuscitation Background ................................................................................................................................................. 5

National picture............................................................................................................................................ 6
EOL End of Life
Structured Judgement Reviews: an overview................................................................................................. 7
HSMR Hospital Standardised Mortality Ratio
Structured Judgement Reviews: the component parts................................................................................... 8
LD Learning Disability
Undertaking a Structured Judgement Review................................................................................................ 9
LFD Learning from Deaths
Operational process.................................................................................................................................... 10
NHSE NHS England
Case study of a Structured Judgement Review............................................................................................ 11
NHSI NHS Improvement
Where do Structured Judgement Review outcomes fit?............................................................................... 15
NMCRR National Mortality Case Record Review How to embed SJR into your organisation................................................................................................... 19
PPI Patient and Public Involvement Spreading SJR within your organisation....................................................................................................... 18
RCP Royal College of Physicians Known challenges....................................................................................................................................... 22

SJR Structured Judgement Reviews West of England AHSN Experience ............................................................................................................. 28

SMHI Summary Hospital-level Mortality Yorkshire & Humber AHSN Improvement Academy Experience.................................................................... 29
Indicator Learning and Quality Improvement............................................................................................................. 30
TEP Treatment Escalation Plan Authors ..................................................................................................................................................... 36

Resources................................................................................................................................................... 37

3
About Academic Health Science Networks (AHSNs)

What is the role of AHSNs? partnership body that bring together all partners
across a regional health economy to improve the
As the only bodies that connect NHS and
health of local communities. We have a remit
academic organisations, local authorities, the third
from NHS England to occupy a unique space
sector and industry, we are catalysts that create
outside of the usual NHS service contract and
the right conditions to facilitate change across
performance management structures. This enables
whole health and social care economies, with a
us to foster collaborative solutions. We use our
clear focus on improving outcomes for patients.
local knowledge and harness the influence of
This means we are uniquely placed to identify and
our partners to drive change on the ground and Yorkshire & Humber
spread health innovation at pace and scale; driving
integrate research into health improvement. We
the adoption and spread of innovative ideas and
are as interested in seeing healthcare businesses
technologies across large populations. Although
thrive and grow, creating jobs and bringing in
small organisations – which ensures we remain
investment to the UK, as we are in seeing the
flexible and responsive to emerging opportunities
healthcare system improve.
and challenges – we lead large regional networks.
Hence our impact rests in our ability to bring Patient Safety Collaboratives
people, resources and organisations together
Each AHSN hosts a Patient Safety Collaborative
quickly, delivering benefits that could not be
(PSC) which is commissioned by NHS
achieved alone.
Improvement. PSCs work with organisations
How are AHSNs different and nationally and regionally to help support and
distinct? encourage a culture of safety, continuous learning West of
England
and improvement.
Everything AHSNs do is driven by two imperatives:
improving health and generating economic
growth in our regions. We are the only

Yorkshire & Humber AHSN Improvement West of England AHSN

Academy [email protected] [email protected]
www.improvementacademy.org www.weahsn.net
www.yhahsn.org.uk Twitter @WEAHSN
Twitter @improve_academy

4
Background

The National Mortality Case Record • NHS Highland (Scotland) The programme is enabling closer work between
Review programme • York Teaching Hospital NHS Foundation Trust AHSNs and healthcare colleagues to address
• Harrogate and District NHS Foundation Trust deficiencies in patient care that are identified,
Most Acute Trusts have systems in place to ensure
• University Hospital of South Manchester NHS through continuous quality improvement, and
patient safety and quality of care. Many of these
Foundation Trust sharing of best practice.
include ways of reviewing hospital deaths, often
• St George’s University Hospitals NHS
by detailed review of the case notes, to identify
Foundation Trust
areas that could be improved and areas of good
• West of England AHSN.
practice that could be expanded. However, it
has been noted that there is often variability Mortality reviews
in such review processes and the extent from
Aim
which learning is gathered and utilised to inform
practice. The NMCRR programme’s aim is to improve Themes
understanding and learning about problems
In order to standardise mortality reviews across
and processes in healthcare associated with
the country the National Mortality Case Record
mortality, and also to share best practice. It Quality improvement initiatives
Review (NMCRR) programme was commissioned
helps healthcare professionals to identify
by Healthcare Quality Improvement Partnership
themes and address deficiencies in processes
(HQIP) and funded by NHS Improvement in 2016.
and patient care.
This programme is being delivered across England
and Scotland by the Royal College of Physicians in The programme aims to introduce the
partnership with the Yorkshire and Humber AHSN standardised and evidence based Structured
Improvement Academy and DATIX. Judgement Review (SJR) methodology for
reviewing case records of adult patients who have
This programme is based on the Yorkshire and
died in acute general hospitals. The primary goal is
Humber mortality review programme set up by
to improve healthcare quality through qualitative
the YHAHSN Improvement Academy in 2014.
analysis of mortality data using a standardised,
The NMCRR pilot phase ran from July 2016 to validated approach linked to quality improvement
January 2017 and the pilot sites were: activity.

5
National picture

Background Learning From Deaths Guidance inpatient deaths:

A 2016 Care Quality Commission (CQC) report 1
The National Quality Board Guidance, published
found that some organisations were not giving 1. Where family or staff concerns have been
in March 2017, sets out the following key
learning from deaths sufficient consideration raised.
requirements which will ensure organisations
and therefore missed valuable opportunities to 2. Where the patient was not expected to die, for
effectively respond to and learn from patient
identify and make improvements in quality of example an elective procedure.
deaths.
care. 3. Where an alarm has been raised such as a Dr
Each Trust should at a minimum ensure there is: Foster alert or CQC concerns.
This review was carried out in response to the 4. Where the learning will inform a provider’s
identification of the low rates of review or • Meaningful engagement and support of quality improvement work e.g. end of life care.
investigations into deaths noted at Southern bereaved families and carers.
Health NHS Foundation Trust.2 Additionally the • The introduction of structured case record There is an expectation from the Department of
review was influenced by reports into care quality reviews when reviewing patient deaths. Health that Trusts will publish quarterly data.
at Mid-Staffordshire NHS Foundation Trust3 and
It is noted that Trusts must have mechanisms to Trusts must also develop a learning from
University Hospitals of Morecambe Bay NHS
review all deaths of people: deaths policy to identify how they will meet the
Foundation Trust.4
requirements outlined in the national guidance.
1. With a Learning Disability
Based upon the findings of the review the
2. With a Serious Mental Health Illness NHS Improvement have released a number
National Quality Board (NQB) published the first
3. Those aged under 18 years of resources to support trusts implement the
edition of the National Guidance on Learning
4. Perinatal and maternal deaths requirements of the national guidance which can
from Deaths for Trusts.5
be accessed at: improvement.nhs.uk/resources/
Additionally it is advised that Trusts review all learning-deaths-nhs

1
Care Quality Commission. Learning, candour and www.england.nhs.uk/south/publications/ind-invest- 5
National Quality Board. National Guidance on
accountability. A review of the way NHS trusts review reports/wessex/southern-health/ Learning from Death: A Framework for NHS Trusts
and investigate the deaths of patients in England. 2016. and NHS Foundation Trusts on identifying, reporting,
www.cqc.org.uk/sites/default/files/20161213-learning- 3
Report of the Mid Staffordshire NHS Foundation Trust investigating and learning from deaths in care.
candour-accountability-full-report.pdf Public Inquiry (“Francis report”). February 2013. March 2017. www.england.nhs.uk/wp-content/
www.midstaffspublicinquiry.com/ uploads/2017/03/nqb-national-guidance-learning-from-
2
Independent review of deaths of people with a deaths.pdf
Learning Disability or Mental Health problem in contact 4
Report of the Morecambe Bay Investgation (“Kirkup
with Southern Health NHS Foundation Trust April 2011 report”). 2015. www.gov.uk/government/organisations/
to March 2015. December 2015. morecambe-bay-investigation

6
Structured Judgement Reviews: an overview

Background Strengths of an SJR How the SJR method works

SJR is a standardised, yet not rigid, case notes The benefits of utilising the SJR methodology A SJR is usually undertaken by an individual
review methodology usable across services, teams is that it provides a structured and replicable reviewing a patient’s death and mainly comprises
and specialties. SJR blends traditional, clinical- process to review deaths, which examines both of two specific aspects; namely explicit judgement
judgement based, review methods with a standard interventions and holistic care giving reviewers a comments being made about the care quality and
format. This approach requires reviewers to make rich data set of information.4 care quality scores being applied. These aspects
safety and quality judgements over phases of care, are applied to both specific phases of care and to
The SJR methodology allows organisations to
to make explicit written comments about care for the overall care received.
ask ‘why’ questions about things that happen to
each phase, and to score care for each phase.1
enable learning and actions where required. The phases of care are as follows:
Since 2014, the Yorkshire & Humber AHSN
SJR allows the identification and feedback of • Admission and initial care – first 24 hours.
Improvement Academy has supported the uptake
good care in the same detail as ‘problematic’ care, • Ongoing care.
of SJR by its acute and mental health trusts.2
which is integral as evidence suggests most care is • Care during a procedure.
The Improvement Academy has successfully
of good or excellent quality and that there is much • Perioperative/procedure care.
standardised mortality review methodology
to be learned from the evaluation of high-quality • End-of-life care (or discharge care).
across all 13 acute trusts in Yorkshire and
care. • Assessment of care overall.
Humber. This work has subsequently led to the
NMCCR programme and to SJR being one of Whilst the principle phase descriptors are noted
the recommended tools for the review of patient above, dependant on the type of care or service
deaths, as outlined by the NHS Improvement the patient received not all phase descriptors may
guidance on implementing the National Quality be relevant or utilised in a review.
Board’s learning from deaths framework.3

1
Hutchinson A, Coster JE, Cooper KL, Pearson M, Bradford, The Yorkshire and the Humber Improvement March 2017. www.england.nhs.uk/wp-content/
McIntosh A, Bath PA. A structured judgement method Academy. www.improvementacademy.org/documents/ uploads/2017/03/nqb-national-guidance-learning-from-
to enhance mortality case note review: development Projects/mortality_review/IA%20SJR%20Report-%20 deaths.pdf
and evaluation. BMJ Quality and Safety 2013;22:1032– 2015.pdf
1040. DOI: 10.1136/bmjqs-2013-001839 4
Royal College of Physicians. Using the Structured
3
National Quality Board. National Guidance on Judgement Review method ­- A guide for reviewers.
2
Hutchinson A, McCooe M & Ryland E. 2015. A Learning from Death: A Framework for NHS Trusts London: RCP, 2017. www.rcplondon.ac.uk/sites/
guide to safety, quality and mortality review using and NHS Foundation Trusts on identifying, reporting, default/files/media/Documents/NMCRR%20guide%20
the structured judgement case note review method. investigating and learning from deaths in care. England_0.pdf

7
Structured Judgement Reviews: the component parts

Explicit Judgement Comments It is recommended that explicit statements use Phase of care scores
judgement words and phrases e.g. ‘good’,
Here the reviewer makes explicit judgement Once explicit judgement comments are made, the
‘unsatisfactory’, ‘failure’ or ‘best practice’.
comments on the phase/overall care reviewed reviewer then applies a phase of care/overall care
which allows the reviewer to concisely describe RCP examples of explicit judgement comments: score.1
and assess the safety and quality of care provided.
Very good care – rapid triage and Only one score is given per phase of care and is
Judgement comments can be made on anything identification of diabetic ketoacidosis with not required for each judgement statement.
the reviewer thinks is pertinent to a particular appropriate treatment.
This allows the reviewer to come to a rounded
case, including technical aspects of care such as
Overall, a fundamental failure to recognise judgement on the phase of care being reviewed,
management plans, whether care meets good
the severity of the patient’s respiratory which is particularly useful when there is a mix of
practice and the interventions undertaken. More
failure. good and poor elements of care.
holistic aspects of care such as end-of-life decision
making and involvement of families are also Therefore a phase of care could identify elements
reviewed. of poor care and still be rated a positive score
overall if there were also elements of care that
were very good.

The following care scores are used:

1. Very poor care

2. Poor care
3. Adequate care
4. Good care
5. Excellent care

1
Royal College of Physicians. Using the Structured
Judgement Review method - Data collection form.
London: RCP, 2017 https://www.rcplondon.ac.uk/
file/5065/download?token=ad_j5n6M

8
Undertaking a Structured Judgement Review

Assessment of problems in Overall care scores and further Problem types

healthcare review
1. Problem in assessment, investigation or
Whilst the explicit judgement comments and care Overall care scores are integral to the review diagnosis (including assessment of pressure
scoring are the main two elements of an SJR, process. A score of 1 or 2 is given when the ulcer risk, venous thromboembolism (VTE)
reviewers will subsequently be asked to make an reviewer judges the care overall is either poor or risk, history of falls).
assessment of problems in healthcare. The reviewer very poor.
2. Problem with medication / IV fluids/
is asked to comment on whether one or more
If a first stage review judges that the overall care electrolytes/ oxygen (other than anaesthetic).
specific types of problems were found and, if so,
score is less than three and either poor (2) or very
identify if it is deemed this led to harm. Problem 3. Problem related to treatment and
poor (1) then the case should be subject to further
types are listed in the box to the right. management plan (including prevention of
scrutiny.
pressure ulcers, falls, VTE).
This may take a number of forms depending upon
4. Problem with infection management.
the detail of the governance structure within
organisations. 5. Problem related to operative/ invasive
procedure (other than infection control).
The purpose of the on-going review in these
circumstances is to define any further action 6. Problem in clinical monitoring (including
needed. Typically poor or very poor care will failure to plan, to undertake, or to recognise
attract an analysis or investigation which aims and respond to changes).
to understand the reasons for poor care and to
provide comment on the possibility of the care 7. Problem in resuscitation following a
having contributed to the death of the patient. cardiac or respiratory arrest (including
cardiopulmonary resuscitation (CPR)).
It is important to note that the SJR cannot comment
on, nor describe, the “avoidability” of a patient’s 8. Problem of any other type not fitting the
death. categories above including communication
and organisational issues.

9
Operational process

This flowchart (Figure 1) provides an example

of the operational processes a trust may follow
Figure 1
when undertaking an SJR. Exclusion criteria
• Cases already in the Serious
It should be noted that this is simply an Incident process or in legal
process/ coroner’s inquests
interpretation of the inclusion and exclusion
criteria, governance processes and feedback
mechanisms a trust could potentially follow
and is not meant to be prescriptive.
Inclusion Criteria
• All deaths in specialities with
<10 deaths/month
• Deaths in specialities with > 10
deaths/month identified through • Investigation
case selection process if deemed
appropriate by
governance
team/mortality
lead.
• Duty of
Case selection process candour • Feedback to
• Must dos from national Problems process. specialities/
guidance – patients with severe in care divisions
mental health illness / learning identified / • Link to M&M
disabilities, elective admission overall care meetings
deaths, concerns from families/ scores <3 • Thank you
carers etc. letters to
Themes/
• Local criteria, for example, staff/teams
SJRs learning
deaths due to specific diagnosis • Link to quality
points
such as sepsis /stroke, deaths improvement/
from cardiac arrests, concerns Good practice patient safety
raised by staff identified teams
• A sample of cases selected out • National
for quality assurance purposes. reports
• Board reports

10
Case study of a Structured Judgement Review

89 Year old male admitted 28/09/2017 - 30/09/2017 continued deterioration of condition

The following case study provides a fictional
23:05 - from Nursing Home. despite treatment including micro recommended
account of a patient death which has
IV antibiotic regime.
undergone an SJR. It outlines the processes for Presenting complaint – Increasing shortness of
case selection, explicit judgement comments breath 01/10/2017 – Developed Type 2 Respiratory
and care scores allocated. Failure, review by ITU – not for Non Invasive
Past Medical History – Myocardial Infarction
Ventilation – ward level ceiling of treatment.
Key learning points from this case include the x2, Hypertension, Type 2 Diabetes Mellitus,
potential need for earlier ceiling of treatment Dementia. Admitted to Nursing Home 10 Re-cannulated for intravenous fluids as cannula
decisions and end of life recognition. weeks previously due to functional decline. No tissued. DNACPR subsequently signed – symptom
community Do Not Attempt CPR (DNACPR) or trigger started and active intervention stopped.
Treatment Escalation Plan (TEP) in place.
Family informed of decision.
Background – GP review 27/09/2017 –
Patient died at 21:35 – 01/10/2017.
Explicit judgement comments from the diagnosed likely Lower Respiratory Tract Infection
reviewer are shown as handwritten (LRTI), started on oral Amoxicillin. 02/10/2017 – Discussion with Bereavement
notes in blue. Office, family raised concerns regarding
Initial Assessment – Observations stable –
involvement in care and end of life decisions.
NEWS 1 – HR -92, purulent sputum – sample sent
to micro, oral antibiotics to continue.

Patient treated for LRTI. Increased confusion Case meets automatic inclusion criteria outlined
noted in morning 29/09/2017 – NEWS 2 – HR in Figure 1 – as family concerns had been raised.
99, Sp02 – 95% on air. Further deterioration in Case therefore subject to an SJR. Due to overall
condition noted in evening 29/09/2017 – NEWS care score and no problems in care identified not
5 – BP 102/70, HR – 110, Temp 38.3, Sp02 – for further review.
92% on air. Sepsis bundle started, consolidation
on chest x-ray – IV Tazocin commenced.
Catheterised to monitor fluid output – although
recording accuracy limited (6 hours without urine
output measure)

11
Phase of care: Admission and initial Phase of care: Ongoing care Phase of care: Care during a procedure
management (approximately the first 24 hours)
Please record your explicit judgements about the Please record your explicit judgements about the
Please record your explicit judgements about the quality of care the patient received and whether quality of care the patient received and whether
quality of care the patient received and whether it was in accordance with current good practice it was in accordance with current good practice
it was in accordance with current good practice (for example, your professional standards or your (for example, your professional standards or your
(for example, your professional standards or your professional perspective). If there is any other professional perspective). If there is any other
professional perspective). If there is any other information you think is important or relevant that information you think is important or relevant that
information you think is important or relevant that you wish to comment on then please do so. you wish to comment on then please do so.
you wish to comment on then please do so.
• Despite increased confusion, medical Not applicable, no procedures performed.
• Thorough admission clerking, with clear team not specifically asked to
Please rate the care received by the patient during
and concise notes and management review patient. Patient only seen by
this phase. Please circle only one score.
plan. junior staff on ward round which is
• Good background history obtained inadequate. 1. Very poor care 2. Poor care 3. Adequate care
from patient and wife. • Good escalation of concerns when 4. Good care 5. Excellent care
• Early senior review on Acute Medical NEWS increased, with senior registrar
Unit with prompt and effective review who commenced sepsis bundle
handover of care to Care of the Elderly as per guidelines, resulting in prompt
ward. administration of IV antibiotics and IV
• Handover from ambulance documented fluids.
no community DNACPR or TEP in • Patient catheterised which was
place, however unfortunately no early adequately documented and clinically
discussion with patient and family indicated for accurate fluid output.
documented regarding escalation However accuracy of fluid output
plans despite patient being an elderly recording in nursing notes was poor.
gentleman with co-morbidities. This is • Relatively timely review requested
suboptimal practice. from ITU.

Please rate the care received by the patient during Please rate the care received by the patient during
this phase. Please circle only one score. this phase. Please circle only one score.

1. Very poor care 2. Poor care 3. Adequate care 1. Very poor care 2. Poor care 3. Adequate care
4. Good care 5. Excellent care 4. Good care 5. Excellent care

12
Phase of care: Perioperative care Phase of care: End of Life Care Phase of care: Overall assessment

Please record your explicit judgements about the Please record your explicit judgements about the Please record your explicit judgements about the
quality of care the patient received and whether quality of care the patient received and whether quality of care the patient received and whether
it was in accordance with current good practice it was in accordance with current good practice it was in accordance with current good practice
(for example, your professional standards or your (for example, your professional standards or your (for example, your professional standards or your
professional perspective). If there is any other professional perspective). If there is any other professional perspective). If there is any other
information you think is important or relevant that information you think is important or relevant that information you think is important or relevant that
you wish to comment on then please do so. you wish to comment on then please do so. you wish to comment on then please do so.

Not applicable, no procedures performed. • It was noted that there was a delay • Patient received generally good
in identifying patient was reaching care during inpatient stay, which
Please rate the care received by the patient during
end of life (EOL), resulting in a delay involved good quality initial clerking
this phase. Please circle only one score.
in DNACPR being signed. Due to delay and deterioration identification and
1. Very poor care 2. Poor care 3. Adequate care patient was unnecessarily cannulated treatment.
4. Good care 5. Excellent care on day of death. • However opportunities were missed
• Whilst family were informed of to discuss treatment escalation plans
decision made by senior medic, they early which resulted in a delay in
were not involved in discussions which patient commencing an EOL pathway.
was inappropriate. Additionally the patient’s family were
• Once patient was identified as EOL a not appropriately involved in this
symptom trigger was commenced which discussion.
was regularly completed resulting in
Please rate the care received by the patient during
patient receiving appropriate EOL care
this overall phase. Please circle only one score.
with symptom control.
1. Very poor care 2. Poor care 3. Adequate care
Please rate the care received by the patient during
4. Good care 5. Excellent care
this phase. Please circle only one score.
Please rate the quality of the patient record.
1. Very poor care 2. Poor care 3. Adequate care
Please circle only one score.
4. Good care 5. Excellent care
1. Very poor 2. Poor 3. Adequate
4. Good 5. Excellent

13
Assessment of problems with healthcare Problem related to treatment and 6. Problem in resuscitation following a cardiac or
management plan (including prevention of respiratory arrest (including cardiopulmonary
In this section, the reviewer is asked to comment
pressure ulcers, falls, VTE) resuscitation (CPR))
on whether one or more specific types of
problem(s) were identified and, if so, to indicate Yes No Yes No
whether any led to harm.
Did the problem lead to harm? Did the problem lead to harm?
Were there any problems with the care of the No Uncertain Yes No Uncertain Yes
patient? (Please tick)
3. Problem with infection management 7. Problem of any other type not fitting the
No (please stop here) categories above
Yes No
Yes No
Yes (please continue below) Did the problem lead to harm?
No Uncertain Yes Did the problem lead to harm?
If you did identify problems, please identify which
problem type(s) from the selection below and 4. Problem related to operative/ invasive No Uncertain Yes
indicate whether it led to any harm. Please tick all procedure (other than infection control)
that relate to the case. Adapted from Hogan H, Zipfel R, Neuberger J,
Yes No
Hutchings A, Darzi A, Black N. Avoidability of hospital
Problem types
Did the problem lead to harm? deaths and association with hospital-wide mortality
1. Problem in assessment, investigation or No Uncertain Yes ratios: retrospective case record review and regression
diagnosis (including assessment of pressure analysis. BMJ 2015;351:h3239/ DOI: 10.1136/bmj.
5. Problem in clinical monitoring (including
ulcer risk, venous thromboembolism (VTE) h3239
failure to plan, to undertake, or to recognise
risk, history of falls)
and respond to changes)
Yes No
Yes No
Did the problem lead to harm?
Did the problem lead to harm?
No Uncertain Yes
No Uncertain Yes
2. Problem with medication / IV fluids/
electrolytes/ oxygen (other than anaesthetic)
Yes No

Did the problem lead to harm?

No Uncertain Yes

14
Where do Structured Judgement Review outcomes fit?

Reviews, SJR and Case Investigations It is important to recognise that neither the
review nor the SJR methodology can generate
The terms review, structured judgement
an outcome which describes if the care that
review or just SJR and case investigation that
was observed was more likely than not to have
appear in this document have clear definitions.
contributed to the death of the patient.
A review of the case notes, which is also in some
Investigations into the quality of care received by
cases referred to as a screening of the case notes
patients’ is therefore a fundamentally different
is any non-validated, variously structured and
process from the retrospective case note reviews
usually relatively brief review of the case notes.
described. An investigation is a formal process
As such these reviews are variable in quality and
where an opinion is formed, usually by a group
cannot create a validated care score. Some simple
of clinicians and clinical governance experts,
reviews of this type may be lengthy and complex
on the standard of care delivered and crucially,
but still do not generate a validated care score.
in the context of this document, whether the
The SJR is a validated research methodology care received was more likely than not to have
which is able to create an overall care score. The contributed to the death of the patient. The
methodology used is explained in more depth investigation will usually draw on evidence
elsewhere in this toolkit. from a variety of sources which will in many
circumstances include the outcome of the
Both simple reviews and the SJR are retrospective validated SJR.
analyses of case notes and both have the ability to
generate comment on the quality of care that is
delivered. In addition, the SJR methodology allows
the reviewer to comment as to whether harm
had occurred. Both methods can be used to “flag
up” poor care and trigger further inquiry into that
quality of care.

15
Serious incident reporting framework Data collection and reporting
and Duty of Candour
A key part of the NMCRR programme has been
As part of the SJR methodology reviewers make to develop and build an on-line platform to
an assessment of problems in healthcare which enable mortality reviews to be aggregated by
may have resulted in harm. Some deaths may Trusts and Health Boards and to conduct analysis
subsequently be identified as being subject to the to facilitate learning and quality improvement
NHS England (NHSE) Serious Incident Reporting initiatives. Since June 2017, the NMCRR core
Framework and the CQC Duty of Candour team has implemented the RCP National Mortality
requirements. Review platform in 15 Trusts and Health Boards
throughout England and Scotland with a further
It is therefore recommended that Trusts undertake
40 signed up to implement during early 2018.
SJRs in a timely manner, ideally within 6 weeks,
We continue to actively recruit Trusts and Health
to ensure Duty of Candour processes can be
Boards wishing to implement the on-line platform.
followed at the most appropriate time.
The data entered into the on-line system will allow
Issues with care which meet the definition of
Trusts to collate cases to enable them to report the
a patient safety incident (any unintended or
numbers and types of reviews undertaken. The
unexpected incident which could have or did
data is not intended to contribute to the national
lead to harm to one or more patients receiving
reporting framework described in Learning from
NHS care) should be reported via local risk
Deaths neither will allow any comparisons of
management systems to the National Reporting
outcomes to be constructed.
and Learning System (NRLS).
More complex governance processes within Trusts
Further information regarding the requirements of
will be required to allow these latter metrics to be
the NHSE Serious Incident Reporting Framework
created and published.
and CQC guidance on the Duty of Candour
can be found at improvement.nhs.uk/uploads/
documents/serious-incidnt-framwrk.pdf and
www.cqc.org.uk/sites/default/files/20150327_
duty_of_candour_guidance_final.pdf

16
How to embed SJR into your organisation

The following pages outline how organisations Table 1: Key representatives to consider
may approach the development of their
• Executive Board sponsorship, including the Trust Medical Director and a Non-Executive Director responsible
learning from problems in care.
for overseeing learning from deaths (as outlined in the National Guidance)

• A project leader, who has change and quality improvement experience. (Ideally a senior clinician)
Identify project team members & roles
• Senior medical representation from each relevant clinical division
and responsibilities
• Managerial representation.
Depending on your organisation, set up your
team. The group could be incorporated as part • Non-medical clinical representation including nursing, allied health professionals and pharmacy
of an existing mortality or patient safety group,
or alternatively it could be established as a • Trust Quality Improvement team member
distinct group.
• Managerial representation
In either case, you will need to ensure it is
• Community representation including a GP, a clinician with experience in mental health and a clinician with
made up of key representatives of groups that
experience with learning disabilities.
will be affected by subsequent changes.
• A safeguarding team member and clinical risk team member.
Table 1 offers some suggestions about who
might be included. The list is not prescriptive • A patient experience team member, bereavement office / patient advice and liaison service team member
and may be dependent on your organisation. and a Chaplain.

• Support function team members including an audit team member, IT professional, administrative support
and legal team member.

• Patient and public representatives

17
Spreading SJR within your organisation: public and family involvement

“As a public contributor on the

It is advisable for your team to include public The PPI team at the West of England AHSN have
Mortality Review Steering Group we
and family representatives, which may include produced a PPI toolkit which provides useful
are, in partnership with our colleagues
existing trust public and patient involvement (PPI) resources for professionals who are looking to
from the acute hospital trusts and the
representatives, who are able to provide the group understand how to best involve the public, patients
West of England AHSN pleased that
with appropriate insights on how the changes and families, available at www.weahsn.net/ wp-
the importance of the public voice in
could best meet the needs of families and carers content/uploads/PPI_Toolkit.pdf
informing the valuable work of the
who suffer bereavement. This involvement can take
The Yorkshire & Humber AHSN Improvement group is recognised. Together, we aim
a number of formats; however it is best if such
Academy has produced three manuals on how to to ensure that a system which reviews
team members are involved in co-producing these
plan for PPI in projects, work with PPI panels and all deaths of elective patients and a
processes.
budgeting available at www.improvementacademy. proportion of those admitted as an
Example 1. Within the West of England AHSN org/about-us/patient-and-public-engagement/ emergency is established by all acute
area, PPI representatives have been present on the hospital trusts in the West of England,
Guidance on developing PPI role descriptions can so that learning from such reviews,
Mortality Review Steering Group and have provided
be found at www.rds-yh.nihr.ac.uk/wp-content/ results, as appropriate, in improved
significant insight and influence on how structured
uploads/2015/01/RDS_PPI-Handbook_2014-v8- health services delivery.”
judgement reviews could be implemented to best
FINAL-11.pdf
meet families and carers needs.
Christine Teller, Public contributor
Example 2. Within the Yorkshire & Humber region, West of England AHSN
a Carers and Relatives Involvement subgroup has
been set up to inform the regional steering group.
See page 25. “It is very encouraging that the public is
involved in this very important work, so
that the mortality review programme
is not only driven by clinical and/or
budget pressures, but the voices of the
family/carer are heard loud and clear.”

Barbara Stephenson, Public contributor

Yorkshire & Humber AHSN

18
Spreading SJR within your organisation: identifying barriers to change

Figure 3: Example barriers to implementation

A large part of the role of the team will be to ensure
that the learning from SJRs translate into improvement
Project related Resource related
actions. The team will need to establish where barriers to
implementation exist and discuss these at team meetings. • Lack of leadership support • Competing priorities
• Weak sponsorship • Time pressure
Commitment planning is a useful way of looking at • Lack of accountability • Work pressure / overloaded
stakeholders’ commitment thus articulating where • Lack of consensus workforce
barriers exist, and prompting where actions may be • Lack of control plan / measures • Shortage of internal resources
required to address these (see Figure 2). • Financial cost

Some barriers can be avoided by the way that the

steering group is established and because of the skills of
the membership. Others are external to the group.

The list in Figure 3 is another approach that can be used

to help you think through the various factors that might Staff related Organisation related
be the cause of the resistance. The list is not exhaustive
• No perceived benefit from • Unanticipated events
and will depend on your context. You may find it helpful implementation • Cultural change
to use a forcefield diagram (see Figure 4) to analyse the • No perceived relevance from • Lack of communication
forces for and resistance to change. change
• Staff fear of change
Figure 2. Commitment planning diagram
• Resistance to change
• Motivation and engagement

Figure 4: Forcefield analysis

Forces for change Proposed

Forces resisting change
change

19
Spreading SJR within your organisation

Culture Training By the end of the training phase we will have

trained around 360 Tier One Trainers throughout
Culture can play a significant influencing role Although this toolkit provides an overview of
England. We are currently working with Scottish
on the speed, effectiveness and lifespan of the processes and benefits of SJRs, it should
colleagues to continue their training throughout
improvement initiatives within organisations. not negate the need to undergo specific SJR
2018.
methodology training.
Whilst the implementation of the practical
Within the Yorkshire & Humber area over 750
processes of SJR may occur relatively quickly, Integral to the NMCRR programme is the training
clinical staff from thirteen acute and four mental
developing an environment in which the learning of healthcare professionals to conduct mortality
health trusts have been trained across specialties,
and actions gathered through the SJR process reviews. Following the publication of Learning
departments and roles from consultants and
are effectively utilised to deliver high quality care from Deaths by the National Quality Board in
registrars to specialist nurses and patient safety
requires an open, honest and learning focused March 2017, the approach to training in England
leads.
culture. changed. In an effort to ensure that capacity
and capability exists to train in-hospital mortality Within the West of England AHSN all six acute
A number of publications located in the
reviewers more quickly, it was decided not to visit trusts within the region, and two outside the
recommended resources section of this toolkit
Trusts and Health Boards to train reviewers but region, have been trained in undertaking SJRs by
explore the role of culture on change in more
instead to hold a minimum number of training West of England AHSN regional Tier One trainers.
detail and include recommendations on how high
sessions throughout August 2017 – January 2018 In total over 135 cascade trainers now exist in the
performing and learning organisational cultures
aimed at training Tier One Trainers. These trainers region with cascade training delivered to over 400
can be developed.
sit regionally as a resource for Trusts to access to clinica staff.
train in-hospital reviewers.

Leadership A list of Tier One Trainers, their locations and

contact details is available via the RCP mortality
It is apparent that effective trust and divisional webpage www.rcplondon.ac.uk/mortality
leadership is integral to the implementation of
SJR, with a specific focus on clinical leadership.
Such leadership will not only be the driving force
for implementing and spreading the use of SJR
but will also be intrinsic in developing the open
and learning focused culture discussed above.

20
Make contact with others The inclusion of such members has enabled Yorkshire & Humber experience
the steering group to develop the processes for
Whilst each Trust is likely to follow a different Within Yorkshire & Humber, Acute and Mental
shared learning across the system, which has been
implementation process with regards to SJRs; Health Trusts’ mortality leads come together
recognised as integral for delivering higher quality
making contact with others and having a quarterly to share learning, achievements and
and safer care as patients are rarely cared for by
collaborative platform for shared learning is highly challenges; shaping the programme bottom-up.
an individual organisation alone.
recommended.
Mortality leads tell us they feel empowered by
The West of England AHSN has supported the
these meetings and being able to share their local
development and sharing of resources for member
challenges and explore solutions as a group.
West of England experience organisations including operational process maps
and educational material. Challenges addressed as a group include the
Within the West of England, Clinical and mortality
development of robust local case selection tools
leads from across the region meet on a quarterly
and systematic identification of learning disabilities
basis, interspersed with monthly Steering
deaths.
Group calls to share progress and the learning
from implementation, as well as the number A separate carers and relatives involvement
and outcomes of reviews. The Collaborative is subgroup informs the steering group.
supported by 2 GPs and 2 public contributors
who aid the discussions on involving relatives
and carers, and how to take the out of hospital
learning forward. The 2 Mental Health trusts in
the region have recently joined the group.

Non-executive and Executives of our participating

organisations were periodically invited to attend
the quarterly face to face meetings to apprise
them of individual and regional progress. This
enabled us to gain buy in at a senior level for this
work that supported those making changes at
team level.

21
Known challenges

What is the challenge? What are the potential ways forward?

During the implementation of SJRs within the When it comes to problem solving through issues, you can use this framework:
Yorkshire and Humber and West of England
• What is the problem?
AHSN regions a number of shared issues and
• Why is it a problem?
challenges became apparent.
• When is it a problem?
Such challenges and potential solutions will be • Where is it a problem?
explored in further detail within this section. • How is it a problem?
• Who is it a problem for?

If you don’t truly understand the problem, you cannot solve it! The cornerstone of any effective root cause
analysis is having an accurately defined problem.

Using robust problem solving techniques will ensure you address the ‘real’ issue – not just the symptoms. It’s not
difficult - just have a questioning attitude. Never stop with the first reason given or the obvious.

There may be multiple root causes for any given problem. Make sure you follow all of them through – they may
all need fixing!

The five whys is a tool that helps to identify the root cause of a problem by verbally questioning the reasons
given. It enables the peeling away of layers through a process of questions repeatedly asking “why” until you
reach the root cause.

22
Known challenge: identifying deaths of people with a learning disability

What is the challenge? What are the potential ways forward?

Often, Acute Trusts have difficulties in The LeDeR review team have produced a number of briefing papers which include the programme’s definition
identifying and tracking people with learning of a learning disability and guidance on identifying the scale of this disability. These resources can be accessed at:
disabilities (LD) through the system to ensure
• www.bristol.ac.uk/media-library/sites/sps/leder/Briefing%20paper%201%20-%20What%20do%20 we%20mean%20
that deaths of people with a LD undergo a by%20learning%20disabilities%20V1.2.pdf
mortality review..
• www.bristol.ac.uk/media-library/sites/sps/leder/12.%20Identifying%20the%20degree%20of%20a%20 person’s%20
Nationally, the Learning Disabilities Mortality learning%20disabilities.pdf
Review (LeDeR) Programme, delivered by the
Trusts within the West of England AHSN steering group are working with Mental Health Trust partners to
University of Bristol, is developing and rolling
identify an agreed definition for the steering group.
out a review process for the deaths of people
with learning disabilities, helping to promote Within the Yorkshire and Humber AHSN region, Acute Trusts work closely with their local LD liaison nurses and
and implement the new review process, coding departments ensuring that patients with LD who have died as an inpatient are flagged up for a SJR.
and providing support to local areas to take For example, in Leeds Teaching Hospitals NHS Trust, all deaths coded as LD are reviewed for appropriateness of
forward the lessons learned in the reviews coding by the lead LD nurse. The nurse, additionally, performs a holistic assessment of care which is triangulated
in order to make improvements to service with the SJR findings. Trust wide learning is reported through the Mortality Improvement Group. All LD reviews
provision. are also referred to the regional LeDeR programme.

Further information can be found at: Nationally, to help with the tracking of people with a learning disability between primary and secondary care,
www.bristol.ac.uk/sps/leder some learning disability liaison teams utilise information contained within the GP QOF learning disability register
to flag people with a learning disability who are admitted to hospital.

Some areas are also utilising summary care records (SCR) with additional information to ensure a person’s
learning disability is included in their record.

Within West of England AHSN , a steering group member is working within their local partners towards
ensuring all who are on the GP QOF learning disability register have a SCR with additional information. It should
additionally be noted that NHS Digital is working to develop a process that will flag the records of people with a
learning disability on the NHS Spine which is accessible to all providers.

23
Known challenge: defining severe mental health illnesses

What is the challenge? What are the potential ways forward?

Under the National Quality Board guidance, NHS Improvement currently recognises that while there is no single definition of the conditions which would
organisations must review deaths of all constitute a SMI, that this is generally restricted to the psychoses such as schizophrenia and bipolar disorder.
patients with severe mental health illnesses.
However, it recognises that personality disorders, eating disorders, obsessive compulsive disorder and substance
However, it is noted that there is no nationally
misuse can be just as severe and disabling. It therefore currently recommends that whilst the former disorders
agreed definition for severe mental health
meet the criteria for a SMI, trusts can also choose to review the deaths of those with other significant mental
illness (SMI) or what methodology should be
health needs, as mentioned, if this can be done proportionately and effectively.
used when undertaking these reviews.
Nationally, work is underway by the Royal College of Psychiatrists to better clarify the expectations of mortality
reviews of people with mental health illnesses, including definitions, and develop a review methodology for
those under the care of mental health and community services. It is currently recommended that Acute Trusts
utilise SJR or another suitable methodology to review the acute care of those with severe mental health needs.

A number of Acute Trusts within the Yorkshire and Humber AHSN and West of England AHSN areas have been
reviewing deaths of patients under the following categories:

• under section,
• under a deprivation of liberty safeguard (DoLS)
• under the care of a secondary care mental health team such as a mental health liaison team.

Within the Yorkshire and Humber AHSN area , the Improvement Academy has been working with four mental
health trusts since 2014, supporting the uptake of SJR for the review of mental health deaths. We have adapted
the SJR tool to create phases of care headings more suitable for mental health reviews such as risk assessment
and allocation of care. It is also important to understand the life lived by the person, the range of comorbidities
and not just what happened at their death. More information on our experience is available in Annex J of the
National Quality Board Guidance:
www.england.nhs.uk/wp-content/uploads/2017/03/nqb-national-guidance-learning-from-deaths.pdf

Additionally, the Improvement Academy has recently set up a separate mental health mortality review
programme using an adapted SJR tool. For more information visit
www.improvementacademy.org/patient-safety/mortality-review-programme.html

24
Known challenge: families/ carers involvement in the review process

What is the challenge? What are the potential ways forward?

Nationally, Trusts have recognised the It is crucial for Trusts to have systems in place to capture concerns and complaints from bereaved families and
challenge of systematically embedding the carers. Within Yorkshire & Humber AHSN region, a number of methods have been adopted by organisations to
voices of bereaved families and carers into allow for families’ and carers’ voices to feed into mortality review processes.
their local mortality review processes.
The format depends largely on local organisational structures and availability of resources. For example, Hull
and East Yorkshire Hospitals NHS Trust issues questionnaires to all bereaved families through its bereavement
office aiming to provide a route for families’ concerns to inform its case selection process. Another Trust sends
letters to families of patients whose deaths have been identified as requiring SJRs. The letter requests families to
feedback to the mortality lead any identified potential issues in the care received by the deceased.

Since November 2017, Yorkshire & Humber AHSN Improvement Academy has also been working with its Carers
and Relatives Involvement Subgroup to develop a flexible framework for the systematic embedding of families’/
carer’s voices at the various steps of the mortality review process including case selection, review and learning.
For more information on when the framework will be available please contact the Improvement Academy.

In the future, changes to the process of death certification, in which deaths will be scrutinised by a medical
examiner, will result in bereaved families being systematically given an opportunity to raise concerns regarding
their relatives care. However, it is noted that the introduction of the medical examiners role is not expected until
April 2019 and therefore Trusts should endeavour to develop effective and appropriate mechanisms for families
to raise concerns. Further information regarding the medical examiner role can be found at: www.gov.uk/
government/uploads/ system/uploads/attachment_data/file/517184/DCR_Consultion_Document.pdf

Where a review identifies problems in care, Duty of Candour processes should be followed. Families should be
offered a genuine apology, be informed and involved in the investigation process, be given an appropriate lead
point of contact to discuss questions and concerns with and finally be informed of the learning and actions
developed from the investigation.

Additional resources regarding family involvement and bereavement support can be found in the recommended
resources section. NHS England are due to develop guidance on how best to engage bereaved families and
carers.

25
Known challenge: clinician engagement

What is the challenge? What are the potential ways forward?

To enable SJR to be effectively utilised for Workload
organisational learning and improvement,
It is recognized that undertaking SJRs often requires significant dedicated organisational resources and reviewers
it is integral to ensure that there is effective
time. Within the Yorkshire & Humber AHSN and West of England AHSN regions, Trusts have found it useful
engagement with clinicians. Such engagement
to develop case selection processes to identify cases that require structured judgement reviews. This ensures
relates to both the workload implications of
all appropriate deaths are reviewed whilst reducing workload on clinicians. It is suggested that trusts review a
routinely undertaking SJRs and the implications
sample of those deaths ‘selected out’ for quality assurance purposes. Some trusts in the West of England have
of feeding back review outcomes to the
spread reviews across specialties to help with workload, whereas others have kept reviews within specialty, but
clinicians/teams involved in the care.
completed by a reviewer independent of care.

Furthermore, within the West of England AHSN steering group, one Trust has utilised a charitable donation
to fund a case selection nurse, who will review all deaths against the selection tool to identify cases to be
taken forward to an SJR, thus, releasing clinician capacity. Finally, the trust also agreed that reviews would
be undertaken at a rate of 2 reviews per clinician per month, with clinicians recording how long they spend
conducting mortality reviews to enable this to be included in subsequent year’s job plans.

Feedback

Feedback is an essential component of learning. Departmental and speciality specific themes can be fed back
through a number of routes including M&M meetings , speciality level dashboards or via divisional leads.
When problems in care are identified feedback should be carried out in a no blame manner. The onus is on the
mortality/governance group rather than the reviewers to carry out this task.

It is important to recognise good care provided. Within the Yorkshire & Humber AHSN region, Doncaster and
Bassetlaw Teaching Hospitals NHS Trust routinely acknowledges exemplary practice by either individuals or ward
teams by way of a letter from the Deputy Medical Director.

Finally, steering group members have recognised the importance of feeding back and reviewing good and
excellent care, alongside poor care, to identify, learn from, and spread examples of high quality care.

26
Known challenge: the community care ­- hospital interface

What is the challenge? What are the potential ways forward?

Hospital teams in both AHSNs have With respect to exploring ways to feed back review outcomes, the West of England AHSN steering group has
highlighted challenges regarding feeding back provided a useful forum to discuss such communication issues and identify potential solutions. Such discussions
review outcomes to external partners including have been enhanced by , for example, the inclusion of representation from local GP’s and a CCG clinician.
community and primary care providers and Similar initiatives have occurred in the YHAHSN.
also reviewing deaths post discharge (within
Some Trusts within the West of England AHSN steering group are now feeding back review outcomes to their
30 days). CCGs and General Practitioners have
local CCG Quality Boards, in addition to mandatory reporting requirements, enabling the dissemination of
been similarly concerned.
information across the wider health community.

Additionally, a number of trusts have agreements in place that where a patient dies within a trust, but aspects of
their care has been delivered in another trust, there is agreement to share this information and undertake a SJR
in the locality where concerns have been raised.

Data exchange can be a challenge across institutional boundaries, despite there being a willingness to
collaborate. One trust has approached the challenge of post-discharge review by developing an area wide data
sharing agreement, which is currently awaiting national agreement. Once agreed this will grant the Trust access
to patient identifiers for post discharge deaths, resulting in such deaths being brought into scope for SJR review.
Another trust has undertaken post-discharge reviews by means of joint reviewing with both General Practioners
and hospital teams accessing their electronic data separately while working in the same setting.

27
West of England AHSN Experience

The West of England Academic Health Science to standardise within organisations never mind Through the review outcomes we have learnt
Network (WEAHSN) has a strong patient safety across them. the importance of timely and compassionate last
portfolio. As part of our work on needless harm phase of life conversations and means we have
This also allowed the other five trusts to accelerate
we partnered with the Royal College of Physicians been able to swiftly move to initiate work on the
their implementation plans when the National
in late 2016 to pilot and be an early adopter of ReSPECT process.1 This directly supports care at
Learning from Deaths guidance was announced in
their Structured Judgement Review (SJR) process. the end of life to ensure that the whole system
March 2017.
seeks to meet the wishes of the patient.
Using our experience in delivering collaborative
We sought regular senior leadership involvement
events and workstreams we approached all acute
that enabled local teams to reconfigure their
hospitals in the region to improve learning from
approach to learning from deaths and worked
deaths with the aim of standardising the mortality
together regionally to review mortality in a
review process, share learning and issues,
standardised way.
triangulating outcome themes and facilitate
1
local and region-wide quality improvement (QI) Recommended Summary Plan for Emergency Care and
The West of England AHSN has delivered the SJR
initiatives. Treatment www.respectprocess.org.uk/
training to eight organisations, which has resulted
in over 135 cascade trainers being trained within
Our Mortality Reviews Breakthrough Collaborative
the region, who directly support the roll-out of the
(using the IHI model) commenced in September
SJR process to their respective Trusts.
2016 and saw two trusts from out of the West
of England region join our six acute trusts in The most consistent theme to emerge from the
establishing the Collaborative Steering Group. West of England Patient Safety Collaborative
The group membership also included two GPs, Mortality Reviews implementation has been the
two public contributors and more recently our two failure to quickly recognise end-of-life palliative
mental health trusts. care across settings.
Whilst all the acute trusts were involved from It has also been identified that patients are being
the outset, we took a step-wise approach using sent to hospital inappropriately, with limited
three trusts as early implementers to refine the conversations happening with the family, patient
method and gain confidence before bringing in or carers about their wishes. Once patients enter
other Trusts. This reflected our awareness that the hospital, there is initially a focus on pathways
mortality reviews can be a challenging process for treatments such as sepsis care.

28
Yorkshire & Humber AHSN Improvement Academy Experience

The Improvement Academy consists of a Locally, Trusts have developed case selection Bassetlaw Teaching Hospitals NHS Trust since
team of improvement scientists, patient processes and since 2014 approximately 7000 SJRs January 2016.
safety experts and clinicians who are have been carried out in the region. A number
A fall in HSMR associated with septicaemia
committed to working with frontline of Trusts have aligned their review processes with
(except in pregnancy) from 139 to 103 in Mid-
services, patients and the public to their local incident reporting systems allowing for
Yorkshire Hospitals NHS Trust over a two year
deliver real and lasting change. It was concerns from staff to be captured.
period.
established as part of the Yorkshire &
Our common themes include:
Humber AHSN in May 2013. 19% reduction in cardiac arrest events
• recognising and managing the deteriorating per 1,000 bed nights in Sheffield Teaching
The Yorkshire & Humber Mortality review
patient, including end of life care. Hospitals NHS Foundation Trust.
programme was set up in 2014 to support the
• communication within organisations,
uptake of the Structured Judgement Review Improved care is seen across the community-
including handover and documentation.
methodology by both our acute and mental hospital interface. For example, a trust is
• recognition and management of sepsis.
health trusts. To date all 13 of our acute trusts collaborating with their ambulance service to
have adopted SJR as review methodology and 4 The Improvement Academy has set up learning improve recognition of ‘red-flag’ sepsis, allowing
of our 6 mental health trusts are using an adapted events bringing together improvement experts prompt administration of life-saving antibiotics on
SJR tool with phases of care to suit mental health and trusts to support the translation of themes the way to hospital.
mortality reviews. into practical improvement steps. Our work is
Our work over the past four years has
also aligned with our regional Patient Safety
The Improvement Academy has trained more demonstrated how standardised retrospective
Collaborative (PSC) programme so that problems
than 750 reviewers from the multidisciplinary mortality case notes review can provide a robust
in care identified through the review process can
team across departments and specialities. Trained method for organisations to assess their care
be tackled through PSC priority themes such as
reviewers include specialist nurses, consultant systems and identify problems in care.
patient deterioration.
surgeons/physicians, senior registrars, and senior
allied health professionals. Our support for organisations in Yorkshire
Systematic analysis of problems in care and
& Humber learning together has yielded
emergent themes feed quality improvement
Mortality leads come together quarterly as the demonstrable benefits to organisations, leading to
initiatives locally, contributing to real and
steering group to share learning ,experience and less organisational isolation and improved patient
sustainable improvements. These include:
challenges, thus shaping the programme bottom experience across the whole healthcare journey.
up. A lay subgroup involving carers and relatives 22% increase in appropriate and timely start
informs the programme steering group. of end-of-life care pathways in Doncaster and

29
Learning and Quality Improvement

Utilising learning and developing actions for The IHI Model for Improvement What are we trying to
improvement are the most important benefits accomplish?
Quality Improvement science is the application
of implementing a structured case note review
of a systematic approach to improvement
methodology.
using specific methods and techniques in
Structured case note reviews will provide trusts order to deliver measurable improvements in
How will we know if a change
with a rich data set from which they can derive quality, care and safety. Our approach uses the
is an improvement?
themes, learn where improvements can be made methodology developed by the Institute for
and ultimately develop improvement plans which Healthcare Improvement called the IHI Model for
will deliver higher quality care. Improvement.

Whilst this toolkit focuses on SJR, the following The model asks three questions:
What changes can we make that
section outlines how Trusts may utilise the
1. What are you trying to accomplish? will result in an improvement?
learning gathered to develop, measure and
2. How will we know if a change is an
evaluate improvement projects.
improvement?
3. What changes can we make that will result in
an improvement?

PL
T

AN
AC
The model then asks you to test out emergent
change ideas using Plan, Do, Study, Act (PDSA)
cycles.

ST

DO
UD
Y
30
Question 1: What are you trying to Once you are confident that you understand your
accomplish? problem, you can move on to agree an aim.

This is made up of three stages: An aim is an explicit description of the team’s

desired outcome. It is important to keep this
1. Understanding your problem
aim as SMART (specific, measurable, achievable,
2. Diagnosing why the problem is occurring
realistic and time-bound) as possible. It should
3. Agreeing the aim of your improvement
be meaningful to staff, patients and families. For
activities.
example, for a patient falls reduction project, the
The learning and themes identified from thematic aim might be ‘to reduce patient falls on Ward A
analysis of cohort of SJRs allow the identification by 50% within 6 months’.
of problems in care, which is the first step in the
‘improvement journey’.

A number of diagnostic tools can be used to help

gain a better understanding of your problem.
Some examples:

• Existing data e.g. local /national audits or

surveys
• New/bespoke data e.g. brief patient/staff
surveys
• Brainstorming
• Process mapping
• Fishbone diagram
• Driver diagrams
• 5 Whys

31
How will we know if a change is an improvement?

This second question relates to the need to

Reasons for measuring: Run charts
measure whether improvement is happening. When we talk about measurement in healthcare A run chart is a tool that measures your progress
there are two types of measurements that over time.
There are different types of measures:
are more familiar to healthcare professionals
Whilst being visually accessible, they are
• Process measures. These relates to the and can cause confusion when we talk about
underpinned by a robust statistical evidence-base
parts of the system that affect delivery of measurement in an improvement context.
that can prove whether or not improvement
the required outcome. In essence, they
• Measurement for judgement: where has occurred. The rules associated with reading
tell us whether the system is behaving the
measures are used to judge us against runcharts can be found here: http://qualitysafety.
way we would wish, e.g. adherence to
performance targets, other Trusts, etc. bmj.com/content/20/1/46
agreed timelines for reviews.
Improvement is not about judgement,
For more information on measuring visit
• Outcome measures. This relates to the however, you can use measures to judge and
MindSetQI on measurement.
aim, so if your aim is to improve sepsis manage your own progress
management, your outcome measure
• Measurement for research: where large
might be time from sepsis diagnosis to
amounts of data are gathered in order to test
antibiotic administration.
a hypothesis.
• Balancing measures. This would be
Measurement for improvement gathers just
included if it was felt that the delivery
enough data to show that improvement is
of one improvement goal could have a
happening and we present this data using run
negative consequence for another part of
chart.
the system e.g. completing SJRs impacting
negatively on clinic numbers.

32
What change can we make that will result in an improvement?

The PDSA cycle

As you go about answering the first two
questions, you are likely to generate a number Once a change idea has been identified, it should be
of change ideas along the way. tested using rapid PDSA cycles.

If you have not, there are a number of sources PDSA is an effective method that helps teams plan
such as the evidence-base and other services/ the actions, test it on a small scale, and review
colleagues. before deciding how to continue.

Using PDSA cycles is a powerful and rapid way

of taking ideas, trying them in practice, learning
what works, and what doesn’t to help you achieve
success.

You can then broaden the scale of the test, or adjust

PL
T

AN
your ideas through more than one PDSA cycle. It

AC
may take a few cycles before the idea starts to work
PL
T

reliably.
AN
AC

ST
For a fun way to introduce a team to quality

DO
UD
PL
T

AN

Y
AC

improvement, check out this blog post

ST

www.weahsn.net/2016/01/anyone-for-tennis/
DO
UD
Y

For an introduction to PDSA cycles watch this video

ST

DO
UD

https://youtu.be/xzAp6ZV5ml4
Y

33
Tools for learning and improvement

It is important for organisations to adopt credible Achieving Behaviour Change (ABC) from within, the behaviours and strategies that
improvement tools and approaches when trying facilitate success are likely to be affordable to
The problems with implementing best practice
to understand the problems in care identified implement, sustainable over time, and acceptable
are well recognised, and interventions to change
through the mortality review process, and to others in the community. More information is
practice, such as education, audit and feedback,
introduce improvements. available in the resource section.
do not consistently lead to change.
It is essential when developing action plans, The Learning from Excellence approach,
The two main issues are:
to try understand what factors might be at developed by the Birmingham Children’s Hospital,
• a failure to understand barriers and levers to
play, including systems factors and behavioural aims to identify, appreciate, study and learn from
implementation of best practice
attitudes. episodes of excellence in frontline healthcare.
• a failure to use behaviour change theory to
www.learningfromexcellence.com
The following tried and tested tools can support design implementation strategies
you to develop your local approaches to improving Yorkshire Contributory Factors
Yorkshire & Humber Improvement Academy,
care. Framework
through the Yorkshire Quality and Safety Group,
Human Factors works with internationally-recognised behaviour In 2012, a systematic review of 83 research studies
change experts to apply psychological insights focusing on the causes of hospital patient safety
Human Factors is an established scientific
to implementation problems where behaviour incidents was conducted. The result of this piece
discipline considered in the design of ‘human
change is required. of work is the first evidence based framework
system interfaces’ in many safety-critical, high-
of accident causation in hospitals: the Yorkshire
reliability industries. Coupling the concepts Yorkshire and Humber ABC for Patient Safety
Contributory Factors Framework. This is a tool
from human factors and patient safety is now Toolkit: http://www.improvementacademy.org/
which has an evidence base for optimizing
widely accepted by patient safety experts. tools-and-resources/abc-for-patient-safety-toolkit.
learning and addressing causes of patient safety
Human factors principles can be applied in the html
incidents (PSIs) by helping clinicians, risk managers
analysis of problems in care and development of
Positive Deviance and patient safety officers identify contributory
improvement actions.
factors of PSIs. Finding the true causes of patient
This asset based approach to quality improvement safety incidents offers an opportunity to address
Yorkshire & Humber Improvement Academy
is built on the premise that solutions to problems systemic flaws effectively, for the benefit of all our
have developed a free Bronze level e-learning to
already exist within communities. Certain future patients.
support front-line staff to improve the safety of
individuals, teams, or organisations – positive
their care available at www.improvementacademy.
deviants – identify these solutions and succeed Available at: www.improvementacademy.org/
org/training-and-events/bronze-human-factors-
despite facing the same constraints as others in tools-and-resources/the-yorkshire-contributory-
training.html
their community. As these solutions are identified factors-framework.html

34
Embed your change

Project Management Sustainability Celebration

Project management tools such as Project Initiation The final challenge when you have identified On project completion, even though there may
Documents (PIDs), Gantt charts, stakeholder and changes that result in improvements is ensuring be a recognition that there is still much to do, it
engagement plans and risks and issues logs may be it becomes sustainable and is embedded into is important to remember celebration.
useful to outline and plan the project dependent everyday practice.
• Celebrate project completion with the team:
on scale. Further information can be accessed at:
The West of England AHSN Quality Improvement
www.weahsn.net/what-we-do/west-of-england- • Ensure the sponsor and stakeholders are
team have identified a number of resources which
academy/improvement-resources-and-tools/the- involved (if possible).
can help sustain and spread a change and can be
improvement-journey/steps-in-the-improvement-
accessed from: www.weahsn.net/what-we-do/ • Acknowledge everybody’s efforts.
journey/step-2-develop-a-shared-purpose/
west-of-england-academy/improvement-resources-
project-management
and-tools/the-improvement-journey/ steps-in-the- • Share and reflect on the positive lessons
improvement-journey/step-5-implement-embed- learned.
Evaluation and-sustain/
• Use corporate recognition systems.
Evaluation allows those undertaking change Training in Quality Improvement • Avoid “institutionalised recognition” – be
to assess whether their change was actually an sincere.
For training in Quality Improvement, the Yorkshire &
improvement, as not all change will lead to an
Humber AHSN’s free Bronze QI e-learning modules
improvement. Evaluation can take a number of • Say “thank you” and mean it.
can be accessed here: www.improvementacademy.
forms and can include different evaluation designs.
org/training-and-events/bronze-quality-
The West of England Academic Health Science improvement-training.html
Network Quality Improvement team have produced
You can find out more about the Model for
a number of resources regarding evaluation which
improvement through the MINDSet quality
can be accessed at: www.weahsn.net/what-we-do/
improvement toolkit. Although aimed at people
west-of-england-academy/improvement-resources-
involved in providing and commissioning services
and-tools/the-improvement-journey/steps-in-the-
for people with mental health projects, it is an
improvement-journey/step-4-test-and-measure-
excellent resource for practical quality improvement
improvement/evaluation-for-a-qi-project/
guidance. Available at http://mindsetqi.net/ as as a
Further evaluation resources can be found in the PDF to download.
recommended resources section.

35
Authors

Authors: Contributors: Acknowledgements:

Dr Usha Appalsawmy Ryan Doherty Special thanks to everyone who has supported this
Mortality programme lead NHS Graduate Management Scheme trainee programme to date, with particular thanks to the
Yorkshire & Humber AHSN Improvement Academy West of England AHSN Patient Safety Collaborative individuals who helped deliver the cascade trainer
sessions across the West of England and Yorkshire &
Kevin Hunter Dr Andrew Gibson Humber AHSN regions.
Patient Safety Programme Manager NMCRR Clinical Lead
West of England AHSN Patient Safety Collaborative Royal College of Physicians
Thank you to Nathalie Delaney at the West of England
Maureen McGeorge Professor Allen Hutchinson
AHSN Patient Safety Collaborative for layout of the
Quality Improvement Trainer Lead Methodologist
toolkit.
Yorkshire & Humber AHSN Improvement Academy Yorkshire & Humber AHSN Improvement Academy

Dr Seema Srivastava MBE Dr Mark Juniper

Clinical Lead Clinical Lead
North Bristol NHS Trust Great Western Hospitals NHS Foundation Trust

Dr Michael McCooe
Associate Clinical Director
Yorkshire & Humber AHSN Improvement Academy

Beverley Slater
Director
Yorkshire & Humber AHSN Improvement Academy

Clare Wade
NMCRR Programme Manager
Royal College of Physicians

36
Resources

www.ombudsman.org.uk/publications/review-quality- www.apcrc.nhs.uk/evaluation/methodology.htm
LifeQI
nhs-complaints-investigations-where-serious-or-avoidable-
harm-has www.qihub.scot.nhs.uk/media/596811/the%20 LifeQI is a Web-Software platform built to
spread%20and%20sustainability%20ofquality%20
www.nice.org.uk/Media/Default/About/what-we-do/ improvement%20in%20healthcare%20pdf%20.pdf support and maintain Quality Improvement work
Into-practice/Support-for-service-improvement-and-audit/ in Health and Social Care. It makes it easy for
How-to-change-practice-barriers-to-change.pdf http://webarchive.nationalarchives.gov.
uk/20160805122935/http://www.nhsiq.nhs.uk/ teams to run QI projects and organisations to
www.kingsfund.org.uk/sites/default/files/field/ media/2757778/nhs_sustainability_model_-_ report on QI activities.
field_publication_file/developing-collective-leadership- february_2010_1_.pdf
kingsfund-may14.pdf
http://resolution.nhs.uk/wp-content/uploads/2017/04/ Across the Patient Safety Collaboratives, a
www.gov.uk/government/uploads/system/uploads/ NHS-Resolution-Saying-Sorry-2017.pdf number of organisations are using LifeQI as the
attachment_data/file/403010/culture-change-nhs.pdf
www.rcplondon.ac.uk/file/7633/download?token=_ platform for recording and sharing data.
www.gov.uk/government/uploads/system/uploads/ manvTUO
attachment_data/file/226703/Berwick_Report.pdf Contact details for LifeQi
www.howsafeisourcare.com/uploads/7/6/0/0/76001935/
www.bps.org.uk/system/files/user-files/Division%20 mmsf_single_pages_7th_stg.pdf
of%20Occupational%20Psychology/public/17689_cat- lifeqisystem.com
1658.pdf http://ihub.scot/media/1844/20170508-mortality-and-
morbidity-reviews_final.pdf
www.health.org.uk/sites/health/files/ [email protected]
MeasuringSafetyCulture.pdf Ladder of engagement, NHS England, located at: https://
www.england.nhs.uk/participation/resources/ladder-of- @lifeqisystem
www.health.org.uk/sites/health/files/ engagement-2/
TheMeasurementAndMonitoringOfSafety_fullversion.pdf
Bradley, E. H., Curry, L. A., Ramanadhan, S., Rowe, L., Contact your Academic Health Science Network
www.rcseng.ac.uk/-/media/files/rcs/library-and- Nembhard, I. M., & Krumholz, H. M. (2009). Research to find out if you have access to the Life System.
publications/non-journal-publications/morbidity-and- in action: using positive deviance to improve quality
mortality--a-guide-to-good-practice.pdf of health care. Implementation Science, 4, 25. doi:
10.1186/1748-5908-4-25
www.health.org.uk/publication/quality-improvement-
made-simple Lawton, R., Taylor, N., Clay-Williams, R., & Braithwaite,
J. (2014). Positive deviance: a different approach
www.hqip.org.uk/resources/guide-to-quality- to achieving patient safety. BMJ Quality & Safety,
improvement-methods/ Published online first 21 July 2014. doi: 10.1136/
bmjqs-2014-003115
www.health.org.uk/sites/health/files/
EvaluationWhatToConsider.pdf

37
Notes

38
Notes

39
 Version 1.3 June 2018
The most recent version of this toolkit and supporting
resources are available at www.weahsn.net
www.improvementacademy.org
www.yhahsn.org.uk
www.rcplondon.ac.uk