Cochrane

Library
Cochrane Database of Systematic Reviews

Early warning systems and rapid response systems for the

prevention of patient deterioration on acute adult hospital wards
(Review)

McGaughey J, Fergusson DA, Van Bogaert P, Rose L

McGaughey J, Fergusson DA, Van Bogaert P, Rose L.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards.
Cochrane Database of Systematic Reviews 2021, Issue 11. Art. No.: CD005529.
DOI: 10.1002/14651858.CD005529.pub3.

www.cochranelibrary.com

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult
hospital wards (Review)
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

TABLE OF CONTENTS
ABSTRACT..................................................................................................................................................................................................... 1
PLAIN LANGUAGE SUMMARY....................................................................................................................................................................... 2
SUMMARY OF FINDINGS.............................................................................................................................................................................. 4
BACKGROUND.............................................................................................................................................................................................. 8
OBJECTIVES.................................................................................................................................................................................................. 9
METHODS..................................................................................................................................................................................................... 9
Figure 1.................................................................................................................................................................................................. 11
RESULTS........................................................................................................................................................................................................ 13
Figure 2.................................................................................................................................................................................................. 17
Figure 3.................................................................................................................................................................................................. 18
DISCUSSION.................................................................................................................................................................................................. 21
AUTHORS' CONCLUSIONS........................................................................................................................................................................... 22
ACKNOWLEDGEMENTS................................................................................................................................................................................ 23
REFERENCES................................................................................................................................................................................................ 24
CHARACTERISTICS OF STUDIES.................................................................................................................................................................. 33
DATA AND ANALYSES.................................................................................................................................................................................... 46
Analysis 1.1. Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 1: 47
Hospital mortality.................................................................................................................................................................................
Analysis 1.2. Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 2: 48
Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death).............................................................
Analysis 1.3. Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 3: 48
Unplanned intensive care unit admission...........................................................................................................................................
Analysis 1.4. Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 4: 49
Length of hospital stay.........................................................................................................................................................................
Analysis 1.5. Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual care, Outcome 5: 49
Adverse events (unexpected cardiac or respiratory arrest)...............................................................................................................
ADDITIONAL TABLES.................................................................................................................................................................................... 50
APPENDICES................................................................................................................................................................................................. 73
WHAT'S NEW................................................................................................................................................................................................. 82
HISTORY........................................................................................................................................................................................................ 83
CONTRIBUTIONS OF AUTHORS................................................................................................................................................................... 83
DECLARATIONS OF INTEREST..................................................................................................................................................................... 83
SOURCES OF SUPPORT............................................................................................................................................................................... 84
DIFFERENCES BETWEEN PROTOCOL AND REVIEW.................................................................................................................................... 84
NOTES........................................................................................................................................................................................................... 84
INDEX TERMS............................................................................................................................................................................................... 84

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) i
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

[Intervention Review]

Early warning systems and rapid response systems for the prevention of
patient deterioration on acute adult hospital wards

Jennifer McGaughey1, Dean A Fergusson2, Peter Van Bogaert3, Louise Rose4

1School of Nursing and Midwifery, Queen's University Belfast, Belfast, UK. 2Clinical Epidemiology Program, Ottawa Hospital Research
Institute, Ottawa, Canada. 3Nursing and Midwifery Sciences, Centre for Research and Innovation in Care, Faculty of Medicine and Health
Sciences, University of Antwerp, Antwerp, Belgium. 4Florence Nightingale Faculty of Nursing, Midwifery and Palliative Care, King's
College London, London, UK

Contact: Jennifer McGaughey, [email protected].

Editorial group: Cochrane Effective Practice and Organisation of Care Group.

Publication status and date: Edited (conclusions changed), published in Issue 11, 2021.

Citation: McGaughey J, Fergusson DA, Van Bogaert P, Rose L. Early warning systems and rapid response systems for the prevention of
patient deterioration on acute adult hospital wards. Cochrane Database of Systematic Reviews 2021, Issue 11. Art. No.: CD005529. DOI:
10.1002/14651858.CD005529.pub3.

Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.

ABSTRACT

Background
Early warning systems (EWS) and rapid response systems (RRS) have been implemented internationally in acute hospitals to facilitate early
recognition, referral and response to patient deterioration as a solution to address suboptimal ward-based care. EWS and RRS facilitate
healthcare decision-making using checklists and provide structure to organisational practices through governance and clinical audit.
However, it is unclear whether these systems improve patient outcomes. This is the first update of a previously published (2007) Cochrane
Review.

Objectives
To determine the effect of EWS and RRS implementation on adults who deteriorate on acute hospital wards compared to people receiving
hospital care without EWS and RRS in place.

Search methods
We searched CENTRAL, MEDLINE, Embase and two trial registers on 28 March 2019. We subsequently ran a MEDLINE update on 15 May
2020 that identified no further studies. We checked references of included studies, conducted citation searching, and contacted experts
and critical care organisations.

Selection criteria
We included randomised trials, non-randomised studies, controlled before-after (CBA) studies, and interrupted time series (ITS) designs
measuring our outcomes of interest following implementation of EWS and RRS in acute hospital wards compared to ward settings without
EWS and RRS.

Data collection and analysis

Two review authors independently checked studies for inclusion, extracted data and assessed methodological quality using standard
Cochrane and Effective Practice and Organisation of Care (EPOC) Group methods. Where possible, we standardised data to rates per 1000
admissions; and calculated risk differences and 95% confidence intervals (CI) using the Newcombe and Altman method. We reanalysed
three CBA studies as ITS designs using segmented regression analysis with Newey-West autocorrelation adjusted standard errors with lag
of order 1. We assessed the certainty of evidence using the GRADE approach.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 1
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Main results
We included four randomised trials (455,226 participants) and seven non-randomised studies (210,905 participants reported in three
studies). All 11 studies implemented an intervention comprising an EWS and RRS conducted in high- or middle-income countries.
Participants were admitted to 282 acute hospitals. We were unable to perform meta-analyses due to clinical and methodological
heterogeneity across studies. Randomised trials were assessed as high risk of bias due to lack of blinding participants and personnel across
all studies. Risk of bias for non-randomised studies was critical (three studies) due to high risk of confounding and unclear risk of bias
due to no reporting of deviation from protocol or serious (four studies) but not critical due to use of statistical methods to control for
some but not all baseline confounders. Where possible we presented original study data which reported the adjusted relative effect given
these were appropriately adjusted for design and participant characteristics. We compared outcomes of randomised and non-randomised
studies reported them separately to determine which studies contributed to the overall certainty of evidence. We reported findings from
key comparisons.

Hospital mortality

Randomised trials provided low-certainty evidence that an EWS and RRS intervention may result in little or no difference in hospital
mortality (4 studies, 455,226 participants; results not pooled). The evidence on hospital mortality from three non-randomised studies was
of very low certainty (210,905 participants).

Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)

One randomised study showed that an EWS and RRS intervention probably results in no difference in this composite outcome (adjusted
odds ratio (aOR) 0.98, 95% CI 0.83 to 1.16; 364,094 participants; moderate-certainty evidence). One non-randomised study suggests that
implementation of an EWS and RRS intervention may slightly reduce this composite outcome (aOR 0.85, 95% CI 0.72 to 0.99; 57,858
participants; low-certainty evidence).

Unplanned ICU admissions

Randomised trials provided low-certainty evidence that an EWS and RRS intervention may result in little or no difference in unplanned
ICU admissions (3 studies, 452,434 participants; results not pooled). The evidence from one non-randomised study is of very low certainty
(aOR 0.88, 95% CI 0.75 to 1.02; 57,858 participants).

ICU readmissions

No studies reported this outcome.

Length of hospital stay

Randomised trials provided low-certainty evidence that an EWS and RRS intervention may have little or no effect on hospital length of stay
(2 studies, 21,417 participants; results not pooled).

Adverse events (unexpected cardiac or respiratory arrest)

Randomised trials provided low-certainty evidence that an EWS and RRS intervention may result in little or no difference in adverse
events (3 studies, 452,434 participants; results not pooled). The evidence on adverse events from three non-randomised studies (210,905
participants) is very uncertain.

Authors' conclusions
Given the low-to-very low certainty evidence for all outcomes from non-randomised studies, we have drawn our conclusions from the
randomised evidence. This evidence provides low-certainty evidence that EWS and RRS may lead to little or no difference in hospital
mortality, unplanned ICU admissions, length of hospital stay or adverse events; and moderate-certainty evidence of little to no difference
on composite outcome. The evidence from this review update highlights the diversity in outcome selection and poor methodological
quality of most studies investigating EWS and RRS. As a result, no strong recommendations can be made regarding the effectiveness of EWS
and RRS based on the evidence currently available. There is a need for development of a patient-informed core outcome set comprising
clear and consistent definitions and recommendations for measurement as well as EWS and RRS interventions conforming to a standard
to facilitate meaningful comparison and future meta-analyses.

PLAIN LANGUAGE SUMMARY

Checklists and specialist teams to recognise and manage sick people in hospital

What was the aim of the review?

Adults in hospital who become unwell need to be identified quickly by clinical staff as requiring help. One way to do this is for doctors
and nurses working on hospital wards to use a checklist of vital signs (e.g. blood pressure, pulse) to help them to recognise signs that

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 2
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

patients are getting worse. These checklists are then used to refer patients to specialist teams of doctors and nurses for rapid assessment
and treatment. We conducted this review to understand if using a checklist and referring to a specialist team reduces the number of
deaths, unplanned intensive care unit (ICU) admissions/readmissions, hospital length of stay, and cardiac or respiratory arrest compared
to hospital wards without these resources.

Key messages

We found evidence that checklists to help doctors and nurses recognise and refer patients who get worse early to specialist teams for
management may result in little or no difference to the number of deaths, unplanned ICU admissions, length of hospital stay or cardiac
arrests in hospital.

What did we study in the review?

This Cochrane Review presents what we know from research on the effect of hospital checklists to help doctors and nurses recognise and
refer patients who are getting worse in hospital to specialist teams for help. Research has shown that patients in acute hospital wards
often show early signs and symptoms, such as changes in breathing and pulse, when their condition is getting worse. It is thought that
if hospital staff could identify and refer those patients who are getting worse earlier to specialist teams with appropriate knowledge and
skills in acute care (active but short-term treatment), then there would be fewer deaths, unplanned ICU admissions, cardiac arrests and
reduced length of stay.

What were the main results of the review?

We identified 11 studies. In total, 666,131 participants in 282 hospitals were included from seven middle- to high-income countries. The
implementation of a checklist to recognise deteriorating patients and a specialist team to rapidly assess and treat presenting symptoms
make little or no difference to deaths, unplanned ICU admissions, length of hospital stay, or cardiac and respiratory arrests compared to
the wards or hospitals without access to the checklist and specialist team. No studies reported ICU readmission rates.

How up-to-date is this review?

The review authors searched for studies published up to May 2020.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 3
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
SUMMARY OF FINDINGS

Summary of findings 1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of patient

Library
Cochrane
deterioration on acute hospital wards

Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of patient deterioration on acute hospital wards

Patient or population: adult on acute hospital wards at risk of clinical deterioration

Better health.
Informed decisions.
Trusted evidence.
Settings: acute wards of hospital in the UK, US, Australia and the Netherlands

Intervention: RRS that included an EWS

Comparison: usual care without any form of EWS or RRS

Outcome Study Relative effect* No of partici- Certainty of the Impact

(95% CI) pants/admissions evidence ‡
(GRADE)

Hospital mortality – randomised trials Priestley 2004 aOR 0.52 (0.32 to 0.85) 2792 ⊕⊕⊝⊝ EWS and RRS may re-
Lowa,b sult in little or no dif-
(follow-up: 3–16 months) Hillman 2005 aOR 1.03 (0.84 to 1.28) 364,094 ference in mortality
among adults at risk of
Jeddian 2016 aOR 1.02 (0.68 to 1.55) 18,684 deterioration.

Haegdorens 2018 aOR 0.82 (0.34 to 1.95) 69,656

Hospital mortality – non-randomised Lighthall 2010 No relative differences are pro- — ⊕⊝⊝⊝ —
studies vided for this study, and there- Very lowb,c,d
fore no data is presented.
(follow-up: 5–48 months)
Rothberg 2012 No relative differences are pro- —
vided for this study, and there-

Cochrane Database of Systematic Reviews

fore no data is presented.

Davis 2015 No relative differences are pro- —

vided for this study, and there-
fore no data is presented.

Ludikhuize 2015 aOR 0.85 (0.64 to 1.00) 57,858

Chen 2016 aRR 0.81 (0.76 to 0.86) —

4
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Menon 2018 RR 1.0 (0.03 to 29.8) 18,954

Library
Cochrane
Composite outcome – randomised tri- Hillman 2005 aOR 0.98 (0.83 to 1.16) 364,094 Moderatea EWS and RRS proba-
als bly results in no differ-
ence in the number of
Composite outcome of incidence participants experienc-
(events divided by number of eligible ing the composite out-
participants admitted to the hospital come.

Better health.
Informed decisions.
Trusted evidence.
during the study period) of cardiac ar-
rests without a pre-existing NFR order,
unplanned ICU admissions, and unex-
pected deaths (deaths without a pre-ex-
isting NFR order)

(follow-up: 6 months)

Composite outcome – non-ran- Ludikhuize 2015 aOR 0.85 (0.72 to 0.99) 57,858 ⊕⊕⊝⊝ —
domised studies Lowc

Composite endpoint of cardiopul-

monary arrest, unplanned ICU admis-
sion or death per 1000 admitted pa-
tients

(follow-up: 5 months)

Unplanned ICU admission – ran- Hillman 2005 aOR 1.04 (0.89 to 1.21) 364,094 ⊕⊕⊝⊝ EWS and RRS may re-
domised trials Lowa,b sult in little or no dif-
Jeddian 2016 aOR 1.15 (0.64 to 2.09) 18,684 ference in unplanned
(follow-up: 3–16 months) ICU admission rates.
Haegdorens 2018 aOR 1.23 (0.91 to 1.65) 69,656

Unplanned ICU admission – non-ran- Aitken 2015 No relative differences provided — ⊕⊝⊝⊝ —

Cochrane Database of Systematic Reviews

domised trial and no data presented. Very lowb,c,d

(follow-up: 5 months)

Ludikhuize 2015 aOR 0.88 (0.75 to 1.02) 57,858

ICU readmission – randomised trials — — — — No randomised trials

reported rates of ICU
readmissions.
5
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
ICU readmission – non-randomised — — — — No non-randomised
studies studies reported rates
of ICU readmissions.

Library
Cochrane
Length of hospital stay – randomised Priestley 2004 HR 0.91 (0.83 to 0.98) 2733 ⊕⊕⊝⊝ EWS and RRS may
trials Lowa,d have little or no effect
Jeddian 2016 aRGM 1.00 (0.97 to 1.03) 18,684 on hospital length of
(follow-up: 3 months) stay.

Better health.
Informed decisions.
Trusted evidence.
Length of hospital stay – non-ran- — — — — No non-randomised
domised studies studies reported
length of hospital stay.

Adverse events (unexpected cardiac Hillman 2005 aOR 0.94 (0.79 to 1.13) 364,094 ⊕⊕⊝⊝ EWS and RRS may re-
or respiratory arrest) – randomised Lowa,b sult in little or no dif-
trials Jeddian 2016 aOR 1.00 (0.69 to 1.48) 18,684 ference in adverse
events among adults
(follow-up: 3–16 months) at risk of deterioration.
Haegdorens 2018 aOR 0.71 (0.33 to 1.52) 69,656

Adverse events (unexpected car- Lighthall 2010 No relative differences provided — ⊕⊝⊝⊝ —
diac or respiratory arrest) – non-ran- and no data presented. Very lowb,c,d
domised studies

(follow-up: 5–48 months)

Rothberg 2012 aOR 1.00 (0.69 to 1.48) —

Aitken 2015 No relative differences provided —

and no data presented.

Davis 2015 No relative differences provided 134,093

and no data presented.

Cochrane Database of Systematic Reviews

Ludikhuize 2015 aOR 0.61 (0.39 to 0.94) 57,858

Chen 2016 aRR 0.55 (0.48 to 0.62) —

Menon 2018 No relative differences provided 18,954

and no data presented.

*The absolute effect is not presented due to lack of unadjusted data.

‡GRADE Working Group grades of evidence

6
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
GRADE Working Group grades of evidence

High: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different‡ is low.

Library
Cochrane
Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different‡ is moderate.

Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different‡ is high.

Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different‡ is very high.

Better health.
Informed decisions.
Trusted evidence.
‡ Substantially different = a large enough difference that it might affect a decision.

aOR: adjusted odds ratio; aRGM: adjusted ratio of geometric means; aRR: adjusted risk ratio; CI: confidence interval; EWS: early warning system; HR: hazard ratio; ICU: in-
tensive care unit; NFR: not-for-resuscitation; RR: risk ratio; RRS: rapid response system.

aDowngraded one level for risk of bias: concerns regarding performance bias or contamination bias, or both.
bDowngraded one level for imprecision of the direction of the effect: due to wide RR interpreted as less than 0.75 to greater than 1.25 which includes appreciable beneficial or
harmful effects.
cDowngraded one level for risk of bias: serious or critical risk of bias due to inadequate adjustment for confounders in observational studies assessed with Risk Of Bias in Non-
randomised Studies – of Interventions (ROBINS-I) tool.
dDowngraded one level for inconsistency: as unable to calculate effect size for all studies.

Cochrane Database of Systematic Reviews

7
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

BACKGROUND A number of track and trigger tools exist. These are either based
on exceeding any one of a set physiological criterion (single
Description of the condition parameter systems or calling criteria) or on the allocation of
points to physiological observations which are summed to give a
Patients who deteriorate on acute hospital wards often exhibit early
score (aggregate weighted scoring systems including EWS, and the
physiological warning signs (such as changes in respiratory rate,
National Early Warning System (NEWS2) (RCP 2017)). All tools have
heart rate, blood pressure, level of consciousness or urine output)
both objective physiological criteria and subjective calling criteria.
prior to cardiac arrest, death or the need for emergent intensive
Standard practice for tracking objective physiological criteria
care unit (ICU) admission (Kause 2004; Fujiwara 2016). These
include the monitoring of heart rate, respiratory rate, systolic
abnormal physiological changes are observable in documented
blood pressure, level of consciousness, oxygen saturation and
vital signs one to four hours prior to cardiac arrest. An increased
temperature, with a minimum frequency of 12-hourly observation
number of abnormal vital signs is associated with increased
recording (NICE 2007; ACSQHC 2017). The subjective calling criteria
hospital mortality (Andersen 2016). Early identification of changes
allow the nurse or doctor to call for help if they have concerns
in vital signs can assist in the detection of people at risk for
regardless of physiological criteria. The type of track and trigger
cardiac arrest, or in the early stages of deterioration, averting the
system selected is associated with the RRS model utilised within
occurrence of serious adverse events. However, these changes
the hospital. Guidelines for standard practice in Australia, UK,
in vital signs are often missed, misinterpreted or mismanaged
Ireland, and the Netherlands recommend that all hospital patients
(Donaldson 2014). As a result, many acutely ill people on general
should have physiological observations recorded on admission,
hospital wards receive suboptimal care (McQuillan 1998; NCEPOD
and should have a clear written plan indicating the parameters,
2005). Suboptimal care is defined as delays in diagnosis, treatment
frequency of observations to be monitored and trigger thresholds
or referral, suboptimal assessment, or inappropriate or inadequate
for referral (NICE 2007; VMS 2008; DoH 2013; ACSQHC 2017).
treatment (Quirke 2011). Evidence suggests that up to 20% to 30%
of hospital ward patients receive substandard care prior to ICU All track and trigger tools have a predetermined physiological
admission, with up to 70% of adverse events being potentially threshold or trigger score that assists hospital staff in recognising
avoidable (Vlayen 2012; Garry 2014). clinical deterioration, and aids decision-making in terms of the
type of response required. These criteria or scores provide a
Description of the intervention mechanism for early intervention and treatment initiation. A rising
Early warning systems (EWS) and rapid response systems (RRS) score indicates substantial deterioration and the need to trigger
have been implemented into practice internationally to improve the RRS. If trigger criteria are met, or threshold value is reached,
the early identification of physiological instability and to initiate a predefined response strategy or referral protocol is initiated. A
an early response mechanism to manage deteriorating patients graded response strategy has been adopted in Australia, UK, and
on acute hospital wards. These systems were developed through Ireland as best practice (NICE 2007; DoH 2013; ACSQHC 2017).
expert consensus as a solution to address suboptimal ward-based Three levels of response (low, medium and high score groups) with
care and to reduce preventable hospital deaths. Essential features maximum response times are specified in the UK (NICE 2007); a
of RRS consist of three inter-related components: an afferent arm, two-tier response (ICU outreach nurses or specialist team) is used
which is the crisis detection and response triggering mechanism; in Australia. These response levels allow actions to be tailored to
an efferent arm, which provides competent, skilled personnel the level of response required, provide a clear indication of the
and resources at the bedside to initiate an appropriate level of action and decisions to be taken to initiate the response for help,
response; and a governance and audit process to co-ordinate and provide standardisation of practice (NICE 2007; DoH 2013;
human and financial resources, evaluate and prevent future ACSQHC 2017). Thus, appropriate and competent staff are alerted
adverse events, and to ensure sustainability of RRS (DeVita 2006). to changes in a patient's physiological condition to manage the
Implementation of all three RRS components ensures deteriorating episode of instability.
hospital patients are identified early through the use of EWS
serving as track and trigger tools; patients are referred early to a To provide an effective RRS response, all staff must be competent
more appropriate level of care using protocolised graded response in assessing and instituting clinical interventions at varying levels.
strategies; and management is by professionals with specialist The provision of staff training and education in acute care ensures
knowledge and skills in acute and critical care. that staff on acute hospital wards are competent to recognise,
refer and manage patient deterioration appropriately. Training
Crisis detection in the afferent arm of the RRS is facilitated by the and education vary across countries and correlate with the RRS
use of a structured EWS tool to detect patient deterioration. These model adopted. In Australia, UK and Ireland, several courses
are known as 'track and trigger' systems. These systems include provide a flexible model of training using simulation, scenario-
assessment tools incorporated into bedside observation charts based training, online and induction training modules for staff
on acute hospital wards that monitor physiological parameters (Smith 2002; HSE 2011; ACT 2016). Training programme content
such as systolic blood pressure, heart rate, respiratory rate, urinary is based on the principles of the Primary Survey Model training
output, temperature and level of consciousness (RCP 2017). 'Track' of Airway, Breathing, Circulation, Disability and Exposure (ABCDE)
refers to the physiological monitoring of observations visually assessment process (Smith 2004).
represented as trends on the observation chart. 'Trigger' refers
to a threshold figure or criteria indicating impending clinical A key component of the RRS is the interprofessional critical care
instability and requiring initiation of an early response to prevent teams, called critical care outreach teams (CCOTs) in the UK (NORF
an adverse event. The purpose of track and trigger tools is to 2003), medical emergency teams (METs) in Australia (Lee 1995)
provide standardisation, thus removing variation associated with (with or without ICU liaison nurses (ICULNs) (Eliott 2008), and
individual clinical decision-making. rapid response teams (RRTs) in the US and Canada (Berwick 2006).

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 8
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

These specialist teams respond to calls from ward staff following METHODS
identification of patients at risk based upon the trigger score or
calling criteria as part of the standardised escalation protocol. Criteria for considering studies for this review
These teams are knowledgeable, experienced critical care staff
Types of studies
competent in managing patient deterioration. CCOTs and ICULNs
also provide follow-up care for discharged ICU patients and provide We considered all randomised trials, non-randomised studies,
staff training and education on aspects of critical care as an integral controlled before-after studies (CBAs) and interrupted time-series
part of their role (DoH 2000; ICS 2002; DeVita 2006). The number (ITS) designs of an EWS and RRS that meet the most recent explicit
of interprofessional members involved, and their availability (five inclusion and quality criteria of the Cochrane Effective Practice
days a week or 24 hours a day, seven days a week), depends on and Organisation of Care (EPOC) Group (EPOC 2017a). CBA studies
the funding model. CCOTs and ICULNs potentially avert the need were eligible if they had at least two intervention and two control
for ICU admissions, enable more timely ICU discharges, and provide sites; ITS studies with a clearly defined point in time when the
educational support to extend the skills of acute ward staff in intervention occurred, and a minimum of three data points before
identifying and managing deteriorating patients (DoH 2000; ICS and after the intervention. Non-randomised studies are typically
2002). used to evaluate the effectiveness of this type of intervention due
to the difficulty in randomising acutely deteriorating patients to an
How the intervention might work EWS or RRS.
The implementation of EWS and RRS uses a systems wide approach
Types of participants
to ensure all patients in hospital who deteriorate are managed
appropriately regardless of location. All patients have their vital We considered studies enrolling all patients who experienced a
signs recorded regularly using track and trigger tools providing deterioration in health and were on an acute adult hospital ward.
an early indicator of deterioration and the need for intervention.
A rising aggregate score or meeting set physiological calling Types of interventions
criteria triggers a graded, protocolised response by a specialist We considered studies that included an intervention with both
team. This ensures that the patient is managed appropriately by components of an EWS and RRS in an acute hospital setting
personnel with specified competencies and skills. The inter-related to identify deteriorating patients on acute wards and compared
components of the EWS and RRS ensure that patient deterioration to usual care on acute wards without an EWS and RRS. Track
is recognised, referred and managed commensurate with the level and trigger tools included any aggregate scoring systems (EWS,
of need required to prevent death, unplanned ICU admission, and Modified Early Warning Systems (MEWS), NEWS, Patient at Risk
cardiac arrest, and to reduce length of hospital stay. (PAR) scores) and single parameter calling criteria as defined by the
individual RRS. RRS included CCOT, MET or RRT.
Why it is important to do this review
In 2007, we published a systematic review examining the Types of outcome measures
effectiveness of EWS and RRS on patient outcomes on acute Primary outcomes
hospital wards (McGaughey 2007) based on the methods in our
published protocol (McGaughey 2005). This review identified only • Hospital mortality
two studies with medium risk of bias (Priestley 2004; Hillman 2005),
Secondary outcomes
which limited this review's conclusions.
• Composite outcome (unexpected cardiac arrests, unplanned
Therefore, this update was needed to ascertain whether the ICU admissions and death)
implementation of EWS and RRS reduces mortality, composite • Unplanned ICU admission
outcome (unexpected cardiac arrests, unplanned ICU admissions
• ICU readmission
and death), unplanned ICU admissions, ICU readmissions, length of
hospital stay or adverse events (unexpected cardiac or respiratory • Length of hospital stay
arrest) of deteriorating patients on acute hospital wards when • Adverse events (defined as an unexpected cardiac or respiratory
compared to patients in hospital wards without an EWS and RRS. arrest)
It is important to reassess the body of evidence and update this
review. This is the first update of a previous Cochrane Review Search methods for identification of studies
(McGaughey 2007). An EPOC information specialist developed the search strategy in
consultation with the review authors. The strategy was based on
OBJECTIVES
a citation search of two included studies from the original review
To determine the effect of EWS and RRS implementation on adults and studies from a scoping search using MeSH terms and free-text
who deteriorate on acute hospital wards compared to people terms related to EWS and RRS. The search strategy was translated
receiving hospital care without EWS and RRS in place. into each database using the appropriate controlled vocabulary.
We applied no language restrictions.

Electronic searches
The EPOC Information Specialist peer reviewed the searches used
to conduct the previous version of this review and revised the
searches for this update. Searches were restricted to retrieve

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 9
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

material published since the date of the last search for the previous Federation of Critical Care Nurses (WFCCN), European federation of
version of this review (McGaughey 2007). Full search strategies are Critical Care Nurses association (EfCCNa), UK Critical Care Research
available in Appendix 1. Group (UKCCRG), UK Critical Care Nurses Network, European
Society of Intensive Care Medicine (ESICM), UK Intensive Care
We searched the following databases for primary studies to 28 Society (ICS), UK National Outreach Forum (NorF), International
March 2019: Liaison Committee on Resuscitation (ILCOR), and Education,
Implementation & Teams (EIT) Taskforce of ILCOR.
• Cochrane Central Register of Controlled Trials (CENTRAL; 2019,
Issue 3) in the Cochrane Library; Data collection and analysis
• MEDLINE Ovid (including Epub Ahead of Print, in process and
other Non-Indexed Citations and Versions); We followed standard methods of Cochrane for conducting a
systematic review as described in the Cochrane Handbook for
• Embase Ovid;
Systematic Reviews of Interventions (Higgins 2011) and adhered to
• CINAHL EBSCO (Cumulative Index to Nursing and Allied Health EPOC methods (EPOC 2017b).
Literature).
Selection of studies
Searching other resources
Three review authors (JM and PVB or DH) independently screened
We conducted a search for references citing the included studies the results of the database searches using Covidence to identify
in this review (also on 28 March 2019) using Science Citation potentially relevant studies. Two review authors (JM and PVB)
Index (Web of Science, Clarivate). These citation searches were not independently assessed the full-text records of potentially relevant
restricted by date. studies. We resolved disagreement or lack of consensus between
review authors regarding study inclusion or exclusion through
We searched for ongoing studies on ClinicalTrials.gov
discussion with another review author (LR). We circulated
(www.clinicaltrials.gov) and the World Health Organization
included studies to nine professional organisations to identify
International Clinical Trials Registry Platform Search Portal (ICTRP)
any additional published or unpublished studies that met our
(www.who.int/trialsearch).
inclusion criteria (see Searching other resources). We documented
We searched reference lists of all included studies and relevant the selection process with a PRISMA flow chart (Figure 1) and
reviews for additional references. Where relevant, we contacted produced a Characteristics of included studies table. All studies
authors of potentially eligible papers for further published and that initially appeared to meet our inclusion criteria but were
unpublished data. We contacted the following nine professional subsequently excluded are detailed in the Characteristics of
critical care organisations to identify any additional or ongoing excluded studies table.
studies: British Association of Critical Care Nurses (BACCN), World

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 10
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Figure 1. Study flow diagram

Data extraction and management Assessment of risk of bias in included studies

Two review authors (JM, PVB) independently extracted data Randomised trials
from each study using an adapted EPOC data collection tool
Two review authors (JM and PVB) independently assessed risk
(EPOC 2017c). We checked the data for any discrepancies and
of bias of randomised trials using the Cochrane risk of bias
collated them. We resolved any discrepancies through discussion
tool according to the six domains outlined in the Cochrane
until review authors reached consensus. We corresponded
Handbook for Systematic Reviews of Interventions (Higgins 2011):
with nine study investigators for further methodological data.
random sequence generation, allocation concealment, blinding
Four replied providing the requested information. We extracted
of participants and healthcare personnel, blinding of outcome
important information with respect to study characteristics,
assessors, incomplete outcome data, selective reporting and other
participant characteristics, intervention characteristics and
sources of bias. Risk of bias judgements were categorised as low
outcome measures.
risk of bias (judged at low risk of bias for all domains for this
result), unclear risk of bias (judged at unclear risk of bias for one
or more key domains for this result), and high risk of bias (judged
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 11
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

at high risk of bias in one or more key domains for this result). We Non-randomised studies
resolved discrepancies in ratings by discussion. We assessed the
We reported adjusted relative effects and absolute differences
methodological information and entered it into the risk of bias table
where we were able to standardise studies for rates per 1000
for each included trial.
admissions/discharge. We calculated 95% confidence intervals
Non-randomised studies (CIs) when data were available.

Two review authors (LR and PVB) independently assessed the Unit of analysis issues
methodological quality of each CBA and ITS study, rather than
Randomised trials
specific outcome results, using the eight domains of the Cochrane
ROBINS-I tool (Sterne 2016). These eight domains include bias Unit of analysis was the cluster for all randomised controlled trials.
due to confounding, bias in selection of participants into the Clusters were hospitals (Hillman 2005) or wards (Priestley 2004;
study, bias in classification of interventions, bias due to deviations Jeddian 2016; Haegdorens 2018). We examined cluster randomised
from intended interventions, bias due to missing data, bias in controlled trials for unit of analysis errors with none detected.
measurement of outcomes, bias in selection of the reported result
and overall bias. Non-randomised studies
The unit of analysis for four non-randomised studies was individual
We considered the following as possible sources of confounding.
patient data (Rothberg 2012; Aitken 2015; Davis 2015; Menon
• Participant characteristics: pre-existing comorbidity, age, 2018) and the cluster at ward level (Lighthall 2010) or hospital
severity of illness during ICU admission. level (Ludikhuize 2015; Chen 2016) for the remaining studies. We
• Organisational characteristics: ICU and hospital census, staffing identified no unit of analysis issues with two ITS studies (Rothberg
levels, patient acuity/case-mix on hospital wards, ward culture 2012; Chen 2016) or two multicentre CBA studies (Davis 2015;
and organisational practices, medical and nursing education Ludikhuize 2015).
levels; ICU discharge policies, ICU readmission policies, and We reanalysed three single-centre CBAs (Lighthall 2010; Aitken
overall predilection to readmit to ICU. 2015; Menon 2018), which provided data at three time points before
• ICU admission/readmission rates. and after the implementation of the intervention using ITS methods
• Prevalence of patients with do not resuscitate status. taking into account unit of analysis issues. These studies provided
• Seasonal variation in patient census. data on the following: hospital mortality per 100 discharges and
cardiac arrest per 1000 discharges (Lighthall 2010); unplanned ICU
We considered the following co-interventions as likely to have admissions and cardiac arrest rates mean per month (Aitken 2015);
impact on outcomes. and hospital mortality and code blue per 1000 hospital admissions
(Menon 2018).
• Patient safety initiatives within the organisation.
• Cardiac arrest team in place as well as the outreach/RRT. To perform the ITS analyses, we received unpublished data
• Education provided to staff specific to management of a from study authors on mortality per 100 discharges (Lighthall
deteriorating patient. 2010), and code blue per 1000 hospital admissions (Menon 2018).
For Aitken 2015, we used published data and the software Engauge
• Policy changes to ICU readmission practices.
(digitizer.sourceforge.net/) to enable ITS analyses of the outcomes
Risk of bias judgements were categorised as 'low risk of of unplanned admission and cardiac arrest. Similarly, we used this
bias' (comparable to a well-performed randomised trial); method to examine cardiac arrest per 1000 discharges in Lighthall
'moderate risk of bias' (sound evidence for a non-randomised 2010. We reanalysed data using segmented regression analysis with
study but cannot be considered comparable to a well-performed Newey-West autocorrelation adjusted standard errors with lag of
randomised trial); 'serious risk of bias' (some important problems); order 1 (Newey 1987). In this analysis we found Lighthall 2010 had
'critical risk of bias' (too problematic to provide any useful evidence employed appropriate ITS analysis methods, not limited to pre–
and should not be included in any synthesis); and 'no information' post comparisons. For Lighthall 2010 and Menon 2018, we reported
on which to base a judgement about risk of bias. We retained standardised rates per 1000 admissions/discharges for hospital
studies at critical risk of bias as our aim was to review all mortality and adverse events with the absolute effect or reanalysed
the available evidence. Review authors resolved discrepancies in ITS data. Data from Aitken 2015 could not be standardised as
certainty ratings by discussion. We assessed the methodological the number of admissions was not provided and, therefore, we
information and entered it into the risk of bias table for each reported author findings or reanalysed ITS data. These reanalyses
included study, and we provided an additional overall table for all did not adjust for confounding.
ROBINS-I assessments.
Dealing with missing data
Measures of treatment effect We contacted authors regarding missing data via several
Randomised trials strategies namely, the corresponding author contact email on the
publication, checking institutional email addresses using Google,
We report adjusted relative effects and absolute differences and via ResearchGate (ResearchGate 2017). We planned to assign
following standardisation across all studies for rates per 1000 studies to the Characteristics of studies awaiting classification table
admissions. We reported length of stay using hazard ratios (HR) when there was insufficient information to determine the study
(Priestley 2004) or adjusted ratio of geometric means (aRGM) design and efforts to contact the authors were unsuccessful.
(Jeddian 2016).

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 12
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Assessment of heterogeneity Sensitivity analysis

The small number of included studies meant we did not examine Given the small number of included studies, a sensitivity analysis
statistical heterogeneity between trials by inspecting forest plots to explore the impact of methodological quality was not possible.
or quantify the impact of heterogeneity using the I2 statistic
(Deeks 2011). As a result, we explored possible causes qualitatively Summary of findings and assessment of the certainty of the
such as design, quality of studies, differences in the participants, evidence
interventions, settings and control groups. We used the TIDieR We created a summary of findings table using the GRADE approach
(Template for Intervention Description and Replication) checklist to (Guyatt 2008), and following specific guidance developed by EPOC
compare interventions across studies (Hoffman 2014). (EPOC 2017d). We included in the summary of findings table the
outcomes of hospital mortality, composite outcome (unexpected
Assessment of reporting biases
cardiac arrests, unplanned ICU admissions and death), unplanned
We were unable to construct a funnel plot for the primary outcome ICU admissions, ICU readmissions, length of hospital stay and
against trial precision (standard error) as there were insufficient adverse events (unexpected cardiac arrest or respiratory arrest).
trials to conduct meta-analyses (Sterne 2011), or test formally for Previous systematic reviews combined data from randomised trials
asymmetry. and non-randomised studies (Chan 2010; Winters 2013; Maharaj
2015; de Jong 2016; Solomon 2016) however, we found no manner
Data synthesis of easily interpretable data presentation.
We summarised search results in a PRISMA study flow diagram To assess the certainty of evidence for each outcome, we used
(Moher 2009). We did not perform meta-analyses due to clinical and five GRADE considerations (risk of bias, inconsistency, indirectness,
methodological heterogeneity of included studies, and, therefore, imprecision and publication bias). Two review authors (JM, LR)
reported data narratively. For each included cluster randomised independently assessed the certainty of evidence (high, moderate,
study, we reported the published adjusted relative effects as these low or very low) following completion of the 'Calculation of
were appropriately adjusted for design and patient characteristics GRADE ratings worksheet' (Table 2). We considered evidence from
(Summary of findings 1). randomised trials as high certainty but downgraded the evidence
For each non-randomised study, we considered data provided per one level for serious (or two levels for very serious) limitations.
1000 admissions (Rothberg 2012; Ludikhuize 2015; Chen 2016; Justification for decisions to downgrade ratings are provided in the
Menon 2018); for two studies, we considered data per 100 (Lighthall footnotes of Summary of findings 1.
2010) or 1000 discharges (Davis 2015). For Lighthall 2010, we For imprecision, we downgraded certainty of evidence one level
standardised per 100 discharges to per 1000 discharges. We due to wide 95% CIs defined as less than 0.50 to more than
reported standardised rates per 1000 admissions or discharges 1.10 (Higgins 2020). For inconsistency, we downgraded certainty of
for mortality, unplanned ICU admission and adverse events with evidence one level when we were unable to determine or calculate
the relative effect where data were available or as reanalysed ITS the effect size of the outcome for all studies.
data. Where rates per 1000 admissions or discharges were not
provided, we used the total number of events and the total number For non-randomised studies, we downgraded all studies one level
of admissions or discharges to calculate a rate per 1000. due to risk of bias associated with no randomisation. We further
downgraded certainty of evidence one level for serious or critical
In addition, we calculated the absolute effects based on the original risk of bias due to confounding as assessed using the ROBINS-I
study data and presented these unadjusted risk differences for tool (Table 3). The ROBINS-I tool was designed to be applied to
non-randomised studies as there were differences in effect due individual results, however in the absence of any meta-analysis we
to lack of unadjusted data as a result of adjustment for multiple applied it to each study as a whole. Prior to future updates we will
time points (Table 1). When possible (i.e. study participant numbers reconsider this aspect of the protocol and look to apply the tool as
were available), we calculated 95% CIs for risk differences using it was designed to specific results from each study.
the Newcombe and Altman method (Newcombe 2000). We were
unable to standardise reanalysed ITS data or calculate absolute RESULTS
difference for unplanned ICU admissions and cardiac arrests
in one single-centre CBA study (Aitken 2015). If available, we Description of studies
reported the published relative effects for non-randomised studies
or reanalysed ITS data for three single-centre CBA studies (Lighthall We identified 11 studies comprising four randomised trials and
2010; Aitken 2015; Menon 2018). seven non-randomised studies.

Subgroup analysis and investigation of heterogeneity Results of the search

The small number and heterogeneity of included studies meant Figure 1 shows the PRISMA flow diagram for screening, selection
it was not possible to conduct subgroup analyses of nurse-led and assessment of studies.
outreach versus physician-led RRS, or by study design (cluster
We identified 5370 studies from all possible sources following the
randomised trial, parallel group randomised trial, CBA and ITS
removal of duplicates and excluded 5262 studies based on review of
studies).
title and abstract. We reviewed 108 full-text articles and identified
nine new studies meeting our inclusion criteria (Lighthall 2010;
Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen
2016; Jeddian 2016; Haegdorens 2018; Menon 2018). We previously

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 13
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

identified two studies in our original review (Priestley 2004; Hillman 13 wards); across 28 wards of seven hospitals (Haegdorens 2018);
2005) (see Characteristics of included studies table). or using matched hospitals (Hillman 2005 with 23 hospitals).

Included studies Sample sizes of randomised trials ranged from 2733 (Priestley 2004)
to 364,094 (Hillman 2005) participants.
Of the 11 studies meeting review inclusion criteria, two were
randomised trials included in the previously published review Settings
(Priestley 2004; Hillman 2005), two were new randomised trials
(Jeddian 2016; Haegdorens 2018), and seven were non-randomised Studies were conducted in the UK (Priestley 2004), Australia
studies (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; (Hillman 2005), Belgium (Haegdorens 2018), and Iran (Jeddian
Ludikhuize 2015; Chen 2016; Menon 2018). We obtained missing 2016). Therefore, all studies were set in middle- to high-income
data from authors on mortality (Davis 2015), study participant countries.
numbers (Hillman 2005), unplanned ICU arrest incidence (Lighthall
Two studies were single-centre general (Priestley 2004)
2010; Menon 2018), and further additional information from
or university-affiliated (Jeddian 2016) hospitals. Haegdorens
published supplementary files (Chen 2016; Davis 2015; Ludikhuize
2018 included seven acute care hospitals with at least 850
2015; Jeddian 2016). Details of are provided in the Characteristics
admissions per year. Hillman 2005 included 23 public hospitals
of included studies table, Table 4, and are briefly summarised
with greater than 20,000 admissions per year (11 control hospitals,
below. Table 5 shows the comparison of interventions using the
median bed number 315, interquartile range (IQR) 229 to 400) and
TIDieR checklist. Table 6 provides data on the clinical heterogeneity
12 MET hospitals (median bed number 364, IQR 182 to 457).
of the type of EWS and RRS.
Hospital wards included general wards (Hillman 2005; Jeddian
Three randomised trials (Priestley 2004; Hillman 2005; Haegdorens
2016), medical and surgical wards (Priestley 2004; Haegdorens
2018), and two non-randomised studies (Aitken 2015; Chen
2018), and elderly medicine wards (Priestley 2004). One study
2016), reported funding. All were investigator-led academic or
excluded patient data from emergency departments (EDs), ICUs,
government grants.
ICU-supervised high dependency units, operating theatres and
Randomised trials postoperative recovery (Hillman 2005). Exclusion criteria were
either not stated (Priestley 2004; Haegdorens 2018), or there were
Study design no patient exclusion criteria (Jeddian 2016).
We included one prospective cluster randomised trial (with the
Interventions
clusters at hospital level) conducted over 12 months in general
inpatient wards in 23 Australian hospitals (Hillman 2005). This trial EWS interventions comprised the PAR score (Priestley 2004), NEWS
adjusted for individual (age/sex) and cluster (bed number/hospital) and SBAR (Situation, Background, Assessment, Recommendation)
characteristics and reported an intraclass cluster coefficient communication tool (Haegdorens 2018), or specific calling criteria
(ICC) for primary and secondary outcomes. We included three (Hillman 2005; Jeddian 2016). PAR score is a multiple parameter
prospective stepped-wedge cluster randomised trials. These trials scoring system, NEWS is an aggregate-weighted scoring system,
introduced a CCOT in 16 acute general wards in one UK hospital and calling criteria are single parameter systems for a response to
(Priestley 2004); a CCOT across 13 general wards in one Iranian patient deterioration.
hospital (Jeddian 2016); and a MET across 28 medical or surgical
wards in seven Belgian hospitals (Haegdorens 2018). RRS interventions included a CCOT (Priestley 2004; Jeddian
2016), or a MET (Hillman 2005; Haegdorens 2018). Two studies
All prospective stepped-wedge cluster randomised trials adjusted introduced the CCOT/MET on a 24 hour, seven day per week basis
for cluster (ward/bed number) and time trends. Haegdorens (Priestley 2004; Haegdorens 2018). Two studies did not report team
2018 applied generalised linear mixed models (GLMM) as cluster availability (Hillman 2005; Jeddian 2016) (Table 5).
sizes varied. Jeddian 2016 used latent ICC.
Team composition varied across the four trials. One study included
Controls in these four trials comprised randomly assigned hospitals a CCOT led by a nurse consultant with a team of nurses supported
that did not receive MET education at any stage (Hillman 2005); by physicians when required (Priestley 2004); one study used a
and wards randomised to sequential roll out of the EWS and nurse-led CCOT consisting of six experienced ICU nurses (Jeddian
RRS intervention with baseline, intervention and postintervention 2016); two studies used a medical-led MET comprising a physician,
phases at different time points (Priestley 2004; Jeddian 2016; and either an ICU or ED nurse (Hillman 2005); or physician and
Haegdorens 2018). a nurse (Haegdorens 2018). At the time of RRT implementation,
two studies indicated the cardiac arrest team was retained
Participants (Hillman 2005; Haegdorens 2018), two studies did not report this
Two studies did not report age inclusion criteria (Priestley 2004; characteristic (Priestley 2004; Jeddian 2016).
Jeddian 2016). The remaining studies excluded participants aged
All randomised trials compared implementation of the EWS and
under 14 years (Hillman 2005) or under 17 years (Haegdorens
RRS intervention with control hospitals or wards that did not have
2018) admitted to participating wards. The mean age of
EWS and RRS. All studies used an educational strategy to introduce
participants ranged from 43 to 65 years across pre-intervention and
the RRS intervention with the training implementation phase
postintervention groups with the percentage of males ranging from
varying from four hours (Haegdorens 2018), one month (Priestley
39% to 54%. In the four randomised trials, allocation of participants
2004), two months (Jeddian 2016), and four months (Hillman
to the intervention or control groups was based on: single-centre
2005). Training consisted of in-service educational programmes on
admission ward (Priestley 2004 with 16 wards; Jeddian 2016 with
recognition of patient deterioration (Priestley 2004; Jeddian 2016),
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 14
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

and EWS or calling criteria (Priestley 2004; Hillman 2005; Jeddian Participants
2016; Haegdorens 2018). There was training for ward nurses Three studies reporting patient inclusion criteria included
(Priestley 2004; Hillman 2005; Jeddian 2016; Haegdorens 2018) participants aged 18 years and over admitted to hospital wards
and doctors (Priestley 2004; Hillman 2005). Hillman 2005 indicated (Ludikhuize 2015; Chen 2016; Menon 2018). Five studies enrolled
education focused on when and how to call the outreach team people admitted to all acute general wards (Lighthall 2010;
and not on the treatment of sick patients. Delivery of the Rothberg 2012; Davis 2015; Chen 2016; Menon 2018). Two studies
education was undertaken using lectures, video tapes and booklets enrolled all people admitted from four (Ludikhuize 2015) or six
(Hillman 2005), or via interactive training (Haegdorens 2018). wards (Aitken 2015) in the participating hospital(s).
Two studies emphasised sharing skills between teams and ward
staff (Priestley 2004; Jeddian 2016). To reinforce implementation, Sample sizes ranged from 18,954 (Menon 2018) to 9,799,081
two studies employed reminders in the form of posters (Hillman participants (Chen 2016). Three studies did not report participant
2005; Haegdorens 2018), identification badges (Hillman 2005), numbers (Lighthall 2010; Rothberg 2012; Aitken 2015). Two studies
champions (Haegdorens 2018), or made resources available via a providing participant demographics reported mean age ranged
study website including a knowledge test for nurses (Haegdorens from 62 to 65 years (Lighthall 2010; Ludikhuize 2015). Two studies
2018). reported the proportion of participants across age categories (aged
less than 18 to greater than 75 years) (Chen 2016; Menon 2018). The
Outcomes
percentage of men ranged from 47% to 96% (Lighthall 2010).
The unit of analysis was the cluster at the hospital level for one
study (Hillman 2005), and at the ward level for remaining studies. Settings
The baseline data collection period was one to four months with Three studies were conducted in the US (Lighthall 2010; Rothberg
postintervention implementation data collected at three to 16 2012; Davis 2015), two in Australia (Aitken 2015; Chen 2016), one
months (Priestley 2004; Hillman 2005; Jeddian 2016; Haegdorens in the Netherlands (Ludikhuize 2015), and one in India (Menon
2018). 2018). Therefore, all studies took place in middle- to high-income
countries.
Selection of outcomes varied across studies. Primary outcomes
included rate of hospital deaths (Priestley 2004; Jeddian 2016) Most studies were single centre, tertiary (Aitken 2015; Menon 2018)
or unexpected deaths (Haegdorens 2018); composite outcome or university-affiliated (Lighthall 2010; Rothberg 2012) hospitals.
of cardiac arrests without a pre-existing not-for-resuscitation Multicentre studies included two (Davis 2015), 12 (Ludikhuize
(NFR) order, unplanned ICU admissions, and unexpected deaths 2015), or 232 (Chen 2016) participating hospitals. All studies
(deaths without a pre-existing NFR order) (Hillman 2005); rate of differed in hospital size and number and type of wards involved.
unplanned ICU admissions (Haegdorens 2018); length of hospital Hospital bed size in single-centre studies ranged from a 226-
stay (Priestley 2004); and rate of cardiac arrests (Jeddian 2016; bed tertiary academic hospital (Menon 2018) to a 750-bed
Haegdorens 2018). Two studies reported individual components tertiary hospital (Aitken 2015). Multicentre studies included two
of their composite endpoint (cardiac arrests, unplanned ICU participating hospitals with an inpatient primary medical centre
admissions and unexpected deaths) (Hillman 2005) or length of stay (392 beds) and all units in a sister campus (119 beds) (Davis 2015);
and ICU admission (Jeddian 2016) as secondary outcomes. two large university hospitals (number of beds 882 to 1000), eight
large teaching hospitals (number of beds 359 to 1070) and two
Non-randomised studies smaller regional hospitals (number of beds 290 to 325) (Ludikhuize
Study design 2015); and 232 hospitals in New South Wales (Chen 2016).
We included seven non-randomised studies: three single-centre Two studies included acute surgical or medical wards (Ludikhuize
prospective CBAs analysed as ITS (Lighthall 2010; Aitken 2015; 2015; Menon 2018). Three studies included admissions to all
Menon 2018); one two-centre CBA (Davis 2015); one prospective, hospital wards (Rothberg 2012; Davis 2015; Chen 2016). Patient
multicentre CBA (Ludikhuize 2015); and two ITS studies (Rothberg data from EDs (Lighthall 2010; Aitken 2015; Davis 2015) and
2012; Chen 2016). For the single-centre studies, we used the ICU (Aitken 2015) were excluded in three studies or were not
data provided per 1000 admissions (Menon 2018), standardised reported (Rothberg 2012; Ludikhuize 2015; Chen 2016; Menon
published data to per 1000 discharges when the study had 2018). Lighthall 2010 excluded ED cardiac arrest events but
employed appropriate ITS analysis (Lighthall 2010), or reported included deaths in ED, ICU and general wards. All studies excluded
reanalysed ITS results for absolute difference when reanalysed hospice or nursing units within the hospital.
ITS data could not be standardised or relative effect could not
be calculated (Aitken 2015). In the pre-intervention period, five Interventions
studies used wards as controls (Lighthall 2010; Rothberg 2012;
EWS interventions comprised the MEWS, an aggregated weighted
Aitken 2015; Davis 2015; Menon 2018); and two studies used
score (Ludikhuize 2015) or single parameter calling criteria
hospitals as controls (Ludikhuize 2015; Chen 2016). Three studies
(Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016;
described usual care to comprise availability of a cardiac arrest
Menon 2018). In six studies, EWS interventions included objective
team (Lighthall 2010; Rothberg 2012; Menon 2018); one study
physiological criteria and the subjective criterion of staff concern or
described usual care as access to a medical registrar using MET
worry to trigger a call for help (Lighthall 2010; Rothberg 2012; Aitken
criteria and a cardiac arrest team (Aitken 2015). Three studies did
2015; Davis 2015; Ludikhuize 2015; Chen 2016). The Amrita Warning
not describe usual practice prior to the implementation of RRS
Criteria developed in India and used in one study did not include
(Davis 2015; Ludikhuize 2015; Chen 2016).
a subjective criterion of staff concern (Menon 2018). Two studies
included participant/family concern as a criterion to escalate a call
for help (Davis 2015; Chen 2016).
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 15
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

RRS interventions included RRT in four studies (Aitken 2015; Davis 2015; Menon 2018). In the remaining studies, the unit of
Davis 2015; Ludikhuize 2015; Chen 2016), or MET in three analysis was the cluster at either the hospital (Ludikhuize 2015;
studies (Lighthall 2010; Rothberg 2012; Menon 2018). Two studies Chen 2016), or ward level (Lighthall 2010).
introduced the RRS intervention on a 24 hour, seven day-a-week
basis (Rothberg 2012; Aitken 2015). The remaining five studies did The baseline data collection period was from five months
not state team availability (Lighthall 2010; Davis 2015; Ludikhuize (Ludikhuize 2015), nine months (Lighthall 2010), 12 months (Menon
2015; Chen 2016; Menon 2018). At the time of RRS implementation, 2018,) and beyond (Rothberg 2012; Aitken 2015; Davis 2015; Chen
the cardiac arrest team was retained (Lighthall 2010; Rothberg 2016). Studies reported postintervention implementation data at
2012; Davis 2015), considered as replaced by the RRS (Aitken 2015), five months (Ludikhuize 2015), 24 to 27 months (Lighthall 2010;
or availability was not stated (Ludikhuize 2015; Chen 2016; Menon Menon 2018), or 36 months and beyond (Rothberg 2012; Aitken
2018). 2015; Davis 2015; Chen 2016).

Team composition varied with teams comprising a medical-led Selection of outcomes varied across studies. The primary outcome
MET (Lighthall 2010; Rothberg 2012; Menon 2018); medical-led of non-randomised studies included mortality rates (Lighthall 2010;
RRT (Ludikhuize 2015; Chen 2016), or nurse-led RRT (Aitken 2015; Rothberg 2012; Davis 2015; Chen 2016; Menon 2018); composite
Davis 2015). Team membership also demonstrated substantial endpoint comprising cardiopulmonary arrest, unplanned ICU
heterogeneity (Table 6). admission, or death (Ludikhuize 2015); incidence of unplanned
ICU admission (Aitken 2015; Davis 2015); and cardiac arrest rates
All seven non-randomised studies compared the EWS and RRS (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Chen
intervention with a control group comprising standard practice 2016; Menon 2018). One study reported individual components
of acute care management before EWS and RRS implementation of their composite endpoint (cardiopulmonary arrest, unplanned
(Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize ICU admission or death) as secondary outcomes (Ludikhuize
2015; Chen 2016; Menon 2018). Intervention hospitals/wards all 2015). Other outcomes included staff satisfaction, prevalence
used an educational strategy to introduce the EWS and RRS of deteriorating patients (Aitken 2015); patient acuity, hospital
intervention, with length of the training implementation phase discharge (Davis 2015); Charlson Comorbidity Index (Lighthall
ranging from several hours as part of annual training (Davis 2010); number of RRS calls (Rothberg 2012; Davis 2015; Menon
2015), one month (Menon 2018), four months (Lighthall 2010), 2018); MET process variables and staff questionnaire (Menon 2018).
to seven months for the implementation of EWS and SBAR as No studies reported ICU readmission rates.
part of the first phase of a two-stage RRS strategy (Ludikhuize
2015). Three studies did not describe the training timeframe Excluded studies
(Rothberg 2012; Aitken 2015; Chen 2016). Training consisted of Ninety-seven studies did not meet our inclusion criteria and
in-service educational programmes on calling criteria (Lighthall are included in the Characteristics of excluded studies table.
2010), purpose of RRT and how to activate team (Rothberg The primary reasons for exclusion was study design due to an
2012), recognition of patient deterioration and annual advanced inadequate number of data points before and after the intervention
resuscitation training programme (Davis 2015), MEWS and SBAR to meet the EPOC criteria for CBAs, followed by failing to include
communication tool (Ludikhuize 2015), and standardised state- both EWS and RRS as components of the intervention.
wide Between the Flags programme (Chen 2016). Two studies
reported educational sessions were tailored for coronary care unit One author replied to a request for further information (Rashid
(CCU), ED, ward nurses, consultants and registrars (Aitken 2015), or 2014), and we excluded the study as a retrospective single-centre
were trained using standardised tool-kits (Ludikhuize 2015). Two CBA. We excluded one paper, which we translated from Czech to
studies did not report on educational content (Aitken 2015; Menon English, based on study design criteria (Martin 2009). Requests
2018). One study reported the mode of delivery was via meetings, for additional studies from professional organisations identified
email, communication boards and posters (Rothberg 2012). To three studies from the European federation of Critical Care Nurses
reinforce implementation, four studies employed reminders in association (EfCCNa) (Laurens 2011; Chen 2015; Petersen 2016),
the form of posters (Lighthall 2010; Rothberg 2012; Aitken 2015; and one study from Critical Care National Network Nurse Leads
Ludikhuize 2015), identification badge holders with calling criteria Forum (CC3N) (Churpek 2017). We excluded all four studies, two
(Lighthall 2010), pocket cards (Rothberg 2012; Ludikhuize 2015; based on design criteria (Laurens 2011; Chen 2015), and two as the
Menon 2018), calling criteria lanyards and screen-saver reminders wrong intervention (Petersen 2016; Churpek 2017).
(Aitken 2015), workflow handouts (Menon 2018), and activation
number on all ward telephones (Rothberg 2012). Risk of bias in included studies
Outcomes Randomised trials

Individual patient data (i.e. patient admissions to study wards) was Risk of bias for the four randomised trials is presented in
the unit of analysis in four studies (Rothberg 2012; Aitken 2015; the Characteristics of included studies table detailing our
judgements for each domain (see Figure 2).

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 16
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Figure 2. Risk of bias summary: review authors' judgements about each risk of bias item for each included
randomised study.

Allocation before the transition period commenced (Jeddian 2016). We

considered two studies as unclear risk of bias (Priestley 2004;
We considered three studies at low risk of bias for random sequence
Haegdorens 2018) as the method of allocation concealment was
generation as they undertook block randomisation using computer
not described explicitly.
software stratifying hospitals by teaching status and number of
beds (Hillman 2005); they conducted randomisation at a fixed time Blinding
point independent of the trial team (Jeddian 2016), or an individual
not involved in further study conduct performed computerised We rated performance bias in all randomised trials at high risk of
randomisation (Haegdorens 2018). One study did not describe bias as blinding of participants and personnel to group allocation
the method of random sequence generation explicitly and was at was not feasible and therefore not performed.
unclear risk of bias (Priestley 2004).
We rated all studies at low risk of bias for blinding of outcome
We considered two studies at low risk of bias for allocation assessment as outcomes were objective measurements collected
concealment as an independent statistician concealed allocation by trained data collectors/independent data team (Hillman 2005),
from investigators/participating hospitals (Hillman 2005), or or used hospital databases (Priestley 2004; Jeddian 2016), or both
information on ward sequence was revealed two to three days (Haegdorens 2018).
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 17
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Incomplete outcome data as authors stated findings were confirmed using data set three but
provided no details or data for data set three to confirm findings
We rated three studies at low risk of bias as no study reported
(Priestley 2004).
incomplete data or loss to follow-up (Priestley 2004; Hillman 2005;
Jeddian 2016); and one study at high risk of bias as 50% of Other potential sources of bias
enrolled hospitals withdrew from the study (Haegdorens 2018).
Three studies reported using intention-to-treat analyses (Priestley We rated three studies at unclear risk of bias as there was potential
2004; Hillman 2005; Jeddian 2016); the remaining study did not for contamination due to publicity of the intervention across
report this (Haegdorens 2018). control wards (Priestley 2004; Jeddian 2016; Haegdorens 2018);
and the remaining study at high risk of contamination bias as the
Selective reporting benefits of the MET system were widely reported in the media,
which may have impacted on the behaviour of the control hospitals
We identified two published trial registrations with the
(Hillman 2005).
same outcomes identified a priori as published in the final
report (Jeddian 2016, IRCT201107187053N1; Haegdorens 2018, Non-randomised studies
NCT01949025). For the two earlier studies for which we were unable
to identify a trial registration or published protocol, we considered The domain-level judgements for risk of bias for all seven non-
one study included all expected outcomes and rated as low risk of randomised studies are presented (see Figure 3). Further details
bias (Hillman 2005); the remaining study was at high risk of bias and our supporting statements for domain-level judgments for risk
of bias are available (Table 3) and summarised (Table 7).

Figure 3. Risk of bias summary: review authors' judgements about each risk of bias item for each included non-
randomised study.

Confounding bias ICU and hospital census; staffing levels; patient acuity/case-mix
on hospital wards; ward culture and organisational practices;
We established the following confounding variables a priori: pre-
medical and nursing education levels; ICU discharge policies, ICU
existing comorbidity; age; severity of illness during ICU admission;
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 18
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

readmission policies, and overall predilection to readmit to ICU; Effects of interventions

ICU admission/readmission rates; prevalence of patients with NFR
order; seasonal variation in patient census; patient safety initiatives See: Summary of findings 1 Early warning systems (EWS) and rapid
within the organisation; cardiac arrest team in place as well as the response systems (RRS) compared with usual care for prevention of
outreach/RRT; education provided to staff specific to management patient deterioration on acute hospital wards
of a deteriorating patient; policy changes to ICU readmission
See: Summary of findings 1.
practices.
We present effect sizes for each study for the following outcomes.
We considered three studies at critical risk of confounding bias
(Aitken 2015; Davis 2015; Menon 2018), and four studies at serious Hospital mortality
risk due to use of statistical methods to control for some but not all
baseline confounders (Lighthall 2010; Rothberg 2012; Ludikhuize Ten studies reported on mortality.
2015; Chen 2016).
Randomised trials
Selection bias We identified four studies recruiting 455,226 participants reporting
We considered all seven studies at low risk of selection bias as we on hospital mortality. The measurement of hospital mortality
saw no evidence of selective recruitment (Lighthall 2010; Rothberg varied across studies; Priestley 2004 and Jeddian 2016 reported
2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon hospital mortality without exclusions while Haegdorens
2018). All admitted patients were included in the seven studies 2018 and Hillman 2005 reported unexpected mortality as either
based on time of exposure, that is, before or after the intervention NFR order, palliative or terminal care, family attending during
commenced as opposed to participant characteristics observed the process of dying, cessation or limiting of active therapy in
after the commencement of the intervention. For each participant, untreatable disease' (Haegdorens 2018) or as all inpatient deaths
start of follow-up and start of intervention coincided. without a pre-existing NFR order (Hillman 2005).

Bias in classification of interventions One study identified a reduction in hospital mortality (adjusted
odds ratio (aOR) 0.52, 95% CI 0.32 to 0.85) (Priestley 2004). The
We considered all seven studies at low risk of bias in classification remaining three studies identified no effect on hospital mortality
of interventions as intervention status was clearly defined in terms (aOR 1.03, 95% CI 0.84 to 1.28 (Hillman 2005); aOR 1.02, 95% CI 0.68
of time of exposure, that is, admission to hospital before or after to 1.55 (Jeddian 2016); aOR 0.82, 95% CI 0.34 to 1.95 (Haegdorens
the EWS and RRS intervention was introduced (Lighthall 2010; 2018)).
Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize 2015; Chen
2016; Menon 2018). All randomised trials adjusted for cluster (ward/bed number); three
studies adjusted for time trends (Priestley 2004; Jeddian 2016;
Bias due to deviations from intended interventions Haegdorens 2018).
For all seven studies, we were unable to assess if there was
The implementation of EWS and RRS may result in little or no
deviation from the intended intervention due to lack of reporting
difference in hospital mortality (low-certainty evidence).
(Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Ludikhuize
2015; Chen 2016; Menon 2018). Non-randomised studies
Bias due to missing data We identified six studies, three recruited 210,905 participants
(number of participants in remaining three studies was unclear
We considered all seven studies at low risk of bias due to missing
due to reporting bed occupancy (Lighthall 2010); number
data (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015;
of participants not provided (Rothberg 2012); and participants
Ludikhuize 2015; Chen 2016; Menon 2018).
grouped annually (Chen 2016)).
Bias in measurement of outcomes
One ITS study identified a reduction in hospital mortality (adjusted
Measured outcomes including cardiac arrest, death and unplanned risk ratio (aRR) 0.81, 95% CI 0.76 to 0.86 (Chen 2016)). Two studies
ICU admission are all objective and not subject to measurement showed no effect (aOR 0.85, 95% CI 0.64 to 1.00 (Ludikhuize
bias. Therefore, we considered all seven studies at low risk of 2015); risk ratio (RR) 1.0, 95% CI 0.03 to 29.8 (Menon 2018)).
outcome measurement bias (Lighthall 2010; Rothberg 2012; Aitken We were unable to calculate the relative effect on mortality for
2015; Davis 2015; Ludikhuize 2015; Chen 2016; Menon 2018). the remaining three studies (Lighthall 2010; Rothberg 2012; Davis
2015). Reanalysed ITS data for one single-centre CBA showed an
Bias in selection of the reported result absolute increase of 0.19 deaths per 100 discharge at 24 months'
We found no evidence of selective reporting and rated risk as low for postintervention (0.19, 95% CI −0.83 to 1.22; P = 0.02; Lighthall
all seven studies (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2010).
2015; Ludikhuize 2015; Chen 2016; Menon 2018).
We are uncertain whether this intervention reduces mortality
Overall risk of bias because the certainty of this evidence was very low (Davis 2015;
Ludikhuize 2015; Chen 2016; Menon 2018).
We rated three studies at critical overall risk of bias (Aitken 2015;
Davis 2015; Menon 2018), and four studies at serious risk of bias Non-randomised studies adjusted for time (Rothberg 2012;
(Lighthall 2010; Rothberg 2012; Ludikhuize 2015; Chen 2016). Davis 2015; Chen 2016), patient acuity (Davis 2015); individual
characteristics (age, sex, marital status, country of birth,

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 19
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

socioeconomic status, private health insurance), geographical 95% CI −15.24 to 2.20; Aitken 2015). We were unable to calculate
locations of hospitals (urban versus rural), hospital type (Chen the absolute difference or standardise reanalysed ITS data in this
2016); time trends and seasonality (time series data), CCI score and study as the number of admissions was not provided (Aitken
periodic or secular variations (Lighthall 2010); and age, sex, hospital 2015). Therefore, we do not know whether EWS and RRS reduces
and admission type (Ludikhuize 2015). unplanned ICU admissions (very low-certainty evidence).

Composite outcome Ludikhuize 2015 adjusted for individual (age, sex), emergency
admission and individual hospital characteristics; Aitken
Two studies reported on a composite outcome as either the
2015 adjusted for time differences.
composite outcome of incidence (events divided by number of
eligible patients admitted to the hospital during the study period) Intensive care unit readmission
of cardiac arrests without a pre-existing NFR order, unplanned
ICU admissions, and unexpected deaths (deaths without a pre- No studies reported ICU readmission rates and therefore the effect
existing NFR order) (Hillman 2005) or composite endpoint of on ICU readmission of an EWS and RRS intervention is unknown.
cardiopulmonary arrest, unplanned ICU admission, or death per
Length of hospital stay
1000 admitted patients (Ludikhuize 2015).
Two randomised trials reported length of hospital stay (Priestley
Randomised trials 2004; Jeddian 2016). No non-randomised studies reported this
One study recruited 364,094 participants and found no effect in outcome.
composite outcome during the six-month study period (aOR 0.98,
Randomised trials
95% CI 0.83 to 1.16; Hillman 2005), adjusting for individual (sex, age)
and cluster (bed number, hospital teaching status) characteristics We identified two studies recruiting 21,417 participants. Priestley
(ICC 0.067, 95% CI 0.05 to 0.08). The implementation of EWS and 2004 reported an increase in mean length of hospital stay in the
RRS probably results in little or no difference in composite outcome EWS and RRS intervention arm compared to control (hazard ratio
(moderate-certainty evidence). (HR) 0.91, 95% CI 0.83 to 0.98). However, there was no difference
following adjustment for clustering and in further sensitivity
Non-randomised studies analyses examining ward characteristics and time trends. Jeddian
One prospective, multicentre non-randomised study recruited 2016 found a reduction in the median days of hospital stay (6, IQR
57,858 participants and found a reduction in composite outcome 3 to 10 with intervention versus 4, IQR 2 to 8 with no intervention;
(aOR 0.85, 95% CI 0.72 to 0.99; Ludikhuize 2015), adjusting for aRGM 1.00, 95% CI 0.97 to 1.03). However, there was no difference
individual (age, sex), emergency admission and individual hospital when adjusted for individual (age, sex, Simplified Acute Physiology
characteristics. The implementation of EWS and RRS may result Score (SAPS II), admission type), cluster (ward) and time effects
in little or no difference in composite outcome (low-certainty (aRGM 1.00, 95% CI 0.97 to 1.03; ICC 0.10, 95% CI 0.01 to 0.18).
evidence). Therefore, an EWS and RRS intervention may result in little or no
difference in hospital length of stay (low-certainty evidence).
Unplanned intensive care unit admission
Non-randomised studies
Five studies reported unplanned ICU admissions as a primary
(Aitken 2015; Haegdorens 2018) or secondary outcome (Hillman No non-randomised studies reported length of hospital stay.
2005; Ludikhuize 2015; Jeddian 2016).
Adverse events (unexpected cardiac or respiratory arrest)
Randomised trials Ten studies measured cardiac arrest rates as either a primary
We identified three studies recruiting 452,434 participants. All three (Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 2015; Chen
studies identified no effect on the incidence of unplanned ICU 2016; Jeddian 2016; Menon 2018; Haegdorens 2018) or secondary
admissions (aOR 1.04, 95% CI 0.89 to 1.21 (Hillman 2005); aOR outcome (Hillman 2005; Ludikhuize 2015).
1.15, 95% CI 0.64 to 2.09 (Jeddian 2016); aOR 1.23, 95% CI 0.91 to
Randomised trials
1.65 (Haegdorens 2018)), adjusting for cluster (ward/bed number)
and time trends (Hillman 2005; Jeddian 2016; Haegdorens 2018). We identified three studies recruiting 452,434 participants. All three
RRS and EWS intervention may result in little or no difference in studies showed no difference in adverse event rates when adjusted
unplanned ICU admissions (low-certainty evidence). for cluster (ward/bed number) and time effects (aOR 0.94, 95% CI
0.79 to 1.13 (Hillman 2005); aOR 1.00, 95% CI 0.69 to 1.48 (Jeddian
Two studies reported ICC (0.09, 95% CI 0.076 to 0.12 (Hillman 2005); 2016); aOR 0.71, 95% CI 0.33 to 1.52 (Haegdorens 2018)). The
0.01, 95% CI 0.0 to 0.26 (Jeddian 2016)). implementation of EWS and RRS may result in little or no difference
to adverse events (low-certainty evidence).
Non-randomised studies
We identified two studies, one reported recruiting 57,858 Two studies reported ICC (0.02, 95% CI 0.01 to 0.07 (Hillman 2005);
participants. One prospective, multicentre CBA found no effect 0.06, 95% CI 0.00 to 0.12 (Jeddian 2016)).
in incidence of unplanned ICU admission (aOR 0.88, 95% CI 0.75
Non-randomised studies
to 1.02, Ludikhuize 2015). Reanalysed ITS data from a single-
centre CBA showed no evidence of change in trend due to the We identified seven studies, three reported recruiting 210,905
intervention (P = 0.10), with an absolute decrease of 6.52 unplanned participants and the number of participants in remaining four
ICU admissions per month after 24 months intervention (−6.52, studies was unclear due to reporting bed occupancy (Lighthall

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 20
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

2010), number of admissions/participants not reported (Rothberg From non-randomised studies, we found very low-certainty
2012; Aitken 2015), and number of participants grouped yearly evidence of little to no effect of EWS and RRS interventions on
(Chen 2016). hospital mortality, unplanned ICU admissions or adverse events.
We found low-certainty evidence of little to no effect on composite
Two studies reported a reduction in adverse events (aOR 0.61, 95% outcome. No non-randomised studies reported hospital length of
CI 0.39 to 0.94 (Ludikhuize 2015); aRR 0.55, 95% CI 0.48 to 0.62 (Chen stay or ICU readmissions and, therefore, we are unable to make a
2016)). One study reported no effect on adverse event rates (aOR statement as to the effect of EWS and RRS interventions on these
1.00, 95% CI 0.69 to 1.48; Rothberg 2012). outcomes.
Reanalysed data from single-centre CBA studies showed no See Summary of findings 1 for more information on our judgements
evidence of change in trend due to the EWS and RRS intervention for the certainty of evidence for our outcomes of interest.
24 months postintervention (20.3, 95% CI −18.56 to 59.19; P =
0.62 (Lighthall 2010); −1.29, 95% CI −8.49 to 5.91; P = 0.58 (Aitken Overall completeness and applicability of evidence
2015); 3.44, 95% CI −10.35 to 17.23; P = 0.25 (Menon 2018)). As the
certainty of this evidence was very low, we do not know whether The 11 included studies were conducted in large urban hospitals
this intervention reduces cardiac arrests. in middle- or high-income countries, with diverse patient
populations. Therefore, we can make no comment on the
Non-randomised studies adjusted for time (Lighthall 2010; applicability of our findings to hospitals in lower-income countries.
Rothberg 2012; Aitken 2015; Davis 2015; Chen 2016; Menon 2018), These 11 studies highlight heterogeneity in EWS systems between
patient acuity (Davis 2015); individual (age, sex, marital status, aggregated weighted scores and single parameter criteria, RRS
country of birth, socioeconomic status, private health insurance), leadership and composition, RRS dose and implementation
geographical area of hospitals (urban versus rural), major hospital processes. These factors and the variation in context and adherence
peer groups (Chen 2016); and age, sex, hospital and admission type to EWS and RRS make generalisations difficult. The choice of EWS
(Ludikhuize 2015). tool was linked to the type of RRS specialist team used (i.e. CCOT,
RRT or MET). In general, aggregate scoring systems were used by
Reanalysed data CCOTs; single parameter systems were used by METs/RRTs. Single
parameter criteria were more commonly used as the EWS trigger
The cluster randomised study results are reported as the published
mechanism, despite being less accurate than aggregated weighted
adjusted relative effects as these were appropriately adjusted for
scores (Churpek 2016; Smith 2016). Further evidence is required to
design and patient characteristics (Summary of findings 1). The
understand the effect of the type of trigger mechanism used on
reanalysed absolute effects are presented in a separate table
patient outcomes.
as there were noticeable differences in effect sizes between the
relative effect and absolute effect (Table 1). These differences were RRS composition and number of individuals in the team varied
attributed to errors as the absolute effect values were unadjusted across countries with either a nurse-led, or physician-led team
and the ICC was not derived from the studies. supported by various other supporting team members. The impact
of the team composition is unclear. We had planned a subgroup
Non-randomised study results provide relative effects obtained
analysis of nurse-led outreach versus medical-led MET. However,
from the original study data where possible (Summary of findings
this was not possible due to the limited number of studies.
1) or as reanalysed ITS data for three single-centre CBA studies
Most studies did not state the availability of the team. Structure
(Lighthall 2010; Aitken 2015; Menon 2018) (Table 1). There were no
and content of ward staff training also varied across studies. To
data where relative differences could not be obtained from study
inform further understanding of the effectiveness of EWS and
authors or calculated from the original study data.
RRS interventions, standardisation of practices and guidelines for
DISCUSSION the composition and dose of RRS would allow for more robust
comparisons.
Summary of main results
Due to strict inclusion criteria relating to study design, that is,
We identified 11 studies meeting our inclusion criteria and exclusion of single-centre CBA without three data points before
reporting on the effectiveness of an EWS and RRS intervention and after intervention implementation, we were only able to
for reducing hospital mortality, composite outcome (unexpected identify a small number of relevant studies. Despite these studies
cardiac arrests, unplanned ICU admissions and death), unplanned including 666,131 participants, no strong recommendations as to
ICU admissions, length of hospital stay and adverse events the effectiveness of EWS and RRS interventions can be made based
(unexpected cardiac arrest or respiratory arrest). We found no on the evidence currently available. Inconclusive findings from the
studies reporting on ICU readmission rates. review limit the applicability of the findings. Our inclusion of non-
randomised studies provided an important understanding of the
Meta-analyses across studies were not possible due to clinical limitations of combining study designs, as results differed from the
heterogeneity in terms of the intervention, setting and study results of randomised trials (Reeves 2020). No studies evaluated
designs. From randomised trials, we found low-certainty evidence costs and future work should determine the economic implications
that an EWS and RRS intervention may result in little or no of implementing RRS.
difference in hospital mortality, unplanned ICU admission rates,
hospital length of stay or adverse events; and moderate-certainty The evidence from this review update highlights some diversity
evidence of little to no effect on composite outcome. in outcome selection and moderate-to-poor methodological
quality of studies investigating EWS and RRS interventions. The
moderate- to very low-certainty evidence makes it difficult to

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 21
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

draw conclusions regarding the effectiveness of EWS and RRS. with more rigorous designs and findings that are more cautious
The inconsistent implementation of EWS and RRS criteria, team given the limited number of high-quality studies.
composition, dose and processes require further description to
understand intervention fidelity across contexts and countries. AUTHORS' CONCLUSIONS
There is a need for development of a patient-informed core
outcome set comprising clear and consistent definitions (COMET Implications for practice
Initiative), and recommendations for measurement as well as There is low- to moderate-certainty evidence from randomised
EWS and RRS interventions conforming to a standard to facilitate controlled trials that early warning system (EWS) and rapid
meaningful comparison and future meta-analyses. Further studies response system (RRS) interventions have little to no effect on
utilising alternative designs to randomised trials is warranted to hospital mortality, composite outcome, unplanned ICU admission
determine the effectiveness of EWS and RRS. rates, hospital length of stay or adverse events. However, the
widespread adoption of EWS and RRS suggests there are perceived
Quality of the evidence benefits in practice that currently are not evidenced in measured
We judged the certainty of evidence from randomised controlled objective patient outcomes. For example, having the ability to call
trials as low for hospital mortality, unplanned ICU admission, a specialist team for urgent review may provide benefits for ward
length of stay and adverse events, and moderate for the composite staff in terms of reduced stress, and improved well-being. This may
outcome (Summary of findings 1). We judged the certainty of lead to improved staff retention. To ascertain the perceived benefits
evidence from non-randomised studies as very low for hospital for healthcare staff, patients, relatives and organisations, further
mortality, unplanned ICU admissions and adverse events, and low research is required that measures staff-, patient- and relative-
certainty for the composite outcome (Summary of findings 1). reported outcomes as well as a better understanding of effective
Because we applied the ROBINS-I tool to studies as a whole rather processes to implement and sustain an EWS and RRS intervention.
than to specific results as is recommended, the bias assessments
may not accurately reflect bias of the individual results of the Implications for research
studies. EWS and RRS have been implemented internationally with no
evidence of cost-effectiveness. Future research studies need to
Potential biases in the review process build in an economic evaluation of EWS and RRS to ensure the
We believe the potential for bias in our review process is low. implementation of EWS and RRS is cost-effective given the low-
We adhered to procedures outlined by Cochrane (Higgins 2011); to moderate-certainty evidence from randomised controlled trials
used a comprehensive search strategy designed to minimise the that EWS and RRS interventions have little to no effect on hospital
risk of language and publication bias; and conducted independent mortality, unplanned ICU admission rates, hospital length of stay or
study inclusion screening, data extraction, risk of bias assessment adverse events.
and grading of evidence certainty by two review authors. We
made modifications to the review from the original protocol The variation in context, adherence, and the type and combination
(see Differences between protocol and review), but we do not of EWS and RRS interventions restricts direct comparisons. Further,
believe these modifications introduced bias into the review evidence suggests EWS with standardised protocols are being
process. implemented in hospitals within existing referral systems (i.e.
without the implementation of an RRS). Implementation of the
Agreements and disagreements with other studies or afferent arm of the RRS without the efferent arm may be due
reviews to a lack of equivocal evidence or based on economic reasons.
Interpreting the effect of multiple interventions introduced at the
We identified seven systematic reviews on EWS and RRS same time is difficult and further understanding of the components
interventions. Comparison of our findings with three reviews of the afferent and efferent arm for effective RRS implementation
highlight similar issues of poor-quality studies with high risk of is required. Additionally, evidence underpinning the process and
bias and the variability of evidence on patient outcomes (Chan context of EWS and RRS implementation suggests that staffing
2010; McNeill 2013; Tirkkonen 2017). Four other systematic reviews ratios and resources impact on the fidelity of the intervention and
undertook meta-analyses including randomised trials, prospective the effect on measured outcomes (McGaughey 2017a; McGaughey
and retrospective observational studies (before and after) and ITS 2017b). Therefore, future studies of EWS and RRS interventions
designs. These broader inclusion criteria in terms of study design, should be accompanied by a process evaluation (Moore 2015) using
single centre studies and adult / paediatric populations resulted in a framework that helps understand the contextual drivers of the
inclusion of 26 studies (Winters 2013), 29 studies (Maharaj 2015), 32 implementation process.
studies (de Jong 2016), and 30 studies (Solomon 2016). These four
systematic reviews all reported that an EWS and RRS intervention The potential risk of contamination bias is an issue with EWS
reduces hospital mortality (Winters 2013; Maharaj 2015; de Jong and RRS research. This was specifically highlighted by Sandroni
2016; Solomon 2016), and cardiac arrests (Winters 2013; Maharaj 2015 with reference to the Hillman 2005 study which noted that
2015; Solomon 2016). although authors made every effort to prevent contamination, the
benefits of the RRS system were reported in the media during
A major difference of our review was the strict inclusion criteria in the study period which could have affected personal behaviour
terms of study design and adult population limiting the number of in the control hospitals and explain why the rates of in-hospital
included studies. Further, we chose not to meta-analyse data from cardiac arrest decreased more in the hospitals of the control
studies that we considered to have considerable heterogeneity in group than in those of the interventional group between the two
terms of the structure and composition of the study interventions. study periods. Large randomised trials of complex interventions
As a result, the findings of our review update are based on studies
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 22
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

with an embedded process evaluation may provide insight into comments and assistance in preparing this updated review (Sasha
the potential risks and strategies that need to be considered and Shepperd, Co-ordinating Editor; Chris Cooper, Daniela Gonçalves
monitored to mitigate against contamination in the conduct of Bradley and Julia Worswick, Managing Editors; Paul Miller,
trials (Robinson 2020). Information Specialist; Andrew Hutchings, Statistical Editor).

In summary, research implications arising from this review include We would also like to thank reviewers for their helpful comments
the need to: on earlier drafts (Christiana Kartsonaki, Stats Referee; Villyen
Motaze, Internal Editor; Charles Ameh, External Referee; Duncan
• understand decisions regarding why different afferent and Smith, External Referee; Ivana Turudic, Consumer; Tess Moore,
efferent components are adopted in practice; Andrew Black and Kerry Dwan, Cochrane Methods Support Unit);
• measure fidelity of future EWS and RRS intervention trials; Dawn Harbison, Research Assistant, for preliminary screening
• determine the most effective dose and reach of EWS and RRS of the literature; and statisticians for reanalysis of single-
interventions; centre controlled before-after studies, standardisation of data and
• establish consensus on a core outcome measurement set to summary of findings table (Ranjeeta Mallick, Senior Statistian; Josh
facilitate comparisons of outcomes across studies; and Montray, Clinical Research Associate Ottawa Hospital Research
Institute).
• conduct cost-effectiveness studies.
National Institute for Health Research, via Cochrane Infrastructure
ACKNOWLEDGEMENTS funding to the EPOC Group. The views and opinions expressed
herein are those of the authors and do not necessarily reflect those
The review authors would like to thank members of the Effective
of the Systematic Reviews Programme, National Institute for Health
Practice and Organisation of Care (EPOC) team for their helpful
Research, National Health Service or the Department of Health.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 23
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

REFERENCES

References to studies included in this review Menon 2018 {published and unpublished data}10.1097/
QMH.0000000000000160
Aitken 2015 {published data only}org/10.1016/
j.aucc.2014.10.044 Menon VP, Prasanna P, Edathadathil F, Balachandran S, Moni M,
Sathyapalan D, et al. A quality improvement initiative to
Aitken LM, Chaboyer W, Vaux A, Crouch S, Burmeister E,
reduce "out-of-ICU" cardiopulmonary arrests in a tertiary
Daly M, et al. Effect of a 2-tier rapid response system on
care hospital in India: a 2-year learning experience. Quality
patient outcome and staff satisfaction. Australian Critical Care
Management in Health Care 2018;27(1):39-49. [DOI: 10.1097/
2015;28(3):107-14. [DOI: org/10.1016/j.aucc.2014.10.044]
QMH.0000000000000160]]
Chen 2016 {published data only}10.1016/
Priestley 2004 {published data only}
j.resuscitation.2016.07.240
Priestley G, Watson W, Rashidian A, Mozley C, Russell D,
Chen J, Ou L, Flabouris A, Hillman K, Bellomo R, Parr M. Impact
Wilson J, et al. Introducing critical care outreach: a ward-
of a standardized rapid response system on outcomes in a large
randomised trial of phased introduction in a general hospital.
healthcare jurisdiction. Resuscitation 2016;107:47-56. [DOI:
Intensive Care Medicine 2004;30:1398-404. [DOI: 10.1007/
10.1016/j.resuscitation.2016.07.240]
s00134-004-2268-7]
Davis 2015 {published and unpublished data}10.1002/jhm.2338
Rothberg 2012 {published data only}
Davis DP, Aguilar SA, Graham PG, Lawrence B, Sell RE,
Rothberg MB, Belforti R, Fitzgerald J, Friderici J, Keyes M. Four
Minokadeh A, et al. A novel configuration of a traditional rapid
years' experience with a hospitalist-led medical emergency
response team decreases non-intensive care unit arrests
team: an interrupted time series. Journal of Hospital Medicine
and overall hospital mortality. Journal of Hospital Medicine
2012;7(2):98-103. [DOI: 10.1002/jhm.953]
2015;10(6):352-7. [DOI: 10.1002/jhm.2338]

Haegdorens 2018 {published data only}

References to studies excluded from this review
Haegdorens F, van Bogaert P, Roelant E, De Meester K,
Misselyn M, Wouters K, et al. The introduction of a rapid Al Qahtani 2013 {published data only}
response system in acute hospitals: a pragmatic stepped Al-Qahtani S, Al-Dorzi HM, Tamim HM, Hussain S, Fong L,
wedge cluster randomised controlled trial. Resuscitation Taher S, et al. Impact of an intensivist-led multidisciplinary
2018;129:127-34. [DOI: 10.1016/j.resuscitation.2018.04.018] extended rapid response team on hospital-wide
cardiopulmonary arrests and mortality. Critical Care Medicine
Hillman 2005 {published data only} 2013;41(2):506-17. [DOI: 10.1097/CCM.0b013e318271440b]
Hillman K, Chen J, Cretikos M, Bellomo R, Brown D, Doig G,
et al, MERIT study Investigators. Introduction of the medical Aneman 2015 {published data only}
emergency team (MET) system: a cluster-randomised control Aneman A, Frost SA, Parr MJ, Hillman KM. Characteristics and
trial. Lancet 2005;365(9477):2091-7. [PMID: 15964445] outcomes of patients admitted to ICU following activation of
the medical emergency team: impact of introducing a two-tier
Jeddian 2016 {published data only} response system. Critical Care Medicine 2015;43(4):765-73. [DOI:
Jeddian A, Hemming K, Lindenmeyer A, Rashidian A, Sayadi L, 10.1097/CCM.0000000000000767]
Jafari N, et al. Evaluation of a critical care outreach service in a
middle-income country: a stepped wedge cluster randomized Austin 2014 {published data only}
trial and nested qualitative study. Journal of Critical Care Austin CA, Hanzaker C, Stafford R, Mayer C, Culp L, Lin FC,
2016;36:212-7. [DOI: 10.1016/j.jcrc.2016.07.018] et al. Utilization of rapid response resources and outcomes
in a comprehensive cancer center. Critical Care Medicine
Lighthall 2010 {published and unpublished data} 2014;42(4):905-9. [DOI: 10.1097/CCM.0000000000000098]
Lighthall GK, Parast LM, Rapoport L, Wagner TH. Introduction
of a rapid response system at a United States veterans affairs Bannard-Smith 2016 {published data only}
hospital reduced cardiac arrests. Anesthesia and Analgesia Bannard-Smith J, Lighthall GK, Subbe CP, Durham L, Welch J,
2010;111(3):679-86. [DOI: 10.1213/ANE.0b013e3181e9c3f3] Bellomo R, et al. Clinical outcomes of patients seen by rapid
response teams: a template for benchmarking international
Ludikhuize 2015 {published data only}10.1097/ teams. Resuscitation 2016;107:7-12.
CCM.0000000000001272
Ludikhuize J, Brunsveld-Reinders AH, Dijkgraaf MG, Barrett 2010 {published data only}
Smorenburg SM, de Rooij SE, Adams R, et al, for the Cost Barrett J, Reeves J, Bucknall T, Levinson M. Medical emergency
and Outcomes of Medical Emergency Teams Study Group. team use and outcomes in an Australian private hospital.
Outcomes associated with the nationwide introduction of rapid Critical Care 2010;14:S89. [DOI: 10.1186/cc8492]
response systems in the Netherlands. Critical Care Medicine
2015;43(12):2544-51. [DOI: 10.1097/CCM.0000000000001272] Barwise 2014 {published data only}
Barwise A, Thongprayoon C, Herasevich V, Pickering B, Gajic O,
Jensen J. Outcomes of delayed RRT activation in patients

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 24
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

transferring to the ICU. Critical Care Medicine 2014;42(12 Suppl Cabrini 2012 {published data only}
1):A1417. [DOI: 10.1097/01.ccm.0000457730.99332.4f] Cabrini L, Monti G, Plumari VP, Landoni G, Turi S, Laura P, et al.
Observed versus predicted hospital mortality in general wards
Barwise 2016 {published data only}
patients assisted by a medical emergency team. Signa Vitae
Barwise A, Thongprayoon C, Gajic O, Jensen J, Herasevich V, 2012;7(1):38-42. [DOI: 10.22514/SV71.042012.7]
Pickering BW. Delayed rapid response team activation is
associated with increased hospital mortality, morbidity, and Calzavacca 2010 {published data only}
length of stay in a tertiary care institution. Critical Care Medicine Calzavacca P, Licari E, Tee A, Mercer I, Haase M, Haase-
2016;44(1):54-63. [DOI: 10.1097/CCM.0000000000001346] Fielitz A, et al. Features and outcome of patients receiving
multiple medical emergency team reviews. Resuscitation
Beitler 2010 {published data only}
2010;81(11):1509-15. [DOI: 10.1016/j.resuscitation.2010.06.017]
Beitler JR, Chong DH, Bails DB, Hurdle K, Link N. Reduction in
hospital-wide mortality following implementation of a rapid Cerchiari 2010 {published data only}
response team program. American Journal of Respiratory and Cerchiari EL, Semeraro F, Gambale G, Terzitta M, Vezzani A,
Critical Care Medicine 2010;181:A5102. Barbagallo M, et al. MIER project "Hospital mortality and
containment strategy" – in-hospital rapid response system
Beitler 2011 {published data only}
(RRS) implementation multicenter prospective study in Emilia-
Beitler JR, Link N, Bails DB, Hurdle K, Chong DH. Reduction Romagna (ER). Resuscitation 2010;81(2 Suppl 1):S43. [DOI:
in hospital-wide mortality after implementation of a rapid 10.1016/j.resuscitation.2010.09.183]
response team: a long-term cohort study. Critical Care
2011;15(6):R269. [DOI: 10.1186/cc10547] Chan 2008 {published data only}
Chan PS, Khalid A, Longmore LS, Berg RA, Kosiborod M,
Bergamasco 2017 {published data only}
Spertus JA. Hospital-wide code rates and mortality before
Bergamasco E, Paula R, Tanita MT, Festti J, Queiroz Cardoso LT, and after implementation of a rapid response team. Journal
Carvalho Grion CM. Analysis of readmission rates to American Medical Association 2008;300(21):2506-13. [DOI:
the intensive care unit after implementation of a rapid 10.1001/jama.2008.715]
response team in a University Hospital. Medicina Intensiva
2017;41(7):411-7. Chen 2009 {published data only}
Chen J, Bellomo R, Flabouris A, Hillman K, Finfer S, MERIT
Bingham 2015 {published data only}
study investigators for the Simpson Centre, ANZICS Clinical
Bingham G, Fossum M, Barratt M, Bucknall T. Clinical review Trials Group. The relationship between early emergency
criteria and medical emergency teams: evaluating a two- team calls and serious adverse events. Critical Care Medicine
tier rapid response system. Critical Care Resuscitation 2009;37(1):148-53.
2015;17(3):167-73.
Chen 2014a {published data only}
Bittman 2020 {published data only}
Chen J, Ou L, Hillman K, Flabouris A, Bellomo R, Hollis SJ,
Bittman J, Nijjar AP, Tam P, Khan N. Early warning scores et al. The impact of implementing a rapid response system:
to predict noncritical events overnight in hospitalized a comparison of cardiopulmonary arrests and mortality
medical patients: a prospective case cohort study. Journal among four teaching hospitals in Australia. Resuscitation
of Patient Safety 2020;16(3):e169-73. [DOI: 10.1097/ 2014;85(9):1275-81. [DOI: 10.1016/j.resuscitation.2014.06.003]
PTS.0000000000000292]
Chen 2014b {published data only}
Blotsky 2012 {published data only}
Chen J, Ou L, Hillman KM, Flabouris A, Bellomo R, Hollis SJ,
Blotsky A, Mardini L, Jayaraman D. Benefits of a local et al. Cardiopulmonary arrest and mortality trends, and
medical emergency team in a Canadian tertiary care their association with rapid response system expansion.
centre. Critical Care Medicine 2012;40(12 Suppl 1):169. [DOI: Medical Journal of Australia 2014;201(3):167-70. [DOI: 10.5694/
10.1097/01.ccm.0000425605.04623.4b] mja14.00019]
Bucknall 2013 {published data only} Chen 2015 {published data only}
Bucknall TK, Jones D, Bellomo R, Staples M, Rescue Chen J, Bellomo R, Flabouris A, Hillman K, Assareh H, Ou L.
Investigators. Responding to medical emergencies: Delayed emergency team calls and associated hospital
system characteristics under examination (RESCUE). A mortality: a multicenter study. Critical Care Medicine
prospective multi-site point prevalence study. Resuscitation 2015;43(10):2059-65. [DOI: 10.1097/CCM.0000000000001192]
2013;84(2):179-83. [DOI: 10.1016/j.resuscitation.2012.06.015]
Choi 2011 {published data only}
Butcher 2010 {published data only}
Choi SH, Son JS, Seo HS, Shin YJ, Lee JM, Han MJ, et al.
Butcher B, Maselli J, Ranji S, Auerbach A. Impact of a rapid Implementation of the medical emergency team in patients
response team on intensive care unit readmission rate. Journal with severe sepsis at the general ward for 3 years. Intensive Care
of Hospital Medicine 2010;5:15-6. [DOI: 10.1002/jhm.705] Medicine 2011;37:S231.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 25
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Churpek 2017 {published data only} Gao 2007 {published data only}
Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, Gao H, Harrison DA, Parry GJ, Daly K, Subbe CP, Rowan K. The
et al. Quick sepsis-related organ failure assessment, systemic impact of the introduction of critical care outreach services in
inflammatory response syndrome, and early warning scores England: a multicentre interrupted time-series analysis. Critical
for detecting clinical deterioration in infected patients Care 2007;11(5):R113.
outside the intensive care unit. American Journal Respiratory
Critical Care Medicine 2017;195(7):906-11. [DOI: 10.1164/ Gilmore 2014 {published data only}
rccm.201604-0854OC] Gilmore AJ, Kelly M, Elmore J, Mahambrey T. The first year
of a new medical emergency team (MET): implementation
Considine 2019 {published data only} and service evaluation. Journal of the Intensive Care Society
Considine J, Street M, Bucknall T, Rawson H, Hutchison AF, 2014;15(1 Suppl 1):S39. [DOI: 10.1177/17511437140151S106]
Dunning T, et al. Characteristics and outcomes of emergency
interhospital transfers from subacute to acute care for clinical Goncales 2012 {published data only}
deterioration. International Journal for Quality in Health Care Goncales PD, Polessi JA, Bass LM, de Santos P, Yokota PK,
2019;31(2):117-24. Laselva CR, et al. Reduced frequency of cardiopulmonary
arrests by rapid response teams. Einstein 2012;10(4):442-8.
Danesh 2012 {published data only}
Danesh V, Guerrier L, Health O, Jimenez E. Proactive vs. Gray 2011 {published data only}
reactive rapid response systems: decreasing unplanned ICU Gray R, Morgan L, Smith H, Benson M, Morgan P. The outcomes
transfers. Critical Care Medicine 2012;40(12 Suppl 1):1. [DOI: following the introduction of a medical emergency team.
10.1097/01.ccm.0000425605.04623.4b] Intensive Care Medicine 2011;37:S226.

Danesh 2019 {published data only} Harm 2012 {published data only}
Danesh V, Neff D, Jones TL, Aroian K, Unruh L, Andrews D, Harm F, Ummenhofer W, Luethy M, Zuercher M. In-hospital
et al. Can proactive rapid response team rounding improve resuscitation calls – prospective follow-up of cause, locations
surveillance and reduce unplanned escalations in care? A and outcome. Resuscitation 2012;83:e14. [DOI: 10.1016/
controlled before and after study. International Journal of j.resuscitation.2012.08.037]
Nursing Studies 2019;91:128-33.
Harrison 2010 {published data only}
Fernando 2018 {published data only} Harrison DA, Gao H, Welch CA, Rowan KM. The effects of critical
Fernando SM, Reardon PM, McIsaac DI, Eagles D, Murphy K, care outreach services before and after critical care: a matched-
Tanuseputro P, et al. Outcomes of older hospitalized cohort analysis. Journal of Critical Care 2010;25(2):196-204.
patients requiring rapid response team activation for acute [DOI: 10.1016/j.jcrc.2009.12.015]
deterioration. Critical Care Medicine 2018;46(12):1953-60.
Hassan 2015 {published data only}
Fernando 2019 {published data only} Hassan S, Patel A, Kirk H, Ullah A, Hamid T. Implementation of a
Fernando SM, Reardon PM, Scales DC, Murphy K, Tanuseputro P, medical emergency team leads to a reduction in cardiac arrest
Heyland DK, et al. Prevalence, risk factors, and clinical calls and enhanced patient care in the critically ill. European
consequences of recurrent activation of a rapid response team: Heart Journal 2015;36:1195-6. [DOI: 10.1093/eurheartj/ehv402]
a multicenter observational study. Journal of Intensive Care
Medicine 2019;34(10):782-9. [DOI: 10.1177/0885066618773735] Hatlem 2011 {published data only}
Hatlem T, Jones C, Woodard EK. Reducing mortality
Frost 2015 {published data only} and avoiding preventable ICU utilization: analysis of a
Frost SA, Chapman A, Aneman A, Chen J, Parr MJ, Hillman K. successful rapid response program using APR DRGs. Journal
Hospital outcomes associated with introduction of a two-tiered for Healthcare Quality 2011;33(5):7-16. [DOI: 10.1111/
response to the deteriorating patient. Critical Care Resuscitation j.1945-1474.2011.00084.x]
2015;17(2):77-82.
Hellervik 2012 {published data only}
Gagne 2018 {published data only} Hellervik SM, Landsperger JS, Chassan CB, Rice TW,
Gagne C, Fetzer S. Early warning score communication Wheeler AP. Comparison of characteristics, interventions
bundle: a pilot study. American Journal of Critical Care and outcomes of patients with single versus multiple rapid
2018;27(3):238-42. response team (RRT) calls during a hospital admission.
Critical Care Medicine 2012;40(12 Suppl 1):210. [DOI:
Galhotra 2010 {published data only} 10.1097/01.ccm.0000425605.04623.4b]
Galhotra S, Devita MA, Dew MA, Simmons RL. A 5-year
analysis of rapid response system activation at an in- Jaderling 2013 {published data only}
hospital haemodialysis unit. Quality & Safety in Health Care Jaderling G, Bell M, Martling CR, Ekbom A, Bottai M, Konrad D.
2010;19(6):e38. [DOI: 10.1136/qshc.2008.031666] ICU admittance by a rapid response team versus conventional
admittance, characteristics, and outcome. Critical Care Medicine
2013;41(3):725-31. [DOI: 10.1097/CCM.0b013e3182711b94]

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 26
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Javeri 2013 {published data only} Kodama 2014 {published data only}
Javeri Y, Nasa P, Singh O, Juneja D, Rashid MF, Garg S. Kodama T Fujitani S. The effect of the medical emergency
To evaluate the impact of rapid response team (RRT) team in preventing the deterioration of inpatients.
implementation on patient outcome during medical Critical Care Medicine 2014;42(12 Suppl 1):A1579. [DOI:
emergencies. Indian Journal of Critical Care Medicine 10.1097/01.ccm.0000458407.26564.bc]
2013;17:15.
Kollef 2014 {published data only}
John 2010 {published data only} Kollef MH, Chen Y, Heard K, LaRossa GN, Lu C, Martin NR, et al. A
John S, Corwin H, Robison C, Houston D, Surgenor S. randomized trial of real-time automated clinical deterioration
Impact of hospitalists and a rapid response team on ICU alerts sent to a rapid response team. Journal of Hospital
readmissions. Critical Care Medicine 2010;38:A154. [DOI: Medicine 2014;9(7):424-9. [DOI: 10.1002/jhm.2193]
10.1097/01.ccm.0000390903.16849.8c]
Konrad 2010 {published data only}
Jokela 2015 {published data only} Konrad D, Jaderling G, Bell M, Granath F, Ekbom A, Martling CR.
Jokela K, Setala P, Virta J, Huhtala H, Yli-Hankala A, Hoppu S. Reducing in-hospital cardiac arrests and hospital mortality by
Using a simplified pre-hospital 'MET' score to predict in-hospital introducing a medical emergency team. Intensive Care Medicine
care and outcomes. Acta Anaesthesiologica Scandinavica 2010;36(1):100-6. [DOI: 10.1007/s00134-009-1634-x]
2015;59(4):505-13. [DOI: 10.1111/aas.12499]
Laurens 2011 {published data only}
Jones 2005 {published data only} Laurens N, Dwyer T. The impact of medical emergency teams
Jones D, Bellomo R, Bates S, Warrillow S, Goldsmith D, Hart G, on ICU admission rates, cardiopulmonary arrests and mortality
et al. Long term effect of a medical emergency team on cardiac in a regional hospital. Resuscitation 2011;82(6):707-12. [DOI:
arrests in a teaching hospital. Critical Care 2005;9(6):R808-15. 10.1016/j.resuscitation.2010.11.031]

Jones 2008 {published data only} Lawless 2013 {published data only}
Jones D, George C, Hart GK, Bellomo R, Martin J. Introduction Lawless B, Hill A. Impact of a medical emergency teams: a
of medical emergency teams in Australia and New Zealand: a component of a strategy to improve quality of critical care.
multi-centre study. Critical Care 2008;12(2):R46. [DOI: 10.1186/ American Journal of Respiratory and Critical Care Medicine
cc6857] 2013;187:A5315.

Karpman 2013 {published data only} Lee Ekblad 2012 {published data only}
Karpman C, Keegan MT, Jensen JB, Bauer PR, Brown DR, Lee-Ekblad D, Mastenbrook S, Wilt J, Birchmeier P, Ekblad G,
Afessa B. The impact of rapid response team on outcome of Beckwith W. Evaluating clinical outcomes after implementation
patients transferred from the ward to the ICU: a single-center of an advanced practice provider led rapid response team.
study. Critical Care Medicine 2013;41(10):2284-91. [DOI: 10.1097/ Critical Care Medicine 2012;40(12 Suppl 1):198. [DOI:
CCM.0b013e318291cccd] 10.1097/01.ccm.0000425605.04623.4b]

Karvellas 2012 {published data only} Le Guen 2015 {published data only}
Karvellas CJ, de Souza IA, Gibney RT, Bagshaw SM. Association Le Guen MP, Tobin AE, Reid D. Intensive care unit admission in
between implementation of an intensivist-led medical patients following rapid response team activation: call factors,
emergency team and mortality. BMJ Quality & Safety patient characteristics and hospital outcomes. Anaesthesia and
2012;21(2):152-9. [DOI: 10.1136/bmjqs-2011-000393] Intensive Care 2015;43(2):211-5.

Kawaguchi 2015 {published data only} Liberti 2012 {published data only}
Kawaguchi R, Nakada T, Oshima T, Abe R, Matsumura Y, Liberti D, Di Maria C, De Luca P, Alberico M, Popa C, Sagliocco O,
Oda S. Reduction of unexpected serious adverse events after et al. In-hospital rapid response system: effects on outcome and
introducing medical emergency team. Acute Medicine & Surgery workload. Critical Care 2012;16:S180. [DOI: 10.1186/cc11113]
2015;2(4):244-9. [DOI: 10.1002/ams2.101]
Lim 2011 {published data only}
Kim 2013 {published data only} Lim SY, Park SY, Park HK, Kim M, Park HY, Lee B, et al. Early
Kim HC, Yoo JW, Lim SY, Suh GY, Koh SO, Na S, et al. Mortality impact of medical emergency team implementation in a
after in-hospital cardiopulmonary resuscitation: multicenter country with limited medical resources: a before-and-after
analysis in Korea. Journal of Critical Care 2013;28(6):942-6. [DOI: study. Journal of Critical Care 2011;26(4):373-8. [DOI: 10.1016/
10.1016/j.jcrc.2013.07.048] j.jcrc.2010.08.019]

Kodama 2013 {published data only} Lorencio Cardenas 2014 {published data only}
Kodama T, Fujitani S. Trial of reducing unexpected hospital Lorencio Cardenas C, Gonzalez Londono J, Tache Sala A, Pujol
mortality by introducing rapid response system in Japan. Valverde P, Garcia Fragua A, Baro Serra A, et al. Implementation
Critical Care Medicine 2013;41(12 Suppl 1):A143. [DOI: of a rapid response team (RRT) in a tertiary hospital. Outcomes
10.1097/01.ccm.0000439825.74448.88] after 4 years of experience. Intensive Care Medicine 2014;40(1
Suppl 1):S161-2. [DOI: 10.1007/s00134-013-3451-5]

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 27
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Ludikhuize 2013 {published data only} Palomba 2014 {published data only}
Ludikhuize J, Dijkgraaf MG, Smorenburg SM, de Rooij SE, Palomba H, Piza F, Jaures M, Capone A. Hospital mortality
Brunsveld-Reinders AH, Tangkau P, et al, the COMET study predictive factors following rapid response team activation.
group. Cost and Outcome of Medical Emergency Teams Critical Care 2014;18:S29. [DOI: 10.1186/cc13272]
(COMET) study. Design and rationale of a Dutch multi-
center study. British Journal of Medicine & Medical Research Panico 2011 {published data only}
2013;3(1):3-28. Panico M, Pisani M, Jenq G, Araujo K, Honiden S. Outcomes
following a rapid response and the impact on transfer to the
Ludington 2011 {published data only} intensive care unit. American Journal of Respiratory and Critical
Ludington MC, Graham P, Lawrence B, Sell R, Husa RD, Care Medicine 2011;183:A4725.
Minokadeh A, et al. A "front-loaded" rapid response team
decreases non-ICU arrests and overall hospital mortality. Petersen 2016 {published data only}
Circulation 2011;124(21 (Suppl 1)):A269. Petersen JA, Antonsen K, Rasmussen LS. Frequency of
early warning score assessment and clinical deterioration
Magnuson 2015 {published data only} in hospitalized patients: a randomized trial. Resuscitation
Magnuson R, Lent D, Pietropaoli A, Ott M, Valcin KE, 2016;101:91-6. [DOI: 10.1016/j.resuscitation.2016.02.003]
Rasul S, et al. Effect of a "re-invigorated" RRT on in-
hospital cardiopulmonary arrest rates and mortality. Pirret 2015 {published data only}
Critical Care Medicine 2015;43(12 Suppl 1):208. [DOI: Pirret AM, Takerei SF, Kazula LM. The effectiveness of a patient
10.1097/01.ccm.0000474654.31694.73] at risk team comprised of predominantly ward experienced
nurses: a before and after study. Intensive Critical Care Nursing
Martin 2009 {published data only} 2015;31(3):133-40. [DOI: 10.1016/j.iccn.2014.10.005]
Martin P, Lukas D, Vaclav Z, Tomas B, Pavel S, Vladimir S.
Early identification of critically ill patients with the Medical Pisa 2012 {published data only}
Emergency System in St. Ann University Hospital, Brno. Pisa M, Collins T, Robertson M, Sicoutris C, Saucier J,
Anesteziologie a Intenzivni Medicina 2009;20(4):185-91. Bushan N, et al. Use of a critical care nurse practitioner-
driven critical care outreach initiative to reduce readmission
Martin 2017 {published data only} mortality. Critical Care Medicine 2012;40(12 Suppl 1):204. [DOI:
Martin C, Jones D, Wolfe R. State-wide reduction in in-hospital 10.1097/01.ccm.0000425605.04623.4b]
cardiac complications in association with the introduction
of a national standard for recognising deteriorating patients. Prchalova 2013 {published data only}
Resuscitation 2017;12:172-8. Prchalova M, Donnelly M, Hayes N, Barnes AM, Kidney E.
Evaluation and impact of implementation of an early warning
McDonnell 2013 {published data only} scoring system (EWS)/medical emergency team (MET).
McDonnell A, Tod A, Bray K, Bainbridge D, Adsetts D, Walters S. Intensive Care Medicine 2013;39:S226-7. [DOI: 10.1007/
A before and after study assessing the impact of a new model s00134-013-3095-5]
for recognizing and responding to early signs of deterioration in
an acute hospital. Journal Advanced Nursing 2013;69(1):41-52. Radeschi 2014 {published data only}
[DOI: 10.1111/j.1365-2648.2012.05986.x] Radeschi G, Mina A, Berta G, Di Pietrantonj C, Rubulotta F.
Effectiveness of a rapid response system following the
Mitchell 2010 {published data only} introduction of an observation chart. Intensive Care Medicine
Mitchell IA, McKay H, van Leuvan C, Berry R, McCutcheon C, 2014;40(1 Suppl 1):S40-1. [DOI: 10.1007/s00134-013-3451-5]
Avard B, et al. A prospective controlled trial of the effect
of a multi-faceted intervention on early recognition and Rashid 2014 {published data only}
intervention in deteriorating hospital patients. Resuscitation Rashid MF, Imran M, Javeri Y, Rajani M, Samad S, Singh O.
2010;81(6):658-66. [DOI: 10.1016/j.resuscitation.2010.03.001] Evaluation of rapid response team implementation in medical
emergencies: a gallant evidence based medicine initiative in
Mullany 2016 {published data only} developing countries for serious adverse events. International
Mullany DV, Ziegenfuss M, Goleby MA, Ward HE. Improved Journal Critical Illness & Injury Science 2014;4(1):3-9. [DOI:
hospital mortality with a low MET dose: the importance of 10.4103/2229-5151.128005]
a modified early warning score and communication tool.
Anaesthesia and Intensive Care 2016;44(6):734-41. Reilly 2010 {published data only}
Reilly S, Pamplin J, Pineda L, Hebert R, Marquez D, Chung K.
O'Connell 2016 {published data only} Decreased rates of code team activation following the
O'Connell A, Flabouris A, Kim SW, Horwood C, Hakendorf P, introduction of rapid response teams to an academic military
Thompson CH. A newly designed observation and response medical center. Critical Care Medicine 2010;38:A26. [DOI:
chart's effect upon adverse inpatient outcomes and 10.1097/01.ccm.0000390903.16849.8c]
rapid response team activity. Internal Medicine Journal
2016;46(8):909-16. Sabahi 2012 {published data only}
Sabahi M, Fanaei SA, Ziaee SA, Falsafi FS. Efficacy of a rapid
response team on reducing the incidence and mortality of

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 28
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

unexpected cardiac arrests. Trauma Monthly 2012;17(2):270-4. Sutherasan 2018 {published data only}
[DOI: 10.5812/traumamon.4170] Sutherasan Y, Theerawit P, Suporn A, Nongnuch A, Phanachet P,
Kositchaiwat C. The impact of introducing the early warning
Salvatierra 2014 {published data only}
scoring system and protocol on clinical outcomes in tertiary
Salvatierra G, Bindler RC, Corbett C, Roll J, Daratha KB. Rapid referral university hospital. Therapeutics and Clinical Risk
response team implementation and in-hospital mortality. management 2018;14:2089-95.
Critical Care Medicine 2014;42(9):2001-6. [DOI: 10.1097/
CCM.0000000000000347] Taguti Pda 2013 {published data only}
Taguti Pda S, Dotti AZ, de Araujo KP, de Pariz PS, Dias GF,
Santamaria 2011 {published data only}
Kauss IA, et al. The performance of a rapid response team in
Santamaria JD, Tobin AE, Holmes J. Medical emergency the management of code yellow events at a university hospital.
teams of greater than 2 years service are associated with Revista Brasileira de Terapia Intensiva 2013;25(2):99-105. [DOI:
reduced mortality in a major metropolitan health network. 10.5935/0103-507X.20130020]
American Journal of Respiratory and Critical Care Medicine
2011;183:A4109. Tridente 2015 {published data only}
Tridente A, Elmore J, Varia R, Mahambrey T. Successful
Sebat 2018 {published data only}
implementation of a medical emergency team: 2-year
Sebat F, Vandegrift MA, Childers S, Lighthall GK. A novel experience in a teaching hospital. Critical Care 2015;19:S144-5.
bedside-focused ward surveillance and response system. [DOI: 10.1186/cc14490]
Joint Commission Journal on Quality and Patient Safety
2018;44(2):94-100. Trottier 2010 {published data only}
Trottier S, Patel D. Decreased codes outside of the intensive care
Simmes 2012 {published data only}
unit: effect of the rapid response team. Critical Care Medicine
Simmes FM, Schoonhoven L, Mintjes J, Fikkers BG, van 2010;38:A184. [DOI: 10.1097/01.ccm.0000390903.16849.8c]
der Hoeven JG. Incidence of cardiac arrests and unexpected
deaths in surgical patients before and after implementation of White 2016 {published data only}
a rapid response system. Annuals Intensive Care 2012;2(1):20. White K, Scott IA, Bernard A, McCulloch K, Vaux A, Joyce C,
[DOI: 10.1186/2110-5820-2-20] et al. Patient characteristics, interventions and outcomes
of 1151 rapid response team activations in a tertiary
Simmes 2015 {published data only}
hospital: a prospective study. Internal Medicine Journal
Simmes F. The assessment of critically ill patients: evaluation 2016;46(12):1398-406.
of the rapid response system in a university medical centre.
Netherlands Journal of Critical Care 2015;23(5):13-7. Yu 2016 {published data only}
Yu Y, Si J, Liu G, Qi S, Xian H. A validation study of national early
Smith 2013 {published data only}
warning score in evaluation of death risk in elderly patients
Smith M, Donuur K, Koochana S, Acquah SO. Clinical outcomes with critical illness. Zhonghua Wei Zhong Bing Ji Jiu Yi Xue
of patients admitted to the medical intensive care unit after 2016;28(5):387-90.
a rapid response team intervention. American Journal of
Respiratory and Critical Care Medicine 2013;187:A5314.
Additional references
So 2018 {published data only}
ACSQHC 2017
So HM, Yan WW, Chair SY. A nurse-led critical care outreach
program to reduce readmission to the intensive care unit: Australian Commission on Safety and Quality in Health
a quasi-experimental study with a historical control group. Care. National Consensus Statement: Essential Elements
Australian Critical Care 2019;32(6):494-501. [DOI: org/10.1016/ for Recognising and Responding to Acute Physiological
j.aucc.2018.11.005] Deterioration. 2nd edition. Sydney, Australia: Australian
Commission on Safety and Quality in Health Care, 2017. [IBSN:
Stelfox 2012 {published data only} 978-0-9806298-8-0]
Stelfox HT, Hemmelgarn BR, Bagshaw SM, Gao S, Doig CJ,
ACT 2016
Nijssen-Jordan C, et al. Intensive care unit bed availability
and outcomes for hospitalized patients with sudden clinical ACT Government Health Australia. COMPASS.
deterioration. Archives Internal Medicine 2012;172(6):467-74. www.healthact.gov.au/professionals/compass (accessed 11
[DOI: 10.1001/archinternmed.2011.2315] March 2018).

Stewart 2014 {published data only} Andersen 2016

Stewart J, Carman M, Spegman A, Sabol VK. Evaluation Andersen LW, Kim WY, Chase M, Berg K, Mortensen SJ,
of the effect of the modified early warning system on the Moskowitz A, et al, for the American Heart Association’s
nurse-led activation of the rapid response system. Journal Get With The Guidelines – Resuscitation Investigators. The
Nursing Care Quality 2014;29(3):223-9. [DOI: 10.1097/ prevalence and significance of abnormal vital signs prior to
NCQ.0000000000000048] in-hospital cardiac arrest. Resuscitation 2016;98:112-7. [DOI:
10.1016/j.resuscitation.2015.08.016]

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 29
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Berwick 2006 Donaldson 2014

Berwick DM, Calkins DR, McCannon CJ, Hackbarth AD. The 100 Donaldson LJ, Panesar SS, Darzi A. Patient-safety-related
000 Lives Campaign: setting a goal and a deadline for improving hospital deaths in England: thematic analysis of incidents
health care quality. Journal American Medical Association reported to a National database, 2010–2012. PLoS Medicine
2006;295(3):324-7. [DOI: 10.1001/jama.295.3.324] 2014;11(6):e1001667. [DOI: 10.1371/journal.pmed.1001667]

Chan 2010 Eliott 2008

Chan PS, Jain R, Nallmothu BK, Berg RA, Sasson C. Rapid Eliott SJ, Ernest D, Doric AG, Page KN, Worrall-Carter LJ,
response teams: a systematic review and meta-analysis. Thalib L, et al. The impact of an ICU liaison nurse
Archives of Internal Medicine 2010;170(1):18-26. [DOI: 10.1001/ service on patient outcomes. Critical Care Resuscitation
archinternmed.2009.424] 2008;10(4):296-300. [PMID: 19049479]

Churpek 2016 EPOC 2017a

Churpek MM, Edelson DP. Moving beyond single parameter Effective Practice, Organisation of Care (EPOC). What study
early warning scores for rapid response system activation. designs should be included in an EPOC review and what
Critical Care Medicine 2016;44(12):2283-5. [DOI: 10.1097/ should they be called? EPOC resources for review authors.
CCM.0000000000002105] www.epoc.cochrane.org/resources/epoc-resources-review-
authors (accessed 3 May 2018).
COMET Initiative
COMET Initiative. Core Outcome Measures in Effectiveness EPOC 2017b
Trials. www.comet-initiative.org/ (accessed 8 June 2018). Cochrane Effective Practice and Organisation of Care (EPOC).
Screening, data extraction and management. EPOC Resources
Covidence [Computer program] for review authors. epoc.cochrane.org/resources/epoc-
Covidence. Melbourne, Australia: Veritas Health Innovation. resources-review-authors (accessed 18 May 2018).
Available at covidence.org.
EPOC 2017c
Deeks 2011 Cochrane Effective Practice and Organisation of Care (EPOC).
Deeks JJ, Higgins JP, Altman DG. Chapter 9: Analysing data Data collection form. EPOC Resources for review authors.
and undertaking meta-analyses. In: Higgins JP, Green S, epoc.cochrane.org/resources/epoc-resources-review-authors
editor(s). Cochrane Handbook for Systematic Reviews of (accessed 15 June 2018).
Interventions Version 5.1.0 (updated March 2011). The Cochrane
Collaboration, 2011. Available from training.cochrane.org/ EPOC 2017d
handbook/archive/v5.1/. Effective Practice, Organisation of Care (EPOC). EPOC
Worksheets for preparing a Summary of Findings (SoF)
de Jong 2016 table using GRADE. EPOC Resources for review authors.
de Jong A, Jung B, Daurat A, Chanques G, Mahul M, Monnin M, epoc.cochrane.org/resources/epoc-resources-review-authors
et al. Effect of rapid response systems on hospital mortality: a (accessed 21 July 2018).
systematic review and meta-analysis. Intensive Care Medicine
2016;42(4):1-3. [DOI: 10.1007/s00134-016-4263-1] EPOC 2017e
Effective Practice and Organisation of Care (EPOC). Analysis
DeVita 2006 in EPOC reviews. EPOC Resources for review authors.
DeVita MA, Bellomo R, Hillman K, Kellum J, Rotondi A, Teres D, epoc.cochrane.org/resources/epoc-resources-review-authors
et al. Findings of the first consensus conference on medical (accessed 7 June 2018).
emergency teams. Critical Care Medicine 2006;34(9):2463-78.
[PMID: 16878033] Fujiwara 2016
Fujiwara S, Koike T, Moriyasu M, Nakagawa M, Atagi K, Lefor AK,
DoH 2000 et al, and IHCA study group. A retrospective study of in-hospital
Department of Health. Comprehensive critical cardiac arrest. Acute Medicine & Surgery 2016;3(4):320-25. [DOI:
care. A review of adult critical care services. 10.1002/ams2.193]
webarchive.nationalarchives.gov.uk/20070403042444/http://
www.dh.gov.uk/en/Publicationsandstatistics/Publications/ Garry 2014
PublicationsPolicyAndGuidance/DH_ 4006585 (accessed 15 Garry DA, McKechnie SR, Culliford DJ, Ezra M, Garry PS,
March 2018). Loveland RC, et al, on behalf of the PREVENT group. A
prospective multicentre observational study of adverse
DoH 2013 iatrogenic events and substandard care preceding intensive
Department of Health Ireland. National Early Warning Score. care unit admission (PREVENT). Anaesthesia 2014;69(2):137-42.
National Clinical Guideline No. 1. Dublin, Ireland: Department of [DOI: 10.1111/anae.12535]
Health, 2013. [ISSN 2009-6259 ]
Guyatt 2008
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-
Coello P, et al, for the GRADE Working Group. GRADE: an

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 30
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

emerging consensus on rating quality of evidence and strength McGaughey 2017a

of recommendations. BMJ 2008;336:924. [DOI: 10.1136/ McGaughey J, O'Halloran P, Porter S, Blackwood B. Early
bmj.39489.470347.AD] warning systems and rapid response to the deteriorating
patient in hospital: a systematic realist review. Journal
Higgins 2011
of Advanced Nursing 2017;73(12):2877-91. [DOI: 10.1111/
Higgins JP, Altman DG. Chapter 8: Assessing risk of bias in jan.13398]
included studies. In: Higgins JP, Green S, editor(s). Cochrane
Handbook for Systematic Reviews of Interventions Version McGaughey 2017b
5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. McGaughey J, O'Halloran P, Porter S, Trinder J, Blackwood B.
Available from training.cochrane.org/handbook/archive/v5.1/. Early warning systems and rapid response to the deteriorating
patient in hospital: a realist evaluation. Journal of Advanced
Higgins 2020
Nursing 2017;73(12):3119-32. [DOI: 10.1111/jan.13367]
Schünemann HJ, Vist GE, Higgins JP, Santesso N, Deeks JJ,
Glasziou P, et al. Chapter 15: Interpreting results and McNeill 2013
drawing conclusions. In: Higgins JP, Thomas J, Chandler McNeill G, Bryden D. Do either early warning systems or
J, Cumpston M, Li T, Page MJ, et al., editor(s). Cochrane emergency response teams improve hospital patient survival?
Handbook for Systematic Reviews of Interventions Version A systematic review. Resuscitation 2013;84(12):1652-67. [DOI:
6.1 (updated September 2020). Cochrane, 2020. Available from 10.1016/j.resuscitation.2013.08.006]
training.cochrane.org/handbook/archive/v6.1.
McQuillan 1998
Hoffman 2014
McQuillan P, Pilkington S, Allan A, Taylor B, Short A, Morgan G, et
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, al. Confidential inquiry into the quality of care before admission
Moher D, et al. Better reporting of interventions: Template for to intensive care. BMJ 1998;316(7148):1853-8. [PMID: 9632403]
Intervention Description and Replication (TIDieR) checklist and
guide. BMJ 2014;348:g1687. [DOI: 10.1136/bmj.g1687] Moher 2009
Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA
HSE 2011
Group. Preferred Reporting Items for Systematic Reviews
Health Service Executive. Training Manual for The National and Meta-Analyses: the PRISMA Statement. PLoS Medicine
Early Warning Score and associated Education Programme. 3rd 2009;6(7):e1000097. [10.1371/journal.pmed.1000097]
edition. Dublin, Ireland: HSE, 2011. [AVAILABLE AT: www.rcsi.ie/
files/facultyofnursingmidwifery/20160811103824_5.2%20NEWS Moore 2015
%20Training.pdf] Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W,
et al. Process evaluation of complex interventions: Medical
ICS 2002
Research Council guidance. BMJ 2015;350:h1258. [DOI: 10.1136/
Intensive Care Society. Guidelines for the Introduction of bmj.h1258]
Outreach Services. Standards and Guidelines. London,
UK: Intensive Care Society, 2002. [AVAILABLE AT: NCEPOD 2005
anaesthesiaconference.kiev.ua/downloads/ICU_standards- National Confidential Enquiry into Patient Outcome and
outreach_2002.pdf] Death (NCEPOD). An Acute Problem? A Report of the National
Confidential Enquiry into Patient Outcome and Death. London,
Kause 2004
UK: NCEPOD, 2005.
Kause J, Smith G, Prytherch D, Parr M, Flabouris A, Hillman K,
Intensive Care Society (UK), Australian and New Zealand Newcombe 2000
Intensive Care Society Clinical Trials Group. A comparison Newcombe RG, Altman DG. Proportions and their differences.
of antecedents to cardiac arrests, deaths and emergency In: Altman DG, Machin D, Bryant TN, Gardner MJ, editors(s).
intensive care admissions in Australia and New Zealand, and Statistics with Confidence. 2nd edition. London, UK: BMJ Books,
the United Kingdom: The ACADEMIA study. Resuscitation 2000. [ISBN: 978-0-727-91375-3]
2004;62(3):275-82. [PMID: 15325446]
Newey 1987
Lee 1995
Newey WK, West KD. A simple, positive semi-definite,
Lee A, Bishop G, Hillman KM, Daffurn K. The medical emergency heteroskedasticity and autocorrelation consistent covariance
team. Anaesthesia and Intensive Care 1995;23(2):183-6. [PMID: matrix. Econometrica 1987;55(3):703-8.
7793590]
NICE 2007
Maharaj 2015
National Institute for Health and Clinical Excellence. Acutely
Maharaj R, Raffaele I, Wendon J. Rapid response systems: a Ill Patients in Hospital: Recognition of and Response to Acute
systematic review and meta-analysis. Critical Care 2015;19:254. Illness in Adults in Hospital. Draft Consultation Document.
[DOI: 10.1186/s13054-015-0973-y] London, UK: Department of Health, 2007.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 31
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

NORF 2003 of a range of adverse clinical outcomes. Critical Care Medicine

National Outreach Forum (NORF). Critical Care Outreach 2003: 2016;44(12):2171-81. [DOI: 10.1097/CCM.0000000000002000]
Progress in Developing Services. London, UK: Department
Solomon 2016
of Health/NHS Modernisation Agency, 2003. [AVAILABLE AT:
www.modern.nhs.uk/criticalcare/5021/7117/78001-DoH- Solomon RS, Corwin GS, Barclay DC, Quddusi SF,
CareOutreach.pdf] Dannenberg MD. Effectiveness of rapid response teams on
rates of in-hospital cardiopulmonary arrest and mortality:
Quirke 2011 a systematic review and meta-analysis. Journal of Hospital
Quirke S, Coombs M, McEldowney R. Suboptimal care of the Medicine 2016;11(6):438-45. [DOI: 10.1002/jhm.2554]
acutely unwell ward patient: A concept analysis. Journal
Sterne 2011
of Advanced Nursing 2011;67(8):1834-45. [DOI: 10.1111/
j.1365-2648.2011.05664.x] Sterne JA, Egger M, Moher D. Chapter 10: Addressing reporting
biases. In: Higgins JP, Green S, editor(s). Cochrane Handbook
RCP 2017 for Systematic Reviews of Interventions Version 5.1.0 (updated
Royal College of Physicians. National Early Warning Score March 2011). The Cochrane Collaboration, 2011. Available from
(NEWS) 2: Standardising the assessment of acute-illness training.cochrane.org/handbook/archive/v5.1/.
severity in the NHS. Updated report of a working party.
Sterne 2016
www.rcplondon.ac.uk/projects/outputs/national-early-
warning-score-news-2 (accessed 23 March 2018). Sterne JA, Hernán MA, Reeves BC, Savović J, Berkman ND,
Julian PT, et al. ROBINS-I: a tool for assessing risk of bias in non-
Reeves 2020 randomised studies of interventions. BMJ 2016;355:i4919. [DOI:
Reeves BC, Deeks JJ, Higgins JP, Shea B, Tugwell P, Wells GA. 10.1136/bmj.i4919]
Chapter 24: Including non-randomized studies on intervention
Tirkkonen 2017
effects. In: Higgins JP, Thomas J, Chandler J, Cumpston M, Li T,
Page MJ, et al., editor(s). Cochrane Handbook for Systematic Tirkkonen J, Tamminen T, Skrifvars MB. Outcome of adult
Reviews of Interventions Version 6.1 (updated September patients attended by rapid response teams: a systematic review
2020). Cochrane, 2020. Available from training.cochrane.org/ of the literature. Resuscitation 2017;112:43-52. [DOI: 10.1016/
handbook/archive/v6.1. j.resuscitation.2016.12.023]

ResearchGate 2017 Vlayen 2012

ResearchGate. Social network service for scientists. Vlayen A, Verelst S, Bekkering G, Schrooten W, Hellings J,
www.researchgate.net (accessed 12 December 2016). Claes N. Incidence and preventability of adverse events
requiring intensive care admission: a systematic review. Journal
Robinson 2020 of Evaluation in Clinical Practice 2012;18(2):485-97. [DOI:
Robinson K, Allen F, Darby J, Fox C, Gordon AL, Horne JC, et 10.1111/j.1365-2753.2010.01612.x]
al. Contamination in complex healthcare trials: the falls in
VMS 2008
care homes (FinCH) study experience. BMC Medical Research
Methodology 2020;20:46. [DOI: 10.1186/s12874-020-00925-z] VMS Safety Programme. Early recognition and treatment
of the vitally endangered patient [Vroege herkenning
Sandroni 2015 en behandeling van de vitaal bedreigde patiënt].
Sandroni C, D'Arrigo S, Antonelli M. Rapid response systems: are www.vmszorg.nl/wp-content/uploads/2017/11/
they really effective? Critical Care 2015;19(1):104. [DOI: 10.1186/ web_2009.0100_praktijkgids_vitaalbedreigdpatient.pdf
s13054-015-0807-y] (accessed prior to 12 December 2019).

Smith 2002 Winters 2013

Smith GB, Osgood VM, Crane S. ALERT – a multiprofessional Winters BD, Weaver SJ, Pfoh ER, Yang T, Pham JC, Dy SM. Rapid-
training course in the care of the acutely ill adult patient. response systems as a patient safety strategy. A Systematic
Resuscitation 2002;52(3):281-6. [PMID: 11886734] review. Annuals Internal Medicine 2013;158(502):417-25. [DOI:
10.7326/0003-4819-158-5-201303051-00009]
Smith 2004
Smith GB, Poplett N. Impact of attending a 1-day multi-
References to other published versions of this review
professional course (ALERT) on the knowledge of acute care
in trainee doctors. Resuscitation 2004;61(2):117-22. [PMID: McGaughey 2005
15135187] McGaughey J, Alderdice F, Fowler R, Kapila A, Moutray M.
Outreach and early warning systems (EWS) for the prevention
Smith 2016
of intensive care admission and death of critically ill adult
Smith GB, Prytherch DR, Jarvis S, Kovacs C, Meredith P, patients on general hospital wards. Cochrane Database of
Schmidt P, et al. A comparison of the ability of the physiologic Systematic Reviews 2005, Issue 4. Art. No: CD005529. [DOI:
components of Medical Emergency Team Criteria and the U.K. 10.1002/14651858.CD005529]
National Early Warning Score to discriminate patients at risk

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 32
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

McGaughey 2007 prevention of intensive care admission and death of critically

McGaughey J, Alderdice F, Fowler R, Kapila A, Mayhew A, ill adult patients on general hospital wards. Cochrane Database
Moutray M. Outreach and early warning systems (EWS) for the of Systematic Reviews 2007, Issue 3. Art. No: CD005529. [DOI:
10.1002/14651858.CD005529.pub2]

CHARACTERISTICS OF STUDIES

Characteristics of included studies [ordered by study ID]

Aitken 2015
Study characteristics

Methods Quasi-experimental pretest–post-test design

Participants Patients on 6 randomly selected wards for point prevalence assessment. Inpatients across all hospital
wards for incidence of ICU unplanned admission and cardiac arrest and random selection of patients
for chart review on predetermined dates.

Age: NR

Sex: NR

Setting: tertiary metropolitan hospital (750 bed), Australia

Interventions Intervention: a 2-tier RRS consisting of ICUON and RRT, MEC criteria

Team composition: ICUON to assist with stabilisation and care of deteriorating patients; multidiscipli-
nary RRT. After hours, the team included medical resident, ICU junior registrar, ICUON and a CCU or ED
nurse

Timeline: point prevalence assessment 1 day prior to and 8 months following RRS implementation; in-
cidence of ICU unplanned admissions and cardiac arrests 20 months prior and 48 months after RRS im-
plementation

Comparison: standard ward care included MEC calling criteria, medical registrar assistance, cardiac ar-
rest team

Outcomes Primary outcomes: prevalence of deteriorating patients, incidence of unplanned admission to ICU and
cardiac arrests (mean/month), staff satisfaction

Notes Funded by Princess Alexandra Hospital Private Practice Trust Fund

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk See Table 4.

tion (selection bias)

Chen 2016
Study characteristics

Methods Interrupted time series (population based)

Participants Participants: all patients admitted to hospital

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 33
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Chen 2016 (Continued)

Age groups: before: ≥ 18 to < 35 years: 18.4%; ≥ 35 and < 55 years: 22.8%; ≥ 55 to < 75 years: 32.2%; ≥
75 years: 26.6%; after ≥ 18 to < 35 years: 16.7%; ≥ 35 to < 55 years: 21.7%; ≥ 55 to < 75 years: 33.4%; ≥ 75
years: 28.2%

Sex: before 1,232,046 (47.3%) male; after 714,346 (48.1%) male

Setting: 232 public hospitals across NSW, Australia (principal group: > 25,000 admissions; major hospi-
tals: > 10,000 and < 25,000 admissions; district: > 2000 and < 10,000 admissions; hospitals: < 2000 ad-
missions; all other public health facilities including subacute and residential: 5,635,947

Interventions Intervention: standardised, hospital-wide RRT and single parameter EWS

Team composition: ICU clinicians

Timeline: baseline 2007–2009; run in 2010; after 2011–2013

Comparison: retrospective data (admissions database)

Outcomes Primary outcomes: IHCA rate (per 1000 admissions), IHCA related mortality rate

Hospital mortality rate (per 1000 admissions), failure to rescue, deaths in low mortality diagnostic
groups, 1 year postmortality of IHCA

Notes Clinical Excellence Commission implemented a standardised RRS in the state of NSW in January 2010.
Funded by National Health and Medical Research Council of Australia.

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk See Table 4.

tion (selection bias)

Davis 2015
Study characteristics

Methods 2 centre controlled before-after study

Participants Participants: inpatients across all units

Age: NR

Sex: NR

Setting: primary medical centre (392 beds) and sister campus (119 beds) (134,093 participants), US

Interventions Intervention: RRT and single-parameter EWS

Team composition: dedicated critical care nurse and respiratory therapist. The third member of the
team was the unit charge nurses who was not a dedicated primary responder but acted only if the re-
sponse was activated in their specific unit.

Timeline: non-ICU CPA: July 2005–June 2011; hospital discharge and mortality: July 2006–June 2011;
Code Blue activation: July 2005–June 2011

Comparison: retrospective baseline data (electronic records)

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 34
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Davis 2015 (Continued)

Outcomes Primary outcomes: number and year-over-year Code Blue and RRT activations over time, hospital dis-
charge (per 1000 discharges), overall hospital mortality (%), patient acuity (CMI), yearly incidence of
non-CPAs, incidence of ICU CPAs

Notes No funding stated

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk See Table 4.

tion (selection bias)

Haegdorens 2018
Study characteristics

Methods Stepped wedge cluster randomised controlled trial

Participants Participants: inpatients on 28 wards

Age: intervention: mean 59.9 (SD 18.2) years; control: mean 58.9 (SD 18.6) years

Sex: intervention: 35,389 (51%) male; control: 34,267 (49%) male

Setting: 7 hospitals (> 850 admissions per year) across Belgium (69,656 participants)

Interventions Intervention: MET and NEWS

Team composition: NR

Timeline: baseline (T0) 4 months; postintervention period (T1–T4) 16 months

Control: 28 matched and paired surgical/medical wards moved from control (standard care) to imple-
mentation via training period

Outcomes Primary outcomes: unexpected death, cardiac arrest with CPR, unplanned ICU admission (per 1000 ad-
missions)

Notes Belgium Federal Government sponsorship. Trial registration: NCT01949025

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Low risk Comment: 1 surgical and 1 medical ward per hospital were randomly paired
tion (selection bias) and assigned as a block to the intervention. In total, 56 wards were enrolled
and randomly allocated to 4 groups using computerised randomisation which
as performed by KW who was not involved in the further conduct of the study.

Allocation concealment Low risk Comment: communication with authors indicated allocation concealment
(selection bias) process was undertaken by KW using randomised computer process. Trial reg-
istration indicated allocation as randomised.

Blinding of participants High risk Comment: unable to blind staff as received training/education prior to imple-
and personnel (perfor- menting NEWS/SBAR.
mance bias)

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 35
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Haegdorens 2018 (Continued)

All outcomes

Blinding of outcome as- Low risk Comment: hospitals were blinded for the collection date of comorbidity.
sessment (detection bias)
All outcomes Databases from each hospital and used a standardised electronic checklist to
collect objective data (unexpected death, cardiac arrest, unplanned ICU ad-
mission).

The researchers reviewed each patient record in case of a crude outcome Indi-
cator. When uncertain 2 independent researchers (FH and MM) agreed.

Incomplete outcome data High risk Quote: "… seven hospitals were excluded from the analysis because of un-
(attrition bias) available data".
All outcomes
Comment: patients with incomplete data were excluded prior to data analysis
(Figure 1 Consort trial profile, p 4).

Selective reporting (re- Low risk Comment: no evidence of selective reporting. All outcomes stated were re-
porting bias) ported.

Other bias Unclear risk Comment: potential risk of contamination across wards.

Hillman 2005
Study characteristics

Methods Prospective cluster randomised controlled trial

Participants Participants: inpatients on general wards

Age: intervention: mean 55.4 (SD 19.9) years; control: mean 56.9 (SD 20.8) years

Sex: intervention: 33,965 (50%) male; control: 26,775 (47%) male

Setting: 23 hospitals in Australia, 12 intervention hospitals (68,376 participants, median 364 bed) and
11 control hospitals (56,756 participants, median 315 bed)

Interventions Intervention: hospital-wide MET and single parameter calling criteria

Team composition: staff designated to form the MET varied between participating hospitals, with a
minimum of 1 doctor and a nurse from ED or ICU to meet protocol requirement

Timeline: 2-month baseline period, 4-month implementation period and 6-month after period in both
control and intervention hospitals

Control: randomised controlled hospitals did not receive MET education and cardiac arrest teams con-
tinued unchanged during the implementation and study period

Outcomes Primary outcome: composite outcome of the incidence (events divided by the number of eligible pa-
tients admitted to the hospital during the study period) of cardiac arrests without a pre-existing NFR
order, unplanned ICU admission, and unexpected deaths (deaths without a pre-existing NFR) (per 1000
admissions)

Secondary outcomes: individual patient data (cardiac arrests without pre-existing NFR, unplanned ICU
admissions, unexpected deaths and length of hospital stay per 1000 admissions)

Notes If a patient had > 1 event during their hospital stay, only 1 event was included in the composite mea-
sure.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 36
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Hillman 2005 (Continued)

Funded by grants from the Australian National Health and Medical Research Council, the Australian
Council for Quality and Safety in Healthcare, and the Australian and New Zealand Intensive Care Foun-
dation.

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Low risk Comment: participating hospitals identified via yearbook and those which met
tion (selection bias) the criteria were randomised using SAS version 6.12 blocked by the number of
beds and stratified by teaching status.

Allocation concealment Low risk Quote: "… an independent statistician (who had no other involvement in the
(selection bias) study) randomly assigned hospitals to receive standardised MET implementa-
tion or to be controls" (p 2091).

Comment: randomisation was concealed from investigators and hospitals.

Blinding of participants High risk Comment: blinding unfeasible as 4-month implementation period (MET edu-
and personnel (perfor- cational strategy, regular reminders, poster, name badges) in intervention hos-
mance bias) pitals.
All outcomes
Comment: no blinding, but outcome unlikely to be influenced

Blinding of outcome as- Low risk Comment: objective outcomes measured (cardiac arrest, ICU admissions, un-
sessment (detection bias) expected death). Optical automated scanning data entry.
All outcomes

Incomplete outcome data Low risk Comment: no hospitals lost to follow-up (Figure Trial Profile, p 2093).
(attrition bias)
All outcomes

Selective reporting (re- Low risk Comment: protocol unavailable but no evidence of selective reporting.
porting bias)

Other bias High risk Quote: "Control hospitals did not receive any education about the MET at any
stage' (pg 2091); 'study was not publicised in the control hospitals" (p 2092).

Comment: there was potential for risk of contamination as the benefits of the
MET system were widely reported in the media, which may have impacted on
the behaviour of the control hospitals.

Jeddian 2016
Study characteristics

Methods Stepped wedge cluster randomised trial

Participants Participants: all patients on 13 adult general wards

Age: intervention: mean 43 (SD 19) years; unexposed: mean 44 (SD 20) years

Sex: intervention: 4266 (39%) male; unexposed: 3732 (48%) male

Setting: university and public teaching hospital (18,682 participants, 800 bed), Iran

Interventions Intervention: CCOS and single parameter calling criteria

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 37
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Jeddian 2016 (Continued)

Team composition: 6 experienced ICU nurses

Timeframe: baseline for 3 periods (12 weeks), roll-out 2 wards every 2 periods (6 steps of 8 weeks each),
transition phase for each ward (8 weeks), and postintervention data collection for 3 periods (12 weeks).
Total of 18 periods (72 weeks)

Control: 13 wards moved from control (usual care) to intervention via training period

Outcomes Primary outcomes: in-hospital mortality (%), CPR (number, %)

Secondary outcomes: length of stay (median days), ICU admission (%)

Notes No funding reported. Trial registration: IRCT201107187053N1

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Low risk Quote: "Randomisation was carried out at a fixed point in time independent of
tion (selection bias) the trial team" (p 214).

Allocation concealment Low risk Quote: "… information on ward sequence was revealed 2 to 3 days before start
(selection bias) of the transition period" (p 214).

Blinding of participants High risk Quote: "The intervention was delivered without blinding" (p 214).
and personnel (perfor-
mance bias)
All outcomes

Blinding of outcome as- Low risk Quote: "Mortality and length of stay data were obtained from the hospital elec-
sessment (detection bias) tronic information systems. Data on cardiopulmonary resuscitation and ad-
All outcomes missions to the intensive care unit were obtained from nursing office and CCOT
records by the CCOT in exposed wards and by the independent data team in
unexposed wards. For these outcomes, data collection was, therefore, not
blind to exposure status".

Comment: objective outcome measures recorded by staff in exposed wards

and independent data team in unexposed wards.

Incomplete outcome data Low risk Quote: "The primary analysis was by intention to treat … For the fully adjust-
(attrition bias) ed analysis, less than 2% of patients had incomplete data so the missing data
All outcomes methods were not warranted" (p 214).

Selective reporting (re- Low risk Comment: protocol unavailable and all outcomes stated are reported.
porting bias)

Other bias Unclear risk Comment: potential risk of contamination across wards.

Lighthall 2010
Study characteristics

Methods Single-centre controlled before-after study

Participants Participants: adult inpatients in all acute care settings of an American veteran hospital

Age: before: 65.2 (SEM 0.08) years; after 65.5 (SEM 0.08) years

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 38
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Lighthall 2010 (Continued)

Sex: before 96% male; after: 96% male

Setting: tertiary medical centre (240 bed), US

Interventions Intervention: high capability MET and single parameter EWS

Team composition: physician led MT

Timeframe: before 9 months and 27 months after (analysed in 9-month blocks); mortality 27 months
after

Comparison: prospective baseline cardiac arrest data 9 months before, retrospective baseline mortali-
ty data 3.5 years before

Outcomes Primary outcomes: mortality rates (per 100 discharges), cardiac arrests rates (per 1000 discharges)

Notes Cardiac arrests in the ED and areas not served by the MET were not included in the analysis. No funding
reported.

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk See Table 4.

tion (selection bias)

Ludikhuize 2015
Study characteristics

Methods Prospective, pragmatic before-after multicentre trial

Participants Participants: adult patients admitted to COMET study wards

Age: before: mean 62.2 (SD 18) years; after: mean 62.3 (SD 18) years

Sex: before: 49.2% male; after 50.1% male

Setting: 2 large university hospitals (882–1000 beds), 8 teaching hospitals (359–1070 beds), 2 regional
hospitals (290–325 beds) (54,479 participants; Table 3), the Netherlands

Interventions Intervention: RRT and MEWS

Composition: the RRT included both an ICU nurse and a physician who was at least trained in funda-
mental critical care

Timeframe: baseline 5 months, MEWS and SBAR implementation (7 months), RRT implementation (12
months), after 5 months (same months of year as the before period)

Comparison: prospective baseline data

Outcomes Primary outcome: composite endpoint of cardiopulmonary arrest, unplanned ICU admission or death
per 1000 admissions

Secondary outcomes: individual patient data (cardiopulmonary arrest, unplanned ICU admission, or
death per 1000 admissions)

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 39
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Ludikhuize 2015 (Continued)

Notes Patients who were readmitted to the hospital were not excluded from the analysis. ICU admission did
not include medium care or other high-dependency units. Intensive care was defined according to the
criteria from the Dutch National Intensive Care Evaluation registry. No funding reported.

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk See Table 4.

tion (selection bias)

Menon 2018
Study characteristics

Methods Prospective before-after study

Participants Participants: all patients

Age: before: 11 at 18–40 years, 30 at 41–70 years, 10 at > 70 years; after: 6 at 18–40 years, 41 at 41–70
years, 16 at > 70 years

Sex: before: 35 (68.6%) male; after 40 (66%) male

Setting: tertiary academic hospital (28,053 participants, 226 bed), India

Interventions Intervention: physician-led MET and AWC

Team composition: MDT lead by internal medicine, charge nurse, nursing supervisor, phlebotomist, res-
piratory therapist, ECG technician

Timeframe: pre-implementation phase January–March 2013 (AWC and MET), 1-month education, be-
fore period April 2013–March 2014, after April 2014–March 2016

Comparison: prospective baseline data

Outcomes Primary outcomes: non-ICU cardiac arrests (per 1000 admissions), Code Blue mortality (per 1000 ad-
missions)

Secondary outcomes: MET activations and process (number), staff questionnaire

Notes No funding reported.

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk See Table 4.

tion (selection bias)

Priestley 2004
Study characteristics

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 40
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Priestley 2004 (Continued)

Methods Prospective stepped wedge randomised trial

Participants Participants: all patients admitted to 16 acute adult wards

Age: intervention: mean 65.2 (95% CI 64.3 to 66.2) years; control: mean 57.4 (95% CI 56.3 to 58.5) years

Sex: intervention: 804 (54.7%) male; control: 611 (43.1%) male

Setting: general hospital (6481 participants, 800 bed), UK

Interventions Intervention: CCOT and PAR score

Team composition: nurse consultant with a team of experienced nurses and medical support as re-
quired

Timeframe: 4 weeks' training each ward after which CCOS was fully operational, phased implementa-
tion over 32-week period

Control: control wards moved from control to intervention wards via the training period

Outcomes Primary outcome: rate of in-hospital deaths (number), length of hospital stay (mean)

Notes A small number of participants were admitted to the hospital more than once during the study period
and were included more than once in the sample.

Funded by the York Research Innovation Fund (York Hospitals NHS Trust).

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk Comment: the process of randomisation for the ward pairings and order of in-
tion (selection bias) troduction to the 4 groups of wards was not specified. Probably done.

Quote: "One from each pair of wards was randomised to the earlier phase of
outreach introduction"; "The order of introduction to the four groups of wards
was also randomly determined".

Allocation concealment Unclear risk Comment: randomisation was done by a study investigator and the method of
(selection bias) allocation is unclear.

Quote: "Randomisation was done by DR alone, based on ward pairings and

risk estimates provided by the rest of the study team".

Blinding of participants High risk Quote: "No blinding was possible".

and personnel (perfor-
mance bias)
All outcomes

Blinding of outcome as- Low risk Comment: objective outcomes measured that were unlikely to be influenced
sessment (detection bias) by lack of blinding, e.g. in-hospital mortality, length of hospital stay.
All outcomes

Incomplete outcome data Low risk Comment: patients with incomplete data excluded prior to data analysis. All
(attrition bias) patients were included in the analysis (data set 1).
All outcomes

Selective reporting (re- High risk Comment: authors stated findings were confirmed using data set 3 but no de-
porting bias) tails or data for data set 3 were provided to confirm findings. No protocol avail-
able and all outcomes stated are reported.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 41
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Priestley 2004 (Continued)

Other bias Unclear risk Comment: potential risk of contamination across wards.

Rothberg 2012
Study characteristics

Methods Interrupted time series

Participants Participants: all hospitalised patients

Age: NR

Sex: NR

Setting: tertiary care referral centre (154,382 admissions; 670 bed), US

Interventions Intervention: hospitalist-led MET and single parameter EWS

Team composition: critical care nurse, respiratory therapist, intravenous therapist and patient's physi-
cian or ICU physician as back up team member

Timeframe: before 24 months (January 2004–December 2006), implementation period first quarter and
second quarter 2006, after 41 months (July 2006–December 2009)

Control: retrospective data

Outcomes Primary outcomes: MET calls and process (n), Cardiac arrests (per 1000 admissions), Code deaths (per
1000 admissions), hospital mortality (per 1000 admissions)

Notes A separate "code" team for cardiovascular arrests included ICU medical resident and intern, critical
care nurse, anaesthetist, respiratory therapist, staff nurse and house supervisor.

No funding reported.

Risk of bias

Bias Authors' judgement Support for judgement

Random sequence genera- Unclear risk See Table 4.

tion (selection bias)

APDC: Admitted Patient Data Collection; AWC: Amrita Warning criteria; CCOT: critical care outreach team; CCU: coronary care unit; CMI:
Case Mix Index; COMET: Cost and Outcomes analysis of MET; CPA: cardiopulmonary arrest; CPR: cardiopulmonary resuscitation; ECG:
electrocardiograph; ED: emergency department; EWS: early warning system; ICU: intensive care unit; ICUON: intensive care unit outreach
nurse; IHCA: in-hospital cardiopulmonary arrest; MDT: multidisciplinary team; MEC: Medical Emergency Criteria; MET: medical emergency
team; MEWS: modified early warning system; NEWS: National Early Warning System; NFR: not-for-resuscitation; NR: not reported; RRS:
rapid response system; RRT: rapid response team; SBAR: Situation, Background, Assessment, Recommendation communication tool; SD:
standard deviation; SEM: standard error of the mean.

Characteristics of excluded studies [ordered by study ID]

Study Reason for exclusion

Al Qahtani 2013 Wrong study design.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 42
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Study Reason for exclusion

Aneman 2015 Before-after study with historical control group.

Austin 2014 Wrong study design.

Bannard-Smith 2016 Wrong study design.

Barrett 2010 Wrong study design.

Barwise 2014 Wrong study design.

Barwise 2016 Wrong study design.

Beitler 2010 Before-after study with historical control group.

Beitler 2011 Before-after study with historical control group.

Bergamasco 2017 Wrong setting.

Bingham 2015 Wrong study design.

Bittman 2020 Wrong intervention.

Blotsky 2012 Wrong study design.

Bucknall 2013 Wrong study design.

Butcher 2010 Wrong study design.

Cabrini 2012 Wrong study design.

Calzavacca 2010 Wrong study design.

Cerchiari 2010 Wrong outcome.

Chan 2008 Wrong study design.

Chen 2009 Post-hoc analysis.

Chen 2014a Wrong study design.

Chen 2014b Wrong study design.

Chen 2015 Retrospective data from MERIT study.

Choi 2011 Wrong study design.

Churpek 2017 Wrong intervention (qSOFA vs EWS).

Considine 2019 Wrong design.

Danesh 2012 Wrong study design.

Danesh 2019 Wrong intervention.

Fernando 2018 Wrong design.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 43
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Study Reason for exclusion

Fernando 2019 Wrong design.

Frost 2015 Wrong study design.

Gagne 2018 Wrong intervention.

Galhotra 2010 Wrong study design.

Gao 2007 Wrong study design.

Gilmore 2014 Wrong study design.

Goncales 2012 Wrong study design.

Gray 2011 Wrong study design.

Harm 2012 Wrong study design.

Harrison 2010 Before-after study with historical control group.

Hassan 2015 Wrong study design.

Hatlem 2011 Wrong study design.

Hellervik 2012 Wrong study design.

Jaderling 2013 Wrong study design.

Javeri 2013 Wrong study design.

John 2010 Wrong study design.

Jokela 2015 Wrong study design.

Jones 2005 Before-after study. Historical controls.

Jones 2008 Wrong study design.

Karpman 2013 Wrong study design.

Karvellas 2012 Wrong study design.

Kawaguchi 2015 Wrong study design.

Kim 2013 Wrong study design.

Kodama 2013 Wrong study design.

Kodama 2014 Wrong study design.

Kollef 2014 Wrong intervention.

Konrad 2010 Wrong study design.

Laurens 2011 Cohort before-after study in single hospital.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 44
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Study Reason for exclusion

Lawless 2013 Wrong study design.

Lee Ekblad 2012 Wrong study design.

Le Guen 2015 Wrong study design.

Liberti 2012 Wrong study design.

Lim 2011 Wrong study design.

Lorencio Cardenas 2014 Wrong study design.

Ludikhuize 2013 Wrong intervention.

Ludington 2011 Wrong study design.

Magnuson 2015 Wrong study design.

Martin 2009 Wrong study design.

Martin 2017 Wrong design.

McDonnell 2013 Wrong outcomes.

Mitchell 2010 Wrong study design.

Mullany 2016 Wrong design.

O'Connell 2016 Wrong design.

Palomba 2014 Wrong study design.

Panico 2011 Wrong study design.

Petersen 2016 Wrong intervention (EWS only).

Pirret 2015 Wrong study design.

Pisa 2012 Wrong study design.

Prchalova 2013 Wrong study design.

Radeschi 2014 Wrong study design.

Rashid 2014 Wrong study design.

Reilly 2010 Wrong study design.

Sabahi 2012 Wrong study design.

Salvatierra 2014 Wrong study design.

Santamaria 2011 Wrong study design.

Sebat 2018 Wrong design.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 45
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Study Reason for exclusion

Simmes 2012 Wrong study design.

Simmes 2015 Wrong study design.

Smith 2013 Wrong study design.

So 2018 Wrong design.

Stelfox 2012 Wrong study design.

Stewart 2014 Wrong study design.

Sutherasan 2018 Wrong design.

Taguti Pda 2013 Wrong study design.

Tridente 2015 Wrong study design.

Trottier 2010 Wrong study design.

White 2016 Wrong design.

Yu 2016 Wrong intervention

EWS: early warning system; MERIT: Multi-center ESG Randomized Interventional Trial; qSOFA: quick sepsis-related organ failure
assessment.

DATA AND ANALYSES

Comparison 1. Early warning systems (EWS) and rapid response systems (RRS) versus usual care

Outcome or subgroup title No. of studies No. of partici- Statistical Effect size
pants method

1.1 Hospital mortality 10 Other data No numeric data

1.1.1 Randomised controlled trials (RCT) 4 Other data No numeric data

1.1.2 Non-randomised controlled trials 6 Other data No numeric data

1.2 Composite outcome (unexpected cardiac 2 Other data No numeric data

arrests, unplanned ICU admissions and death)

1.2.1 Randomised controlled trials (RCT) 1 Other data No numeric data

1.2.2 Non-randomised controlled trials 1 Other data No numeric data

1.3 Unplanned intensive care unit admission 5 Other data No numeric data

1.3.1 Randomised controlled trials (RCT) 3 Other data No numeric data

1.3.2 Non-randomised controlled trial 2 Other data No numeric data

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 46
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Outcome or subgroup title No. of studies No. of partici- Statistical Effect size
pants method

1.4 Length of hospital stay 2 Other data No numeric data

1.4.1 Randomised controlled trials (RCT) 2 Other data No numeric data

1.5 Adverse events (unexpected cardiac or res- 10 Other data No numeric data
piratory arrest)

1.5.1 Randomised controlled trials (RCT) 3 Other data No numeric data

1.5.2 Non-randomised controlled trials 7 Other data No numeric data

Analysis 1.1. Comparison 1: Early warning systems (EWS) and rapid

response systems (RRS) versus usual care, Outcome 1: Hospital mortality
Hospital mortality
Study Absolute effect (EWS Absolute effect (con- Relative effect (95% CI) No. participants/admis- Notes
and RRS) trol) sions
Randomised controlled trials (RCT)
Haegdorens 2018 0.7/1000 1.5/1000 aOR 0.82 (0.34 to 1.95) 69,656 0.8 more participants per
1000 died without EWS
and RRS (95% CI 0.4 to
1.1)
Step wedge cluster RCT;
NEWS, medical-led MET
Hillman 2005 1.1/1000 1.2/1000 aOR 1.03 (0.84 to 1.28) 364,094 0.1 more participants per
1000 died without EWS
and RRS (95% CI −0.2 to
0.5)
Cluster RCT; calling crite-
ria, medical-led MET
Jeddian 2016 35.3/1000 47.4/1000 aOR 1.02 18,684 12.2 more participants
(0.68 to 1.55) per 1000 died without
EWS and RRS (95% CI 6.4
to 18.1)
Step wedge cluster RCT;
calling criteria, nurse-led
CCOT
Priestley 2004 49.1/1000 65.1/1000 aOR 0.52 (0.32 to 0.85) 2792 16.0 more participants
per 1000 died without
EWS and RRS (95 % CI 6.0
to 26.0)
Prospective step wedge
RCT, absolute effect
based on Dataset 1 and 2
patients combined; PAR
score, nurse-led CCOT
Non-randomised controlled trials
Chen 2016 14.5/1000 17.8/1000 aRR 0.81 (0.67 to 0.86) NR ITS, number of partic-
ipants grouped yearly
and not possible to de-
termine number of par-
ticipants or CI; calling
criteria, medical-led RRT
Davis 2015 0.2/1000 0.2/1000 — 134,093 0.04 more participants
per 1000 died without
EWS and RRS (95% CI
−0.1 to 0.3)
Number of participants
unclear and CI calculat-
ed from Table 2
2 centre before-after;
calling criteria, nurse-led
RRT

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 47
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Lighthall 2010 22.4/1000 27.1/1000 — NR Single-centre before-af-

ter; calling criteria, med-
ical-led MET
Number of participants
unclear and absolute ef-
fect based on 'bed occu-
pancy'
Ludikhuize 2015 17.7/1000 20.4/1000 aOR 0.85 (0.64 to 1.00) 57,858 Prospective, pragmatic
before-after multicentre
study; EWS, medical-led
RRT
Menon 2018 2.4/1000 4.93/1000 RR 1.0 (0.03 to 29.8) 18,954 2.49 more participants
per 1000 died without
EWS and RRS (95% CI 0.7
to 4.6)
Prospective single-cen-
tre before-after; calling
criteria, medical-led MET
Rothberg 2012 22/1000 22/1000 — NR ITS. Number of partici-
pants unclear; calling cri-
teria, medical-led MET

Analysis 1.2. Comparison 1: Early warning systems (EWS) and rapid response systems (RRS) versus usual
care, Outcome 2: Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)
Composite outcome (unexpected cardiac arrests, unplanned ICU admissions and death)
Study Absolute effect (EWS Absolute effect (con- Relative effect (95% CI) No. participants/admis- Notes
and RRS) trol) sions
Randomised controlled trials (RCT)
Hillman 2005 5.3/1000 5.9/1000 aOR 0.98 (0.83 to 1.16) 364,094 0.6 more participants
per 1000 experienced
the composite outcome
without EWS (95% CI
−0.3 to 1.4)
Cluster RCT; calling crite-
ria, medical-led MET
Non-randomised controlled trials
Ludikhuize 2015 32.9/1000 37.1/1000 aOR 0.85 (0.72 to 0.99) 57,858 4.2 more participants
per 1000 experienced
the composite outcome
without EWS (95% CI 1.2
to 7.2)
Prospective, pragmat-
ic before-after multi-
centred trial; EWS, med-
ical-led RRT

Analysis 1.3. Comparison 1: Early warning systems (EWS) and rapid response
systems (RRS) versus usual care, Outcome 3: Unplanned intensive care unit admission
Unplanned intensive care unit admission
Study Absolute effect (EWS Absolute effect (con- Relative effect No. participants/admis- Notes
and RRT) trol) sions
Randomised controlled trials (RCT)
Haegdorens 2018 10.3/1000 6.5/1000 aOR 1.23 (0.91 to 1.65) 69,656 3.8 fewer participants
per 1000 experienced un-
planned ICU admission
without EWS and RRS
(95% CI 2.3 to 5.0)
Step wedge cluster RCT;
NEWS, medical-led MET
Hillman 2005 4.2/1000 4.7/1000 aOR 1.04 (0.89 to 1.21) 364,094 0.49 more participants
per 1000 unplanned ICU
admission without EWS
and RRS (95% CI −0.2 to
1.2)

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 48
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Cluster RCT; calling crite-

ria, medical-led MET
Jeddian 2016 12.8/1000 12.3/1000 aOR 1.15 (0.64 to 2.09) 18,684 0.5 fewer participants
per 1000 experienced un-
planned ICU admission
without EWS and RRS
(95% CI −2.9 to 3.7)
Step wedge cluster RCT;
calling criteria, nurse-led
CCOT
Non-randomised controlled trial
Aitken 2015 NR NR −6.52 (−15.24 to 2.20) NR Relative effect calculated
from reanalysed ITS, ab-
solute difference could
not be calculated and re-
analysed ITS could not
be standardised as num-
ber of admissions not
provided
Quasi-experimental, sin-
gle-centre pretest–post-
test, reanalysed ITS; call-
ing criteria, nurse-led
RRT
Ludikhuize 2015 17.1/1000 19.8/1000 aOR 0.88 (0.75 to 1.02) 57,858 2.6 more participants
per 1000 experienced un-
planned ICU admission
without EWS and RRS
Prospective, pragmatic
before-after multicentre
trial; EWS, medical-led
RRT

Analysis 1.4. Comparison 1: Early warning systems (EWS) and rapid

response systems (RRS) versus usual care, Outcome 4: Length of hospital stay
Length of hospital stay
Study Absolute effect (EWS Absolute effect (con- Relative effect No. participants/admis- Notes
and RRT) trol) sions
Randomised controlled trials (RCT)
Jeddian 2016 Median (IQR) 4 (2 to 8) Median (IQR) 6 (3 to 10) aRGM 1.00 18,684 Length of stay was re-
days days (0.97 to 1.03) duced by 2 days with
EWS
Step wedge cluster RCT;
calling criteria, nurse-led
CCOT
Priestley 2004 NR NR HR 0.91 (0.83 to 0.98) 2733 Prospective step wedge
RCT; PAR score, nurse-
led CCOT

Analysis 1.5. Comparison 1: Early warning systems (EWS) and rapid response systems (RRS)
versus usual care, Outcome 5: Adverse events (unexpected cardiac or respiratory arrest)
Adverse events (unexpected cardiac or respiratory arrest)
Study Absolute effect (EWS Absolute effect (con- Relative effect (95% CI) No. participants/admis- Notes
and RRS) trol) sions
Randomised controlled trials (RCT)
Haegdorens 2018 1.0/1000 1.3/1000 aOR 0.71 69,656 0.3 more patients per
(0.33 to 1.52) 1000 experienced an ad-
verse event without EWS
and RRS (95% CI −0.7 to
1.4)
Step wedge cluster RCT;
NEWS, medical–led MET
Hillman 2005 1.3/1000 1.6/1000 aOR 0.94 (0.79 to 1.13) 364,094 03 more patients per
1000 experienced an ad-

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 49
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

verse event without EWS

and RRS (95% CI −0.1 to
0.8)
Cluster RCT; calling crite-
ria, medical-led MET
Jeddian 2016 48.6/1000 36.1/1000 aOR 1.00 18,684 12.5 fewer participants
(0.69 to 1.48) per 1000 experienced an
adverse event without
EWS and RRS (95% CI 9.5
to 15.5)
Step wedge cluster RCT;
calling criteria, nurse-led
CCOT
Non-randomised controlled trials
Aitken 2015 NR NR −1.29 NR Quasi-experimental, sin-
(−8.49 to 5.91) gle-centre pretest–post-
test; reanalysed ITS; call-
ing criteria, nurse-led
RRT
Chen 2016 1.3/1000 2.4/1000 aRR 0.55 NR 1.1 more participant per
(0.48 to 0.62) 1000 experienced an ad-
verse event without EWS
and RRS
ITS; calling criteria, med-
ical-led RRT
Davis 2015 1.1/1000 2.7/1000 NR 134,093 1.6 more participants per
1000 experienced an ad-
verse event without EWS
(95% CI 1.1 to 2.2)
2 centre before-after;
calling criteria, nurse-led
RRT
Lighthall 2010 4.4/1000 10.1/1000 20.3 (−18.56 to 59.19) NR 5.7 more participants per
1000 experienced an ad-
verse event without EWS
and RRS
Single-centre before-af-
ter; reanalysed ITS; call-
ing criteria, medical-led
MET
Ludikhuize 2015 1.2/1000 1.9/1000 aOR 0.61 57,858 0.7 more participant per
(0.39 to 0.94) 1000 experienced an ad-
verse event without EWS
and RRS (95% CI 0.1 to
1.4)
Prospective, pragmatic
before-after multicentre
trial; EWS, medical-led
RRT
Menon 2018 3.1/1000 6.8/1000 3.44 (−10.35 to 17.23) 18,954 Prospective single-cen-
tre before-after; re-
analysed ITS; calling cri-
teria, medical-led MET
Rothberg 2012 3.1/1000 4.7/1000 aOR 1.00 NR 1.6 more participants per
(0.69 to 1.48) 1000 experienced an ad-
verse event without EWS
and RRS
ITS, number of partici-
pants unclear; calling cri-
teria, medical-led MET

ADDITIONAL TABLES

Table 1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of
patient deterioration on acute hospital wards – absolute effects
Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of patient deteriora-
tion on acute hospital wards

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 50
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of
patient deterioration on acute hospital wards – absolute effects (Continued)
Patient or population: adults on acute hospital wards at risk of clinical deterioration

Settings: acute wards of hospital in the UK, US, Australia and the Netherlands

Intervention: RRS that included an EWS

Comparison: usual care without any form of EWS or RRS

Outcomes Study Absolute effect* No of partici- Reanalysed

pants/admis- ITS data
(95% CI) sions

Assumed risk Corresponding risk

Without EWS and RRS With EWS and RRS

Hospital mortality – Priestley 2004 65.1 per 1000 49.1 per 1000 2792 —
randomised trials
16.0 more participants per 1000 died without EWS (95% —
(follow-up: 3–16 CI 6 to 26)
months)
Hillman 2005 1.18 per 1000 1.06 per 1000 364,094 —

0.12 more participants per 1000 died without EWS (95% —

CI 0 to 0)

Jeddian 2016 47.4 per 1000 35.3 per 1000 18,684 —

12.2 more participants per 1000 died without EWS (95% —

CI 6 to 18)

Haegdorens 1.5 per 1000 0.7 per 1000 69,656 —

2018
0.8 more participants per 1000 died without EWS (95% —
CI 0 to 1)

Hospital mortality Lighthall 2010 27.1 per 1000 22.4 per 1000 0.19 (−0.83 to
– non-randomised 1.22)
studies —
(follow-up: 5–48
Rothberg 2012 22 per 1000 22 per 1000
months)
—

Davis 2015 0.21 per 1000 0.17 per 1000 134,093 —

0.04 more participants per 1000 died without EWS (95%

CI −0.1 to 0.3)

Ludikhuize 20.4 per 1000 17.7 per 1000 57,858

2015
2.7 more participants per 1000 died without EWS (95%
0.5 to 4.9)

Chen 2016 18 per 1000 14 per 1000 — —

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 51
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of
patient deterioration on acute hospital wards – absolute effects (Continued)
3.4 more participants per 1000 died without EWS (P <
0.001)

Menon 2018 4.9 per 1000 2.4 per 1000 18,954 —

2.5 more participants per 1000 died without EWS (95%

CI 0.7 to 4.6)

Composite outcome Hillman 2005 5.9 per 1000 5.3 per 1000 364,094 —
– randomised trials
0.6 more participants per 1000 experienced the com-
Composite outcome posite outcome without EWS (95% CI −0.3 to 1.4)
of incidence (events
divided by number of
eligible

participants admit-
ted to the hospital
during the study pe-
riod) of cardiac ar-
rests

without a pre-exist-
ing NFR order, un-
planned ICU admis-
sions and unexpect-
ed deaths (deaths
without a pre-exist-
ing NFR order)

(follow-up: 6
months)

Composite outcome Ludikhuize 37.1 per 1000 32.9 per 1000 57,858 —
– non-randomised 2015
studies 4.2 more participants per 1000 experienced the com-
posite outcome without EWS (95% CI 1.2 to 7.2)
Composite endpoint
of cardiopulmonary
arrest, unplanned
ICU admission or
death

(follow-up: 5
months)

Unplanned ICU Hillman 2005 4.7 per 1000 4.2 per 1000 364,094
admission – ran-
domised trials 0.5 more participants per 1000 unplanned ICU admis-
sion without EWS (95% CI −0.2 to 1.2)
(follow-up: 3–16
months)
Jeddian 2016 12.3 per 1000 12.8 per 1000 18,684 —

0.5 fewer participants per 1000 experienced unplanned

ICU admission without EWS (95% CI −2.9 to 3.7)

Haegdorens 6.5 per 1000 10.3 per 1000 69,656 —

2018

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 52
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of
patient deterioration on acute hospital wards – absolute effects (Continued)
3.8 fewer participants per 1000 experienced unplanned
ICU admission without EWS (95% CI 2.3 to 5.0)

Unplanned ICU ad- Aitken 2015 NR NR — −6.52 (−15.24

mission – non-ran- to 2.20)
domised studies —
(follow-up: 5
Ludikhuize 19.8 per 1000 17.1 per 1000 57,858 —
months)
2015
2.6 more participants per 1000 experienced unplanned
ICU admission without EWS (95% CI 0.5 to 4.9)

ICU readmission – No randomised trials reported on the rates of unplanned ICU readmission.
randomised trials

ICU readmission – No non-randomised studies reported on the rates of unplanned ICU readmission.
non-randomised
studies

Length of hospital No randomised trials reported length of hospital stay.

stay – randomised
trials

Length of hospi- Priestley 2004 NR NR 2733 —

tal stay – non-ran-
domised studies —
(follow-up: 3
Jeddian 2016 Median 6 days (IQR 3 to Median 4 days (IQR 2 to 8) 18,684 —
months)
10)

Length of stay was reduced by 2 days with EWS

Adverse events (un- Hillman 2005 1.6 per 1000 1.3 per 1000 364,094 —
expected cardiac or
respiratory arrest) – 0.3 more participants per 1000 experienced an adverse —
randomised trials event without EWS (95% CI −0.1 to 0.8)
(follow-up: 3–16
Jeddian 2016 36.1 per 1000 48.6 per 1000 18,684 —
months)
12.5 fewer participants per 1000 experienced an ad- —
verse event without EWS (95% CI 9.5 to 15.5)

Haegdorens 1.3 per 1000 1.0 per 1000 69,656 —

2018
0.3 more participants per 1000 experienced an adverse —
event without EWS (95% CI −0.7 to 1.4)

Adverse events (un- Lighthall 2010 10.1 per 1000 4.4 per 1000 — 20.3 (−18.56 to
expected cardiac or 59.19)
respiratory arrest) 5.7 more participants per 1000 experienced an adverse
– non-randomised event without EWS
studies
Rothberg 2012 4.7 per 1000 3.1 per 1000 — —
(follow-up: 5–48
months)

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 53
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 1. Early warning systems (EWS) and rapid response systems (RRS) compared with usual care for prevention of
patient deterioration on acute hospital wards – absolute effects (Continued)
1.6 more participants per 1000 experienced an adverse
event without EWS

Aitken 2015 NR NR — 1.29 (−8.49 to

5.91)
—

Davis 2015 2.7 per 1000 1.1 per 1000 134,093 —

1.6 more participants per 1000 experienced an adverse

event without EWS (95% CI 1.1 to 2.2)

Ludikhuize 1.9 per 1000 1.2 per 1000 57,858 —

2015
0.7 more participants per 1000 experienced an adverse
event without EWS (95% CI 0.1 to 1.4)

Chen 2016 2.4 per 1000 1.3 per 1000 — —

1.1 more participants per 1000 experienced an adverse

event without EWS

Menon 2018 6.8 per 1000 31 per 1000 18,954 3.44 (−10.35 to
17.23)
3.7 more participants per 1000 experienced an adverse
event without EWS (95% CI 1.5 to 6.0)

*The risk WITHOUT the intervention is based on the USUAL CARE control group risk IN EACH STUDY. The corresponding risk WITH the
intervention (and the 95% confidence interval for the difference) is based on the overall relative effect (and its 95% confidence inter-
val).

CI: confidence interval; EWS: early warning system; ICU: intensive care unit; IQR: interquartile range; NFR: not-for-resuscitation; NR: not
reported; RRS: rapid response system.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 54
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 2. Calculation of GRADE ratings worksheet
Number of studies Design Risk of bias Inconsisten- Indirectness Imprecision Publication Certainty‡
cy bias

Library
Cochrane
(overall
score)

Hospital mortality – randomised trials

Priestley 2004; Hillman 2005; Jeddian 2016; 4 RCT Serious (−1) Not serious Not serious Serious (−1) Not serious ⊕⊕⊝⊝

Better health.
Informed decisions.
Trusted evidence.
Haegdorens 2018 Lowa,b
(4)

Hospital mortality – non-randomised studies

Lighthall 2010; Rothberg 2012; Davis 2015; 4 CBA Serious (−1) Serious (−1) Not serious Serious (−1) Not serious ⊕⊝⊝⊝
Ludikhuize 2015; Chen 2016; Menon 2018 Very lowb,c,d
2 ITS

(3)

Composite outcome – randomised trials

Hillman 2005 1 RCT Serious (−1) None Not serious None Not serious ⊕⊕⊕⊝
Moderatea
(4)

Composite outcome – non-randomised studies

Ludikhuize 2015 1 CBA Serious (−1) None Not serious None Not serious ⊕⊕⊝⊝
Lowc
(3)

Unplanned ICU admission – randomised trials

Cochrane Database of Systematic Reviews

Hillman 2005; Jeddian 2016; Haegdorens 2018 3 RCT Serious (−1) Not serious Not serious Serious (−1) Not serious ⊕⊕⊝⊝
Lowa,b
(4)

Unplanned ICU admission – non-randomised studies

Aitken 2015; Ludikhuize 2015 CBA re- Serious (−1) Serious (−1) Not serious Serious (−1) Not serious ⊕⊝⊝⊝
analysed as Very lowb,c,d
ITS

1 ITS
55
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 2. Calculation of GRADE ratings worksheet (Continued)
(3)

Library
Cochrane
ICU readmission – randomised trials Not reported

ICU readmission – non-randomised studies Not reported

Length of hospital stay – randomised trials

Priestley 2004; Jeddian 2016 2 RCT Serious (−1) Serious (−1) Not serious Not serious Not serious ⊕⊕⊝⊝

Better health.
Informed decisions.
Trusted evidence.
Lowa,d
(4)

Length of hospital stay – non-randomised Not reported

studies

Adverse events – randomised trials

Hillman 2005; Jeddian 2016; Haegdorens 2018 3 RCTs Serious (−1) Not serious Not serious Serious (−1) Not serious ⊕⊕⊝⊝
Lowa,b
(4)

Adverse events – non-randomised studies

Lighthall 2010; Rothberg 2012; Aitken 2015; Davis 5 CBA Serious (−1) Serious (−1) Not serious Serious (−1) Not serious ⊕⊝⊝⊝
2015; Ludikhuize 2015; Chen 2016; Menon 2018 Very lowb,c,d
2 ITS

(3)

‡GRADE Working Group grades of evidence

High: this research provides a very good indication of the likely effect. The likelihood that the effect will be substantially different‡ is low.

Cochrane Database of Systematic Reviews

Moderate: this research provides a good indication of the likely effect. The likelihood that the effect will be substantially different‡ is moderate.

Low: this research provides some indication of the likely effect. However, the likelihood that it will be substantially different‡ is high.

Very low: this research does not provide a reliable indication of the likely effect. The likelihood that the effect will be substantially different‡ is very high.

‡Substantially different = a large enough difference that it might affect a decision.

CBA: controlled before-after study; ICU: intensive care unit; ITS: interrupted time series; RCT: randomised controlled trial.

aDowngraded one level for risk of bias: concerns regarding performance bias, contamination bias, or both.
56
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
bDowngraded one level for imprecision of the direction of the effect: due to wide risk ratio interpreted as less than 0.75 to greater than 1.25, which includes appreciable beneficial
or harmful effects.
cDowngraded one level for risk of bias: serious or critical risk of bias due to inadequate adjustment for confounders in observational studies assessed with the Risk Of Bias in

Library
Cochrane
Non-randomised Studies – of Interventions (ROBINS-I) tool.
dDowngraded one level for inconsistency: as unable to calculate effect size for all studies.

Better health.
Informed decisions.
Trusted evidence.
Cochrane Database of Systematic Reviews
57
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 3. Quality assessment judgements (ROBINS-I) of non-randomised studies

Lighthall 2010

Bias Authors' Support for judgement

judgement

D1: confounding bias Serious risk Comment: serious but not critical risk due to use of statistical methods to con-
trol for some, but not all, baseline confounders.

D2: selection bias Low risk Comment: no evidence of selective recruitment.

D3: bias in classification Low risk Comment: intervention status was clearly defined in terms of time of expo-
of interventions sure, i.e. admission to hospital before or after the RRS.

D4: bias due to devia- No information Comment: deviations from protocol not reported.
tion from intended in-
tervention

D5: bias due to missing Low risk Comment: no evidence of missing data.
data

D6: bias in measure- Low risk Comment: outcomes were objective and not subject to interpretation.
ment of outcomes

D7: bias in selection of Low risk Comment: no evidence of selective reporting.

reported outcomes

Overall risk of bias Serious risk Comment: due to serious risk of confounding and unclear risk of protocol devi-
ation.

Rothberg 2012

Bias Authors' judgement Support for judgement

D1: confounding bias Serious risk Comment: serious but not critical risk due to use of statistical methods to con-
trol for some, but not all, baseline confounders.

D2: selection bias Low risk Comment: no evidence of selective recruitment.

D3: bias in classification Low risk Comment: intervention status was clearly defined in terms of time of expo-
of interventions sure, i.e. admission to hospital before or after the RRS.

D4: bias due to devia- No information Comment: deviations from protocol not reported.
tion from intended in-
tervention

D5: bias due to missing Low risk Comment: no evidence of missing data.
data

D6: bias in measure- Low risk Comment: outcomes were objective and not subject to interpretation.
ment of outcomes

D7: bias in selection of Low risk Comment: no evidence of selective reporting.

reported outcomes

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 58
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 3. Quality assessment judgements (ROBINS-I) of non-randomised studies (Continued)

Overall risk of bias Serious risk Comment: serious but not critical risk due to use of statistical methods to con-
trol for some, but not all, baseline confounders.

Aitken 2015

Bias Authors' judgement Support for judgement

D1: confounding bias Critical risk Comment: there was no adjustment for patient or organisational characteris-
tics.

No baseline confounders were measured, or analyses performed for confound-

ing. Authors used process control methods, which helps to at least assess for
time-varying confounding but there are no analysis methods that control for
confounding.

D2: selection bias Low risk Comment: participant characteristics were not used as selection criteria after
the intervention. No evidence of selective recruitment.

D3: bias in classification Low risk Comment: interventions were defined in terms of time of exposure – either be-
of interventions fore or after the intervention was implemented.

D4: bias due to devia- No information Comment: deviations from intended intervention not reported.
tion from intended in-
tervention

D5: bias due to missing Low risk Comment: no evidence of missing data.
data

D6: bias in measure- Low risk Comment: measured outcomes including cardiac arrest, death and unplanned
ment of outcomes ICU admission were all objective and not subject to measurement bias.

D7: bias in selection of Low risk Comment: no evidence of selective outcome reporting.
reported outcomes

Overall risk of bias Critical risk

Davis 2015

Bias Authors' judgement Support for judgement

D1: confounding bias Critical risk Comment: there was no attempt to control for patient-associated confounders
such as acuity, comorbidity or age. Changes in ICU admission/readmission
rates could have confounded results, i.e. fewer non-ICU cardiac arrest because
more early ICU admissions (non-RRT initiated); however, this was not reported
on or adjusted for.

No evidence of analyses controlling for confounding.

There were no postintervention variables that need to be controlled for.

D2: selection bias Low risk Comment: no evidence of selective recruitment.

D3: bias in classification Low risk Comment: intervention status was clearly defined in terms of time of expo-
of interventions sure, i.e. admission to hospital before or after the RRS.

D4: bias due to devia- No information Comment: deviations from intended intervention not reported.
tion from intended in-
tervention

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 59
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 3. Quality assessment judgements (ROBINS-I) of non-randomised studies (Continued)

D5: bias due to missing Low risk Comment: no evidence of missing data.
data

D6: bias in measure- Low risk Comment: cardiac arrest rates and mortality were objective outcomes and
ment of outcomes were not subject to measurement bias.

D7: bias in selection of Low risk Comment: no evidence of selective reporting.

reported outcomes

Overall risk of bias Critical risk Comment: Rated as critical due to high risk if confounding bias and unclear
risk of bias due to no reporting of deviation from protocol.

Ludikhuize 2015

Bias Authors' judgement Support for judgement

D1: confounding bias Serious risk Comment: serious but not critical risk due to use of statistical methods to con-
trol for some, but not all, baseline confounders.

D2: selection bias Low risk Comment: no evidence of selective recruitment

D3: bias in classification Low risk Comment: intervention status was clearly defined in terms of time of expo-
of interventions sure, i.e. admission to hospital before or after the RRS.

D4: bias due to devia- No information Comment: deviations from protocol not reported.
tion from intended in-
tervention

D5: bias due to missing Low risk Comment: no evidence of missing data.
data

D6: bias in measure- Low risk Comment: outcomes were objective and not subject to interpretation.
ment of outcomes

D7: bias in selection of Low risk Comment: no evidence of selective reporting.

reported outcomes

Overall risk of bias Serious risk Comment: serious but not critical risk due to use of statistical methods to con-
trol for some, but not all, baseline confounders.

Chen 2016

Bias Authors' judgement Support for judgement

D1: confounding bias Serious risk Comment: evaluated changes in baseline characteristics by calendar year to
account for hospital cluster effect and assessed intervention effect using re-
gression to measure monthly trends.

Analysis adjusted for year, age, sex, marital status, country of birth, socioeco-
nomic status, hospital (urban vs rural), health insurance and major hospital
peer groups. There were no postintervention variables that need to be con-
trolled for.

D2: selection bias Low risk Comment: population-based study that used NSW APDC database to obtain
data on all admissions to 232 hospitals between 2007 and 2013.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 60
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 3. Quality assessment judgements (ROBINS-I) of non-randomised studies (Continued)

D3: bias in classification Low risk Comment: intervention status was clearly defined in terms of time of expo-
of interventions sure, i.e. admission to hospital before or after the RRS.

D4: bias due to devia- No information Comment: deviations from intended intervention not reported.
tion from intended in-
tervention

D5: bias due to missing Low risk Comment: no evidence of missing data.
data

D6: bias in measure- Low risk Quote: "Patient outcomes and other related variables were derived from the
ment of outcomes NSW APDC database, which includes demographic and diagnostic information
for each public and private hospital admission episode".

Comment: databases coded hospital mortality and cardiac arrest rates as ob-
jective outcomes and were not subject to measurement bias.

D7: bias in selection of Low risk Comment: no evidence of selective reporting.

reported outcomes

Overall risk of bias Serious risk Comment: rated as serious due to high risk of confounding bias due to use of
statistical methods to control for some, but not all, baseline confounders.

Menon 2018

Bias Authors' judgement Support for judgement

D1: confounding bias Critical risk Comment: there was no attempt to control for patient-associated confounders
such as acuity, comorbidity, age or time variation.

No evidence of analyses controlling for confounding.

There were no postintervention variables that need to be controlled for.

D2: selection bias Low risk Comment: no evidence of selective recruitment.

D3: bias in classification Low risk Comment: intervention status was clearly defined in terms of time of expo-
of interventions sure, i.e. admission to hospital before or after the RRS.

D4: bias due to devia- No information Comment: deviations from intended intervention not reported.
tion from intended in-
tervention

D5: bias due to missing Low risk Comment: no evidence of missing data.
data

D6: bias in measure- Low risk Code Blue events recorded before and after MET.
ment of outcomes
Comment: Code Blue rates were objective outcomes and were not subject to
measurement bias.

D7: bias in selection of Low risk Comment: no evidence of selective reporting. Deviations from protocol not re-
reported outcomes ported.

Overall risk of bias Critical risk Comment: rated as critical due to high risk of confounding bias due lack of sta-
tistical methods to control for baseline confounders.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 61
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

ICU: intensive care unit; MET: medical emergency team; NSW APDC: New South Wales Admitted Patient Data Collection; ROBINS-I: Risk Of
Bias in Non-randomised Studies – of Interventions; RRS: rapid response system.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 62
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 4. Summary of included study characteristics
Study ID Design Country Partic- Partic- Hospital Interven- Before After Time- Outcomes
ipants ipants tion frame

Library
Cochrane
(before) (after)

Randomised trials

Priestley Prospective step UK 3,391 3,090 800-bedded gener- PAR score, 4 weeks 28 weeks 32 weeks Primary: in-hospital
2004 wedge RCT al hospital (Dataset education, deaths, LOS

Better health.
Informed decisions.
Trusted evidence.
2: 2903 partici- 24/7 CCOT
pants)

Hillman Cluster RCT Australia 68,376 56,756 23 public hospitals Calling cri- 2 6 12 Primary: composite
2005 (total 125,132 par- teria, educa- months months months outcome (CAs, un-
ticipants) tion, MET planned ICU admis-
sions, unexpected
deaths)

Secondary outcome:
CAs, unplanned ICU
admissions, unexpect-
ed deaths

Jeddian Step wedge cluster Iran 10,880 7,802 1 university and Calling cri- 12 weeks 12 weeks July Primary: in-hospital
2016 RCT public teaching teria, educa- 2010– mortality, CAs
hospital (total tion CCOT Decem-
18,682 partici- ber 2011 Secondary: LOS, ICU
pants) admissions

Haeg- Step wedge cluster Belgium 35,389 34,267 7 hospitals (to- NEWS, 4 16 October Primary: unexpect-
dorens RCT tal 69,656 partici- SBAR, train- months months 2013– ed deaths, CAs, un-
2018 pants) ing, 24/7 May planned ICU admis-
MET 2015 sions

Cochrane Database of Systematic Reviews

Secondary: total ward
mortality, ward mor-
tality without NFR
code, hospital mortal-
ity

Non-randomised studies

Lighthall 2010 Sin- USA NR NR Tertiary medical Calling cri- 9 27 June Hospital mortality, CAs
gle-cen- centre 240 beds teria, edu- months months 2005–
tre (total number of cation, high (CA), Septem-
CBA, re- participants NR) ber 2007
63
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 4. Summary of included study characteristics (Continued)
analysed capability 42
ITS MET months

Library
Cochrane
(mortal-
ity)

Rothberg 2012 ITS USA NR NR 670-bed tertiary Calling cri- 24 41 January Hospital mortality, CAs
care referral cen- teria, educa- months months 2004–
tre (total number tion, MET (January (July Decem-

Better health.
Informed decisions.
Trusted evidence.
of participants NR) 2004– 2006– ber 2009
Decem- Decem-
ber ber
2006) 2009)

Aitken 2015 Qua- Australia NR NR 750 bed tertiary MEC call- 20 48 January Unplanned ICU admis-
si-exper- hospital (total ing criteria, months months 2008– sions, CAs, prevalence
imental, number of partici- education, (ICU ad- (ICU Octo- of deteriorating pa-
pretest– pants NR) 2-tier RRS mission admis- ber 2013 tients, staff satisfac-
post-test (ICUON, 24 and CA); sion and (ICU ad- tion
single hours per preva- CAs); mission
centre day/7 days lence: 1 Preva- and CAs)
CBA, re- per week day pri- lence: 8
analysed RRT) or to im- months
ITS plemen- follow-
tation ing

Davis 2015 2-centre USA 40,806 93,287 Primary medical Calling crite- 28 48 July CAs (non-ICU), hospital
CBA centre (392 beds) ria, training, months months 2005– mortality
and sister campus RRT June
(119 beds) (total 2011
134,093 partici-
pants)

Cochrane Database of Systematic Reviews

Ludikhuize 2015 Prospec- Nether- 28,298 29,560 2 large universi- MEWS, 5 5 April/Ju- Primary: composite
tive, lands ty hospitals (882– SBAR, edu- months months ly 2009– outcome (CAs, un-
prag- 1000 beds), 8 cation, RRT Au- planned ICU admis-
mat- teaching hospitals gust/No- sion, death)
ic be- (359–1070 beds) vember
fore-af- and 2 regional hos- 2011 Secondary: CAs, un-
ter mul- pitals (290–325 planned ICU admis-
ticentre beds) (total 57,858 sion, death
study participants)

Chen 2016 ITS Australia 2,602,351 3,033,596 232 hospitals Calling cri- 3 years 3 years January Primary: IHCA rate, IH-
in NSW (total teria, educa- 2007– CA-related mortality
64

9,799,081 partici- tion, RRT rate, hospital mortali-

Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 4. Summary of included study characteristics (Continued)
pants pre, run-in, Decem- ty rate, FTR, DLMDRG,
post) ber 2013 1-year postdischarge

Library
Cochrane
mortality

Menon 2018 Prospec- India 7495 11,459 226-bed tertiary AWC call- 12 24 April Primary: CAs (non-
tive sin- academic hospital ing criteria, months months 2013– ICU), mortality
gle-cen- (total 18,954 partic- education, March
tre ipants) MET 2016
CBA, re-

Better health.
Informed decisions.
Trusted evidence.
analysed
ITS

AWC: Amrita Warning Criteria; CAs: cardiac arrests; CBA: controlled before-after study; CCOT: Critical Care Outreach Team; CCU: critical care unit; FTR: failure-to-rescue; DLMDRG:
deaths in low mortality diagnostic related groups; NFR: not-for-resuscitation; ED: emergency department; EWS: early warning system; ICU: intensive care unit; ICUON: intensive
care unit outreach nurse; IHCA: in-hospital cardiac arrest; ITS: interrupted time series; LOS: length of stay; MEC: medical emergency calling criteria; MET: medical emergency
team; MEWS: Modified Early Warning Score; NEWS: National Early Warning Score; NR: not reported; NSW: New South Wales; PAR: Patient At Risk score; RCT: randomised controlled
trial; RRS: rapid response system; RRT: rapid response team; SBAR: Situation, Background, Assessment, Response communication tool.

Table 5. TIDieR framework of intervention description

Study EWS Cri- Type Avail- RRS process Training Du- Content Train- Mode Lo- In- Re- Organisational Ar-
ID teria of abili- ra- er/ed- ca- ter- minders structures rest
RRS ty tion uca- tion ven- team
tor tion
mod-
ified

Randomised trials

Priest- PAR Ob- CCOT 24 PAR referral to CCOT Train- 4 Manage- CCOT NR NR NR NR Obtained fund- NR
ley score jec- hours/7 and admitting con- ing for weeks ment of ing for CCOT
2004 tive days sultant, graded re- all ward critical- with require-

Cochrane Database of Systematic Reviews

cri- sponse strategy, sup- nurses ly ill pa- ment for opera-
teria port and advise staff and doc- tients tionalisation by
and or transfer tors, em- and PAR April 2001
con- phasis score
cern on shar-
ing skills
and col-
labora-
tion
65
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 5. TIDieR framework of intervention description (Continued)

Hill- Call- Ob- MET NR Staff education and Stan- 4 Calling NR Lec- NR NR Iden- Management, Ar-
man ing jec- reminders regarding dard- months criteria, tures, tifica- ethics and re- rest

Library
Cochrane
2005 crite- tive calling criteria, stan- ised ed- be- impor- MET tion suscitation team
ria cri- dardised implemen- ucation fore tance of video- badges committee ap- main-
teria tation strategy strate- im- criteria tape, and proval obtained tained
and gy for ple- to iden- book- posters from control in
con- all nurs- men- tify pa- lets with hospitals con-
cern ing and ta- tients list of trol
medical tion at risk, calling hos-

Better health.
Informed decisions.
Trusted evidence.
staff need criteria pitals
to call and
quickly, MET
how to equiv-
call MET alent
to
pre-
exist-
ing
ar-
rest
team

Jed- Call- Ob- CCOT NR Patient showing no Training 8 Assess- NR NR NR NR NR Overseen by NR

dian ing jec- improvement after for ward weeks ment, committee
2016 crite- tive 30 minutes were re- nurses iden-
ria cri- ferred to team, CCOT tifica-
teria assessed using com- tion, and
and posite scoring sys- manage-
con- tem and managed or ment of
cern transferred acute-
ly ill pa-
tients

Cochrane Database of Systematic Reviews

Haeg- NEWS Ob- MET 24 Team-directed im- Training 4 Mea- 2 ex- Inter- NR NR Web- Project manag- NR
dorens and jec- hours/7 plementation strate- for ward hours sure- peri- ac- site, er per hospital
2018 SBAR tive days gy, graded response nurses ment enced tive knowl- to ensure uni-
cri- protocol, maximum and in- prac- train- edge formity
teria waiting times, con- terpre- tising ing test for
and tact numbers, back- tation nurs- nurses
con- up procedures of vital es
cern signs,
clinical
obser-
vation,
commu-
66
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 5. TIDieR framework of intervention description (Continued)
nication
skills,

Library
Cochrane
and
practical
tips and
tricks in
handling
NEWS
and

Better health.
Informed decisions.
Trusted evidence.
SBAR

Non-randomised studies

LighthallCall- Ob- Physi- NR eTeam paged and In-ser- 4 NR NR NR NR NR Posters Steering com- Ar-
2010 ing jec- cian-led, primary care team vice ed- months and mittee of major rest
crite- tive high summoned ucation be- iden- academic and team
ria cri- capa- for nurs- fore tifica- clinical service in
teria bility es, med- oper- tion departments place
and MET ical and ation badge established at
con- surgical hold- criteria, team time
cern depart- ers composition
ments with and operation;
calling eTeam steer-
criteria ing and resus-
citation com-
mittee meet-
ings including
physician and
nurse leaders
to discuss event
rates, provider
conduct, effica-
cy and patient

Cochrane Database of Systematic Reviews

outcomes

Roth- Call- Ob- MET 24- MET activation via Hospi- NR Purpose NR Meet- NR Team Emails, MET imple- Separate
berg ing jec- hour pager tal-wide of team ings, com- posters, mented in ac- 'code'
2012 crite- tive cov- educa- and how email, posi- pocket cordance with team
ria cri- erage tion of to acti- com- tion cards, IHI recommen-
teria by unit vate the mu- and acti- dations, over-
and hos- staff, all team nica- pro- vation sight commit-
con- pital- levels of tion tocol num- tee met biweek-
cern ists manage- boards, ad- ber ly
ment posters just- on all
ments ward
67
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 5. TIDieR framework of intervention description (Continued)
and using tele-
physi- PDSA phones

Library
Cochrane
cians cy-
cles

Aitken MEC Ob- 2-tier 24 ICUON activated via Exten- NR Tailored NR NR NR NR Posters, Project plan de- Re-
2015 jec- RRS hours/7 direct call for con- sive in- sessions lan- veloped, phys- placed
tive (ICUON, days cern, RRT via switch- service for var- yards ical resources MEC
cri- RRT) (ICUON board and pager sys- training ious and organised, poli- sys-

Better health.
Informed decisions.
Trusted evidence.
teria pro- tem for physiological for CCU, stake- screen cies revised; tem
and vided criteria ED, ward holders savers evaluation and and
con- 7–11 nurses, meetings with ar-
cern p.m. consul- stakeholder rest
ser- tants, groups team
vice, regis-
RRT trars
8–5
p.m.,
out-
of-
hours
ser-
vice
11
p.m.–
8
a.m.)

Davis Call- Ob- RRT NR Charge nurse con- RRT ed- Sev- Patient Crit- Con- NR Fre- NR Hospital-wide NR
2015 ing jec- ducts 'rounds' each ucation eral surveil- ical cep- quent- advanced re-
crite- tive shift, algorithm for on pa- hours lance, care tual ly suscitation
ria cri- arrest and non-ar- tient sur- as resusci- physi- mod- mod- training curricu-
teria rest, RRT activa- veillance part tation cians el ified lum based on

Cochrane Database of Systematic Reviews

and tion criteria and RRT and of training, and based 5 key compo-
nurse nurse responds with recogni- an- recogni- RRT/ on nents
or code blue physician tion of nual tion of Code insti-
fam- lead available patient train- deterio- Blue tu-
ily deterio- ing ration nurs- tion-
con- ration es al
cern CQI
data

Ludikhuize
MEWS Ob- RRT NR MEWS > 3, Call physi- Educa- EWS Trained NR NR NR NR Plasti- No details NR
2015 and jec- cian using SBAR, tion to and using cised
SBAR tive Physician responds SBAR stan- hand-
68
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 5. TIDieR framework of intervention description (Continued)
cri- within 30 minutes, each car- 7 dardised held
teria no effect to therapy er months tool kits cards,

Library
Cochrane
and 60 minutes, call RRT be- posters,
con- fore in pa-
cern im- tient
ple- charts,
men- feed-
ta- back
tion session

Better health.
Informed decisions.
Trusted evidence.
and
face-
to-face
com-
muni-
cation

Chen Call- Ob- RRT NR 2-tier response strat- Educa- NR BTF pro- NR NR NR NR NR State-wide gov- NR
2016 ing jec- egy (yellow red red), tion pro- gramme ernment led
crite- tive response strategy gramme 5 component
ria cri- with minimum skills aimed at strategy: EWS,
teria and minimum re- all staff RRS, educa-
and sponse times tion, gover-
nurse nance structure
or and minimum
fam- dataset
ily
con-
cern

Menon AWC Ob- MET NR Dedicated telephone Educa- 1 NR Se- NR NR NR Con- Expert review Not
2018 jec- number, activate via tion and month nior cise by clinicians stat-
tive hospital intercom training physi- pock- and senior ad- ed. In
crite- system, pager to MET sessions cians et MET ministrators de- place

Cochrane Database of Systematic Reviews

ria member, respond × and card, veloped call- pri-
15 mins nurse work- ing criteria or to
ad- flow (AWC) and MET im-
min- hand- process ple-
istra- outs men-
tors ta-
tion

AWC: Amrita Warning Criteria; BTF: Between the Flags; CCOT: critical care outreach team; CCU: coronary care unit; CQI: continuous quality improvement; ED: emergency
department; EWS: Early Warning Score; ICUON: intensive care unit outreach nurse; IHI: Institute for Health Improvement; MEC: Medical Emergency Calling criteria; MET: medical
emergency team; MEWS: Modified Early Warning Score; min: minute; NEWS: National Early Warning Score; NR: not reported; PAR: Patient at Risk score; PDSA: Plan, Do, Study,
69
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews
Act cycle; resus: resuscitation; RCT: randomised controlled trial; RRS: rapid response system; RRT: rapid response team; SBAR: Situation, Background, Assessment, Response
communication tool.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 70
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Table 6. Clinical heterogeneity

Study ID Design EWS trigger Type of RRS Composition
mechanism

Randomised trials

Priestley 2004 Prospective step Aggregated Nurse-led CCOT Led by a nurse consultant with a team of nurses
wedge cluster RCT weighted score supported by physicians when required.

Hillman 2005 Cluster RCT Single parameter Medical-led MET Varied across participating hospitals with study
criteria protocol requiring at least the equivalent of pre-
existing cardiac arrest team consisting of a physi-
cian, and either an ICU or ED nurse.

Jeddian 2016 Step wedge clus- Single parameter Nurse-led CCOT 6 experienced ICU nurses.
ter RCT criteria

Haegdorens Step wedge clus- Aggregated Medical-led MET Physician and nurse.
2018 ter RCT weighted score

Non-randomised studies

Lighthall 2010 Single-centre con- Single parameter Medical-led MET A high capability, physician led e-Team.
trolled before-af- criteria
ter

Rothberg 2012 ITS Single parameter Medical-led MET An ICU nurse, respiratory therapist, intravenous
criteria therapist, and the patient's physician, surgeon or
resident.

Aitken 2015 Quasi-experimen- Single parameter Nurse-led RRT ICU outreach nurse and interprofessional team
tal, single-centre criteria with an after-hours team comprising medical resi-
pretest–post-test dent, ICU junior registrar, ICUON, and a CCU or ED
nurse.

Davis 2015 2-centre con- Single parameter Nurse-led RRT A dedicated ICU nurse, respiratory therapist and
trolled before-af- criteria charge nurse on each inpatient unit.
ter

Ludikhuize 2015 Prospective, prag- Aggregated Medical-led RRT An ICU nurse and a physician.
matic controlled weighted score
before-after multi-
centre study

Chen 2016 ITS Single parameter Medical-led RRT ICU clinicians with advanced life support skills.
criteria

Menon 2018 Prospective, sin- Single parameter Medical-led MET Physician-led team with charge nurse, nursing su-
gle-centre con- criteria pervisor, phlebotomist, respiratory therapist and
trolled before-af- ECG technician.
ter

CCOT: critical care outreach team; CCU: coronary care unit; ECG: electrocardiogram; ED: emergency department; EWS: early warning
system; ICU: intensive care unit; ICUON: intensive care unit outreach nurse; ITS: interrupted time series; MET: medical emergency
team; RCT: randomised controlled trial; RRS: rapid response system; RRT: rapid response team.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 71
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review)
Table 7. Quality assessment summary (ROBINS-I) of non-randomised studies
Study ID D1: confound- D2: selec- D3: inter- D4: intervention deviation D5: missing D6: out- D7: result Overall
ing tion vention data come mea- reporting

Library
Cochrane
classifica- surement
tion

Lighthall 2010 Serious Low Low No information Low Low Low Serious

Rothberg 2012 Serious Low Low No information Low Low Low Serious

Better health.
Informed decisions.
Trusted evidence.
Aitken 2015 Critical Low Low No information Low Low Low Critical

Davis 2015 Critical Low Low No information Low Low Low Critical

Ludikhuize 2015 Serious Low Low No information Low Low Low Serious

Chen 2016 Serious Low Low No information Low Low Low Serious

Menon 2018 Critical Low Low No information Low Low Low Critical

Low risk of bias: comparable to a well-performed randomised trial.

Moderate risk of bias: sound evidence for a non-randomised study but cannot be considered comparable to a well-performed randomised trial.
Serious risk of bias: some important problems; critical risk of bias: too problematic to provide any useful evidence and should not be included in any synthesis.
No information: no information on which to base a judgement about risk of bias.
ROBINS-I: Risk Of Bias in Non-randomised Studies – of Interventions.

Cochrane Database of Systematic Reviews

72
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

APPENDICES

Appendix 1. Search strategies

MEDLINE (Ovid)
Search date: 28 March 2019

includes In-Process & Other Non-Indexed Citations

No. Search terms Results

1 hospital rapid response team/ 662

2 (critical care adj2 outreach).ti,ab. 131

3 (rapid response adj2 (team? or system?)).ti,ab. 963

4 medical emergency team?.ti,ab. 487

5 code team*.ti,ab. 67

6 crash team*.ti,ab. 7

7 medical crisis team?.ti,ab. 2

8 hospital at night.ti,ab. 42

9 patient at risk.ti,ab. 1034

10 "track and trigger".ti,ab. 98

11 or/1-10 2878

12 (warning system? or warning scor* or early warning).ti,ab. 5888

13 exp intensive care units/ 76387

14 emergencies/ 38957

15 emergency medicine/ 12628

16 exp emergency medical services/ 128798

17 exp critical care/ 53880

18 critical illness/ 25433

19 ((critical* or acute*) adj ill*).ti,ab. 51970

20 ((acute or intensive or critical) adj care).ti,ab. 163347

21 emergen*.ti,ab. 360711

22 (clinical* adj deteriorat*).ti,ab. 4310

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 73
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

23 exp monitoring, physiologic/ 163747

24 exp vital signs/ 396027

25 monitor*.ti,ab. 729007

26 vital sign?.ti,ab. 12593

27 or/13-26 1779978

28 12 and 27 2662

29 11 or 28 5339

30 randomized controlled trial.pt. 478827

31 controlled clinical trial.pt. 92995

32 multicenter study.pt. 247591

33 pragmatic clinical trial.pt. 1011

34 (randomis* or randomiz* or randomly).ti,ab. 819388

35 groups.ab. 1894087

36 (trial or multicenter or multi center or multicentre or multi centre).ti. 232146

37 (intervention? or effect? or impact? or controlled or control group? or (be- 8881821

fore adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post
test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or
pseudoexperiment* or evaluat* or time series or time point? or repeated mea-
sur*).ti,ab.

38 non-randomized controlled trials as topic/ 469

39 interrupted time series analysis/ 549

40 controlled before-after studies/ 380

41 or/30-40 9907346

42 exp animals/ 2.2E+07

43 humans/ 1.8E+07

44 42 not (42 and 43) 4563407

45 review.pt. 2494049

46 meta analysis.pt. 98900

47 news.pt. 194141

48 comment.pt. 761592

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 74
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

49 editorial.pt. 485437

50 cochrane database of systematic reviews.jn. 14093

51 comment on.cm. 761534

52 (systematic review or literature review).ti. 127756

53 or/44-52 8188805

54 41 not 53 6954531

55 29 and 54 2207

MEDLINE (Ovid)

Updated search date: 15 May 2020

1 hospital rapid response team/ 297

2 (critical care adj2 outreach).ti,ab. 16

3 (rapid response adj2 (team? or system?)).ti,ab. 326

4 medical emergency team?.ti,ab. 103

5 code team*.ti,ab. (14) 14

6 crash team*.ti,ab. 0

7 medical crisis team?.ti,ab. 0

8 hospital at night.ti,ab. 10

9 patient at risk.ti,ab. 112

10 "track and trigger".ti,ab. 36

11 1 or 2 or 3 or 4 or 5 or 6 or 7 or 8 or 9 or 10 665

12 (warning system? or warning scor* or early warning).ti,ab. 1641

13 exp intensive care units/ 17464

14 emergencies/ 3128

15 emergency medicine/ 2261

16 exp emergency medical services/ 26195

17 exp critical care/ 8463

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 75
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

18 critical illness/ 7433

19 ((critical* or acute*) adj ill*).ti,ab. 9708

20 20 ((acute or intensive or critical) adj care).ti,ab. 30725

21 emergen*.ti,ab. 69753

22 (clinical* adj deteriorat*).ti,ab. 755

23 exp monitoring, physiologic/ 26102

24 exp vital signs/ 33510

25 monitor*.ti,ab. 121303

26 vital sign?.ti,ab. 2892

27 or/13-26 271347

28 12 and 27 839

29 11 or 28 1417

30 randomized controlled trial.pt. 78514

31 controlled clinical trial.pt. 2784

32 multicenter study.pt. 62265

33 pragmatic clinical trial.pt. 844

34 (randomis* or randomiz* or randomly).ti,ab. 155702

35 groups.ab. 316764

36 (trial or multicenter or multi center or multicentre or multi centre).ti. 53468

37 (intervention? or effect? or impact? or controlled or control group? or (be- 1402946

fore adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post
test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or
pseudoexperiment* or evaluat* or time series or time point? or repeated mea-
sur*).ti,ab.

38 non-randomized controlled trials as topic/ 533

39 interrupted time series analysis/ 570

40 controlled before-after studies/ 293

41 or/30-40 1545468

42 exp animals/ 2702941

43 humans/ 2290095

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 76
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

44 42 not (42 and 43) 412846

45 review.pt. 355413

46 meta analysis.pt. 42457

47 news.pt. 12275

48 comment.pt. 158816

49 editorial.pt. 81909

50 cochrane database of systematic reviews.jn. 1362

51 comment on.cm. 158815

52 (systematic review or literature review).ti. 52187

53 or/44-52 998690

54 41 not 53 1109544

55 29 and 54 720

56 limit 55 to (yr="2019 -Current") 163

Embase (Ovid)

Search date: 28 March 2019

No. Search terms Results

1 rapid response team/ 1972

2 (critical care adj2 outreach).ti,ab. 250

3 (rapid response adj2 (team? or system?)).ti,ab. 1492

4 medical emergency team?.ti,ab. 717

5 code team*.ti,ab. 175

6 crash team*.ti,ab. 12

7 medical crisis team?.ti,ab. 2

8 hospital at night.ti,ab. 64

9 patient at risk.ti,ab. 1473

10 "track and trigger".ti,ab. 146

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 77
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

11 or/1-10 4797

12 (warning system? or warning scor* or early warning).ti,ab. 7773

13 intensive care unit/ 135245

14 emergency/ 51406

15 emergency medicine/ 38107

16 emergency health service/ 88646

17 exp intensive care/ 638450

18 critical illness/ 27067

19 ((critical* or acute*) adj ill*).ti,ab. 76379

20 ((acute or intensive or critical) adj care).ti,ab. 237772

21 emergen*.ti,ab. 496642

22 (clinical* adj deteriorat*).ti,ab. 7025

23 exp physiologic monitoring/ 5375

24 vital sign/ 19753

25 monitor*.ti,ab. 993187

26 vital sign?.ti,ab. 24569

27 or/13-26 2162766

28 12 and 27 3803

29 11 or 28 8238

30 randomized controlled trial/ 538031

31 controlled clinical trial/ 458827

32 quasi experimental study/ 5390

33 pretest posttest control group design/ 374

34 time series analysis/ 22533

35 experimental design/ 16656

36 multicenter study/ 209033

37 (randomis* or randomiz* or randomly).ti,ab. 1135425

38 groups.ab. 2593764

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 78
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

39 (trial or multicentre or multicenter or multi centre or multi center).ti. 319997

40 (intervention? or effect? or impact? or controlled or control group? or (be- 11207074

fore adj5 after) or (pre adj5 post) or ((pretest or pre test) and (posttest or post
test)) or quasiexperiment* or quasi experiment* or pseudo experiment* or
pseudoexperiment* or evaluat* or time series or time point? or repeated mea-
sur*).ti,ab.

41 or/30-40 12508639

42 (systematic review or literature review).ti. 152716

43 "cochrane database of systematic reviews".jn. 13125

44 exp animals/ or exp invertebrate/ or animal experiment/ or animal model/ or 25551124

animal tissue/ or animal cell/ or nonhuman/

45 human/ or normal human/ or human cell/ 19467168

46 44 not (44 and 45) 6138161

47 42 or 43 or 46 6302571

48 41 not 47 9604704

49 29 and 48 4269

CENTRAL (in the Cochrane Library)

Search date: 28 March 2019

No. Search terms Results

#1 [mh "hospital rapid response team"] 13

#2 ("critical care" near/2 outreach):ti,ab 7

#3 ("rapid response" near/2 (team* or system*)):ti,ab 43

#4 medical emergency team*:ti,ab 28

#5 code team*:ti,ab 5

#6 crash team*:ti,ab 0

#7 medical crisis team*:ti,ab 0

#8 hospital at night:ti,ab 4

#9 patient at risk:ti,ab 298

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 79
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

#10 track and trigger:ti,ab 7

#11 {or #1-#10} 391

#12 (warning NEXT system* or warning NEXT scor* or early NEXT warning*):ti,ab 173

#13 [mh "intensive care units"] 3233

#14 [mh emergencies] 1095

#15 [mh "emergency medicine"] 243

#16 [mh "emergency medical services"] 3494

#17 [mh "critical care"] 1911

#18 [mh "critical illness"] 1829

#19 ((critical* or acute*) next ill*):ti,ab 5627

#20 ((acute or intensive or critical) next care):ti,ab 14638

#21 emergenc*:ti,ab 18503

#22 (clinical* next deteriorat*):ti,ab 391

#23 [mh "monitoring, physiologic"] 11469

#24 [mh "vital signs"] 35118

#25 monitor*:ti,ab 55068

#26 (vital NEXT sign*):ti,ab 6109

#27 {or #13-#26} 128421

#28 #12 and #27 94

#29 #11 or #28 476

CINAHL

Search date: 28 March 2019

No. Search terms Results

S1 (critical care N2 outreach) 135

S2 (rapid response N2 (team* or system*)) 1,009

S3 medical emergency team* 545

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 80
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

S4 code team* 218

S5 crash team* 16

S6 medical crisis team* 17

S7 "hospital at night" 35

S8 "patient at risk" 1,268

S9 "track and trigger" 66

S10 (warning system* or warning scor* or early warning) 2,017

S11 S1 OR S2 OR S3 OR S4 OR S5 OR S6 OR S7 OR S8 OR S9 OR S10 5,009

S12 S11 Limiters - Exclude MEDLINE records 1,955

S13 S12 Limiters - Published Date: 20160101-20191231 908

S14 PT randomized controlled trial 87,226

S15 PT clinical trial 86,261

S16 PT research 1,957,254

S17 (MH "Randomized Controlled Trials") 80,176

S18 (MH "Clinical Trials") 143,516

S19 (MH "Intervention Trials") 8,000

S20 (MH "Nonrandomized Trials") 389

S21 (MH "Experimental Studies") 23,330

S22 (MH "Pretest-Posttest Design+") 38,767

S23 (MH "Quasi-Experimental Studies+") 13,090

S24 (MH "Multicenter Studies") 105,085

S25 (MH "Health Services Research") 12,992

S26 TI ( randomis* or randomiz* or randomly) OR AB ( randomis* or randomiz* or 247,332

randomly)

S27 TI (trial or effect* or impact* or intervention* or before N5 after or pre N5 post 1,670,088
or ((pretest or "pre test") and (posttest or "post test")) or quasiexperiment* or
quasi W0 experiment* or pseudo experiment* or pseudoexperiment* or evalu-
at* or "time series" or time W0 point* or repeated W0 measur*) OR AB (trial or
effect* or impact* or intervention* or before N5 after or pre N5 post or ((pretest
or "pre test") and (posttest or "post test")) or quasiexperiment* or quasi W0 ex-
periment* or pseudo experiment* or pseudoexperiment* or evaluat* or "time
series" or time W0 point* or repeated W0 measur*)

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 81
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

(Continued)

S28 S14 OR S15 OR S16 OR S17 OR S18 OR S19 OR S20 OR S21 OR S22 OR S23 OR 2,701,725
S24 OR S25 OR S26 OR S27

S29 S13 AND S28 557

WHO International Clinical Trials Registry Platform (ICTRP)

Search date: 28 March 2019

Search terms Results

critical care outreach 0

rapid response team 13

medical emergency team 13

medical crisis team 0

track and trigger 7

warning system 24

warning score 27

early warning 105

Total 189

ClinicalTrials.gov

Search date: 28 March 2019

Field Searches terms

Other terms “critical care outreach” OR “rapid response team” OR “medical emergency team” OR “medical cri-
sis team” OR "track and trigger" OR "warning system" OR "warning score" OR "early warning"

WHAT'S NEW

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 82
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

Date Event Description

29 October 2020 New citation required and conclusions This is the first update of the Cochrane Review originally pub-
have changed lished in 2007. We conducted a new search and updated oth-
er content according to the methodological expectations of
Cochrane Intervention Reviews (MECIR). The review now in-
cludes 11 studies.

HISTORY
Protocol first published: Issue 4, 2005
Review first published: Issue 3, 2007

Date Event Description

15 May 2020 New search has been performed Updated MEDLINE search identified 163 studies which were
screened with no further studies meeting our inclusion criteria
identified.

12 November 2008 Amended Minor changes

22 April 2008 Amended Converted to new review format.

23 May 2007 New citation required and conclusions Substantive amendment

have changed

CONTRIBUTIONS OF AUTHORS
JM led the writing of the protocol and all previous review authors (Fiona Alderdice, Rob Fowler, Atul Kapila, Marianne Moutray) provided
comment and feedback.

For the updated review

JM and PVB screened records for eligibility, extracted data and assessed the certainty of randomised trials.

LR was an independent arbitrator and resolved disagreement.

PVB and LR independently assessed and agreed the quality of non-randomised studies.

JM undertook the analysis, interpreted the results and wrote the review.

PVB and LR contributed to the interpretation of results and writing of the review.

DF provided statistical advice.

DECLARATIONS OF INTEREST
JM: none.

PVB: none.

DF: none.

LR: none.

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 83
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.
 Cochrane Trusted evidence.
Informed decisions.
Library Better health. Cochrane Database of Systematic Reviews

SOURCES OF SUPPORT

Internal sources
• Queens University Belfast, UK

External sources
• The initial review was funded by a Research and Development Office and Health Research Board All Ireland Cochrane Fellowship, UK

DIFFERENCES BETWEEN PROTOCOL AND REVIEW

We updated the review title to reflect international terminology, refined the search strategy to increase precision and updated the eligibility
criteria of study designs to align with Effective Practice and Organisation of Care (EPOC) study design inclusion criteria. This updated
review includes the main author of initial review (JM) with the addition of three new authors (LR, PVB, DF).

We updated the databases searched and the search strategies. We revised the types of outcome measures to differentiate between primary
and secondary outcomes, reflect the change in the use of outcome terms and to include 'composite outcomes' given this was a decision
taken at the initial review and not subsequently clarified. We added a description of the 'Unit of analysis' under the methods section to
reflect Cochrane's recommendations (Higgins 2011) and Cochrane EPOC (EPOC 2017e) for conducting and reporting analysis in reviews and
added a summary of findings table and GRADE ratings, which were not a requirement when we published the original protocol and review.

We revised the data analysis for this updated review from the original protocol. We planned to perform quantitative analysis using an
intention-to-treat analysis, utilising risk ratio (RR) with 95% confidence intervals as our measure of effect for both individual and pooled
effect estimates, but, due to the heterogeneity of outcome measures, this was not possible. We planned to undertake meta-analysis by
computing a pooled RR using a random effects model and adjust study-level findings for the study design effect by adopting the approach
recommended by Cochrane (design effect = 1 + (M – 1) × ICC where M is the mean cluster size and ICC is the intracluster correlation
coefficient), but, due to statistical and clinical heterogeneity across studies, this was not possible.

NOTES
Prior to submission for publication, an updated MEDLINE search was run (see Appendix 1). This search identified 163 studies, which were
screened with no further studies meeting our inclusion criteria. Consequently, the network editor approved publication without an update
search for this update.

INDEX TERMS

Medical Subject Headings (MeSH)

Critical Care; Hospital Mortality; *Hospitalization; *Hospitals; Length of Stay

MeSH check words

Adult; Humans

Early warning systems and rapid response systems for the prevention of patient deterioration on acute adult hospital wards (Review) 84
Copyright © 2021 The Cochrane Collaboration. Published by John Wiley & Sons, Ltd.