Monitoring quality of care: making the most of data
Chris Sherlaw-Johnson, Senior Research Analyst, Nuffield Trust
© Nuffield Trust
A selection of what we are doing at the Nuffield Trust
QualityWatch: http://www.qualitywatch.org.uk/
Evaluating the impact of social action on hospital activity
Evaluating the impact of new primary care services in outer North East London
Understanding the capability of routine data for identifying where people have
experienced severe harm
Providing evidence and tools to support the delivery of Combined Geriatric
Assessment (CGA) in hospitals
Understanding the interface between ambulance services and acute trusts by
using linked data
Using data to understand effective initiatives for the care of older people
© Nuffield Trust
© Nuffield Trust
Monitoring activity
Regulation of services - CQC
Clinical standards and national performance within a clinical
specialty - national audits
Performance within hospitals
Performance within clinical units
Evaluation of new services or procedures, e.g. Vanguards
© Nuffield Trust
What we
want to
measure
Available
data
Indicators
“Not everything that can be counted counts, and not
everything that counts can be counted.” (Einstein)
© Nuffield Trust
20
40
60
80
100
120
140
160
180
0 200 400 600 800 1000 1200 1400
Standardisedratio
Expected admissions
Emergency admissions for patients with fractured
neck of femur by local authority: 2013/14
Source: Hospital Episode Statistics
© Nuffield Trust
Headlines
“NHS hit by new hospital deaths cover up” (Daily Mail)
“2,500 die at horror hospital” (Sunday Express)
“Killing season on NHS wards” (Daily Mail)
“13,000 died needlessly at 14 worst NHS trusts”
“We have 6,000 avoidable deaths every year” (Jeremy Hunt)
© Nuffield Trust
Following up potential concerns
Data
monitoring
Potential
concern Clear false
positive
Can we correct
the data?
Can we enhance the
monitoring
methodology?
No concerns
identified
Investigate
Areas of
improvement
identified
Improvement
actions
© Nuffield Trust
Continuous monitoring tools
Shewhart-type charts (e.g. c-chart, p-chart, g-chart)
Exponentially weighted moving average (EWMA)
Variable life-adjusted display (VLAD)
Cumulative sum (CUSUM)
Sequential Probability Ratio Test (SPRT)
Resetting Sequential Probability Ratio Test (RSPRT)
Scan charts
© Nuffield Trust
0
2
4
6
8
10
12
0 2 4 6 8 10 12
Month
Observed
Expected
Example: Emergency admissions to hospital from 40-
bed nursing home
© Nuffield Trust
0
2
4
6
8
10
12
0 2 4 6 8 10 12
CUSUM
Control limit
Alert
Reset to zero
© Nuffield Trust
Continuous monitoring
Why?
Making use of time series rather than isolated snapshots
Monitoring/evaluating as soon as data become available
Detecting problems or improvements quickly
Could be valuable when data sparse or limited
Uses
Monitoring of outcomes
Evaluation of changes to care delivery
Comparisons with others / against one’s own past history
© Nuffield Trust
Conclusions
In most instances, health service data does not measure quality
of care but tells you where to look for potential problems
If data is there, it can be helpful to use it, even if not entirely
accurate. Using data may provide the biggest incentive to
improving it
Continuous monitoring techniques are valuable when the data
can be regularly updated
Enabling local understanding of methods and their
implementation is highly important

More Related Content

PDF
Learning from the Care Quality Commission
PDF
Real-time monitoring and the data trap
PDF
Evaluating new models of care: Improvement Analytics Unit
PDF
Providing actionable healthcare analytics at scale: Insights from the Nationa...
PDF
Local and national uses of data
PDF
Providing actionable healthcare analytics at scale: Understanding improvement...
PPTX
Cono Ariti: matched control studies
PPTX
Shaping Informatics for Allied Health - Refining our voice
Learning from the Care Quality Commission
Real-time monitoring and the data trap
Evaluating new models of care: Improvement Analytics Unit
Providing actionable healthcare analytics at scale: Insights from the Nationa...
Local and national uses of data
Providing actionable healthcare analytics at scale: Understanding improvement...
Cono Ariti: matched control studies
Shaping Informatics for Allied Health - Refining our voice

What's hot (20)

PPTX
Using the EMR in early recognition and management of sepsis
PPTX
Making surgical practice improvement easy
PDF
Providing actionable healthcare analytics at scale: A perspective from stroke...
PPTX
Professor Tony Marson - International Business Festival 2018
PPTX
Surveillance of social media: Big data analytics
PDF
Elective care conference: the Endoscopy Improvement Programme
PPTX
The Austin Health Diabetes Discovery Initiative: Using technology to support ...
PPTX
Clinician Satisfaction Before and After Transition from a Basic to a Comprehe...
PDF
Effectiveness of the current dominant approach to integrated care in the NHS:...
PDF
Elective Care Conference: developing & implementing an RTT training strategy
PDF
A Smart Alternative For Specimen Collection - Patient Service Center Visits
PDF
Elective care conference: rules recap & effective management of diagnostic wa...
PDF
Clinicians Satisfaction Before and After Transition from a Basic to a Compreh...
PDF
Elective care conference: imaging demand and capacity
PDF
Beyond the Audit: Measuring MedRec Processes for Quality Improvement
PDF
1325 keynote yale_pdf shareable
PDF
Overcoming Big Data Bottlenecks in Healthcare - a Predictive Analytics Case S...
PDF
Elective care conference: recovery planning & trajectory development
PPTX
ED Safety Checklist Masterclass Presentation
PPTX
Medicines Breakthrough Collaborative 1
Using the EMR in early recognition and management of sepsis
Making surgical practice improvement easy
Providing actionable healthcare analytics at scale: A perspective from stroke...
Professor Tony Marson - International Business Festival 2018
Surveillance of social media: Big data analytics
Elective care conference: the Endoscopy Improvement Programme
The Austin Health Diabetes Discovery Initiative: Using technology to support ...
Clinician Satisfaction Before and After Transition from a Basic to a Comprehe...
Effectiveness of the current dominant approach to integrated care in the NHS:...
Elective Care Conference: developing & implementing an RTT training strategy
A Smart Alternative For Specimen Collection - Patient Service Center Visits
Elective care conference: rules recap & effective management of diagnostic wa...
Clinicians Satisfaction Before and After Transition from a Basic to a Compreh...
Elective care conference: imaging demand and capacity
Beyond the Audit: Measuring MedRec Processes for Quality Improvement
1325 keynote yale_pdf shareable
Overcoming Big Data Bottlenecks in Healthcare - a Predictive Analytics Case S...
Elective care conference: recovery planning & trajectory development
ED Safety Checklist Masterclass Presentation
Medicines Breakthrough Collaborative 1

Similar to Monitoring quality of care: making the most of data (20)

PDF
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014
PPTX
MLS13 QI Workshop
PPT
Qipp increasing productivity using existing resources
PDF
Cono Ariti: matched control studies
PDF
Challenges and improvements in diagnostic services across seven day services
PPTX
Measurement for improvement
PDF
Quality system
PPTX
SCIE Investor Presentation January 2017
PPTX
Martin Bardsley: integration and innovation in health
PDF
MODERN REQUIREMENTS OF AN ASSISTED REPRODUCTIVE CENTER OF EXCELLENCE
PDF
Bernie Harrison - Australian Council Healthcare Standards
PPTX
Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...
PPTX
Dan Wellings' HARC presentation
PDF
Martin Bardsley: Introducing QualityWatch
PPTX
Introduction - The Emergency Laparotomy Collaborative
PPT
Sutton Leiden Presentation Dec 2008 Ds4
 
PDF
STARSurg DISCOVER Summary Protocol
PPTX
NIHR partnering with industry
PDF
Heather Dawe: Applications of risk estimation
PPTX
Reading between the lines
Dr Martin Bardsley: Use of Retrospective Matching Methods 30 June 2014
MLS13 QI Workshop
Qipp increasing productivity using existing resources
Cono Ariti: matched control studies
Challenges and improvements in diagnostic services across seven day services
Measurement for improvement
Quality system
SCIE Investor Presentation January 2017
Martin Bardsley: integration and innovation in health
MODERN REQUIREMENTS OF AN ASSISTED REPRODUCTIVE CENTER OF EXCELLENCE
Bernie Harrison - Australian Council Healthcare Standards
Evaluating Service Innovations for Older People - Martin Bardsley, Nuffield T...
Dan Wellings' HARC presentation
Martin Bardsley: Introducing QualityWatch
Introduction - The Emergency Laparotomy Collaborative
Sutton Leiden Presentation Dec 2008 Ds4
 
STARSurg DISCOVER Summary Protocol
NIHR partnering with industry
Heather Dawe: Applications of risk estimation
Reading between the lines

More from Nuffield Trust (20)

PDF
Transforming outpatient services - Nuffield Trust/NHS Improvement Event
PDF
13 reasons to spend more on health and social care
PDF
Energising your workforce in the face of adversity
PDF
Shifting the balance of care: great expectations
PDF
Automation, Employment, and Health Care
PDF
Public perspectives on the NHS and social care
PDF
Evaluation of the Integrated Care and Support Pioneers Programme
PDF
Ensuring success for new models of care
PDF
Effectiveness of the current dominant approach to integrated care in the NHS
PDF
Applied use of CUSUMs in surveillance
PDF
Engaging with data
PDF
New Models of General Practice: Practical and policy lessons
PDF
Coping with change for a sustainable future
PDF
Helen Parker: Lessons from New Zealand
PDF
The Primary Care Home: A solutions focused new care model provider
PDF
Early findings from Collaborate
PDF
Effective governance of multi-site general practice organisations
PDF
Extended Primary Care Access in Southwark
PDF
Arvind Madan, NHS England
PDF
Has clinical commissioning found its voice? GP perspectives on their CCGs
Transforming outpatient services - Nuffield Trust/NHS Improvement Event
13 reasons to spend more on health and social care
Energising your workforce in the face of adversity
Shifting the balance of care: great expectations
Automation, Employment, and Health Care
Public perspectives on the NHS and social care
Evaluation of the Integrated Care and Support Pioneers Programme
Ensuring success for new models of care
Effectiveness of the current dominant approach to integrated care in the NHS
Applied use of CUSUMs in surveillance
Engaging with data
New Models of General Practice: Practical and policy lessons
Coping with change for a sustainable future
Helen Parker: Lessons from New Zealand
The Primary Care Home: A solutions focused new care model provider
Early findings from Collaborate
Effective governance of multi-site general practice organisations
Extended Primary Care Access in Southwark
Arvind Madan, NHS England
Has clinical commissioning found its voice? GP perspectives on their CCGs

Recently uploaded (20)

PPTX
Head Spine trauma assesment and managementATLS Final.pptx
PPT
12.Biomechanical Preparation final (2).ppt
PPTX
Emotional Well Being & Conflict Resolution_VKV.pptx
PPTX
Airway Management Emergency department presentation .pptx
PPTX
Health and wellness (Chapter 7).pptx for dpt
PPTX
Tuberculosis Control Bangladesh Current 2025.pptx
PPTX
ATIBACTERIAL. PENICILLINS AND CEPHALOSPORINS.pptx
PPTX
Writing clinical trial SPIRIT guidelines
PPTX
INTRODUCTION TO BIOLOGY AND THE BRANCHES OF BIOLOGY
PDF
Dental Implants Review : A detailed Review
PPTX
Clinical Placement of the Operation Theatre.pptx
PPTX
BP504T_PGPC_UNIT_II (1) (1)PHARMACOGNOSY AND PHYTOCHEMISTRY 2
PPT
01 Occupational Health and safety General Industry.ppt
PPTX
Anti-depressant ggggghhhhhhhhhhhhhhhhhhh
PPTX
OBSTRUCTIVE SLEEP APNOEA- features and management
PPTX
4. Musculoskeletal X ray For health student
PPTX
Seizures in paediatrics as a big cause of morbidity.pptx
PPTX
"Pancha Mahabhoota, and Tridoshass".pptx
PDF
Chapter 8. HHS Facility Design and Construction _ HHS.gov.pdf
PDF
Mobile Radiography: Techniques, Applications, and Safety Measures
Head Spine trauma assesment and managementATLS Final.pptx
12.Biomechanical Preparation final (2).ppt
Emotional Well Being & Conflict Resolution_VKV.pptx
Airway Management Emergency department presentation .pptx
Health and wellness (Chapter 7).pptx for dpt
Tuberculosis Control Bangladesh Current 2025.pptx
ATIBACTERIAL. PENICILLINS AND CEPHALOSPORINS.pptx
Writing clinical trial SPIRIT guidelines
INTRODUCTION TO BIOLOGY AND THE BRANCHES OF BIOLOGY
Dental Implants Review : A detailed Review
Clinical Placement of the Operation Theatre.pptx
BP504T_PGPC_UNIT_II (1) (1)PHARMACOGNOSY AND PHYTOCHEMISTRY 2
01 Occupational Health and safety General Industry.ppt
Anti-depressant ggggghhhhhhhhhhhhhhhhhhh
OBSTRUCTIVE SLEEP APNOEA- features and management
4. Musculoskeletal X ray For health student
Seizures in paediatrics as a big cause of morbidity.pptx
"Pancha Mahabhoota, and Tridoshass".pptx
Chapter 8. HHS Facility Design and Construction _ HHS.gov.pdf
Mobile Radiography: Techniques, Applications, and Safety Measures

Monitoring quality of care: making the most of data

  • 1. Monitoring quality of care: making the most of data Chris Sherlaw-Johnson, Senior Research Analyst, Nuffield Trust
  • 2. © Nuffield Trust A selection of what we are doing at the Nuffield Trust QualityWatch: http://www.qualitywatch.org.uk/ Evaluating the impact of social action on hospital activity Evaluating the impact of new primary care services in outer North East London Understanding the capability of routine data for identifying where people have experienced severe harm Providing evidence and tools to support the delivery of Combined Geriatric Assessment (CGA) in hospitals Understanding the interface between ambulance services and acute trusts by using linked data Using data to understand effective initiatives for the care of older people
  • 4. © Nuffield Trust Monitoring activity Regulation of services - CQC Clinical standards and national performance within a clinical specialty - national audits Performance within hospitals Performance within clinical units Evaluation of new services or procedures, e.g. Vanguards
  • 5. © Nuffield Trust What we want to measure Available data Indicators “Not everything that can be counted counts, and not everything that counts can be counted.” (Einstein)
  • 6. © Nuffield Trust 20 40 60 80 100 120 140 160 180 0 200 400 600 800 1000 1200 1400 Standardisedratio Expected admissions Emergency admissions for patients with fractured neck of femur by local authority: 2013/14 Source: Hospital Episode Statistics
  • 7. © Nuffield Trust Headlines “NHS hit by new hospital deaths cover up” (Daily Mail) “2,500 die at horror hospital” (Sunday Express) “Killing season on NHS wards” (Daily Mail) “13,000 died needlessly at 14 worst NHS trusts” “We have 6,000 avoidable deaths every year” (Jeremy Hunt)
  • 8. © Nuffield Trust Following up potential concerns Data monitoring Potential concern Clear false positive Can we correct the data? Can we enhance the monitoring methodology? No concerns identified Investigate Areas of improvement identified Improvement actions
  • 9. © Nuffield Trust Continuous monitoring tools Shewhart-type charts (e.g. c-chart, p-chart, g-chart) Exponentially weighted moving average (EWMA) Variable life-adjusted display (VLAD) Cumulative sum (CUSUM) Sequential Probability Ratio Test (SPRT) Resetting Sequential Probability Ratio Test (RSPRT) Scan charts
  • 10. © Nuffield Trust 0 2 4 6 8 10 12 0 2 4 6 8 10 12 Month Observed Expected Example: Emergency admissions to hospital from 40- bed nursing home
  • 11. © Nuffield Trust 0 2 4 6 8 10 12 0 2 4 6 8 10 12 CUSUM Control limit Alert Reset to zero
  • 12. © Nuffield Trust Continuous monitoring Why? Making use of time series rather than isolated snapshots Monitoring/evaluating as soon as data become available Detecting problems or improvements quickly Could be valuable when data sparse or limited Uses Monitoring of outcomes Evaluation of changes to care delivery Comparisons with others / against one’s own past history
  • 13. © Nuffield Trust Conclusions In most instances, health service data does not measure quality of care but tells you where to look for potential problems If data is there, it can be helpful to use it, even if not entirely accurate. Using data may provide the biggest incentive to improving it Continuous monitoring techniques are valuable when the data can be regularly updated Enabling local understanding of methods and their implementation is highly important