3 Frequent Mistakes in Healthcare Data Analytics

3 Frequent Mistakes in Healthcare
Data Analytics
By John Wadsworth
© 2014 Health Catalyst
www.healthcatalyst.com Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.
www.healthcatalyst.com
Proprietary. Feel free to share but we would appreciate a Health Catalyst citation.

Healthcare Data Analytics
Health systems and the healthcare
industry in general are exploring the
possibilities of healthcare data
analytics.
Big Data, population health
management, or accountable care
are all hot topics, viewed by many as
tenets for healthcare reform.
Underlying each of these themes is
the concept of analytics. Without
analytics, it is difficult (if not
impossible) to manage population
health effectively or determine how
risk should be shared.

Data Analytics Mistake 3
Visualizations
Healthcare Data Analytics
While healthcare analytics is critical, it is
important to note that there is no such thing
as a magic bullet for analytics. It is surprising
how often healthcare organizations view
analytics to cure all their woes.
Generally there are three common mistakes
that consistently plague analytic endeavors.
Analytics
Whiplash
Coloring the
Truth
Deceitful

Analytics Whiplash
1
Every master fly fisherman knows this very important fact:
A fish needs some time to study the fly. It must be confident
that if it is going to make the effort to chase a fly, it will catch
it. For the fish to get comfortable, a fisherman must
flawlessly present the fly on the water.
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
River guides call it “Perfect
presentation,” a technique that
every good fisherman takes the
time to master.
Far too many fishermen make the
mistake of impatiently not giving
fish sufficient time to study the fly.
Hurriedly, they abandon what
might be the perfect fishing-hole
on a river bend.

1
Like the impatient fly fisherman many data analysts and BI developers often feel like
they are being whiplashed from one analysis to the next. The cycle goes like this:
Analytics Whiplash
Mgmt
requests
problem
analysis
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
Management engages a data analyst
to begin studying a problem and just
when the analysis is beginning to
bear fruit, management wants to
move on to the next problem.
It is frustrating for an analyst to feel
leadership is impatiently casting to
and fro without allowing the analyst
enough time to firmly and fully define
a problem that could yield a big catch
in the way of process improvement.
Analyst
begins to
grasp
problem
Analyst
assigned
to study
problem
Analyst
directed to
a new
problem

1
Analytics projects are most successful when the analyst can follow an iterative process through cycles of analysis,
measurement, adjustment, followed by more evaluation and readjustment to zero in on specific process improvements.
Analytics Whiplash
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
Producing half-baked analyses
will lead senior management
and/or process owners to be
uncertain about whether they
can trust the information.
Adding more analysts may only
compound the issue because
the organization will have a
larger capacity to generate even
more incomplete analysis.

Analytics Whiplash
MISTAKES OF
1
HEALTHCARE
DATA ANALYTICS
Prioritization from Leadership Is Key
Leadership needs to become proficient with prioritization. Not everything can be priority number one. Furthermore, analysts shouldn’t be put into a position of determining what comes first. That’s a function of leadership.
If they haven’t already done so,
senior management must
collectively take a step back as
a group, determine which
projects have the highest value
(as well as which can wait), and
then commit to seeing the
highest-value projects through
to completion – even if a shiny
new object comes along.

Coloring the Truth
2
Which is more important — telling senior management what they want to hear or reporting bad news accurately?
In the short run, telling your senior leadership what you think they want to hear may seem like the easier path.
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
However, it won’t help them
make meaningful quality
improvements within the
hospital.
It will ultimately engender
mistrust in IT and analytics.

2
An example of this type of flawed thinking is illustrated below. Up until 2014, Acute care hospitals were required by the Centers for Medicare and Medicaid Services (CMS) to report on Central Line-Associated Bloodstream Infections (CLABSIs) acquired during a patient’s
hospital stay.
Coloring the Truth
Under that regulation, only ICUs within the hospitals were required to report CLABSI rates to CMS. As CLABSI rates influenced future reimbursement from CMS, hospitals kept a very close watch on those rates.
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
Example Scenario

2
One day you discover that what you have been reporting as CLABSI incidence is limited only to the ICU, but you have now discovered an alarming number of CLABSIs occur outside of the ICU.
Coloring the Truth
Imagine for a moment that you are an analyst tasked with outcomes reporting for CLABSI.
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
Example Scenario
In your mind, you begin to weigh the pros
and cons of sharing this information.
You tell yourself there is no future
financial penalty associated with these
incidents (from a payer perspective).
You remind yourself that the hospital is
not required to report on these newly
identified cases.

Coloring the Truth
Example Scenario
MISTAKES OF
2
HEALTHCARE
DATA ANALYTICS
So, what do you do?
Do you go ahead and share this new information with senior leadership? You find yourself repeatedly asking yourself how you think they, the senior leadership, will respond to this kind of bad news. At length, you decide to let a sleeping dog lie.
You figure that because there is no
financial imperative and that leadership
may not react well to the news, it’s just
better that they don’t know.
“What they don’t know, won’t hurt ‘em”
you tell yourself.

Coloring the Truth
2
Analysts Must Be Comfortable Sharing Bad News
This scenario has played out with many hospital systems. It’s symptomatic of a much bigger problem. In effect, is senior leadership unknowingly incentivizing analysts to lie to them?
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
To incent analysts to act as real analysts, senior
management has to be willing to hear the bad
news with the good and include all the data.
If the goal is to use data analytics to become a
high-performing organization, you need to take
all of the data into consideration. It may cause
uncomfortable or even painful moments early-on,
but it will be much more effective in helping
the organization become a high-performing,
data-driven health system.

Deceitful visualizations
3
Politicians and the media often
presents good data with deliberately
misleading visualizations, thereby
violating Edward Tufte’s six
principles of graphical integrity.
This manipulates the public to see
what they want them to see instead
of what’s really there.
MISTAKES OF
HEALTHCARE
DATA ANALYTICS

3
For example, the image below (courtesy of Political Math) shows the
difference between an accurate scale and a misleading scale. The
graph on the left is deceptive because it gives the impression that there
was a large increase in deductibles over two years.
Deceitful visualizations
MISTAKES OF
HEALTHCARE
DATA ANALYTICS
The average family deductible increased 30%
in two years, from $1,034 to $1,344. This effect
is more pronounced for small firms, where PPO
deductibles increased from $1,439 to $2,367.
- A rise of 64%
The average family deductible increased 30%
in two years, from $1,034 to $1,344. This effect
is more pronounced for small firms, where PPO
deductibles increased from $1,439 to $2,367.
- A rise of 64%
The reality, shown on the
right, is a more modest
increase.
While both graphs show
accurate information, the
scale has been manipulated
to give a different
impression.
This practice is a disservice
to the data and to the
organization.
WRONG RIGHT

When data analysts are empowered to spend time with the data
and be open and honest with senior leadership and use accurate,
truthful visualizations — it can make the difference between failing
and thriving in a value-based healthcare environment.
Empowered Healthcare Data Analysts
Lead to Usable Healthcare Analytics

Healthcare Analytics Adoption Model
A framework and guide for assessing and implementing healthcare analytics in an
organization
More about this topic
Using Healthcare Analytics for Improvement Projects: Where to Start
Eric Just, Vice President, Technology
Getting the Most Out of Your Data Analysts
Russ Staheli, Vice President, Analytics
The Best System for Healthcare Analytics Is Not a Point Solution
Ken Trowbridge, Vice President
Advanced Healthcare Analytics Case Study: Improving Appendectomy Care
A Success Story from Texas Children’s Hospital

For more information:

Other Clinical Quality Improvement Resources
Click to read additional information at www.healthcatalyst.com
John Wadsworth joined Health Catalyst in September 2011 as a senior
data architect. Prior to Catalyst, he worked for Intermountain Healthcare
and for ARUP Laboratories as a data architect. John has a Master of
Science degree in biomedical informatics from the University of Utah,
School of Medicine.

3 Frequent Mistakes in Healthcare Data Analytics

More Related Content

What's hot (20)

Viewers also liked (14)

Similar to 3 Frequent Mistakes in Healthcare Data Analytics (20)

More from Health Catalyst (20)

Recently uploaded (20)

3 Frequent Mistakes in Healthcare Data Analytics