F E AT U R E A RT I C L E

Treating the Patient with

Evidence. Can We Get
There From Here?
Ian D. Coulter, PhD
From the UCLA School of Dentistry, Division of Public Health and
Community Dentistry, and Southern California
Evidence-Based Practice Center, RAND

Within medicine there is considerable debate about how much of clinical practice is
evidence-based. The initial estimates by the Office of Technology Assessment in 19791
and 19832 were that only approximately 10% to 20% of medicine could claim to be
evidence-based. As noted by Imrie and Ramey,3 this figure was simply an estimate.4
They further noted that other commentators have given figures as low as 15% for prac-
tices based on any evidence. The problem of establishing any figure is that of first
needing to define what will constitute the evidence.

WHAT IS EVIDENCE?
Imrie and Ramey3 also noted that for some things
One widely used method to define evidence is to the intervention is so self-evident that no evidence
base the definition on the existence of at least one is required from an RCT (eg, blood transfusions,
well-conducted randomized controlled trial. restarting a heart, stopping massive bleeding). In
Through use of that criteria, published studies dentistry, intervening for extensive caries might be
show that an average of 37% of medical interven- considered a self-evident intervention, although
tions are supported by evidence from a random- intervening for incipient caries might be more
ized controlled trial (RCT), but 76% were debatable.
supported by some compelling evidence,3 where
compelling evidence means evidence below that of Part of the challenge in determining
an RCT, such as prospective or comparative the correct rate of evidence-based
studies, retrospective case series, and so on. practice lies in choosing the
Furthermore, the percent varies by the area being focus for the study.
considered. Areas such as ENT surgery, burn
therapy, retinal breaks, lattice degeneration, and For many it is the clinical procedure itself.5 Under
pediatric surgery3 are not strongly based on RCT this approach, Dubinsky and Ferguson6 concluded
evidence. Some areas such as psychiatric interven- that 21% of the technologies for medicine were
tions (65% supported by an RCT) and general prac- based on evidence. However, a second method is to
tice interventions (53% supported by an RCT) have calculate how many patients receive interventions
studies showing a strong evidence base. based on evidence for commonly used therapies

J Evid Base Dent Pract 2002;2:83-88

© 2002 Mosby, Inc. All rights reserved.
1532-3382/2002/$35.00 + 0 77/1/123018
doi:10.1067/med.2002.123018

83
and for common conditions. In one such study in a usually established by proponents of evidence-
department of internal medicine,4 the result was based practice (EBP), expert opinion is given the
that 21% of the interventions could be supported by lowest rating.12 While this is more pertinent to the
placebo-controlled randomized trials and a further opinion of individual experts, the status of expert
44% by head-to-head trials. The latter method is panel knowledge as evidence is also subject to
probably of more use to practitioners in that it disagreement. Hemingway et al13,14 have shown
better estimates the actual care given in a practice that in a prospective study of coronary revascular-
that is evidence-based. ization procedures, they found the indications for
appropriateness of the expert panel had clinical
DETERMINING WHAT EVIDENCE TO USE IN PRACTICE validity in measuring underuse of the procedure
The use of a single RCT—no matter how good the and that underuse was associated with adverse
study—does pose some problems if it is the only clinical outcomes. They also found that the ratings
basis for making a decision. Single studies can be were predictive of the number of diseased vessels,
contradicted by later studies. By its nature, a RCT helped identify patients at high risk of mortality,
tests a procedure or therapy under ideal conditions and were associated with subsequent revasculariza-
and therefore may have limited relevance for actual tion rates.15
practice because of the feasibility of applying the
practice in a real setting. It deals with efficacy, not • First, the recommendations can be applied to
effectiveness. nearly all patients and not just to highly
To overcome the problem of a single study, some selected patients represented in trials.
have suggested that only a systematic literature • Second, a panel might be better than an indi-
review should form the basis for judging whether a vidual practitioner in assessing evolving
procedure, or a treatment, is evidence-based. A results of trials.
review has the advantage that it is based on a body • Third, the mixed nature of specialists on a
of work and incorporates, for the most part, a panel guards against the referral bias of the
method for evaluating the quality of the studies. specialists (“referring patients to them-
selves”).
However, a hierarchy of evidence may • Fourth, because the panel focuses on appro-
relegate to a lower status weaker priateness without reference to cost or expe-
study designs that have greater clin- diency, the ratings represent “pure” clinical
ical significance and greater feasibility decisions.
in being applied in clinical practice.7
There are several advantages of using the indica-
Where a meta-analysis is done, the pooling of tions of the expert panel, as aforementioned.
data provides much stronger evidence for effi- However, as Anderson and Brown16 noted, the
cacy.8,9 It should be noted, however, that examples problem with expert panel indications for appropri-
of misleading meta-analysis have already been ateness is there is no way of knowing whether they
documented in the literature.10 Furthermore, are linked to evidence as there is, for example, in the
studies with negative results are less likely to be development of clinical guidelines. In developing
published—very often resulting in an overestimate guidelines the expert panel indicates for each guide-
of benefit. line the level of evidence supporting it. Therefore,
There is also the question of what role is to be any health provider reading such a guideline knows
given to the conclusions generated by expert the strength of the evidence and can judge those that
panels. What value do their pronouncements have have greater salience for practice.
as evidence? Where the opinion of experts has been
included, as in one study of internal medicine,11 the TREATING THE PATIENT WITH EVIDENCE. CAN WE
rate of appropriate care increased drastically. In this GET THERE FROM HERE?
study, 50% of the diagnosis/intervention combina- The dilemma arises from the fact that case studies
tions had the support of a RCT and an additional are the lowest rank for evidence within EBP and
34% were supported by consensus opinions of inde- have been critiqued within such fields as ethics as
pendent experts for a combined rate of more than being potentially very misleading.18 As noted by
80%. However, in the hierarchies of evidence Godlee,17 the “research literature is poorly orga-

84 Coulter Journal of Evidence-Based Dental Practice

March 2002
 One of the dilemmas of trying to create Table 1. Questions for assessing the quality of the
study itself21
EBP is that clinical practice is essen-
tially case-based.17
Question 1: Was the study original?

nized, largely of poor quality and irrelevant to clin- Question 2: Who is the study about?
ical practice, often conflicting, and often not there at • How were they recruited?
all.” Even when it does exist, it may be on a patient
• Who was included?
sample quite dissimilar from the one treated by the
provider. In dentistry, a large amount of the • Who was excluded?
published work on caries prevention and treatment Question 3: Was the design sensible?
has been done on children. Its relevance as a guide to • What was the intervention and what was it compared
treating adults and the elderly may be questionable. with?
• What outcome was measured and how?
DEVELOPING AN EVIDENCE-BASED CLINICAL
PRACTICE Question 4: Was systematic bias avoided or minimized?

Let us begin with the assumption that you want Question 5: Was the assessment blind?
to make your practice evidence-based. As noted by Question 6: Were preliminary statistical questions dealt with?
one commentator, “having the evidence in your
• Sample size?
hand is just a start—but a good one.”19 There are a
variety of ways for a health provider to obtain such • Duration of follow up?
evidence, including the following: • Completeness of follow-up?
• online access through such Internet portals
as MEDLINE
• purchasing evidence-based journals such appropriate for a practice in which no patients
as the Journal of Evidence-Based Dental are at high risk?
Practice 3. The next step is to distinguish “signal from
• consulting primary research journals such noise.”7 By signal is meant, “is the question being
as the Journal of Dental Research. studied clinically significant?” Noise refers to the
However, this presupposes the ability of the reader design of the study. As Edwards et al7 noted, a
to critically assess the studies. With published well-designed study in a systematic review will
research results of RCTs, what criteria should the be given more weight than a study whose clin-
provider use to determine whether the study ical significance is greater but whose design is
should inform the treatment of their patients, if it is not so strong. They suggest that signal can be
appropriate? determined by noting the effect size, the rele-
There are several questions that can be posed before vance of the findings, and the applicability to a
even reviewing the quality of the study. clinical problem.
1. The first question should be: Is the benefit from 4. If a study passes these initial assessments, then
the treatment studied sufficiently large in the next question is the quality of the study itself.
comparison with the risks to make it even worth Greenhalgh21 has provided a set of questions for
considering?20 For some treatments, both the answering this that are presented in the Table in
cost and the patient’s preference/values may summary form.
also factor into this decision.
2. Assuming that a RCT has shown a considerable However, conducting the critical appraisal your-
benefit, the next question should be “Is it self is not necessary. Few providers would have the
applicable to my patients?” Is the patient popu- time to do this extensively; therefore, it is possible
lation in the study of sufficient similarity to the to rely on preappraised work.22 Numerous publica-
provider’s patients? For example, if the study tions exist now that do the critical appraisal for the
shows only sufficient benefit for patients who provider, with JEBDP being one such example. In
are at high risk for a disease but modest to low Great Britain, the National Health Service provided
benefit for those not at high risk, would it be 33,000 general practitioners in England with copies

Journal of Evidence-Based Dental Practice Coulter 85

Volume 2, Number 1
of Clinical Evidence and 400,000 doctors in America moderate periodontal involvement with no pain,
also received free copies.19 sharp pain or dull throbbing pain; or severe peri-
However, all of this presupposes a provider who odontal involvement with no pain, sharp pain, or
will still go to considerable effort to read material dull throbbing pain. The next category could be a
in a format that is still very removed from the patient 35 to 55 years who is an irregular user of
reality of practice. A new format has emerged dental care (with the same subcategories, and so on).
called the evidence-based case report, which is now to The objective is to create indications that include
be found in the British Medical Journal and its adequate detail so that patients characterized in a
American journal, the Western Journal of Medicine given category are relatively homogeneous but also
(now the WJM).17 Here a case report is presented to ensure that the therapy/procedure would be
that illustrates how evidence can be used in appropriate for all patients who would meet the
patient care in a particular case. In a sense this is criteria identified in the indication. The procedure
EBP in action. The advantage is that the situation should be comprehensive enough to include all the
has a degree of realism that a provider can identify indications for doing the procedure that occur in
with (they are real cases). Their intent is to illus- practice. The indications should be detailed,
trate the process. To create them, the authors are comprehensive, and manageable.26 However, they
asked to search the literature for studies, apply the may be considerable in number (up to 4000 in some
information to a case, audit the outcome, and instances). They also try to cover every logical
write up the report. An added bonus of this combination of risk factor and illness.
approach might be that the clinician can also see if This gives them an advantage over the previous
the EBP management of the case did result in a evidence in that the probability of a provider iden-
better outcome for a patient. tifying patients of his/her own who meet the indi-
Glasziou et al,23 in looking at using evidence from cation are quite high. Whereas a particular clinical
both systematic reviews and trials, ask 4 questions: trial might not include either the type of patient
1. Is my patient different from those in the study? commonly seen in practice or the type of therapy
2. Is the treatment feasible in my clinic? commonly used, the indications for appropriate-
3. What are the likely benefits and harms from the ness developed by expert panels are much more
treatment? likely to resemble cases seen in practice (and the
4. How will my patient’s values influence the deci-
sion?
The expert panels provide a different type of
This has some justification in
evidence, but some of the same questions can be that a patient has the right, and
asked of them. First, was a systematic literature possibly the expectation, that the
review, particularly a meta-analysis, done before health provider will inform her of
the panel and provided to the members? Although the most appropriate care for
this is not proof the panel members actually based a given health problem.
their ratings of the appropriateness of procedures
on evidence, it does ensure that at least the evidence
was available. therapy) and to provide an extensive range of cases
In a RAND expert panel,24 a comprehensive set of with slight variations that make a difference in
indications is created for sub-categories of patients treatment (the age of the patient, irregular versus
that is logically exhaustive. That is, the set of indi- regular use of care, high risk for caries, high versus
cations should allow for all the logical sets within a low previous caries, and so on). Naylor27 noted that
subcategory of patients. The indications categorize where trials are useful in telling a clinician what to
patients in terms of their symptoms, history, diag- do in practice, expert panels help determine what
nosis, stage of disease, and so on. they should not do and to identify uncertain indi-
So, for example, a category may refer to a patient cations and, therefore, areas for future research.27
35 to 55 years of age who is a regular user of dental However, this view ignores the fact that such panels
care.25 Then, within this category, it can be a patient do provide extensive indications for what a clini-
with advanced caries in a patient at high risk for cian should also do over a much broader range of
caries with no or minimal periodontal involvement possible categories of patients than do trials. The
with no pain, sharp pain, dull throbbing pain or latter, by necessity, are only relevant to the partic-

86 Coulter Journal of Evidence-Based Dental Practice

March 2002
ular category of patients meeting the inclusion of recommending the care that is the most appro-
criteria, which, given the demands and costs of a priate. Professional autonomy means the autonomy
trial, is likely to be fairly restrictive. to decide what is best professionally and to recom-
mend it to the patient—and that patient autonomy
CONCLUSION means the right to decide if they will accept the
Quite clearly the data concerning appropriate recommendation.
care can help provide EBP in practice if it can be
brought to bear on real clinical problems. What I REFERENCES
have described in this article, however, might be 1. US Congress, Office of Technology Assessment. Assessing the
termed the judgment of appropriateness as determined efficacy and safety of medical technologies. Washington, DC:
by the health professional (researchers, expert panels, Congress, Office of Technology Assessment; September 1978.
2. US Congress, Office of Technology Assessment. The impact of
and, ultimately, the health care provider). randomized clinical trials on health care policy and medical
However, there is some question about whether practice. Washington, DC: Congress, Office of Technology
the patient actually wants the health professional to Assessment; August 1983. OTA-BP-H-22.
3. Imrie R, Ramey DW. The evidence for evidence-based medi-
make the final choice. Studies have shown that cine. Complement Ther Med 2000;8:123-6.
patients and their health providers often have quite 4. Michaud G, McGowan JL, van der Jagt R, Wells G, Tugwell P.
different conceptions of what is appropriate care.28 Are therapeutic decisions supported by evidence from health
care research? Arch Intern Med 1998;158:1665-8.
Furthermore, in real practice, the patient values (eg, 5. Ellis J, Mulligan I, Rowe J, Sackett DL. Inpatient medicine is
concerning the economical factors) may play a large evidence based. A-Team, Nuffield Department of Clinical
role in the final decision. This has led some Medicine. Lancet 1995;346:407-10.
6. Dubinsky M, Ferguson JH. Analysis of the National Institutes
commentators to argue that the expert panels of Health Medicare coverage assessment. Int J Technol Assess
should incorporate patient values and economic Health Care 1990;6:480-8.
factors. As Sharpe and Faden28 noted, in clinical 7. Edwards AGK, Russell IT, Stott NCH. Signal versus noise in
the evidence base for medicine: an alternative to hierarchies of
practice the recommendation of a evidence? Fam Prac 1998;15:319-22.
procedure/therapy will depend on the patient 8. Lau J, Ioannidis JP, Schmid CH. Quantitative synthesis in
profile, the physician’s (and team’s) skill, the systematic reviews. Ann Intern Med 1997;127:820-6.
9. Bland CJ, Meurer LN, Maldonado G. A systematic approach
quality of the supporting evidence, and the proce- to conducting a non-statistical meta-analysis of research liter-
dures benefit/risk ratio. ature. Acad Med 1995;70:642-53.
In a sense this simply reminds us that clinical 10. Egger M, Smith GD. Misleading meta-analysis [editorial]. Br
Med J 1995;310:752-4.
practice is complex and at any given time, a host of 11. Nordin-Johansson A, Asplund K. Randomized controlled
factors may go into making clinical recommenda- trials and consensus as a basis for interventions in internal
tions. But given that, there is a place for what might medicine. J Intern Med 2000;247:94-104.
12. Tonelli MR. In defense of expert opinion. Acad Med
be termed a pure judgment: 1999;74:1187-92
13. Hemingway H, Crook AM, Dawson JR, Edelman J,
Edmondson S, Feder G, et al. Rating the appropriateness of
• where it is assumed that the patient can coronary angiography, coronary angioplasty and coronary
afford the procedure29 artery bypass grafting: the ACRE study. J Public Health Med
• where the benefit is worth it, irrespective of 1999;21:421-9.
14. Hemingway H, Crook AM, Feder G, Banerjee S, Dawson JR,
the costs, Magee P, et al. Underuse of coronary revascularization proce-
• where the clinician is skilled in performing dures in patients considered appropriate candidates for revas-
the therapy, cularization. N Engl J Med 2001;344:645-54.
15. Hemingway H, Crook AM, Banerjee S, Dawson JR, Feder G,
• where the setting and the services, Magee PG, et al. Hypothetical ratings of coronary angiog-
including the supportive services and raphy appropriateness: are they associated with actual angio-
professionals, are appropriate. graphic findings, mortality, and revascularization rate? The
ACRE study. Heart 2001;85:672-9.
16. Anderson GM, Brown AD. Appropriateness ratings: overuse,
underuse, or misuse [commentary]? Lancet 2001;358:1475-6.
In this way we can at least identify what should 17. Godlee F. Applying research to individual patients. Evidence
constitute best practice even if also recognizing that based case reports will help [editorial]. Br Med J
1998;316:1621-2.
they may be difficult to achieve in practice. It is also 18. Pattison S, Dickerson D, Parker M, Heller T. Do case studies
a way of meeting the professional responsibilities. mislead about the nature of reality? J Med Ethics 1999;25:42-6.
While the patient may have the right to choose the 19. Barton S. Using clinical evidence. Having the evidence in
your hand is just a start—but a good one [editorial]. Br Med
therapy they wish to be subjected to, a health J 2001;322:503-4.
professional should have the ethical responsibility 20. Sheldon TA, Guyatt GH, Haines A. Getting research findings

Journal of Evidence-Based Dental Practice Coulter 87

Volume 2, Number 1
 into practice. When to act on the evidence. Br Med J manipulation and mobilization of the cervical spine. Top
1998;317:139-42. Clin Chiro 1995;2:54-62.
21. Greenhalgh T. Assessing the methodological quality of 25. Coulter ID, Marcus M, Freed JR. Consistency across panels
published papers. Br Med J 1997;315:305-8 of ratings of appropriateness of dental care treatment proce-
22. Guyatt GH, Meade MO, Jaesche RZ, Cook DJ, Haynes RB. dures. Community Dent Health 1998;15:97-104.
Practitioners of evidence based care. Not all clinicians need 26. Brook RH. The RAND/UCLA appropriateness method. In
to appraise evidence from scratch but all need some skills McCormack KA, Moore SR, Siegel RA, editors. Clinical prac-
[editorial]. Br Med J 2000;320:954-5. tice guideline development. Methodology perspectives. US
23. Glasziou P, Guyatt GH, Dans AL, Dans LF, Straus S, Department of Health and Human Services, AHCPR; 1994.
Sackett DL. Applying the results of trials and systematic p. 59-65.
reviews to individual patients [editorial]. ACP J Club 27. Naylor CD. What is appropriate care [editorial]? N Engl J
1998;129:A15-16. Med 1998;338:1918-20.
24. Coulter ID, Shekelle PG, Mootz RD, Hansen DT. The use of 28. Sharpe VA, Faden AI. Appropriateness in patient care: a new
expert panel results: the RAND panel for appropriateness of conceptual framework. Milbank Q 1996;74:115-38.

88 Coulter Journal of Evidence-Based Dental Practice

March 2002