Archived version from NCDOCKS Institutional Repository http://libres.uncg.

edu/ir/asu/

What Is Clinical Evidence In Speech-Language Pathology?

A Scoping Review
By: Schea Fissel Brannick, George W. Wolford, Laura L. Wolford,
Kayleigh Effron, and Jennifer Buckler

Abstract
Purpose: Two disparate models drive American speech-language pathologists' views of evidence-based practice (EBP): the
American Speech-Language-Hearing Association's (2004a, 2004b) and Dollaghan's (2007). These models discuss evidence
derived from clinical practice but differ in the terms used, the definitions, and discussions of its role. These concepts, which we
unify as clinical evidence, are an important part of EBP but lack consistent terminology and clear definitions in the literature.
Our objective was to identify how clinical evidence is described in the field. Method: We conducted a scoping review to
identify terms ascribed to clinical evidence and their descriptions. We searched the peer-reviewed, accessible, speech-
language pathology intervention literature from 2005 to 2020. We extracted the terms and descriptions, from which three
types of clinical evidence arose. We then used an open-coding framework to categorize positive and negative descriptions
of clinical expertise and summarize the role of clinical evidence in decision making. Results: Seventy-eight articles included a
description of clinical evidence. Across publications, a single term was used to describe disparate concepts, and the same
concept was given different terms, yet the concepts that authors described clustered into three categories: clinical opinion,
clinical expertise, and practice-based evidence, with each described as distinct from research evidence, and separate from
the process of clinical decision making. Clinical opinion and clinical expertise were intrinsic to the clinician. Clinical opinion was
insufficient and biased, whereas clinical expertise was a positive multidimensional construct. Practice-based evidence was
extrinsic to the clinician—the local clinical data that clinicians generated. Good clinical decisions integrated multiple sources of
evidence. Conclusions: These results outline a shared language for SLPs to discuss their clinical evidence with researchers,
families, allied professionals, and each other.

Brannick, S., Wolford, G., Wolford, L., Effron, K., & Buckler, J. (2022). What Is Clinical Evidence in Speech-Language
Pathology? A Scoping Review, American Journal of Speech-Language Pathology. Vol. 31, issue 6, November 2022, pages
2943-2958. Publisher version of record available at: https://doi.org/10.1044/2022_AJSLP-22-00203
Review Article

What Is Clinical Evidence in Speech-Language

Pathology? A Scoping Review
Schea Fissel Brannick,a George W. Wolford,a,b Laura L. Wolford,a,c Kayleigh Effron,a
and Jennifer Bucklera
a
Speech-Language Pathology Program, Midwestern University, Glendale, AZ b Department of Rehabilitation Sciences, Beaver College of Health
Sciences, Appalachian State University, Boone, NC c Department of Communication Sciences and Disorders, MGH Institute of Health
Professions, Boston, MA

ARTICLE INFO ABSTRACT

Article History: Purpose: Two disparate models drive American speech-language pathologists’
Received June 21, 2022 views of evidence-based practice (EBP): the American Speech-Language-
Revision received August 3, 2022 Hearing Association’s (2004a, 2004b) and Dollaghan’s (2007). These models
Accepted August 4, 2022 discuss evidence derived from clinical practice but differ in the terms used, the
definitions, and discussions of its role. These concepts, which we unify as clini-
Editor-in-Chief: Erinn H. Finke cal evidence, are an important part of EBP but lack consistent terminology and
Editor: Holly L. Storkel clear definitions in the literature. Our objective was to identify how clinical evi-
dence is described in the field.
https://doi.org/10.1044/2022_AJSLP-22-00203 Method: We conducted a scoping review to identify terms ascribed to clinical
evidence and their descriptions. We searched the peer-reviewed, accessible,
speech-language pathology intervention literature from 2005 to 2020. We
extracted the terms and descriptions, from which three types of clinical evidence
arose. We then used an open-coding framework to categorize positive and nega-
tive descriptions of clinical expertise and summarize the role of clinical evidence
in decision making.
Results: Seventy-eight articles included a description of clinical evidence.
Across publications, a single term was used to describe disparate concepts,
and the same concept was given different terms, yet the concepts that authors
described clustered into three categories: clinical opinion, clinical expertise, and
practice-based evidence, with each described as distinct from research evi-
dence, and separate from the process of clinical decision making. Clinical opin-
ion and clinical expertise were intrinsic to the clinician. Clinical opinion was
insufficient and biased, whereas clinical expertise was a positive multidimen-
sional construct. Practice-based evidence was extrinsic to the clinician—the
local clinical data that clinicians generated. Good clinical decisions integrated
multiple sources of evidence.
Conclusions: These results outline a shared language for SLPs to discuss their
clinical evidence with researchers, families, allied professionals, and each other.
Clarification of the terminology, associated definitions, and the contributions of
clinical evidence to good clinical decision-making informs EBP models in
speech-language pathology.
Supplemental Material: https://doi.org/10.23641/asha.21498546

Correspondence to Schea Fissel Brannick: [email protected]. Schea Fissel Brannick is an Associate Professor at Midwestern University. George
W. Wolford was under contract as an Assistant Professor at Appalachian State University while revising this review article. Laura L. Wolford was
under contract as an Assistant Professor in the Department of Communications Sciences and Disorders, MGH Institute of Health Professions,
Boston, MA while revising this article. Kayleigh Effron is currently a practicing speech-language pathologist working in the greater Phoenix, AZ,
area. Schea Fissel Brannick and George W. Wolford equally share the first author position, as each contributed essential and substantive ideas,
content, and writing efforts to the development and submission of this article. Disclosure: The authors have declared that no competing financial or
nonfinancial interests existed at the time of publication.

American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022 • Copyright © 2022 The Authors 2943
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
 Recommendations from the preeminent models of across practitioners over time, making “individual clinical
evidence-based practice (EBP) in speech-language pathology expertise” very difficult to define, measure, or appraise.
(American Speech-Language-Hearing Association [ASHA], The ambiguity of “individual clinical expertise” was fur-
n.d.-a, 2004a, 2004b; Dollaghan, 2007) suggest that clini- ther complicated by Sackett et al.’s (1996, p. 71) introduc-
cians should identify and critically appraise evidence from tion of other evidence terms that were described as clinical
research, clinical, and patient sources, and then integrate (e.g., external clinical evidence, which Sackett used to refer
these to make the best possible treatment decision. How- to as “clinically relevant” intervention research). Confu-
ever, ASHA’s (2004a, 2004b) and Dollaghan’s (2007) sion about the precise nature and role of clinical expertise,
models differ markedly in their use of language to describe as well as other terms of clinical evidence, permeated the
the sources of evidence that are clinical in nature (e.g., translation of EBM to EBP for SLPs.
“clinical opinion,” “clinical expertise,” “evidence internal In the mid-2000s, three influential publications on
to clinical practice”). As a result, authors of peer-reviewed EBP were disseminated to SLPs in the United States. Two
EBP papers in the field define clinical sources of evidence of these sources were from the professional organization
differently and describe the role of clinical sources of responsible for supporting and empowering SLPs in the
evidence inconsistently. Without clarity or consistency in United States, ASHA. ASHA synthesized Sackett’s frame-
the terminology or process, speech-language pathologists work into a technical report (ASHA, 2004a) and a joint
(SLPs) cannot optimally identify, use, or communicate coordinated committee report (ASHA, 2004b). These
about the evidence they generate through clinical practice reports promoted the idea of EBP within the professional
as part of EBP. community and set goals for the organization. The third
In this review, we sought to clarify the sources of source was the book, The Handbook for Evidence-Based
evidence that are clinical, which we unify under the term Practice in Communication Disorders, in which Dollaghan
“clinical evidence,” and identify the role of clinical evi- (2007) proposed an E3BP framework and heavily focused
dence in evidence-based decision-making processes. To do on critical appraisal of evidence.
this, we conducted a scoping review of peer-reviewed pub- ASHA’s joint coordinated committee report (2004b)
lications, relevant and accessible to American SLPs that largely adopted Sackett’s evidence-based decision-making
included a description of terms related to clinical evidence. model, which they represented as an equilateral triangle
Our primary purpose was to establish a shared language with “current best evidence,” “clinical expertise,” and “cli-
for clinicians to discuss their clinical evidence with ent values” as the vertices (see Figure 1, left). They
researchers, allied professionals, families/clients, and each defined the goal of EBP as “the integration of (a) clinical
other. Our secondary purpose was to clarify the congru- expertise, (b) best current evidence, and (c) client values to
ency of our findings on clinical evidence with prominent provide high-quality services” (ASHA, 2004b, p. 1). Like
models of EBP. Sackett, ASHA defined “clinical expertise” according to
the general characteristics of a clinician but did not clarify
Evidence-Based Medicine to EBP how to measure/appraise this source of evidence and did
not define how “clinical expertise” should be integrated
In the early 1990s, the term evidence-based medicine with other sources of evidence to make good clinical deci-
(EBM; Guyatt, 1991) was used to unify the terms and sions. The phrase “best current evidence” stands without
procedures that outlined how to systematically gather, either a “clinical” or “research” modifier, which is also
appraise, and use the “best possible evidence” to inform observed in the triangle graphic. However, throughout the
medical decisions (Sackett & Rosenberg, 1995, p. 620). 2004b report, the term “evidence” is described implicitly
Early papers by Sackett (1997), Sackett and Rosenberg or explicitly as meaning research-based evidence, such as
(1995), and Sackett et al. (1996) on EBM suggested that in the statement, “the integration of clinical expertise, the
the “best possible evidence” was a practitioner’s integra- best current research evidence, and individual client
tion of high-quality medical research with high-quality values” (p. 2). ASHA (2004a) presents a similarly
clinical expertise. By the 1990s, high-quality medical research-focused characterization of “evidence,” stating
research was well defined according to measurable, gold that, “It is extremely rare for a single study to provide the
standard metrics of randomized controlled trial designs definitive answer to a scientific or clinical question, but a
(Sackett, 1997; Sackett & Rosenberg, 1995). However, body of evidence comprising high quality investigations
high-quality “individual clinical expertise” was not defined can be synthesized to approach a definitive answer even
according to the design or outcomes of clinical practice when, as is likely, results vary across study” (para 8). The
but according to the characteristics of the practitioner focus of “evidence” here is on amassing research evidence
themself, including their skills, (tacit) knowledge, and that converges to answer a clinical question and implies
decision-making abilities (Sackett et al., 1996, p. 71). Prac- that research evidence will always clarify clinical uncer-
titioner characteristics are dynamic, varying within and tainty (Dodd, 2007). Overall, ASHA’s (2004a, 2004b)

2944 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022

Figure 1. American Speech-Language-Hearing Association (ASHA) evidence-based practice (EBP) models (note: the diagrams are reprinted
with permission).

conceptualization of “evidence” specifically refers to is not the data generated during standard clinical evalua-
research. tion and treatment. Instead, she recommends appraising E2
The E3BP model (Dollaghan, 2007) conceptualized using single-subject research design metrics. Dollaghan’s
“evidence” differently. In the introduction, Dollaghan model suggests that when a clinician is uncertain about a
(2007) proposes that there are three separate types of evi- treatment decision, they should examine a clinical interven-
dence, which should be integrated: “(1) best available tion using metrics such as blinding and metrics for compar-
external evidence from systematic research, (2) best avail- ing baseline versus treatment phases, and suggests calculat-
able evidence internal to clinical practice, and (3) best ing Cohen’s d to evaluate the magnitude of treatment
available evidence concerning the preferences of a fully effects. This experimental view of clinical evidence differs
informed patient” (p. 2). In her conceptualization of from others who argue for the utility of routine data collec-
E3BP, Dollaghan removed clinical expertise from the tion as a deciding factor in evidence-based decision making
model itself (running contrary to Sackett et al., 1996, and (e.g., Olswang & Bain, 1994).
ASHA, 2004a, 2004b), because “clinical expertise is not a
separate piece of the E3BP puzzle but rather the glue by Recent Reworkings of Clinical Evidence
which the best available evidence of all three kinds is inte- in Speech-Language Pathology
grated in providing optimal care” (p. 3). Critically, Dollaghan
identifies “clinical expertise” as partially encompassing or EBP within speech-language pathology has evolved
synthesizing research evidence, as well as family or patient since the publication of the ASHA (2004a, 2004b) and
values. As an expansion of Sackett’s “individual clinical Dollaghan (2007) models. The relatively recent reworking
expertise,” Dollaghan’s view suggests that the totality of a of the EBP models on ASHA’s EBP website (ASHA, n.d.-a)
clinician’s knowledge (including the research evidence they launched between 2019 and 2020, which highlights the
know), decisions, skills, and abilities is “clinical expertise.” lack of clarity or consensus about the terms associated
However, in Dollaghan’s EBP model, clinical expertise is with clinical evidence. The most striking evolution is that
not its own source of evidence, which is a substantial depar- the E3BP model (Dollaghan, 2007) and the ASHA
ture from Sackett’s and ASHA’s models. This discrepancy (2004a, 2004b) model have become intertwined in many
in the role of clinical expertise fundamentally means that of ASHA’s subsequent nonrefereed resources—despite the
the E3BP model is incongruent with the ASHA (2004a, two models being substantially different. This entangle-
2004b) triangle. Subsequent work that uses the term “clini- ment can be seen in the EBP triangle, which was updated
cal expertise” rarely clarifies its meaning or which model (if from the traditional three points: “current best evidence,”
either) is referenced. “clinical expertise,” and “client values” (see Figure 1, left)
Another incongruency relates to the second compo- to a new triangle that includes “client perspectives,” “clini-
nent of the E3BP model. Dollaghan’s (2007) description cal expertise,” and “evidence” (external and internal; see
of the new term “evidence internal to clinical practice,” or Figure 1, right). Dollaghan’s (2007) evidence internal to
E2, explicitly states that, “E2 is not a synonym for routine clinical practice was a distinct source of evidence in her
measures of patient performance” (p. 115). Dollaghan’s E2 E3BP model. This source was absent from the original

Fissel Brannick et al.: Clinical Evidence in Speech-Language Pathology 2945

ASHA (2004a, 2004b) model, but it now appears grouped Seymour, 2004). Even when research does match and
with research evidence in the revised ASHA EBP triangle inform intervention decisions based on a client’s diagno-
(ASHA, n.d.-a). sis, service delivery model, or background, the interven-
In addition, within the 2019–2020 time frame, a post tion methods are too often insufficiently detailed to allow
from the ASHA Journals Academy by Higginbotham and clinicians to translate research to practice (Dodd, 2007;
Satchidanand (2019) proposed a diamond EBP model that Ludemann et al., 2017; McCurtin & Roddham, 2012). In
separated out “clinical expertise & opinion,” “external sci- these cases, understanding the different types of clinical
entific evidence,” “client–patient–caregiver perspectives,” evidence and its role in making a good clinical decision
and “internal evidence.” Unlike Dollaghan’s (2007) model, is essential.
Higginbotham and Satchidanand (2019) retained clinical To clarify and establish the language that clinicians
expertise as part of the model, adding in opinion. Addi- may use to discuss clinical evidence as part of EBP in
tionally, their model suggests “internal evidence” arises speech-language pathology, our objectives were to delineate
from the evaluation of client data and stresses the impor- the language used by authors to describe clinical evidence
tance of collecting client data as part of the ongoing ther- and its role in good clinical decision-making processes. To
apeutic process, which is a departure from Dollaghan’s accomplish these objectives, we conducted a scoping review
(2007) definition of this term. This reconceptualization of of the peer-reviewed EBP intervention literature that was
“internal evidence” coincided with ASHA’s 2019–2020 relevant and accessible to American SLPs.
website revisions that now describe internal evidence as
“the data that you systematically collect directly from
your clients to ensure that they’re making progress. This Method
data may include subjective observations of your client as
well as objective performance data compiled across time” Design
(ASHA, n.d.-b, para 2).
While ASHA’s EBP model revisions attempt to clar- A scoping review methodology (Arksey & O’Malley,
ify the components of clinical evidence, many questions 2005, p. 22) includes five steps: (1) developing broad
are left unanswered: Should Dollaghan’s (2007) view research questions, (2) conducting a search of the relevant
that “clinical expertise is not a separate piece of evidence literature, (3) applying inclusion and exclusion criteria, (4)
in the E3BP puzzle” be upheld or revised? How is representing the data visually, and (5) summarizing the
Higginbotham and Satchidanand’s (2019) conceptualiza- data in a meaningful way. Scoping reviews, compared to
tion of “clinical expertise” separate from or similar to systematic reviews, are recommended when attempting to
“clinical opinion”? How does “clinical expertise” differ broadly identify concepts and definitions of concepts, par-
from “internal evidence”? Which description of “internal ticularly in new or emerging bodies of literature (Munn
evidence” should we accept, Dollaghan’s (2007) or et al., 2018).
Higginbotham and Satchidanand’s (2019)? Most impor-
tantly: How should clinicians weigh and integrate these Step 1: Research Questions
different types of clinical evidence into evidence-based
decision-making? Q1: What terms and descriptions of clinical evidence
are presented in the intervention literature?
Statement of Need and Objectives Q2: What are the types of clinical evidence that are
discussed in the intervention literature?
Without clearly or consistently defining the various Q3: What are the attributes of clinical expertise that
sources of clinical evidence, these constructs cannot be are discussed as positive or negative in the intervention
meaningfully discussed or used by clinicians seeking to literature?
implement EBP. Without a shared language for clinicians Q4: What is the role of clinical evidence in making
to discuss clinical evidence, it is subject to being auto- good clinical decisions, as described within the intervention
matically accepted or dismissed according to an individ- literature?
ual’s own biases, the very thing that EBP was designed
to guard against (Sackett & Rosenberg, 1995). Clear Step 2: Search Strategy
understanding of clinical evidence is especially important
for practicing SLPs who commonly find that their search We aimed to represent the language used by authors
and appraisal of external research evidence is incongru- who described clinical evidence in the peer-reviewed
ent with their client’s diagnosis (Roberts et al., 2020), speech-language pathology intervention literature. We also
service delivery model (Justice et al., 2008), or cultural- wanted to represent descriptions of clinical evidence con-
linguistic and socioeconomic backgrounds (Fannin, 2017; tained in publications relevant and accessible to practicing

2946 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022

SLPs in the United States. Our inclusion criteria were complete a scoping study in a timely fashion, on the
intervention articles that were published in peer-reviewed other” (Daudt et al., 2013, p. 5).
speech-language pathology journals between 2005 and The search of ASHAWire included the following
2020, and a description of clinical evidence. We operation- journals: (a) American Journal of Speech-Language Pathol-
alized a “description of clinical evidence” as any language ogy, (b) Journal of Speech and Hearing Disorders, (c) Jour-
that explicitly stated what the keyword is, includes, or is nal of Speech, Language, and Hearing Research, (d) Lan-
characterized by, and further detail this process under guage, Speech, and Hearing Services in the Schools, (e)
Step 4. Online article searches were conducted through Contemporary Issues in Communication Science Disorders,
ASHAWire (an electronic database of ASHA Journal (f) Journal of Speech and Hearing Research, and (g) all
publications) and Google Scholar (an academic search Perspectives journals published in 2017 or later (at which
engine). ASHAWire was selected for its relevance to time the Perspectives journals were considered peer
American SLPs and its accessibility for ASHA members. reviewed; Beverly et al., n.d). The ASHAWire search was
Google Scholar was selected because this search engine is conducted using the same search string as in the Google
what many practicing SLPs report using to access search but was conducted all at once. The term “SLP”
research evidence (Muttiah et al., 2011; Thome et al., was also omitted from the search strings because of the
2020). Therefore, we adjusted this step of the Arksey and field-specific nature of the database. Terms included
O’Malley (2005) framework to include Google Scholar as “intervention” AND (“clinical opinion” OR “clinical
part of our search strategy. While a limitation of this expertise” OR “practice-based evidence” OR “practice-
approach is that those seeking to verify our search proce- based research” OR “science-based practice” OR “clinical
dures may find slightly different results (Rovira et al., science”). We piloted these search strings with the word
2019), Google Scholar has been increasingly used in the “therapy” rather than “intervention” in the ASHA jour-
search strategies of published scoping reviews (Daudt nals, which resulted in 130 results that were all duplicates
et al., 2013), has been identified as a positive supplement or met exclusion criteria, except for one article that con-
to traditional database searches (Haddaway et al., 2015), tained a definition that was already referenced within the
and aligns with the traditional goal of scoping reviews, “to corpus. Therefore, we did not pursue additional synony-
map rapidly the key concepts underpinning a research area mous search terms.
and the main sources and types of evidence available”
(Mays et al., 2001, p. 189). Our search strategy yielded Step 3: Study Selection
many articles (i.e., 972) suggesting our search captured
most relevant and accessible articles that contained a Article titles and citations were first screened for the
description of clinical evidence in the peer-reviewed, following exclusion criteria: publication prior to 2005 or
speech-language pathology literature. after 2020, publication in a book or thesis/dissertation,
The article searches were completed in January and publication in a non–peer-reviewed source (which included
February of 2021. For Google Scholar, eight searches the Perspectives journals before 2017; Beverly et al., n.d.),
were completed starting with the search line, “SLP” AND written in a language other than English, or related to
“Intervention,” followed by one of the exact phrases: a field other than speech-language pathology. Articles
“clinical opinion,” “clinical expertise,” “clinical science,” were also excluded if they studied the application of
“clinical evidence,” “internal evidence,” “practice-based EBP only related to assessment without mention of
evidence,” “practice-based research,” and “science-based intervention.
practice.” We adapted a method of narrowing search Articles that were not excluded by title and citation
results found in Graham et al. (2006) to include only the screening were then abstract appraised for inclusion criteria.
first 100 results (10 Google search pages) per search A search was conducted for keywords commonly associ-
string. This process is further detailed under Step 4. The ated with aspects of clinical evidence: “opinion,” “expert/ise,”
final search strategy was also refined following several ini- “clinic/al/ician,” “practitioner,” “practice,” “knowledge,”
tial pilots of the strategy, which revealed that (a) “speech- “science,” “evidence,” and “internal.” Articles were included
language pathologist” instead of “SLP” resulted in fewer for full-text appraisal if they contained one or more of the
found publications, (b) the term “evidence-based practice” above terms within the abstract. Articles that met title screen-
yielded far too many irrelevant publications that did not ing but did not contain an abstract were automatically
discuss clinical evidence, and (c) for each search string, included for full-text appraisal.
the most relevant results were contained in the first four During full-text appraisal, articles were included in
Google search pages, representing the first 30–40 search the final corpus if they contained a description of clinical
results. This search strategy was selected to balance “the evidence within the full text. To focus on articles that
laborious nature of study identification and the need for defined clinical evidence for intervention, articles were
comprehensiveness on the one hand, with the need to excluded if the full text focused exclusively on educational

Fissel Brannick et al.: Clinical Evidence in Speech-Language Pathology 2947

methods for teaching EBP, were systematic or meta- Step 4: Data Mapping
analytic reviews of research evidence, focused exclusively
on patient/family preferences, or were restricted to We collected and mapped the data from the corpus
assessment/diagnostic concepts in speech-language pathol- in four stages. In the first stage, we used the keyword
ogy. Finally, we conducted hand searches of the reference search strategy described above to find and extract the
lists contained in articles that were retained in the final cor- terms and language that authors used to describe clinical
pus by using the same keyword search process described evidence. In the second stage, we searched for definitions
above. The complete search strategy is represented in of the types of clinical evidence being described. In
Figure 2 (adapted from Moher et al., 2009), which may be the third stage, we topic coded (Saldaña, 2009) the defi-
found in the results. nition data for contextualized, positive, and negative

Figure 2. Search strategy.

2948 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022

descriptions of clinical expertise. In the fourth stage, we Data Mapping III: Positive and Negative Attributes
coded the definition data to summarize how authors of Clinical Expertise
reported that clinical evidence should be used to make To answer Research Question 3, we hand coded
good clinical decisions. We used NVivo (QSR International the extracted data for the (positively or negatively) polar-
Pty Ltd., 2020) to code the data in the third and fourth ized descriptions of clinical expertise. In context, these
stages. were often phrased as recommendations to clinicians.
For example, the following quote from Ebbels (2017, p.
Data Mapping I: Locating Definitions and Terms 221) is phrased as a recommendation (e.g., “clinicians
of Clinical Evidence need to. . .”) and contains several words/phrases (e.g., just
As described in Step 3, we used the keyword search anecdotes, flawed, mistakes) that signal negative polarity:
strategy described above to locate any descriptions of “Thus, clinicians need to recognize that clinical practice
clinical evidence within the included publications. When which relies on just anecdotes and experience could be
a keyword was found, the two first authors read the par- flawed and lead to clinical experience which consists of
agraph containing the keyword to determine if a descrip- ‘making the same mistakes with increasing confidence’”.
tion of clinical evidence was present. If a description of In the next example, the word important signals positive
clinical evidence was found, we extracted both the verba- polarity, “Evidence from real world clinical practice can
tim language of the description (typically one to five sen- add important data to the E3BP knowledge base” (Chan
tences found proximal to the keyword) and the key clini- et al., 2013, p. 335). Both excerpts were included in the
cal evidence term/s that were referenced in the descrip- corpus.
tion. For example, a search for the keyword “internal” The process of mapping the data aligned with
might have led to the term “internal evidence,” which grounded theory methodology (Chun Tie et al., 2019). We
authors may have explicitly defined as evidence from used an open-coding framework (Williams & Moser,
clinical practice experiences. This would be considered a 2019) to code all the data for statements that described
term related to clinical evidence. If the authors used a positive and negative aspects of clinical expertise or what
keyword (e.g., “internal”) in a way that was not related clinicians should or should not do. The first three authors
to clinical evidence (e.g., “internal medicine” or “ratings completed three rounds of coding using the constant com-
were internally consistent”), those were not considered a parative method (Kolb, 2012). They independently coded
description of terms. Many articles contained more than the data, met to develop a common codebook, used the
one term and multiple descriptions of terms. All term- codebook to recode the data, and then met to compare
description combinations were compiled into a database coding. In the first round, the authors independently
by author and year of publication. These comprised the developed topic codes (Saldaña, 2009) for each qualifying
raw data of the study. statement. A given statement could be assigned multiple
codes, as commonly occurred with lists of positive or
Data Mapping II: Determining the Types of negative descriptors of clinical evidence. After each
Clinical Evidence round of coding, the first three authors met to discuss
To determine the types of clinical evidence described the data and develop a shared codebook. This process
in the literature, we summarized the raw definition data repeated after the coders generated the second round of
according to the source of clinical evidence, or how the codes. After the third round of coding, the frameworks
evidence was generated. The two first authors indepen- converged for both codebooks. During the third meeting,
dently read through the definitions of clinical evidence authors came to consensus for all codes. Finally, axial
and applied topic coding procedures to generate a list of coding was conducted by grouping conceptually similar
categories that reflected how the clinical evidence was codes into broader categories (Williams & Moser, 2019)
generated. They then compared their frameworks to that described the positive and negative aspects of clini-
finalize a set of four categories that described the context cal expertise. Thus, the final conceptual hierarchy was
in which the clinical evidence was generated, the role of categories that contained codes.
the person or persons generating the clinical evidence,
and the procedures that were used to generate the clinical Data Mapping IV: The Role of Clinical Evidence
evidence. At this level of analysis, we evaluated the inter- in Decision Making
rater reliability of these methods to classify the types of To answer Research Question 4, we used the same
clinical evidence. A total of 38.5% of definitions/ methodology as in Data Mapping III, to code for descrip-
descriptions (30/78) were coded for interrater reliability tions of the role of clinical evidence in relation to other
at 96.7% agreement, which was interpreted as strong. types of evidence when making a clinical decision. We
Discrepancies were resolved through discussion and con- operationalized descriptions of the “role of clinical evi-
sensus by the first three authors. dence” as its function or recommended/expected use in

Fissel Brannick et al.: Clinical Evidence in Speech-Language Pathology 2949

evidence-based decision making. Function statements were interdisciplinary publications describing the SCIRehab
descriptive; they described how clinicians commonly use Project (Brougham et al., 2011; Gassaway et al., 2009;
one or more aspects of clinical evidence in decision- Gordan et al., 2009; Horn et al., 2015; Whiteneck &
making processes. Statements of recommended/expected Gassaway, 2012, 2013; Whiteneck et al., 2009) described
use, on the other hand, were prescriptive and described “practice-based evidence” as a “research methodology” by
how clinicians should use one or more aspects of clinical which aspects of the scientific method were used to collect
evidence to make evidence-based decisions. For example, retrospective data from practice contexts—not as a
if an author stated that clinicians should prioritize research method that is used to collect data in session by practi-
evidence over clinical evidence that would be considered a tioners. In contrast, publications in speech-language
statement of recommended/expected use, as it describes pathology journals (n = 7) described “practice-based evi-
how the author believes clinical evidence should be used in dence” as procedures by which SLPs generate clinical
decision-making. As in Data Mapping III, codes that were hypotheses and systematically collect data during treat-
conceptually similar were grouped into broader categories ment to test hypotheses (Baker & McLeod, 2011; Crooke
(Williams & Moser, 2019) that described how clinical evi- & Olswang, 2015; Donaldson & Stahmer, 2014; Riedeman
dence is used to make good clinical decisions. & Turkstra, 2018; Smith, 2018; Swift et al., 2017). Of
these seven articles, four suggested that “practice-based
evidence” also encompassed application of the scientific
Results method, knowledge integration, clinician skill, decision-
making, and/or internal verification/validation of clinical
A total of 972 articles were identified, which data.
included 259 articles identified through ASHAWire and Authors also described different terms analogously
713 articles identified through Google Scholar. After or nested terms in varying ways within an EBP hierarchy.
removing duplicates, the application of inclusion and For instance, Baker and McLeod (2011) describe clinical
exclusion criteria returned 78 published works, which were expertise as the integration of research, clinical, and
reviewed in full (see Figure 2 and Supplemental Material patient/family evidence. Others suggested this term encom-
S1). Of note, the exclusion criteria barred some influential passed various aspects of clinician skills, decision-making,
works from the corpus that were not peer-reviewed articles experiences, attitudes, and opinions (e.g., Kamhi, 2006;
(e.g., ASHA, n.d.-a, 2004a; Dollaghan, 2007) or were Thome et al., 2020). Iacono and Cameron (2009)
from other fields (e.g., Ericsson & Lehmann, 1996). described “clinical opinion” as a type of “internal evi-
dence” (p. 237), yet Donaldson and Stahmer (2014) pre-
Question 1: Terms and Definitions of sented “internal evidence” as analogous to “practice-based
Clinical Evidence evidence,” which was subsequently defined as “systematic
and repeated data collection” (p. 271). While these various
Terms terms are implicitly linked, the relationships or differences
Across the 78 articles, 98 terms were used to between these clinical evidence terms were not explicitly
describe aspects of clinical evidence (see Supplemental stated for greater than 80% of terms used in the corpus
Material S2). Approximately, one third of articles (23/78) (n = 80/98 terms).
used more than one term. Some terms were used across
authors (e.g., “clinical expertise”), whereas others were Question 2: Types of Clinical Evidence
unique to one publication, (e.g.., “craft-based knowledge,”
Justice, 2010, or “indirect evidence,” Dijkers et al., 2012). In Research Question 2, we sought to evaluate the
Descriptions of clinical evidence were often presented as types of clinical evidence that were described in the litera-
recommendations for ideal clinical practice patterns or ture. Three broad types of clinical evidence arose from the
descriptions of the sort of clinical evidence that should be analysis, although research evidence was at times described
used to inform EBP, rather than thorough investigations of as clinical evidence.1 To describe these types using con-
these concepts. For example, Fey (2006) described the clini- sistent terminology, we selected the most-used terms to
cal importance of clinician self-evaluation and integration
of clinician experience but did not explicitly define what
1
either term meant. The research evidence described as clinical evidence was often
termed practice-based research (n = 29). Practice-based research
Descriptions of Clinical Evidence designs were typically aligned with clinical research trials or research
conducted in clinical settings, including retrospective studies and fea-
The meaning of individual terms was not consistent sibility designs. While this is certainly essential and valuable work,
across articles. Authors used similar terms to represent these designs can already be evaluated based on research evidence
dissimilar concepts. For example, authors in a series of metrics and were thus outside the scope of this review article.

2950 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022

represent each of these types: clinical opinion, clinical aspects of clinicians were identified (see Supplemental
expertise, and practice-based evidence. Material S3): (a) interpersonal skills and attributes, (b)
Clinical opinion (n = 39) described an intrinsic technical clinical skills, (c) experience, (d) means of mea-
construct—the dynamic, implicit viewpoints of researchers suring intervention outcomes, (e) tacit knowledge/bias,
who stated or implied that a given clinician (frequently and (f) systematicity. Operational definitions and counts
themselves) was an expert. Self-proclaimed expert opinion for the codes and categories may be found in Supplemen-
was often accompanied by a limited description of the tal Material S3.
expert’s qualifications (e.g., number of years in practice)
but did not otherwise describe how those qualifications Interpersonal Skills and Attributes
led to expertise. Some authors described a negative per- All 18 publications that described interpersonal skills
ception of clinical opinion, describing it as implicit or did so by noting the value of positive interpersonal skills
biased (e.g., Cardin & Hudson, 2018; Goldstein et al., and attributes. Most of the articles described the desirabil-
2007; Justice, 2010; Mcleod & Baker, 2014; Muñoz, 2017; ity of certain personality traits, such as empathy or com-
Selin et al., 2019). Clinical opinion also included attitudi- passion. Some described expert clinicians as those who
nal constructs like a clinician’s personal values, which have positive communication skills and the ability to work
authors typically described as positively impacting clinical successfully as a member of professional teams (n = 5).
outcomes (e.g., Roulstone, 2011).
Clinical expertise (n = 54) also represented a con- Technical Clinical Skills
struct intrinsic to the clinician, in which they dynamically All 22 articles discussing this category positively
integrate multiple sources of knowledge and gain technical described specific clinical or procedural skills that made
skills to select appropriate measures, engage in consistent clinicians effective, such as clinicians’ ability to work
practice, and collect data. While clinical opinion was often within different practice contexts and the fidelity of inter-
described as lacking rigor, such as having preferences vention (n = 12).
without data, descriptions of clinical expertise often dis-
cussed clinicians as being reflective and self-aware of their Experience
own knowledge. While clinical opinion was related to clini- Most of the 23 articles described experiences that
cians’ biases, personal values, and opinions, clinical exper- were important for expertise development, including inten-
tise described the mixing of a clinician’s knowledge, prior tional mentorship and clinical experiences (n = 13), educa-
clinical experiences, choice to practice systematically, use tional history and clinical training (n = 23), and the
of demonstrable technical skills in intervention, and means growth in clinical proficiency that results from intentional
of measuring intervention outcomes. practice experiences over time (n = 7). Far fewer articles
Practice-based evidence (n = 28) described static (n = 4) described experience in a negative light. These neg-
clinician-generated client data that are interpreted to test a ative descriptions suggested that accumulated clinical
clinical hypothesis or answer a clinical question. Unlike experiences were insufficient for expertise development.
the prior two constructs, practice-based evidence was
described as extrinsic to the clinician: the product that is Measuring Intervention Outcomes
generated from a clinician’s systematic measurement, Articles that referenced measuring outcomes (n =
aggregation, and interpretation of data. While clinical 28) always positively characterized the importance of col-
expertise may include skilled data collection, practice- lecting data on the outcomes of intervention. This cate-
based evidence is the data itself: clinical evidence derived gory was frequently related to the terms “practice-based
from clinical practice. This is unlike practice-based evidence” and “practice-based research.” A subset of pub-
research, which is research evidence and thus generated lications (n = 11) recommended using a research method-
through the steps of the scientific method and externally ology, such as single-case experimental design, to measure
validated. Though practice-based evidence is generated sys- the outcomes of intervention.
tematically, it is clinician generated and does not follow
all steps of the scientific method. Tacit Knowledge and Behavior
This category was very polarized. Nineteen articles
Question 3: Positive and Negative Aspects reported positive, negative, or mixed views of tacit knowl-
of Clinical Expertise edge and behaviors, which were sometimes described as
personal or clinical biases. This construct frequently refer-
We sought to determine how authors described enced the insights, intuitions, and impulses of clinicians.
aspects of clinical expertise positively and negatively. Approximately half of the articles in this category (10/19)
Sixty-eight of 78 articles included a positive/negative described such biases or tacit knowledge as something
description of clinical expertise. Six categories describing valuable or positive—often as an attribute that allowed

Fissel Brannick et al.: Clinical Evidence in Speech-Language Pathology 2951

expert clinicians to respond quickly and fluidly within ses- 2006), context of service delivery (e.g., Swift et al., 2017),
sions or to individualize treatment for their clients. The applicability given local resources (e.g., Gillam & Gillam,
other half of articles (11/19) described tacit knowledge 2006), local policies (e.g., McCauley et al., 2009), the clini-
negatively. Of these, five suggested tacit knowledge is cian’s own analysis and problem-solving (e.g., Goldstein
biased, unvalidated, or unreliable. Six characterized tacit et al., 2007), professional education (e.g., Iacono &
knowledge as a problematic foil to EBP because it leads Cameron, 2009), and practice-based evidence (e.g., Dodd,
to habitual or uncritiqued practice. 2007; Kamhi, 2011). Overall, the articles asserted that cli-
nicians should draw from a broader base of factors to
Systematicity make decisions, which differs from the traditional three
The largest proportion of articles (33/68) described sources of the ASHA (2004a, 2004b) EBP triangle. Authors
the value of clinical practices that are structured, reflec- do not appear to be limiting their recommendations to
tive, or deliberate in nature. All these articles described the former (ASHA, 2004a, 2004b) or current ASHA
systematic practice positively. Nearly half of the 33 articles models (n.d.-a), or the Dollaghan (2007) model of EBP
for this category (n = 16) described aspects of organized when describing the types of evidence involved in EBP.
thinking or stepwise problem-solving that were important
for systematic practice. Twelve articles described the Aligning With Professional Culture
importance of explicit, organized knowledge that devel- The 13 articles with membership in this category
oped from repeated clinical experiences or testing clinical described how clinicians should integrate advice from
hypotheses. Seven articles suggested that self-reflection local clinical expertise or broader expert communities
processes support the development of systematic practice. when making clinical decisions. Five articles described
Six articles suggested that documenting intervention how clinicians should access the clinical expertise of other
methods is important for systematically determining the SLPs in their local practice context. Four recommended
reason for a particular outcome. Few articles (4/68) that clinicians engage in communities of practice or
described how generating clinical questions or hypotheses community–academic partnerships to create new knowl-
supports the organization of explicit clinical knowledge. edge. Three articles described the importance of seeking
other allied professionals’ clinical expertise as part of
Question 4: The Role of Clinical Evidence interprofessional practices.
in Making Good Clinical Decisions
Operating Under Consensus Recommendations
We found that 53 of 78 articles included a descrip- Seven articles identified the importance of aligning
tion of the role of clinical evidence in making a good clini- one’s clinical practices with practice recommendations or
cal decision. Overall, the role of clinical evidence in good guiding statements from professional organizations (e.g.,
clinical decision-making was described in relation to five ASHA Practice Policies).
other categorical constructs. These categories (summarized
in Supplemental Material S4) included (a) integrating mul- Prioritizing Client and Family Values
tiple sources of information, (b) aligning with professional Ten articles discussed the importance of integrating
culture, (c) operating under consensus recommendations, client and family values into decision making. This
(d) prioritizing client and family values, and (e) prioritiz- included recommendations for individualizing interven-
ing research evidence. tion approaches and using family-centered intervention
approaches.
Integrating Multiple Sources of Information
All 28 member articles for this construct described Prioritizing Research Evidence
the importance of clinicians weighing and integrating mul- Most articles (38/58) described the importance of cli-
tiple sources of information to make good clinical deci- nicians knowing, translating, and applying research evi-
sions. Typically, authors who described the importance of dence to make good decisions. Authors often implied, or
integrating multiple sources of information did so by list- in some cases explicitly stated, that research evidence
ing the types of information clinicians should consider. should be weighted more heavily in decision making than
These lists included the types of evidence in the ASHA clinical evidence because of its reliability, validity, and
(2004a, 2004b) EBP model (research evidence, clinical processes of external verification by peer review. For
expertise, and patient/family values), or terms like those instance, 11 of 38 articles specifically stated that research
described in the results to Question 2. However, other evidence was the most important part of the EBP triad,
sources of information in decision making were also and four suggested that when research evidence and clini-
included in these lists, such as the clinician’s theoretical cal evidence are at odds, research evidence should be pri-
perspective (e.g., Fey, 2006), clinical opinions (e.g., Kamhi, oritized. In the articles that discussed prioritizing research

2952 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022

evidence, authors suggested that sources of clinical evi- Discussion
dence should only be used when research evidence was
lacking (n = 8). Many of the articles (21/38) suggested that Our results clarify the language used by authors in
expert clinicians ought to be familiar with current litera- the field to describe clinical evidence and provide initial
ture results and theory. Authors often (12/38) indicated insight into clinical evidence as a distinct and important
that good clinical decisions hinge on clinicians knowing part of the EBP models of speech-language pathology.
how and when to translate research results or theory into Based on our findings, we suggest concepts, terms, and
practice. processes that SLPs may use to begin discussions of clini-
cal evidence with others, and suggest areas of future
Summary of Results research exploring the place and role of clinical evidence
in EBP models.
Across the 78 articles reviewed, authors used 98 dif-
ferent terms to describe aspects of clinical evidence. These Clinical Evidence is Evidence
terms were not used consistently, and they held different
meanings for different authors and articles. Three types of While early models of EBP used “evidence” to refer
clinical evidence were discussed in the literature: clinical implicitly or explicitly to external research evidence, we
opinion (self-proclaimed skill, belief, or personal bias), clin- found that authors described clinical evidence in the litera-
ical expertise (involving demonstrable skill and explicit ture as a distinct source of evidence, one that was separate
knowledge), and practice-based evidence (systematic data from research evidence. While the three types of clinical
or information collection from application of an interven- evidence identified (clinical opinion, clinical expertise, and
tion). Authors often discussed positive and negative practice-based evidence) have been historically enmeshed
aspects of clinical expertise as enmeshed with descriptions in the literature, we found that they are separable and
of the clinician themselves, whereas practice-based evidence definable constructs. By explicitly identifying the compo-
was described as extrinsic to the clinician: the product that nents that differentiate these three types of evidence from
was generated from systematically measuring patient out- each other and from other sources of evidence, clinicians
comes and reflecting on them over time. may identify, collect, appraise, and discuss their clinical
Negative descriptions of clinicians were associated evidence to determine its value relative to other forms of
with clinical opinion (self-proclaimed skill/bias), but posi- available evidence.
tive descriptions were connected to clinical expertise
(demonstrable knowledge/skill). Descriptions of clinical Clinical Evidence Can Be Appraised and
opinion referenced a practitioner’s views based on a single Improved
source of information that lacked rigor, such as an atti-
tude or years of experience. Conversely, descriptions of Clinical Expertise
clinical expertise referenced multiple, positive characteris- Our results support the importance of differentiat-
tics of clinicians who are skilled, are experienced, engage ing between clinical opinion as a poor-quality, unidimen-
in practice systematically, measure the outcomes of inter- sional source of clinical evidence, and clinical expertise
vention, and who are strong communicators with positive as a high-quality, multidimensional source of clinical
interpersonal traits. evidence that is intrinsic to the clinician. Clinical ex-
Good clinical decisions were characterized as those pertise was described as stemming from observable/
that integrated multiple sources of information, capitalized demonstrable categories of clinical skills, knowledge,
on the expertise of others, aligned with professional con- and practices. This finding does not support the merger
sensus statements and professional culture, and prioritized of clinical expertise with clinical opinion in the
research evidence, as well as client/family values. Fre- Higginbotham and Satchidanand (2019) diamond EBP
quently, authors suggested that integrating many sources model but suggests that clinical expertise should be
of information led to good decision making, often recom- appraised and prioritized over clinical opinion as a dis-
mending more than the traditional three sources of evi- tinct source of evidence. As an explicit construct, clinical
dence referenced by ASHA (n.d.-a, 2004a, 2004b) or expertise can be appraised and intentionally improved,
Dollaghan (2007). Poor clinical decisions were described whereas clinical opinion represents a clinical belief or
as based on just one or few sources of evidence. While attitude that is not regularly updated through reflection
different types of clinical evidence took on different and intentional practice. Clinicians may reflect on their
roles within each of the five categories that marked skills, attributes, communication, experience, systemati-
good clinical decisions, positive attributes of clinical city, and measurement practices to self-assess their
expertise were described in all categories of good clinical expertise broadly or to generate new insights and evi-
decision making. dence about a specific population or practice. Clinicians

Fissel Brannick et al.: Clinical Evidence in Speech-Language Pathology 2953

may use this information to explicitly discuss their clini- information from practice-based evidence in clinical
cal expertise for a particular case with other clinicians, decisions?
professionals, researchers, and clients/families. Further-
more, the features that characterize clinical expertise are Good Clinical Decision Making Is
tangible outcomes that may be explicitly taught in pre- Multidimensional and Distinct From
service training programs and continuing education Clinical Expertise
offerings.
Our results identify that authors referenced multiple
Practice-Based Evidence sources of evidence to appraise and integrate when mak-
Authors within the corpus described practice-based ing good clinical decisions, many more than the tradi-
evidence to include reviewing and identifying patterns in tional three sources of evidence referenced in the EBP
local data (e.g., King et al., 2007; Wheeler-Hegland models of speech-language pathology (ASHA, 2004a,
et al., 2009), comparing local data to published research 2004b; Dollaghan, 2007). Decisions based on one or few
outcomes or confidence intervals (Gillam & Gillam, evidence sources were described negatively by authors and
2006), and validating local data/interpretations (e.g., aligned with decisions based on clinical opinion. Clinicians
Cardin & Hudson, 2018; Cirrin & Gillam, 2008; Douglas should avoid making decisions based on any single stream
et al., 2019; Fey, 2006; Kamhi, 2011; McCurtin et al., of evidence (e.g., only personal beliefs, only research evi-
2019). We found broad author support for the regular, dence, only practice-based evidence). High-quality clinical
systematic collection of data from clients, and many evidence (i.e., clinical expertise and practice-based evi-
authors communicated the importance of integrating dence) was included in the multiple sources of evidence to
this source of evidence to make good clinical decisions. consider in the construction of a good evidence-based
Practice-based evidence represents data extrinsic to the decision.
clinician, distinguishing it from intrinsic clinical expertise. We found that authors described clinical expertise as
This source is also different from research evidence important to, but distinct from, the process of clinical
because the scientific method is not strictly followed, and decision making. While EBP models have historically col-
the design/goals of regular data collection necessarily dif- lapsed clinical expertise and clinical decision making into
fer from the design/goals of data collected for research. one construct (e.g., Dollaghan, 2007) or described the pro-
Nevertheless, systematic data-informed clinical decision cess of clinical decision making as an indistinct integration
making is a recommended clinical practice that predates stage (ASHA, 2004a, 2004b), differentiating these two
even the early ASHA statements on EBP (e.g., Olswang constructs is important. Clinical expertise is defined by the
& Bain, 1994) and aligns with early descriptions of collective skills, knowledge, attributes, experiences, and
evidence-based medicine that reference clinical data practice-patterns that accumulate over time as intrinsic
(Sackett & Rosenberg, 1995). Clinicians may find it characteristics of effective clinicians and gradually improve
useful to collect, appraise, and discuss their de-identified through reflection, practice, appraisal, and discussion.
practice-based evidence within communities of practice Alternatively, clinical decision making is a process by which
with other SLPs and allied professionals. As part of clinicians iteratively seek, appraise, weigh, and assemble
community–academic partnerships, clinicians may present multiple sources of evidence that converge toward one, or
data summaries to stakeholders to justify scale-up research sometimes, several paths of action. Without acknowledging
on a method/intervention of consequence or to evaluate clinical expertise as distinct from decision-making processes,
the implementation of a method/intervention in the com- clinicians cannot generate, appraise, or discuss it as a
munity. Preservice training programs and continuing edu- source of evidence. If clinical expertise is not explicit and
cation offerings should (a) teach how to map different distinct, it cannot be integrated with the multiple other
data collection methods to client goals/objectives and sources of evidence that characterize good evidence-based
intervention methods, (b) provide practice opportunities decisions. Therefore, clinical expertise is not the quality of
to check/appraise different forms of practice-based evi- the clinical decision or the act of deciding; it is a form of
dence, and (c) teach descriptive, quantitative, and qualita- evidence to be integrated before clinical decision making
tive methods of aggregating and analyzing data collected may begin.
from multiple clients. Clinical experts may collect and appraise practice-
Although authors described practice-based evidence based evidence as another source of evidence to integrate
clearly and positively, there was little consensus in the lit- when making decisions. Practice-based evidence may be
erature as to its place in the EBP model. Future research collected to monitor the outcomes of an evidence-based
should consider (a) where practice-based evidence fits decision, to test hypotheses, or to serve as the foundation
within current EBP models, (b) how should one appraise of a discussion with others. When the integration of multi-
practice-based evidence, and (c) how should one integrate ple sources of evidence point to more than one possible

2954 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022

decision, referencing clinical expertise and collecting prac- Conclusions
tice-based evidence can clarify uncertainty. If uncertainty
persists, practice-based evidence can be collected to test/ Clinicians may use the language of clinical evidence
monitor the effectiveness of the decision. described in this review article to discuss the quality of
By adopting the shared language of clinical evidence their clinical evidence with other clinicians, researchers,
and clinical decision-making presented in this review article, clients/families, and allied health professionals. Researchers
SLPs may better discuss their clinical evidence with may expand on this work by exploring and writing about
researchers, families, allied professionals, and each other. EBP using the vocabulary defined in this review to ensure
Those responsible for revising and disseminating EBP consistency across researchers and publications. The
models should consider how our findings inform future descriptions of clinical evidence proposed herein should be
model development. Future research should consider testing useful in providing a unifying vocabulary to generate “col-
the acceptability and implementation of these findings by laborative, critical discourse” (Osborne, 2010, p. 463) that
practicing SLPs. supports the knowledge, skill, and decision making of cli-
nicians. Unifying the language of clinical evidence should
improve our ability to investigate and apply EBP models
Strengths and Limitations of This and to engage in conversations about clinical evidence
Scoping Review from a place of shared understanding. It is essential that
researchers and clinicians engage in meaningful conversa-
This review represents the first attempt to clarify tions about clinical evidence that are founded on mutual
and summarize the published language describing clinical respect and appreciation. Critically, we hope that this
evidence, making this work highly relevant to clinicians work leads clinicians and researchers to discuss, appraise,
and researchers interested in EBP. Our methods were and refine sources of clinical evidence and the processes
informed and conducted by a diverse research team repre- for making good clinical decisions, together.
senting researchers with expertise in scoping reviews and
clinicians working in academic and community contexts.
We made every attempt to present our methods with Data Availability Statement
transparency and conduct the review rigorously, with iter-
ative checks in place to reduce bias. However, fully con- The raw data summary tables supporting our results
trolling for reviewer bias is impossible, and our findings are included in this submission as supplemental materials.
must be interpreted with consideration for such biases.
This work emphasized models of EBP developed by
American organizations for American SLPs. Our search
of the ASHA journals aligned with this focus, but our Acknowledgments
results may not be generalizable to the EBP practices of
non-American SLPs. The use of Google Scholar benefited This work was supported by Midwestern University.
the breadth of our search and identified articles not found The authors would like to acknowledge the importance of
in our database search. However, the use of Google each of the authors’ preservice and postservice clinical and
Scholar limits the precise replicability of results via the research training, the open lines of communication between
ranking algorithm of this academic search engine. Because academic and clinical faculty, and the informal and formal
we did not search every known database, our search strat- community-based, clinical–academic partnerships that con-
egy may have overlooked some articles that would have tributed to the development of this review article. Such
met inclusion criteria. While we piloted synonyms for experiences represent the translational and community
search terms (e.g., “therapy”), we did not use all possible ideals of the TAG lab, and were central to promoting com-
search terms (e.g., “implementation”), which may have also munication and perspective taking between clinicians and
limited the number of articles we found. We fulfilled Items researchers to ensure a high-quality, representative review.
1–4 and 6–22 of the Preferred Reporting Items for System-
atic reviews and Meta-Analyses extension for Scoping
Reviews (PRISMA-ScR) checklist but did not meet Item 5, References
as the search protocol was not registered at the start of this
project. Finally, this study evaluated the way clinical evi- An * is used to denote an article that met inclusion criteria within
dence is conceptualized and discussed by researchers. While the corpus.
American Speech-Language-Hearing Association. (n.d.-a). Evidence-
this is a limitation inherent to a scoping review of the liter- based practice (EBP). https://www.asha.org/Research/EBP/
ature, these views may not be indicative of clinicians’ American Speech-Language-Hearing Association. (n.d.-b). Step 2:
perspectives—an irony that is not lost on the authors. Gather evidence. https://www.asha.org/research/ebp/gather-evidence/

Fissel Brannick et al.: Clinical Evidence in Speech-Language Pathology 2955

American Speech-Language-Hearing Association. (2004a). Evidence- Language, Speech, and Hearing Services in Schools, 45(4),
based practice in communication disorders: An introduction 261–276. https://doi.org/10.1044/2014_LSHSS-14-0038
[Technical report]. http://www.asha.org/policy *Douglas, N. F., Squires, K., & Hinckley, J. (2019). Narratives of
American Speech-Language-Hearing Association. (2004b). Report expert speech-language pathologists: Defining clinical exper-
of the joint coordinating committee on evidence-based practice. tise and supporting knowledge transfer. Teaching and Learn-
https://www.asha.org/siteassets/uploadedfiles/jccebpreport04.pdf ing in Communication Sciences & Disorders, 3(2). https://doi.
Arksey, H., & O’Malley, L. (2005). Scoping studies: Towards a org/10.30707/TLCSD3.2Douglas
methodological framework. International Journal of Social *Ebbels, S. (2017). Intervention research: Appraising study
Research Methodology, 8(1), 19–32. https://doi.org/10.1080/ designs, interpreting findings and creating research in clinical
1364557032000119616 practice. International Journal of Speech-Language Pathology,
*Baker, E., & McLeod, S. (2011). Evidence-based practice for chil- 19(3), 218–231. https://doi.org/10.1080/17549507.2016.1276215
dren with speech sound disorders: Part 2 application to Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional
clinical practice. Language, Speech, and Hearing Services performance: Evidence of maximal adaptation to task con-
in Schools, 42(2), 140–151. https://doi.org/10.1044/0161- straints. Annual Review of Psychology, 47(1), 273–305. https://
1461(2010/10-0023) doi.org/10.1146/annurev.psych.47.1.273
Beverly, B., Sandage, M., Konrad-Martin, D., & Domsch, C. (n.d.). Fannin, D. K. (2017). Reporting of underrepresented populations
Perspectives of the ASHA Special Interest Groups. ASHA in autism treatment studies across 25 years. Clinical Archives
Journals Academy. https://academy.pubs.asha.org/asha-journals- of Communication Disorders, 2(3), 250–263. https://dx.doi.org/
author-resource-center/selecting-a-journal/perspectives-of-the-asha- 10.21849/cacd.2017.00129
special-interest-groups/ *Fey, M. E. (2006). Commentary on “Making evidence-based deci-
*Brougham, R., David, D. S., Adornato, V., Gordan, W., Dale, sions about child language intervention in schools” by Gillam and
B., Georgeadis, A. C., & Gassaway, J. (2011). Speech- Gillam. Language, Speech, and Hearing Services in Schools, 37(4),
language pathology treatment time during inpatient spinal 316–319. https://doi.org/10.1044/0161-1461(2006/036)
cord injury rehabilitation: The SCIRehab project. The Jour- *Gassaway, J., Whiteneck, G., & Dijkers, M. (2009). Clinical tax-
nal of Spinal Cord Medicine, 34(2), 186–195. https://doi.org/ onomy development and application in spinal cord injury
10.1179/107902611X12971826988174 research: The SCIRehab Project. The Journal of Spinal Cord
*Cardin, A. D., & Hudson, M. B. (2018). Evidence-based practice Medicine, 32(3), 260–269. https://doi.org/10.1080/10790268.
in the hospital setting: Views of interdisciplinary therapy prac- 2009.11760780
titioners. Journal of Allied Health, 47(2), 81–89. *Gillam, S. L., & Gillam, R. B. (2006). Making evidence-based
*Chan, A. K., McCabe, P., & Madill, C. J. (2013). The imple- decisions about child language intervention in schools. Lan-
mentation of evidence-based practice in the management of guage, Speech, and Hearing Services in Schools, 37(4), 304–
adults with functional voice disorders: A national survey of 315. https://doi.org/10.1044/0161-1461(2006/035)
speech-language pathologists. International Journal of Speech- *Goldstein, H., Schneider, N., & Thiemann, K. (2007). Peer-
Language Pathology, 15(3), 334–344. https://doi.org/10.3109/ mediated social communication intervention. Topics in Lan-
17549507.2013.783110 guage Disorders, 27(2), 182–199. https://doi.org/10.1097/01.
Chun Tie, Y., Birks, M., & Francis, K. (2019). Grounded theory TLD.0000269932.26504.a8
research: A design framework for novice researchers. SAGE *Gordan, W., Spivak-David, D., Adornato, V., Dale, B.,
Open Medicine, 7. https://doi.org/10.1177/2050312118822927 Brougham, R., Georgeadis, A. C., & Gassaway, J. (2009).
*Cirrin, F. M., & Gillam, R. B. (2008). Language intervention SCIRehab Project series: The speech language pathology tax-
practices for school-age children with spoken language disor- onomy. The Journal of Spinal Cord Medicine, 32(3), 307–318.
ders: A systematic review. Language, Speech, and Hearing Ser- https://doi.org/10.1080/10790268.2009.11760784
vices in Schools, 39(1), S110–S137. https://doi.org/10.1044/0161- Graham, I., Logan, J., Harrison, M. B., Straus, S. E., Tetroe, J.,
1461(2008/012) Casswell, W., & Robinson, N. (2006). Lost in knowledge
*Crooke, P. J., & Olswang, L. B. (2015). Practice-based research: translation: Time for a map? The Journal of Continuing Edu-
Another pathway for closing the research–practice gap. Journal cation in the Health Professions, 26(1), 13–24. https://doi.org/
of Speech, Language, and Hearing Research, 58(6), S1871–S1882. 10.1002/chp.47
https://doi.org/10.1044/2015_JSLHR-L-15-0243 Guyatt, G. H. (1991). Evidence-based medicine. ACP Journal
Daudt, H. M., van Mossel, C., & Scott, S. J. (2013). Enhancing Club, 114(2), A16. https://doi.org/10.7326/ACPJC-1991-114-2-A16
the scoping study methodology: A large, inter-professional Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S.
team’s experience with Arksey and O’Malley’s framework. (2015). The role of Google Scholar in evidence reviews and its
BMC Medical Research Methodology, 13(1), 1–9. https://doi. applicability to grey literature searching. PLOS ONE, 10(9),
org/10.1186/1471-2288-13-48 1–17. https://doi.org/10.1371/journal.pone.0138237
*Dijkers, M. P., Murphy, S. L., & Krellman, J. (2012). Evidence- Higginbotham, J., & Satchidanand, A. (2019). From triangle to
based practice for rehabilitation professionals: Concepts and diamond: Recognizing and using data to inform our evidence-
controversies. Archives of Physical Medicine and Rehabilitation, based practice. ASHA Journals: Academy. https://academy.
93(8), S164–S176. https://doi.org/10.1016/j.apmr.2011.12.014 pubs.asha.org/2019/04/from-triangle-to-diamond-recognizing-and-
*Dodd, B. (2007). Evidence-based practice and speech-language using-data-to-inform-our-evidence-based-practice
pathology: Strengths, weaknesses, opportunities and threats. *Horn, S. D., Corrigan, J. D., Bogner, J., Hammond, F. M.,
Folia Phoniatrica et Logopaedica, 59(3), 118–129. https://doi. Seel, R. T., Smout, R. J., Barrett, R. S., Dijkers, M. P., &
org/10.1159/000101770 Whiteneck, G. G. (2015). Traumatic brain injury–practice
Dollaghan, C. A. (2007). The handbook for evidence-based practice based evidence study: Design and patients, centers, treatments,
in communication disorders. Brookes. and outcomes. Archives of Physical Medicine and Rehabilita-
*Donaldson, A. L., & Stahmer, A. C. (2014). Team collaboration: tion, 96(8), S178–S196.e15. https://doi.org/10.1016/j.apmr.2014.
The use of behavior principles for serving students with ASD. 09.042

2956 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022

*Iacono, T., & Cameron, M. (2009). Australian speech-language and meta-analyses: The PRISMA statement. PLOS Medicine,
pathologists’ perceptions and experiences of augmentative and 6(7), Article e1000097. https://doi.org/10.1371/journal.pmed.
alternative communication in early childhood intervention. 1000097
Augmentative and Alternative Communication, 25(4), 236–249. Munn, Z., Peters, M. D. J., Stern, C., Tufunaru, C., McArthur,
https://doi.org/10.3109/07434610903322151 A., & Aromataris, E. (2018). Systematic review or scoping
*Justice, L. M. (2010). When craft and science collide: Improving review? Guidance for authors when choosing between a system-
therapeutic practices through evidence-based innovations. atic or scoping review approach. BMC Medical Research Meth-
International Journal of Speech-Language Pathology, 12(2), odology, 18(1), 143. https://doi.org/10.1186/s12874-018-0611-x
79–86. https://doi.org/10.3109/17549500903373984 *Muñoz, M. L. (2017). Cultural adaptation of evidence-based
Justice, L. M., Nye, C., Schwarz, J., McGinty, A., & Rivera, A. treatments: An example from aphasia. Perspectives of the
(2008). Methodological quality of intervention research in ASHA Special Interest Groups, 2(14), 5–14. https://doi.org/10.
speech-language pathology: Analysis of 10 years of group-design 1044/persp2.SIG14.5
studies. Evidence-Based Communication Assessment and Interven- Muttiah, N., Georges, K., & Brackenbury, T. (2011). Clinical and
tion, 2(1), 46–59. https://doi.org/10.1080/17489530802008839 research perspectives on nonspeech oral motor treatments
*Kamhi, A. G. (2006). Treatment decisions for children with and evidence-based practice. American Journal of Speech-
speech–sound disorders. Language, Speech, and Hearing Ser- Language Pathology, 20(1), 47–59. https://doi.org/10.1044/1058-
vices in Schools, 37(4), 271–279. https://doi.org/10.1044/0161- 0360(2010/09-0106)
1461(2006/031) Olswang, L. B., & Bain, B. (1994). Data collection. American
*Kamhi, A. G. (2011). Balancing certainty and uncertainty in Journal of Speech-Language Pathology, 3(3), 55–66. https://
clinical practice. Language, Speech, and Hearing Services in doi.org/10.1044/1058-0360.0303.55
Schools, 42(1), 59–64. https://doi.org/10.1044/0161-1461(2009/ *Olswang, L. B., & Prelock, P. A. (2015). Bridging the gap
09-0034) between research and practice: Implementation science.
*King, G., Currie, M., Bartlett, D. J., Gilpin, M., Willoughby, C., Journal of Speech, Language, and Hearing Research, 58(6),
Tucker, M. A., Strachan, D., & Baxter, D. (2007). The devel- S1818–S1826. https://doi.org/10.1044/2015_JSLHR-L-14-0305
opment of expertise in pediatric rehabilitation therapists: Osborne, K. (2010). Arguing to learn in science: The role of col-
Changes in approach, self-knowledge, and use of enabling laborative critical discourse. Science, 328(5977), 463–466.
and customizing strategies. Developmental Neurorehabilitation, https://doi.org/10.1126/science.1183944
10(3), 223–240. https://doi.org/10.1080/17518420701302670 QSR International Pty Ltd. (2020). NVivo. https://www.
Kolb, S. M. (2012). Grounded theory and the constant compara- qsrinternational.com/nvivo-qualitative-data-analysis-software/
tive method: Valid research strategies for educators. Journal home
of Emerging Trends in Educational Research and Policy Stud- *Riedeman, S., & Turkstra, L. (2018). Knowledge, confidence,
ies, 3(1), 83–86. and practice patterns of speech-language pathologists working
Ludemann, A., Power, E., & Hoffmann, T. C. (2017). Investigat- with adults with traumatic brain injury. American Journal of
ing the adequacy of intervention descriptions in recent speech- Speech-Language Pathology, 27(1), 181–191. https://doi.org/
language pathology literature: Is evidence from randomized 10.1044/2017_AJSLP-17-0011
trials useable? American Journal of Speech-Language Pathology, Roberts, M. Y., Sone, B. J., Zanzinger, K. E., Bloem, M. E.,
26(2), 443–455. https://doi.org/10.1044/2016_AJSLP-16-0035 Kulba, K., Schaff, A., Davis, K. C., Reisfeld, N., & Goldstein,
Mays, N., Roberts, E., & Popay, J. (2001). Synthesising research H. (2020). Trends in clinical practice research in ASHA journals:
evidence. In N. Fulop, P. Allen, A. Clarke, & N. Black 2008-2018. American Journal of Speech-Language Pathology,
(Eds.), Studying the organisation and delivery of health services: 29(3), 1629–1639. https://doi.org/10.1044/2020_AJSLP-19-00011
research methods (pp. 188–220). Routledge. *Roulstone, S. (2011). Evidence, expertise, and patient preference
*McCauley, R. J., Strand, E., Lof, G. L., Schooling, T., & in speech-language pathology. International Journal of Speech-
Frymark, T. (2009). Evidence-based systematic review: Effects Language Pathology, 13(1), 43–48. https://doi.org/10.3109/
of nonspeech oral motor exercises on speech. American Jour- 17549507.2010.491130
nal of Speech-Language Pathology, 18(4), 343–360. https://doi. Rovira, C., Codina, L., Guerrero-Solé, F., & Lopezosa, C. (2019).
org/10.1044/1058-0360(2009/09-0006) Ranking by relevance and citation counts, a comparative
McCurtin, A., & Roddam, H. (2012). Evidence-based practice: study: Google Scholar, Microsoft Academic, WoS and
SLTs under siege or opportunity for growth? The use and Scopus. Future Internet, 11(9), 202–223. https://doi.org/10.
nature of research evidence in the profession. International 3390/fi11090202
Journal of Language & Communication Disorders, 47(1), 11–26. Sackett, D. L. (1997). Evidence-based medicine. Seminars in Perina-
https://doi.org/10.1111/j.1460-6984.2011.00074.x tology, 21(1), 3–5. https://doi.org/10.1016/S0146-0005(97)80013-4
*McCurtin, A., Murphy, C. A., & Roddam, H. (2019). Moving Sackett, D. L., & Rosenberg, W. M. (1995). The need for
beyond traditional understandings of evidence-based practice: evidence-based medicine. Journal of the Royal Society of Medi-
A total evidence and knowledge approach (TEKA) to treat- cine, 88(11), 620–624. https://doi.org/10.1177/014107689508801105
ment evaluation and clinical decision making in speech- Sackett, D. L., Rosenberg, W. M., Gray, J. A., Haynes, R. B., &
language pathology. Seminars in Speech and Language, 40(5), Richardson, W. S. (1996). Evidence based medicine: What it is
370–393. https://doi.org/10.1055/s-0039-1694996 and what it isn’t. BMJ, 312(7023), 71–72. https://doi.org/10.
*Mcleod, S., & Baker, E. (2014). Speech-language pathologists’ 1136/bmj.312.7023.71
practices regarding assessment, analysis, target selection, inter- Saldaña, J. (2009). The coding manual for qualitative researchers.
vention, and service delivery for children with speech sound SAGE Publications.
disorders. Clinical Linguistics & Phonetics, 28(7–8), 508–531. Seymour, H. N. (2004). The challenge of language assessment for
https://doi.org/10.3109/02699206.2014.926994 African American English-speaking children: A historical per-
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & PRISMA spective. Seminars in Speech and Language, 25(1), 3–12. https://
Group. (2009). Preferred reporting items for systematic reviews doi.org/10.1055/s-2004-824821

Fissel Brannick et al.: Clinical Evidence in Speech-Language Pathology 2957

*Selin, C. M., Rice, M. L., Girolamo, T., & Wang, C. J. (2019). phagia behavioral treatments. Part V–Applications for clinicians
Speech-language pathologists’ clinical decision making for and researchers. Journal of Rehabilitation Research & Develop-
children with specific language impairment. Language, Speech, ment, 46(2), 215–222. https://doi.org/10.1682/JRRD.2008.08.0093
and Hearing Services in Schools, 50(2), 283–307. https://doi. *Whiteneck, G., & Gassaway, J. (2012). The SCIRehab project:
org/10.1044/2018_LSHSS-18-0017 What rehabilitation interventions are most strongly associated
*Smith, P. A. (2018). On the horizon: Older adults with autism with positive outcomes after spinal cord injury? The Journal
in a changing health care environment. Perspectives of the of Spinal Cord Medicine, 35(6), 482–483. https://doi.org/10.
ASHA Special Interest Groups, 3(15), 4–14. https://doi.org/10. 1179/2045772312Y.0000000083
1044/persp3.SIG15.4 *Whiteneck, G. G., & Gassaway, J. (2013). SCIRehab uses
*Swift, M. C., Langevin, M., & Clark, A. M. (2017). Using critical practice-based evidence methodology to associate patient and
realistic evaluation to support translation of research into clinical treatment characteristics with outcomes. Archives of Physical
practice. International Journal of Speech-Language Pathology, Medicine and Rehabilitation, 94(4), S67–S74. https://doi.org/
19(3), 335–343. https://doi.org/10.1080/17549507.2017.1309067 10.1016/j.apmr.2012.12.022
*Thome, E. K., Loveall, S. J., & Henderson, D. E. (2020). A sur- *Whiteneck, G., Gassaway, J., Dijkers, M., & Jha, A. (2009).
vey of speech-language pathologists’ understanding and reported New approach to study the contents and outcomes of spinal
use of evidence-based practice. Perspectives of the ASHA cord injury rehabilitation: The SCIRehab Project. The Journal
Special Interest Groups, 5(4), 984–999. https://doi.org/10.1044/ of Spinal Cord Medicine, 32(3), 251–259. https://doi.org/10.
2020_PERSP-20-00008 1080/10790268.2009.11760779
*Wheeler-Hegland, K., Frymark, T., Schooling, T., McCabe, D., Williams, M., & Moser, T. (2019). The art of coding and the-
Ashford, J., Mullen, R., Hammond, C. S., & Musson, N. matic exploration in qualitative research. International Man-
(2009). Evidence-based systematic review: Oropharyngeal dys- agement Review, 15(1), 45–55.

2958 American Journal of Speech-Language Pathology • Vol. 31 • 2943–2958 • November 2022