761339

research-article2018
BHDXXX10.1177/0198742918761339Behavioral DisordersSplett et al.

Article
Behavioral Disorders

Comparison of Universal Mental Health

2018, Vol. 43(3) 344­–356
© Hammill Institute on Disabilities 2018
Reprints and permissions:
Screening to Students Already Receiving sagepub.com/journalsPermissions.nav
DOI: 10.1177/0198742918761339
https://doi.org/10.1177/0198742918761339

Intervention in a Multitiered System of journals.sagepub.com/home/bhd

Support

Joni W. Splett, PhD1, Kathryn M. Trainor, MEd1, Anthony Raborn, MAE1,

Colleen A. Halliday-Boykins, PhD2, Marlene E. Garzona, BA1,
Melissa D. Dongo, BA1, and Mark D. Weist, PhD3

Abstract
Despite schools increasingly adopting multitiered systems of support (MTSS) for prevention and intervention of mental
health concerns, many are slow to adopt universal mental health screening (UMHS), a core MTSS feature, due to concerns
about their limited capacity to meet the needs of all identified. In this study, we examined differences in the number and
characteristics of students who would be identified for intervention services when UMHS in an MTSS were added to
those students already receiving social, emotional, and behavioral supports. In a sample of 3,744 students in Grades 1 to 5
from six schools, 679 (18.1%) additional students were identified by screening, representing a 180.1% increase in students
identified with behavioral risk or need for mental health interventions. Using a series of stepwise logistic regression
analyses, we identified significant predictors of newly identified students including gender, number of office discipline
referrals, and externalizing, internalizing, and adaptive behavior ratings. Findings are discussed in relation to opportunities
for prevention and the systems needed in an MTSS to meet the needs of newly identified students.

Keywords
screening, behavior assessment, mental health, school, prevention

Along with the increased adoption of multitiered systems of criminal arrest; Darney, Reinke, Herman, Stormont, &
support (MTSS) has come more attention to universal screen- Ialongo, 2013; Hawton, Saunders, & O’Connor, 2012).
ing for academic, behavioral, and mental health concerns. Despite substantial evidence suggesting the need for uni-
Universal screening shifts the focus from a reactive, wait-to- versal mental health screening (UMHS; National Research
fail model to a proactive system in which needs are identified Council and Institute of Medicine, 2009) and the existence of
at their onset and interventions are delivered efficiently to the psychometrically valid screening tools, schools have been
level of need demonstrated (Dowdy et al., 2015). Universal slow to implement; and a range of concerns about how to
screening is a core component of MTSS, along with a con- conduct UMHS responsibly and ethically remain (Chafouleas,
tinuum of evidence-based practices, progress monitoring, Kilgus, & Wallach, 2010). For example, many schools are
data-based decision-making, and team-based implementa- concerned about their ability to respond effectively to an
tion (Jimerson, Burns, & VanDerHeyden, 2016; OSEP increase in students identified as needing intervention
Technical Assistance Center on Positive Behavioral (Chafouleas et al., 2010). It would be unethical to identify
Interventions and Supports, 2015). Beyond the MTSS litera- needs and not offer additional intervention supports, but the
ture, universal screening of mental health concerns has also resources of many schools are already overwhelmed (Baker,
gained increased attention in recent years as a means of clos-
ing the long-standing gap between need for mental health 1
University of Florida, Gainesville, USA
intervention and the receipt of such intervention among 2
Medical University of South Carolina, Charleston, USA
youth. In fact, less than 50% of youth with need for mental 3
University of South Carolina, Columbia, USA
health interventions are in treatment (Merikangas et al.,
Corresponding Author:
2010). Without effective identification and intervention, Joni W. Splett, University of Florida, 1414 Norman Hall, P.O. Box
youth in need are likely to encounter a myriad of deleterious 117050, Gainesville, FL 32608, USA.
outcomes (e.g., poor grades, suspension, drop out, suicidality, Email: [email protected]
Splett et al. 345

Kamphaus, Horne, & Winsor, 2006). However, little research limited to special education, are intended to advance a more
has quantified the increase in identified need following proactive, school-wide system that identifies all students in
UMHS implementation nor thoroughly examined character- need and provide opportunities for prevention and early
istics of newly identified students. Such research is critically intervention (Dowdy et al., 2015). School-wide or universal
needed to help schools plan for changes they might expect in data such as attendance, grades, curriculum-based mea-
the amount and type of identified need following UMHS sures, office discipline referrals (ODRs), suspensions, and
implementation. Toward this end, the current study aims to UMHS are used in the MTSS framework to identify youth
examine the demographic, school functioning, and mental in need of additional supports or intervention (Jimerson
health characteristics of additional students who would be et al., 2016). In the early years of implementing multitiered
identified and served if UMHS was implemented. To provide behavioral frameworks (e.g., School-Wide Positive
context for the study, we describe traditional referral-to-inter- Behavioral Interventions and Supports [SWPBIS]), ODRs
vention methods, UMHS, and the use of both methods within were the most commonly used school-wide data source to
an MTSS. both identify need for intervention and monitor intervention
progress and impact (Sugai, Sprague, Horner, & Walker,
2001). ODRs were popular due to their ease of use as an
Traditional Referral-to-Intervention
existing data source and relationship to future outcomes,
Methods such as increased suspension and dropout probability (Tobin
Traditionally, students with emotional and behavioral concerns & Sugai, 1999).
receiving intervention in schools are identified via teacher However, despite their predictive validity, ODRs do not
referral and served through special education (Kauffman, detect a full range of emotional and behavioral problems.
Mock, & Simpson, 2007). Ysseldyke, Vanderwood, and ODRs are more highly correlated with externalizing behav-
Shriner (1997) found that under this wait-to-fail model more ior problems (e.g., disruptive behavior, attention problems)
than 70% of students referred for a psychoeducational evalua- than with other behavioral and mental health problems
tion were placed in special education. Scholars continue to (e.g., concentration problems, depression, anxiety, adaptive
suggest that this referral and identification method is reactive skills; Walker, Cheney, Stage, Blum, & Horner, 2005). The
and inefficient for identification and intervention of all in need reliance on ODRs to identify at-risk students places the
(Kauffman et al., 2007). In fact, less than 1% of the school focus primarily on students with externalizing behavior
population served through special education is identified in the problems, passing over students at risk of internalizing
emotional disturbance (ED) category, the primary category behavior concerns (Walker et al., 2005). Additional data
designated for children with an emotional/behavioral distur- points, such as UMHS, are needed to conduct school-wide
bance (U.S. Department of Education, National Center for identification of students in need.
Education Statistics, 2011). Yet, nearly 12% of school-age
children or youth have an emotional/behavioral disturbance UMHS
with at least moderate impairment (Forness, Freeman, Within an MTSS, UMHS is an additional system for identi-
Paparella, Kauffman, & Walker, 2012). Furthermore, in one fying students in need of supports and services. In this
study, only one third of students determined by researchers to approach, each student in a school is assessed using a brief,
have the most severe and pervasive emotional and behavioral research-validated screener to determine if they display
concerns in four urban elementary schools were receiving spe- elevated levels of mental health concerns (Essex et al.,
cial education services under any disability category (Baker 2009). This reduces dependence on teacher referrals in a
et al., 2006). In another study, students receiving special educa- traditional, reactive identification system and measures
tion services for problems not related to mental health con- problem behavior across a spectrum of concerns, not just
cerns were 4 times more likely to have social, emotional, and/ externalizing behaviors captured by ODRs. Through
or behavioral difficulties than their peers in general education UMHS, all students are eligible for early identification and
but just as likely to have received mental health services in the intervention and the likelihood an at-risk child will be
past 12 months (Pastor & Reuben, 2009). In both studies, missed (i.e., false negative) is decreased (Dowdy et al.,
researchers suggested systems and services beyond traditional 2015). UMHS aligns with research and policy recommen-
special education are critically needed to meet the mental dations for the use of school-wide prevention systems to
health needs of all youth. address student needs proactively (Dowdy et al., 2015;
Individuals With Disabilities Education Improvement Act,
Identification of Intervention Need in 2004). Evidence shows that UMHS can accurately predict
student behavioral and emotional outcomes while providing
MTSS
valuable information for intervention (Essex et al., 2009).
The MTSS principles of data-based decision making and a Schools adopting UMHS as part of their MTSS have
tiered array of intervention services, inclusive of but not many options of psychometrically valid tools that measure
346 Behavioral Disorders 43(3)

a range of constructs via an array of administration proce- With regard to school functioning, students at risk of mental
dures (Splett, Mulloy, Philip, & Weist, 2014). For example, health problems tend to be absent from school more often
the Behavior Assessment System for Children, Third and have more disciplinary problems than those rated in the
Edition—Behavioral and Emotional Screening System normal range on measures of emotional and behavioral risk
Teacher Form (BASC-3 BESS; Reynolds & Kamphaus, (Gall, Pagano, Desmond, Perrin, & Murphy, 2000; Walker
2015) and Strengths and Difficulties Questionnaire (SDQ; et al., 2005). To better inform implementation procedures of
Goodman, 1997) include student self-, teacher-, and parent- UMHS, research is needed extending the work of Eklund
report forms that measure externalizing (BESS)/conduct and Dowdy (2014) to include these risk indicators with a
problems (SDQ), internalizing (BESS)/emotional symp- larger, real-world sample in schools implementing UMHS
toms (SDQ), adaptive skills (BESS), hyperactivity/inatten- school-wide.
tion (SDQ), peer relationships (SDQ), and prosocial
behaviors (SDQ). Although still reliant on teacher perspec-
tive, these universal screening procedures structure teach-
Present Study
ers’ qualitative perceptions and increase accuracy over that The purpose of the current study was to examine changes in
provided by traditional referral mechanisms (Eklund & the number and characteristics of students who would be
Dowdy, 2014). Furthermore, in an MTSS, UMHS data are identified for intervention services by UMHS but were not
considered in conjunction with other universal data such as already receiving services from the school. We used base-
attendance, grades, curriculum-based measures, and sus- line data from a sample of schools randomly assigned to the
pensions/expulsions to identify a student’s need for inter- intervention condition of a randomized trial where imple-
vention and match the necessary level of support (see Lane, menting UMHS within an MTSS was a component of the
Oakes, Ennis, & Hirsch, 2014). intervention (Award No. 2015-CK-BX-0018). Following
Although this approach is hypothesized to provide early practices modified from Eklund and Dowdy (2014), we
detection, prevention, and intervention opportunities for all used intervention receipt data collected monthly for 4
in need (Dowdy et al., 2015), prior research also suggests it months prior to implementing UMHS and identification
may increase the number of students identified as in need for results of UMHS to categorize students as school-only
services and strain already limited school resources (Baker identified, screener-only identified, both identified, or not
et al., 2006). Eklund and Dowdy (2014) compared students identified. Specifically, we addressed the following research
newly identified by UMHS to students already identified by questions:
traditional school referral and intervention practices. Schools
in their sample identified emotional and behavioral needs by Research Question 1: What percentage of the school
traditional means in 23% of their students and UMHS identi- population was identified by UMHS but not already
fied an additional 11.5% of students. These additional stu- being served by the school (screener only)?
dents identified by UMHS had similar behavioral needs, but Research Question 2: What is the percentage increase
less academic concern than those already identified by the in identified students when those identified only by the
school. Although a useful first step, these findings are lim- screener (screener only) are included in the full sample
ited in several ways. First, Eklund and Dowdy included all of identified students?
students who were referred, assessed, and/or provided inter- Research Question 3: How do the characteristics of stu-
vention for social, emotional, and behavioral concerns. dents not already being served but identified by the
Combining referral, assessment, and intervention receipt screener (screener only) differ from those identified by
into one variable overestimates the number of students actu- the school previously (both and school only)?
ally being served by the school, as some of these students
may have just been referred and/or assessed, but not receiv- Eklund and Dowdy (2014) detected an additional 11.5%
ing intervention. of the school population being identified as in need of men-
Furthermore, a significant body of research has identi- tal health intervention by UMHS, which was an increase of
fied key indicators beyond academic and behavioral func- nearly 50%. However, as previously described, this study
tioning predictive of students’ need for mental health likely overestimated the number of school-identified stu-
intervention. The behavior of students who are male and/or dents by including all students who had been referred,
African American tends to be rated as more problematic or assessed, and/or provided intervention for social, emo-
in need of mental health intervention than their female and/ tional, and behavioral concerns. We limited our categoriza-
or Caucasian counterparts (Rosenfield & Mouzon, 2013; tion of students identified by the school only to those
Splett et al., in press). Behavioral risk has also been found receiving intervention, including special education services,
to increase or intensify over time with students identified in to better reflect the change in service demand that would
kindergarten with any mental health symptom experiencing result from UMHS. Given our more conservative defini-
the greatest impairment by Grade 5 (Essex et al., 2009). tion, we hypothesized the percentage of students identified
Splett et al. 347

Table 1.  Sample Demographics.

Characteristics Total sample Both identified School only Screener only Neither identified
N 3,744 239 138 679 2,688
Percentage of total sample 100 6.38 3.69 18.14 71.79
Student gender (% male) 51.07 78.66 51.45 62.89 45.61
Student race/ethnicity (% White) 62.93 56.49 58.70 60.24 64.40
Student race/ethnicity (% African American) 20.91 27.20 31.16 25.18 18.75
Student race/ethnicity (% other) 16.16 16.32 10.14 14.58 16.85
Mean student age (years) 7.80 (1.80) 8.09 (1.98) 7.98 (1.78) 8.10 (1.76) 7.70 (1.78)
Mean Q1 ODRs 0.59 (2.79) 3.39 (7.50) 0.51 (2.54) 1.37 (3.34) 0.15 (1.30)
Mean Q1 grade point average 2.13 (0.77) 1.44 (0.81) 2.01 (0.79) 1.63 (0.77) 2.33 (0.66)
Mean number of Q1 absences 7.88 (11.11) 12.64 (13.17) 6.34 (8.54) 10.43 (13.49) 6.89 (10.10)
Mean BESS Externalizing Scale Score 3.59 (4.43) 10.11 (4.50) 2.04 (2.48) 9.11 (4.35) 1.69 (2.29)
Mean BESS Internalizing Scale Score 3.36 (3.62) 7.89 (3.89) 2.71 (2.73) 6.36 (4.09) 2.24 (2.63)
Mean BESS Adaptive Scale Score 7.50 (4.42) 2.70 (2.27) 8.33 (3.61) 3.05 (2.43) 9.01 (3.90)

Note. Numbers in parenthesis are standard deviations. Q1 = Quarter 1; ODRs = office discipline referrals; BESS = Behavioral and Emotional Screening System.

only by UMHS and percent increase in students with an a rural area as defined by the 2010 census (U.S. Census
identified need would be substantially more than previously Bureau, 2012). The number of students enrolled in study
detected. With regard to our third research question, we schools ranged from 558 to 794 (M = 710 students) with
hypothesized students not already being served but identi- nearly even distribution across grade levels (M = 14.4%
fied by the screener to have lower academic, behavioral, kindergarten, 16.2% Grade 1, 15.5% Grade 2, 16.9% Grade
and mental health risk than those identified by both meth- 3, 15.6% Grade 4, and 15.5% Grade 5). The percentage of
ods, but higher risk than those only identified by the school. White students enrolled in study schools ranged from 30.5
Such findings may indicate the newly identified students to 88.0 (M = 63.1%). Five of the six study schools were
have subclinical or emerging mental health concerns that composed of mostly White students, whereas the sixth
may be more amenable to early intervention efforts that pre- school had mostly African American students enrolled
vent further psychopathology (Durlak & Wells, 1998). (53.2%). Finally, the percentage of students receiving spe-
cial education services in study schools ranged from 6.1 to
34.8 (M = 18.4%).
Method
Participants and Setting Research Procedures
The sample included students in Grades 1 to 5 in six schools The study draws data from the Project About School Safety,
in two districts in two different Southeastern U.S. states. a 4-year, federally funded, randomized control trial of the
After removing 258 students (6.4%) due to not having a interconnected systems framework (ISF), a multitiered
UMHS score, the final sample included 3,744 students and intervention strategy connecting school mental health with
190 teachers who completed teacher-rated UMHS assess- SWPBIS through effective teams, data-based decision-
ments (100% of teachers in participating schools partici- making (e.g., UMHS), implementation support for evi-
pated). The gender distribution was approximately equal dence-based practices, and ongoing quality improvement to
with 49% female students. A majority of students were assure responsiveness to school and student needs (see
White (63%), and students were evenly distributed across Barrett, Eber, & Weist, 2013). Following a 6-month start-up
grade levels (15.3% kindergarten, 16.9% Grade 1, 16.1% phase, 24 elementary schools in South Carolina and Florida
Grade 2, 18.2% Grade 3, 17.2% Grade 4, and 16.3% Grade were randomized to three conditions and those in the inter-
5). On average, students in the sample were rated below at- vention condition (n = 8) began implementation. Institutional
risk cutoff on the UMHS measure for domains of external- Review Boards at each research and school site approved
izing, internalizing, and adaptive behaviors. Demographic the study procedures with a full waiver of written consent
variables, school functioning, and behavioral and emotional for the collection of data utilized in this study. Data are
risk of all students in the sample are reported in the second taken from the first year of UMHS implementation from
column of Table 1. schools randomized to the intervention condition if they
Students in the sample were enrolled in one of the six met two conditions: (a) had not implemented UMHS in past
study schools inclusive of prekindergarten to Grade 5. Five 5 years to establish a baseline, and (b) achieved at least an
of the six study schools were in urban areas and the sixth in 80% response rate within 2 weeks of monthly requests for
348 Behavioral Disorders 43(3)

intervention receipt forms (see description below). Two extremely elevated (71 and higher) levels of risk (Reynolds
schools, one in each school district, did not meet both crite- & Kamphaus, 2015). We considered students with an Over-
ria and were excluded from the sample. Data were collected all Risk Index T score in the elevated or extremely elevated
from August through November 2016, the first 4 months of ranges identified as at risk for behavior and emotional prob-
intervention implementation, to reduce biases from any lems by the screener. The BESS-Teacher also provides sub-
possible intervention effects. index raw scores, including the Externalizing Risk Index
(including behaviors from the hyperactivity, aggression,
and conduct problems domains of the BASC-3 TRS), Inter-
Implementation Procedures
nalizing Risk Index (including behaviors from the anxiety,
In the 3 years prior to implementing UMHS, all study depression, and somatization domains of the BASC-3 TRS),
schools were implementing a tiered model of prevention and Adaptive Skills Risk Index (including behaviors from
and intervention, including systems for both behavior and the adaptability, activities of daily living, functional com-
academic data review and intervention. In the semester in munication, social skills, leadership, and study skills
which data were collected, study schools measured imple- domains of the BASC-3 TRS). Students’ raw scores on
mentation fidelity of their tiered model via the Tiered each subindex were independent variables in each model
Fidelity Inventory (TFI; Algozzine et al., 2014) and tested.
Benchmarks of Quality (BoQ; Kincaid, Childs, & George,
2005). Both measures are widely used tools for measuring School identified.  Data were collected from two sources to
implementation of SWPBIS with adequate psychometric determine which students were receiving social, emotional,
properties (Mercer, McIntosh, & Hoselton, 2017). Schools and/or behavioral intervention prior to universal screening.
in one district completed all three tiers of the TFI, whereas First, we determined which students were receiving inter-
schools in the other district completed the BoQ for Tier 1 ventions in each participating school for the 4 months (i.e.,
implementation fidelity because of state-level requirements August through November) preceding administration of the
and Tiers 2 and 3 of the TFI for purposes of the larger study. BESS-Teacher via an Intervention Receipt Form. The Inter-
Study schools implemented Tier 1 with 62% to 88% fidelity vention Receipt Form was a project-developed spreadsheet
on the TFI and BoQ. Tier 2 was implemented with 58% to that captured the type, frequency, and duration of social,
88% fidelity on the TFI and Tier 3 76% to 94% fidelity. Two emotional, and behavioral services a student received. We
schools in each district were implementing all three tiers of defined intervention as any type of support or service pro-
PBIS with at least 80% fidelity, whereas the other two vided to students who are at risk of or have been identified
schools in the project were implementing Tiers 1, 2, and 3 as having a social, emotional, and/or behavioral problem.
with 58% to 76% fidelity with better implementation at At the beginning of the school year, all school administra-
Tiers 1 and 3 than Tier 2 in both schools. tors identified personnel who provided social, emotional,
As all study schools were in the intervention condition of and/or behavioral intervention in their school. A grant-
the larger study, all implemented UMHS as part of the ISF. funded district employee emailed a request to identified
Schools selected the BASC-3 BESS—Teacher Form-Child/ school employees to complete the Intervention Receipt
Adolescent (BESS-Teacher; Reynolds & Kamphaus, 2015) Form at the end of each month. The district employee
to implement UMHS. ISF teams in all study schools asked replaced students’ names with ID numbers and returned the
all teachers to complete the BESS-Teacher for all students spreadsheet to the research team. In total, 127 intervention
in their homeroom classrooms via an online system where receipt forms were solicited and 122 were returned (96.1%).
their classroom roster was preloaded. Students with ID numbers listed on any intervention receipt
form during the 4 months preceding administration of the
BESS-Teacher were coded as school identified. The second
Measures
data source for determining which students were receiving
UMHS.  The BESS-Teacher (Reynolds & Kamphaus, 2015) social, emotional, and/or behavioral intervention was extant
is a 20-item behavioral scale with items drawn from the data for special education eligibility. Students coded in the
BASC-3 Teacher Rating Scale (BASC-3 TRS; Reynolds & extant database as receiving special education services in
Kamphaus, 2015). It is completed by teachers for their stu- the categories of ED, other health impairment, autism,
dents, does not require any prior training, and is used for developmental disability, and/or intellectual disability were
students from Grades K to 12. It has been shown to possess included as school identified, as these students are likely to
adequate reliability (i.e., internal consistency, test–retest, receive social, emotional, and/or behavioral intervention
and interrater), construct validity, sensitivity, specificity, given the nature of their disability.
and factor structure (Reynolds & Kamphaus, 2015). Scor-
ing of the BESS-Teacher provides an Overall Risk Index T Extant student data.  The school district also provided student-
score indicating normal (score 30–60), elevated (61–70), or level demographic and behavioral data including grade,
Splett et al. 349

gender, race/ethnicity (African American, Caucasian, and package (Hartig, 2017), the deviance-based goodness of fit
other), Quarter 1 total number of ODRs, Quarter 1 absences chi-squared statistic, and McFadden’s R 2 (McFadden,
(both excused and unexcused), and Quarter 1 grades. Students 1974). These statistics were chosen for their ease of calcula-
received up to four grades in the areas of English language, tion and interpretation, and because they ask different ques-
arts, mathematics, social studies, and science. Quarter 1 tions about the fit of the models. McFadden’s R 2 , which
grades in each area were converted to a 0 to 3 scale, where D’s has a similar interpretation as the R 2 in a linear regression,
and F’s = 0, C’s = 1, B’s = 2, and A’s = 3. Then, these values was calculated by taking the ratio of the log-likelihood
were averaged across the number of grades a student received. value of the fit model and the log-likelihood value of the
intercept-only model and subtracting the result from 1.
Values that fall around 0.2 to 0.4 indicate very good model
Data Analyses fit (McFadden, 1979). For each logistic regression, outliers
Consistent with prior research (Eklund & Dowdy, 2014), with a deviance residual statistic greater than ±3 were
we considered students identified as at risk by both the removed and the model was refit.
school and the BESS-Teacher as both identified, students In addition, we used leave-one-out cross-validation
identified as at risk by the BESS-Teacher only as screener- (LOOCV; Stone, 1974) to assess the classification accuracy
only identified, those identified as at risk by the school only of the stepwise models as compared with the models with
as school-only identified, and students not identified as at each of the 10 selected variables. Briefly, using LOOCV, we
risk by the school or the BESS-Teacher as not identified. iteratively removed one data point, fit the model on the
To determine the characteristics of screener-only identi- remaining points, and used the coefficients of this fitted
fied students in comparison with those already identified by model to predict the classification of the removed data
the school (both and/or school-only identified), we con- point, then averaged the classification accuracy across the
ducted three logistic regression models: (a) both + school- entire data set. This provided an idea of how well the mod-
only versus screener-only identified, (b) both versus els predicted the data.
screener-only identified, and (c) school-only versus
screener-only identified. For each analysis, we used a for-
ward-and-backward stepwise variable selection procedure
Results
based on minimizing the Akaike information criterion As seen in Table 1, most students were not identified with
(AIC) within each step. We chose this procedure because it behavioral risk or in need for mental health interventions by
can automatically determine the variables that are the most the school or the screener (n = 2,688, 71.8%). Of those stu-
salient in predicting differences between groups without dents identified by either method (n = 1,056, 28.2%), most
having to regress on every combination of variables by were identified by the screener only (n = 679, 18.1%), fol-
hand. However, given the risk of biased regression coeffi- lowed by those identified by both the school and the
cients using this technique (Harrell, 2015), we first only screener (n = 239, 6.4%), and, finally, those identified only
included variables with strong empirical and theoretical by the school only (n = 138, 3.7%). Prior to UMHS, partici-
support for answering our research questions despite access pating schools provided intervention to 377 students
to a much larger data set. Second, we used AIC as the step- (10.1%). The 679 students newly identified by the screener
wise criteria instead of the traditional p value approach, represent a 180.1% increase in students with identified risk
which allows for nonsignificant coefficients in the final or need for intervention.
models if they significantly improve the quality of the
model compared with other candidate models. Characteristics of Screener- and School-Identified
Three independent variable categories were included in
the initial model: sociodemographic variables (gender with
Students
reference category being female, race/ethnicity with refer- Model fit.  The results of the logistic regressions, including
ence category African American students, and student age), the fit statistics and sample sizes, can be seen in Table 2.
school functioning variables (ODRs, suspension days, The simulation-based residual analysis indicated that the
grades, and attendance), and problem-type variables (BESS- residuals of the data fit each model adequately, with the K-S
Teacher externalizing, internalizing, and adaptive skills tests indicating that the model residuals were approximately
subindex scores). The logistic regressions were performed uniform for each logistic regression (ranging from .016–
in R version 3.3.2 (R Core Team, 2016) with the glm func- .042 with p values from .981–.126). The deviance-based
tion and the variable selection procedure uses the step func- chi-square statistic shows that each of the models ade-
tion, both in the base stats package. quately explains the data better than random chance.
To determine the fit of these regressions, we computed Finally, the R 2 statistics for the models that fit well had two
three statistics for each: a simulation-based Kolmogorov– groupings: either below the cutoff for very good fit (around
Smirnov (K-S) Test of model residuals with the DHARMa 6% of variance explained) or significantly above it (around
350 Behavioral Disorders 43(3)

Table 2.  Logistic Regression Results for At-Risk Behavior Identification Status.

Retained variables B SE B p Odds

Both + school only (vs. screener only) identified
 Intercept −0.361 0.288 .210 0.697
  Student gender: male 0.518 0.176 .003 1.678
  Number of Q1 ODRs 0.072 0.024 .003 1.075
  Average Q1 grades −0.193 0.115 .093 0.824
  BESS Externalizing Scale Score −0.098 0.021 <.001 0.906
  BESS Internalizing Scale Score −0.035 0.020 .084 0.966
  BESS Adaptive Scale Score 0.129 0.030 <.001 1.138
  χ2 (df = 6) 71.430 <.001  
Both (vs. screener only) identified
 Intercept −2.269 0.461 <.001 0.103
  Student gender: male 0.642 0.181 <.001 1.901
  Student age 0.002 0.043 .966 1.002
  BESS Externalizing Scale Score 0.026 0.018 .142 1.027
  BESS Internalizing Scale Score 0.110 0.020 <.001 1.116
  BESS Adaptive Scale Score −0.087 0.035 .013 0.916
  χ2 (df = 5) 53.525 .000  
School only (vs. screener only) identified
 Intercept 5.157 1.336 <.001 173.556
  Student age −0.406 0.128 .002 0.666
  BESS Externalizing Scale Score −1.011 0.143 <.001 0.364
  BESS Internalizing Scale Score −0.815 0.120 <.001 0.443
  BESS Adaptive Scale Score 1.205 0.174 <.001 3.336
  χ2 (df = 4) 583.581 <.001  

Note. Both Identified = students identified as at risk by both the school and BESS-Teacher; screener only = students identified as at risk by the BESS-
Teacher only; school only = students identified as at risk by the school only. Q1 = Quarter 1; ODRs = office discipline referrals; BESS = Behavioral and
Emotional Screening System.

82% of variance explained). Given these results, we con- each of the remaining variables constant. The coefficient for
cluded that each of the models fit the data adequately the BESS-Teacher internalizing subindex score was not sta-
enough to continue the analysis. tistically significant. This model explained 7.3% of the vari-
ation between groups beyond chance (McFadden’s R 2 = .073
All school identified versus screener-only identified.  As seen in ) with a LOOCV classification accuracy of 73.3%.
Table 2, the model comparing screener-only identified stu-
dents to both and school-only identified students indicated Both versus screener-only identified. The model comparing
that student gender, number of ODRs, and average Quarter 1 screener-only identified students to those both identified
grades were retained predictors of group assignment when contained two insignificant variables that nonetheless pro-
holding all three BESS-Teacher subindex scores constant. vided a reduction in AIC verified by removing the insignifi-
Males and students with larger numbers of ODRs had higher cant variables one at a time; therefore, we left both in for
odds of being identified by the school than the screener analysis. The variables selected explained 5.2% of the vari-
alone. Students with higher grades, however, were less likely ance between the two groups (McFadden’s R 2 = .052) and
to be identified by the school than by the screener only; all had a classification accuracy of 72.9% from the LOOCV. In
other variables equal, the schools were more likely to iden- this model, males had higher odds of being both identified
tify the lower achieving students whereas the students only than only identified by the screener while holding age and
identified by the BESS-Teacher tended to have higher BESS-Teacher externalizing, internalizing, and adaptive
grades. However, although average Quarter 1 grades was a skills subindex scores constant. The direction of the coeffi-
retained variable due to reducing AIC, it is not significant at cients for the BESS-Teacher subindex scores characterized
α = .05 . In addition, students with BESS-Teacher screener-only students as having lower subindex scores
externalizing, internalizing, and adaptive skills subindex
­ than both identified students. However, the BESS-Teacher
scores associated with higher risk had higher odds of being externalizing subindex score coefficient was not ­statistically
­identified by the school than the screener only, while holding significant.
Splett et al. 351

School-only versus screener-only identified. The model com- than those students the school had already identified. Of stu-
paring students only identified by the screener with those dents identified by the screener, those already identified by
only identified by the school showed that student age is the the school were predominately male (78.7%) and those not
best predictor, after accounting for the known differences in already identified by the school were about 63% male, which
BESS-Teacher externalizing, internalizing, and adaptive was still more than the distribution of gender in the entire
skills subindex scores. Older students had higher odds of sample (51.1% male). The overidentification of male stu-
only being identified by the screener. As expected, given the dents by the school is consistent with prior research showing
use of the BESS-Teacher Overall Risk Index Score to define male students are nearly 3.5 times more likely to be identified
groups, higher levels of clinical risk on the BESS-Teacher as having an emotional disability (Coutinho & Oswald, 2005)
subindex scores decreased the odds that a student is identi- and that UMHS results in male students being nominated 3
fied by the school alone. This model fit well, explaining times more often than female students (Young, Sabbah,
about 82% of the variance between groups (McFadden’s Young, Reiser, & Richardson, 2010).
R 2 = .815) and accurately classifying 96.6% of cases. Race/ethnicity was not a significant predictor of being
identified by the screener and/or school in any of our mod-
els. This is despite the fact that the proportion of African
Discussion
American students in all identified groups was higher than
We examined the impact UMHS would have on schools’ their representation in the overall sample and the not identi-
intervention delivery systems by investigating the number fied group (see Table 1). This neither supports nor contra-
and characteristics of students identified by UMHS but not dicts the notion that UMHS mitigates the overrepresentation
already being served. In doing so, we aimed to empirically of racial/ethnic minority students in special education ser-
inform service delivery recommendations and changes for vices (Raines, Dever, Kamphaus, & Roach, 2012). It is pos-
schools implementing UMHS within an MTSS. sible that the same teacher biases present in referrals to
special education are also evident in teacher ratings of stu-
dent behavior. Future research examining this issue is
Increase in Identified Students
needed as UMHS continues to be adopted by schools.
As expected, we detected that UMHS resulted in a substan- Functionally, those identified by the screener only dem-
tial increase of students identified as at risk beyond those onstrated less academic, behavioral, and mental health risk
schools were already serving (180.1%), especially in com- than those identified by both the screener and the school, but
parison with previous findings (50% increase; Eklund & more risk than those only identified by the school. The
Dowdy, 2014). Schools in the current study were only serv- screener-only students had fewer ODRs than those identi-
ing about 10% of students prior to UMHS, but adding those fied by both the school and screener, but descriptively had
only identified by the screener would increase the propor- more disciplinary problems than those only identified by the
tion served to nearly 26%. Both these proportions are incon- school (see Table 1). Students only identified by the screener
sistent with the MTSS heuristic that 15% to 20% of students also had lower risk scores on the BESS-Teacher subindices
will need additional supports beyond the universal curricu- than students identified by both the school and screener.
lum, despite implementation fidelity data indicating strong Ultimately, it appears students only identified by the
implementation of Tier 1 system and practices in nearly all screener have an elevated risk profile, but not as severe aca-
study schools. This suggests traditional school identifica- demic, behavioral, or mental health impairment as those also
tion methods are likely insufficient to identify all in need, identified by the school. This finding is inconsistent with
but also amplifies existing concerns that UMHS may iden- prior research by Eklund and Dowdy (2014), which found
tify more students than schools with already limited no significant academic or mental health differences between
resources are able to serve (Baker et al., 2006; Chafouleas screener-only and both identified students. Findings of the
et al., 2010). Advocates of UMHS should be cautious in current study extend this prior research by comparing groups
recommending UMHS before ensuring schools have the of identified students on multiple data sources typically con-
systems and practices in place to provide those identified by sidered by school intervention teams (e.g., ODRs, suspen-
UMHS with effective interventions. sions, attendance). Simply stated, schools identified those at
most risk, but adding UMHS may provide an opportunity for
prevention and early intervention by detecting a group of
Profile of Screener-Only Students
students with elevated, but less extreme, impairment.
We found students not already being served but identified as
at risk by UMHS were demographically and functionally dif-
Limitations
ferent than those students already receiving intervention from
the schools. Demographically, those identified by the screener Findings of the current study should be interpreted in light of
only were less likely to be male and more likely to be older several limitations. First, the generalizability of our findings
352 Behavioral Disorders 43(3)

is limited because we only sampled from elementary schools (e.g., feeling sad or withdrawn for at least 2 weeks; Jensen
in two school districts and only considered identification of et al., 2011) that are predictive of students with unmet men-
UMHS using one screener when risk in an MTSS is typi- tal health needs. Including such variables in future research
cally identified using multiple sources of data (e.g., aca- may improve interpretability and implications of related
demic, attendance, disciplinary history). We also had to findings.
remove nearly 6.5% of our sample due to missing BESS-
Teacher data. Future research should seek to replicate these
Implications for Future Research and Practice
findings with middle and high schools, especially given the
increased need for mental health services typically identified We found students identified by the BESS-Teacher but not
in middle and high school students (Merikangas et al., 2010), previously receiving intervention to have elevated risk, but
and using additional UMHS tools and indicators of risk. not as extreme as those identified and already receiving
Given that other UMHS tools measure different or addi- intervention services. In addition, those newly identified by
tional constructs in different ways than the BESS-Teacher the screener had fewer ODRs and absences than those
(e.g., self vs. teacher report), the number and characteristics already known, indicating these traditional indicators of
of students identified by the screener may vary. Examining risk are not sufficient to detect all students in need. These
variations across tools may help schools select a UMHS tool findings suggest that using UMHS will proactively detect a
that best fits their population and desired outcomes. greater breadth of risk and provide expanded opportunities
Another limitation is our method for identifying students for prevention and early intervention (Dowdy et al., 2015).
already receiving intervention from the school. We choose However, to meet the increased demand of students in need
to define traditional school identification as those receiving and seize the opportunity for proactive prevention and early
intervention for social, emotional, or behavioral concerns intervention, schools implementing UMHS need to be pre-
because it best matches the concern of school personnel that pared with adequate plans for how they will use results to
they will not be able to adequately meet the intervention evaluate and bolster their Tier 1 and Tier 2 systems and
needs of students newly identified by UMHS. However, practices.
this narrow definition and school-reported nature of the
data collected missed students who had likely been previ- Tier 1 systems and practices.  Given study schools adequately
ously considered for intervention, assessed, and determined implemented Tier 1 of SWPBIS yet identified nearly 26% of
ineligible. Prior studies also surveyed school personnel to students as in need of intervention when using a combina-
identify students referred, assessed, and determined ineli- tion of traditional and UMHS methods, Tier 1 systems and
gible, in addition to those currently receiving intervention. practices should first be evaluated and improved to promote
This likely led to the substantial difference in the increased universal mental wellness and prevent mental health con-
proportion of students identified by UMHS as in need in our cerns. For example, schools implementing SWPBIS estab-
study (from 49.5% in Eklund & Dowdy, 2014, to 180.1% in lish behavioral expectations at Tier 1 (e.g., safe, responsible,
the current study). Future research should compare these and respectful) and implement a school-wide behavioral
methods of defining the school-identified groups using the curriculum to teach and reinforce these expectations across
same sample and analyses to best inform recommendations school environments. We recommend expanding traditional
for referral, assessment, and intervention in an MTSS Tier 1 behavioral curriculum to teach the social and emo-
implementing UMHS. tional skills needed to meet the school’s behavioral expecta-
A final limitation is the small amount of variance tions (e.g., self-management, social skills; see Weist et al.,
explained by two of the three models. The theoretically rel- 2018). Schools can do this by adopting an evidence-based,
evant variables did not create models with excellent model school-wide curriculum focused on social–emotional skill
fit according to McFadden’s R 2 . Despite the retained vari- building such as Promoting Alternative Thinking Strategies
ables having almost the same classification accuracy as the (PATHS; Domitrovich, Cortes, & Greenberg, 2007), Strong
models with all 10 variables, a large proportion of the vari- Kids (Merrell & Gueldner, 2010), Tools for Getting Along
ance in two of our models was left unexplained. This indi- (Smith et al., 2016), and Second Step (Committee for Chil-
cates that the variables that were theoretically relevant dren, 2011). Implementing a school-wide curriculum for
according to the literature were not as strongly predictive of social, emotional, and behavioral skills may be especially
differences between newly identified students and students critical for schools with high rates of mental health concerns
already receiving intervention at school as anticipated. As a to address concerns early and bring the number of students
result, the significance of the retained variables and the with need for Tier 2 and 3 interventions down to a service-
interpretation of their effects in these two models should be able proportion (Baker et al., 2006).
viewed with caution and further research to test and identify
variables with more explanatory power pursued. For exam- Tier 2 practices.  Our results indicate newly identified stu-
ple, prior research has detected specific problem behaviors dents demonstrate risk but less than those already receiving
Splett et al. 353

intervention, suggesting the need for efficient but less inten- intervention. Our results suggest school-based MTSS teams
sive intervention services. For example, for students with using multiple data sources including UMHS must have the
externalizing concerns, substantial evidence supports the organizational structures in place to handle an increase in
efficiency and effectiveness of an intervention strategy the number of students and scope of problems discussed.
commonly referred to as Check, Connect, and Expect They must be prepared to work efficiently with a preplanned
(Cheney, Stage, Hawken, Lynass, Mielenz, & Waugh, agenda, quick access to and understanding of data, and
2009), Check In Check Out (CICO; Hawken & Horner, ongoing accountability of team members to be present, ful-
2003), and/or the Behavior Education Program (Crone, fill assigned roles, and follow up as assigned (Splett et al.,
Horner, & Hawken, 2004), which all involve students meet- 2017). We encourage school teams to evaluate their current
ing with an adult mentor at the start of the day to set behav- team practices against those recommended in an MTSS and
ioral goals, and again at end of the day to reflect on those make plans for improvement. To this end, Splett and col-
goals. leagues (2017) identified recommended teaming practices
Cognitive–behavioral interventions are typically be sug- in an MTSS with a demonstrative case study and then pro-
gested for internalizing concerns such as depression and vided a review of tools school may consider using to evalu-
anxiety (Weersing, Jeffreys, Do, Schwartz, & Bolano, ate and make improvement plans for their current teams.
2017), but few are available that are efficient for less inten-
sive concerns and feasible for school-employed mental
health professionals to implement (Bruhn, Lane, & Hirsch,
Conclusion
2014; Hanchon & Fernald, 2013). One promising interven- UMHS efficiently determines the nature and level of all stu-
tion incorporates cognitive–behavioral strategies with dents’ mental health needs so that timely and appropriate
CICO by modifying the morning checking-in and afternoon intervention can be provided and, hence, reduce the stag-
checking-out time to include instruction, reteaching, and gering unmet need. Results from the current study raise
reinforcement of prosocial replacement behaviors (e.g., eye some possible issues in realizing these benefits. First, the
contact, speaking in appropriate tone of voice) and coping number of students identified as at risk of emotional and
strategies (e.g., challenging negative thoughts to help a stu- behavioral difficulties would likely increase dramatically
dent respond appropriately when something does not go his with the implementation of UMHS, which might over-
or her way, Hunter, Chenier, & Gresham, 2013). For stu- whelm the capacity of schools to provide services. Second,
dents who are unresponsive to this efficient intervention, the characteristics of students newly identified by the
schools can consider using out-of-class time to deliver a screener are likely to differ from those already receiving
manualized curriculum with evidence of behavioral services in ways that have implications for the nature and
improvement for students at risk of internalizing behavior delivery of interventions. These findings necessitate
problems, such as Support for Students Exposed to Trauma expanding the array of interventions provided (e.g., inter-
(Jaycox, Langley, & Dean, 2009), Strong Kids (Merrell & ventions to address internalizing difficulties are likely
Gueldner, 2010), PATHS (Domitrovich et al., 2007), and needed) and the capacity of school systems to manage the
Coping Cat (Kendall, 1990). increased demand for services from students with identified
For students at risk of adaptive behavior problems, needs (e.g., efficient teaming practices to consider interven-
instruction in social, study, organizational, and/or work tion need and monitor progress). To achieve the aims of
completion skills can be integrated into academic instruc- UMHS and make these necessary changes, additional
tion to minimize missed classroom time (Bruhn et al., 2014; research and technical assistance are critically needed.
Rodriguez, Loman, & Borgmeier, 2016). However, if stu-
dents are not responsive to this less intensive and efficient Declaration of Conflicting Interests
intervention, pull-out programs include Skillstreaming
The author(s) declared no potential conflicts of interest with
(McGinnis, 2011) and Stop and Think (Knoff, 2001). respect to the research, authorship, and/or publication of this
article.
Tier 2 systems.  Beyond expanding Tier 1 and 2 practices,
schools must be prepared with adequate systems, such as Funding
teaming, coaching, and technical assistance, to implement
The author(s) disclosed receipt of the following financial support
the entire MTSS with sustained fidelity. In fact, McIntosh
for the research, authorship, and/or publication of this article: This
and colleagues (2013) found teaming practices (e.g., use of project was supported by Award No. 2015-CK-BX-0018, awarded
data) and district-level capacity building (e.g., coaching, by the National Institute of Justice, Office of Justice Programs,
professional development) significantly predicted sustained U.S. Department of Justice. The opinions, findings, and conclu-
implementation of SWPBIS. Specifically, problem-solving sions or recommendations expressed in this publication are those
or referral teams have traditionally been the venue for con- of the author(s) and do not necessarily reflect those of the
sidering the needs of identified students and planning Department of Justice.
354 Behavioral Disorders 43(3)

References School Mental Health, 6, 40–49. doi:10.1007/s12310-013-

9109-1
Algozzine, R. F., Barrett, S., Eber, L., George, H., Horner, R. H.,
Essex, M. J., Kraemer, H. C., Slattery, M. J., Burk, L. R., Thomas
Lewis, T. J., . . . Sugai, G. (2014). SWPBIS Tiered Fidelity
Boyce, W., Woodward, H. R., & Kupfer, D. J. (2009). Screening
Inventory. Eugene, OR: OSEP Technical Assistance Center
for childhood mental health problems: Outcomes and early
on Positive Behavioral Interventions & Supports. Available
identification. Journal of Child Psychology and Psychiatry,
from http://www.pbis.org
50, 562–570. doi:10.1111/j.1469-7610.2008.02015.x
Baker, J. A., Kamphaus, R. W., Horne, A. M., & Winsor, A. P.
Forness, S. R., Freeman, S. F. N., Paparella, T., Kauffman, J. M.,
(2006). Evidence for population-based perspectives on chil-
& Walker, H. M. (2012). Special education implications of
dren’s behavioral adjustment and needs for service delivery
point and cumulative prevalence for children with emotional
in schools. School Psychology Review, 35, 31–46.
or behavioral disorders. Journal of Emotional and Behavioral
Barrett, S., Eber, L., & Weist, M. D. (2013). Advancing educa-
Disorders, 20, 4–18. doi:10.1177/1063426611401624
tion effectiveness: An interconnected systems framework for
Gall, G., Pagano, M. E., Desmond, M. S., Perrin, J. M., & Murphy,
Positive Behavioral Interventions and Supports (PBIS) and
J. M. (2000). Utility of psychosocial screening at a school-
school mental health. Eugene: University of Oregon Press.
based health center. Journal of School Health, 70, 292–298.
Bruhn, A. L., Lane, K. L., & Hirsch, S. E. (2014). A review of
doi:10.1111/j.1746-1561.2000.tb07254.x
tier 2 interventions conducted within multitiered models of
Goodman, R. (1997). The Strengths and Difficulties Questionnaire:
behavioral prevention. Journal of Emotional and Behavioral
A research note. Journal of Child Psychology and Psychiatry,
Disorders, 22, 171–189. doi:10.1177/1063426613476092
Chafouleas, S. M., Kilgus, S. P., & Wallach, N. (2010). 38, 581–586. doi:10.1111/j.1469-7610.1997.tb01545.x
Ethical dilemmas in school-based behavioral screen- Hanchon, T. A., & Fernald, L. N. (2013). The provision of coun-
ing. Assessment for Effective Intervention, 35, 245–252. seling services among school psychologists: An exploration
doi:10.1177/1534508410379002 of training, current practices, and perceptions. Psychology in
Cheney, D., Stage, S. A., Hawken, L. S., Lynass, L., Mielenz, C., the Schools, 50, 651–671. doi:10.1002/pits.21700
& Waugh, M. (2009). A 2-year outcome study of the check, Harrell, F. (2015). Regression modeling strategies: With applica-
connect, and expect intervention for students at risk for severe tions to linear models, logistic and ordinal regression, and
behavior problems. Journal of Emotional and Behavioral survival analysis. New York, NY: Springer. doi:10.1007/978-
Disorders, 17, 226–243. doi:10.1177/1063426609339186 1-4757-3462-1
Committee for Children. (2011). Second step: Violence prevention Hartig, F. (2017). DHARMa: Residual diagnostics for hierarchi-
curriculum (4th ed.). Seattle, WA: Author. cal (multi-level/mixed) regression models (R package ver-
Coutinho, M. J., & Oswald, D. P. (2005). State variation in gender sion 0.1.5). Retrieved from https://CRAN.R-project.org/
disproportionality in special education: Findings and recom- package=DHARMa
mendations. Remedial and Special Education, 26, 7–15. doi:1 Hawken, L. S., & Horner, R. H. (2003). Evaluation of a targeted
0.1177/07419325050260010201 group intervention within a school-wide system of behavior
Crone, D. A., Horner, R. H., & Hawken, L. S. (2004). Responding support. Journal of Behavioral Education, 12, 225–240.
to problem behavior in schools: The behavior education pro- Hawton, K., Saunders, K. E., & O’Connor, R. C. (2012). Self-
gram. New York, NY: Guilford. harm and suicide in adolescents. The Lancet, 379, 2373–2382.
Darney, D., Reinke, W. M., Herman, K. C., Stormont, M., & doi:10.1016/S0140-6736(12)60322-5
Ialongo, N. S. (2013). Children with co-occurring academic Hunter, K. K., Chenier, J. S., & Gresham, F. M. (2013). Evaluation
and behavior problems in first grade: Distal outcomes in of Check In/Check Out for students with internalizing behav-
twelfth grade. Journal of School Psychology, 51, 117–128. ior problems. Journal of Emotional and Behavioral Disorders,
doi:10.1016/j.jsp.2012.09.005 22, 135–148. doi:10.1177/1063426613476091
Domitrovich, C. E., Cortes, R. C., & Greenberg, M. T. (2007). Individuals With Disabilities Education Improvement Act, U. S.
Improving young children’s social and emotional compe- C. (1400). (2004).
tence: A randomized trial of the preschool “PATHS” curricu- Jaycox, L., Langley, A., & Dean, K. L. (2009). Support for stu-
lum. Journal of Primary Prevention, 28, 67–91. doi:10.1007/ dents exposed to trauma: The SSET program. Santa Monica,
s10935-007-0081-0 CA: RAND Corporation.
Dowdy, E., Furlong, M., Raines, T. C., Bovery, B., Kauffman, B., Jensen, P. S., Goldman, E., Offord, D., Costello, E. J., Friedman,
Kamphaus, R. W., . . . Murdock, J. (2015). Enhancing school- R., Huff, B., . . . Roberts, R. (2011). Overlooked and under-
based mental health services with a preventive and promotive served: “Action signs” for identifying children with unmet
approach to universal screening for complete mental health. mental health needs. Pediatrics, 128, 970–979. doi:10.1542/
Journal of Educational and Psychological Consultation, 25, peds.2009-0367
178–197. doi:10.1080/10474412.2014.929951 Jimerson, S. R., Burns, M. K., & VanDerHeyden, A. M. (2016).
Durlak, J. A., & Wells, A. M. (1998). Evaluation of indicated From response to intervention to multi-tiered systems of sup-
preventive intervention (secondary prevention) mental health port: Advances in the science and practice of assessment
programs for children and adolescents. American Journal and intervention. In S. R. Jimerson, M. K. Burns, & A. M.
of Community Psychology, 26, 775–802. doi:10.1023/ VanDerHeyden (Eds.), Handbook of response to interven-
A:1022162015815 tion: The science and practice of multi-tiered systems of sup-
Eklund, K., & Dowdy, E. (2014). Screening for behavioral and port (2nd ed., pp. 1–6). New York, NY: Springer Science +
emotional risk versus traditional school identification methods. Business Media. doi:10.1007/978-1-4899-7568-3_1
Splett et al. 355

Kauffman, J. M., Mock, D. R., & Simpson, R. L. (2007). Problems Pastor, P. N., & Reuben, C. A. (2009). Emotional/behavioral diffi-
related to underservice of students with emotional or behav- culties and mental health service contacts of students in special
ioral disorders. Behavioral Disorders, 33, 43–57. education for non-mental health problems. Journal of School
Kendall, P. C. (1990). Coping cat workbook. Ardmore, PA: Health, 79, 82–89. doi:10.1111/j.1746-1561.2008.00380.x
Workbook Publishing. Raines, T. C., Dever, B. V., Kamphaus, R. W., & Roach, A. T.
Kincaid, D., Childs, K., & George, H. P. (2005). School-wide (2012). Universal screening for behavioral and emotional risk:
benchmarks of quality. Unpublished instrument, University A promising method for reducing disproportionate placement
of South Florida, Tampa. in special education. The Journal of Negro Education, 81,
Knoff, H. M. (2001). The stop & think social skills program. 283–296. doi:10.7709/jnegroeducation.81.3.0283
Longmont, CO: Sopris West. R Core Team. (2016). R: A language and environment for statisti-
Lane, K. L., Oakes, W. P., Ennis, R. P., & Hirsch, S. E. (2014). cal computing. Vienna, Austria: R Foundation for Statistical
Identifying students for secondary and tertiary prevention Computing. Retrieved from https://www.R-project.org/
efforts: How do we determine which students have Tier 2 Reynolds, C. R., & Kamphaus, R. W. (2015). BASC-3 Behavioral
and Tier 3 needs? Preventing School Failure: Alternative and Emotional Screening System manual. Circle Pines, MN:
Education for Children and Youth, 58, 171–182. doi:10.1080/ Pearson.
1045988X.2014.895573 Rodriguez, B. J., Loman, S. L., & Borgmeier, C. (2016). Tier 2
McFadden, D. (1974). Conditional logit analysis of qualitative interventions in positive behavior support: A survey of school
choice behavior. In P. Zarembka (Ed.), Frontiers in econo- implementation. Preventing School Failure: Alternative
metrics (pp. 105–142). New York, NY: Academic Press. Education for Children and Youth, 60, 94–105. doi:10.1080
McFadden, D. (1979). Quantitative methods for analyzing travel /1045988X.2015.1025354
behavior of individuals: Some recent developments. In D. A. Rosenfield, S., & Mouzon, D. (2013). Gender and mental health.
Hensher & P. R. Stopher (Eds.), Behavioral travel modeling In C. S. Aneshensel, J. C. Phelan, & A. Bierman (Eds.),
(pp. 279–318). London, England: Groom Helm. Handbook of the sociology of mental health (2nd ed., pp.
McGinnis, E. (2011). Skillstreaming the elementary school child: 277–296). New York, NY: Springer. doi:10.1007/978-94-
A guide for teaching prosocial skills (3rd ed.). Champaign, 007-4276-5_14
IL: Research. Smith, S. W., Daunic, A. P., Aydin, B., Van Loan, C. L., Barber,
McIntosh, K., Mercer, S. H., Hume, A. E., Frank, J. L., Turri, B. R., & Taylor, G. G. (2016). Effect of tools for getting
M. G., & Mathews, S. (2013). Factors related to sustained along on student risk for emotional and behavioral problems
implementation of schoolwide positive behavior support. in upper elementary classrooms: A replication study. School
Exceptional Children, 79, 293–311. Psychology Review, 45, 73–92. doi:10.17105/SPR45-1.73-92
Mercer, S. H., McIntosh, K., & Hoselton, R. (2017). Splett, J. W., Mulloy, M., Philip, A., & Weist, M. D. (2014).
Comparability of fidelity measures for assessing tier 1 Mental health screening for school children. In B. McPherson
school-wide positive behavioral interventions and supports. & C. Driscoll (Eds.), School health screening systems: The
Journal of Positive Behavior Interventions, 19, 195–204. complete perspective (pp. 137–161). New York, NY: Nova
doi:10.1177/1098300717693384
Science.
Merikangas, K. R., He, J., Burstein, M., Swanson, S. A.,
Splett, J. W., Perales, K., Halliday-Boykins, C. A., Gilchrest, C. E.,
Avenevoli, S., Cui, L., . . . Swendsen, J. (2010). Lifetime
Gibson, N., & Weist, M. D. (2017). Best practices for teaming
prevalence of mental disorders in U.S. adolescents: Results
and collaboration in the interconnected systems framework.
from the National Comorbidity Study-Adolescent Supplement
Journal of Applied School Psychology, 33, 347–368. doi:10.1
(NCS-A). Journal of American Academy of Child Adolescent
080/15377903.2017.1328625
Psychiatry, 49, 980–989. doi:10.1016/j.jaac.2010.05.017
Splett, J. W., Smith-Millman, M., Raborn, A., Warmbold-Brahn,
Merrell, K. W., & Gueldner, B. A. (2010). Preventative inter-
ventions for students with internalizing disorders: Effective K., Maras, M. A., & Flaspohler, P. (in press). Student, teacher
strategies for promoting mental health in schools. In M. R. and classroom predictors of between-teacher variance of stu-
Shinn & H. M. Walker (Eds.), Interventions for achievement dents’ teacher-rated behavior. School Psychology Quarterly.
and behavior in a three-tier model including RTI (3rd ed., pp. Stone, M. (1974). Cross-validation and multinomial prediction.
729–823). Bethesda, MD: National Association of School Biometrika, 61, 509–515. doi:10.2307/2334733
Psychologists. Sugai, G., Sprague, J. R., Horner, R. H., & Walker, H. M. (2001).
National Research Council and Institute of Medicine. (2009). Preventing school violence: The use of office discipline
Preventing mental, emotional, and behavioral disorders referrals to assess and monitor school-wide discipline inter-
among young people: Progress and possibilities (M. E. ventions. In H. M. Walker & M. H. Epstein (Eds.), Making
O’Connell, T. Boat, & K. E. Warner, Eds.). Washington, DC: schools safer and violence free: Critical issues, solutions, and
National Academies Press. Retrieved from https://www.ncbi. recommended practices (pp. 50–57). Austin, TX: PRO-ED.
nlm.nih.gov/books/NBK32784/ Tobin, T. J., & Sugai, G. M. (1999). Using sixth-grade school records
OSEP Technical Assistance Center on Positive Behavioral to predict school violence, chronic discipline problems, and
Interventions and Supports. (2015). Positive Behavioral high school outcomes. Journal of Emotional and Behavioral
Interventions and Supports (PBIS) implementation blueprint: Disorders, 7, 40–53. doi:10.1177/106342669900700105
Part 1: Foundations and supporting information. Eugene: U.S. Census Bureau. (2012). Growth in urban population out-
University of Oregon. Available from www.pbis.org paces rest of nation, Census Bureau reports [Press release].
356 Behavioral Disorders 43(3)

Retrieved from https://www.census.gov/newsroom/releases/ Weist, M. D., Eber., L., Horner, R., Splett, J., Putnam, R., Barrett,
archives/2010_census/cb12-50.html S., . . . Hoover, S. (2018). (2018). Improving multitiered
U.S. Department of Education, National Center for Education systems of support for students with “internalizing” emo-
Statistics. (2011). Digest of education statistics. Washington, DC: tional/behavioral problems. Journal of Positive Behavior
Author. Retrieved from http://nces.ed.gov/programs/digest/d11/ Interventions. Advance online publication. doi:10.1177/
Walker, B., Cheney, D., Stage, S., Blum, C., & Horner, R. H. 1098300717753832
(2005). Schoolwide screening and positive behavior supports: Young, E. L., Sabbah, H. Y., Young, B. J., Reiser, M. L., &
Identifying and supporting students at risk for school failure. Richardson, M. J. (2010). Gender differences and similari-
Journal of Positive Behavior Interventions, 7, 194–204. doi:1 ties in a screening process for emotional and behavioral risks
0.1177/10983007050070040101 in secondary schools. Journal of Emotional and Behavioral
Weersing, V. R., Jeffreys, M., Do, M. C. T., Schwartz, K. T., & Disorders, 18, 225–235. doi:10.1177/1063426609338858
Bolano, C. (2017). Evidence base update of psychosocial Ysseldyke, J. E., Vanderwood, M. L., & Shriner, J. (1997).
treatments for child and adolescent depression. Journal of Changes over the past decade in special education referral
Clinical Child & Adolescent Psychology, 46, 11–43. doi:10.1 to placement probability: An incredibly reliable practice.
080/15374416.2016.1220310 Diagnostique, 23, 193–201.
Reproduced with permission of copyright owner. Further reproduction
prohibited without permission.