Behaviour Research and Therapy 150 (2022) 104028

Contents lists available at ScienceDirect

Behaviour Research and Therapy

journal homepage: www.elsevier.com/locate/brat

Guided internet-based transdiagnostic individually tailored Cognitive

Behavioral Therapy for symptoms of depression and/or anxiety in college
students: A randomized controlled trial
Eirini Karyotaki a, b, *, Anke M. Klein c, f, Marketa Ciharova a, b, Felix Bolinski a, b, d, Lisa Krijnen e,
Lisa de Koning f, Leonore de Wit a, b, Claudia M. van der Heijde j, David D. Ebert g, h,
Heleen Riper a, b, Neeltje Batelaan i, Peter Vonk j, Randy P. Auerbach k, Ronald C. Kessler l,
Ronny Bruffaerts m, Sascha Struijs a, b, n, Reinout W. Wiers f, 1, Pim Cuijpers a, b, 1
a
Department of Clinical, Neuro-, and Developmental Psychology, Vrije Universiteit Amsterdam, Van der Boechorststraat 7, BT 1081 Amsterdam, the Netherlands
b
Amsterdam Public Health Research Institute, Faculty of Behavioural and Movement Sciences, Vrije Universiteit Amsterdam, the Netherlands
c
Developmental and Educational Psychology of the Institute of Psychology, Leiden University, Leiden, the Netherlands
d
Department of Public Mental Health and Prevention, Trimbos Institute (Netherlands Institute of Mental Health and Addiction), Utrecht, the Netherlands
e
Department of Clinical Child and Family Studies, Utrecht University, Utrecht, the Netherlands
f
Addiction, Development, And Psychopathology Lab, Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
g
Psychology and Digital Mental Health Care, Technical University of Munich, Munich, Germany
h
Clinical Psychology and Psychotherapy, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany
i
Department of Psychiatry and Department of Epidemiology and Biostatistics, VU University Medical Centre Amsterdam, Amsterdam Public Health Research Institute,
Amsterdam, the Netherlands
j
Department of Research, Development and Prevention, Student Health Service University of Amsterdam, Amsterdam, the Netherlands
k
Department of Psychiatry, Columbia University, New York, NY, USA
l
Department of Health Care Policy, Harvard Medical School, Boston, MA, USA
m
Universitair Psychiatrisch Centrum, Centre for Public Health Psychiatry, Katholieke Universiteit Leuven, Leuven, Belgium
n
Institute of Psychology, Leiden University, Leiden, the Netherlands

A R T I C L E I N F O A B S T R A C T

Keywords: Common mental disorders, such as depression and anxiety, often emerge in college students during the transition
Depression into early adulthood. Mental health problems can seriously impact students’ functioning, interpersonal re­
Anxiety lationships, and academic achievement. Actively reaching out to college students with mental health problems
College students
and offering them internet-based interventions may be a promising way of providing low-threshold access to
Transdiagnostic
Internet-based CBT
evidence-based treatment in colleges. This randomized controlled trial aimed to assess the effectiveness of a
e-health guided web-based transdiagnostic individually tailored Cognitive Behavioral Therapy (iCBT) in treating college
students with depression and/or anxiety symptoms. Through an online survey that screened college students’
mental health, we recruited 100 college students aged ≥18 years who reported mild to moderate depression and/
or anxiety symptoms and were attending colleges in the Netherlands. Participants were randomly allocated to
guided iCBT (n = 48) or treatment as usual (TAU) control (n = 52). Primary outcomes were symptoms of
depression and anxiety measured at post-treatment (7 weeks post-randomization). We also measured all out­
comes at 6- and 12-months post-randomization. All analyses were based on the intention-to-treat principle and
were repeated using the complete-case sample. We found no evidence of a difference between the effects of
guided iCBT and TAU in any of the examined outcomes (i.e., symptoms of depression and anxiety, quality of life,
educational achievement, and college dropout) across all time points (p > .05). There was no evidence that
effects of iCBT were associated with treatment satisfaction and adherence. More research into transdiagnostic
individually tailored iCBT is necessary. Further, future studies should recruit larger samples to investigate

* Corresponding author. Department of Clinical Psychology and Amsterdam Public Health Institute, VU Amsterdam, Van der Boechorststraat 7, 1081, BT
Amsterdam, the Netherlands.
E-mail address: [email protected] (E. Karyotaki).
1
Reinout W. Wiers and Pim Cuijpers share last authorship.

https://doi.org/10.1016/j.brat.2021.104028
Received 6 June 2021; Received in revised form 17 December 2021; Accepted 27 December 2021
Available online 4 January 2022
0005-7967/© 2022 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

possible smaller but clinically relevant effects of internet-based interventions for college students with depression
and/or anxiety.

1. Introduction smaller among college students, and thus, more clinical trials are needed
to explore ways to increase treatment effectiveness (Bolinski et al., 2020;
The mental health of college students is a growing concern world­ Harrer et al., 2019).
wide. College years are particularly challenging due to the develop­ To optimize the effects of iCBT for depression and anxiety in college
mental transition from late adolescence to young adulthood (Arnett, students, it is necessary to consider individual patient needs and po­
2000; Baghurst & Kelley, 2014). Students face various stressors such as tential comorbidities. In this context, internet-based transdiagnostic and
independent living, adjustment to a new social environment, and aca­ individually tailored interventions have been rapidly emerging
demic pressure, making them particularly susceptible to experiencing (Păsărelu, Andersson, Bergman Nordgren, & Dobrean, 2017). Such in­
mental health problems (Karyotaki et al., 2020; Pedrelli, Nyer, Yeung, terventions simultaneously tackle core processes that contribute to the
Zulauf, & Wilens, 2015). Epidemiological studies have indicated that development and maintenance of multiple disorders (Craske, 2012).
one in three college students has experienced at least one mental dis­ Given the high comorbidity between depression and anxiety (Auerbach
order over the past twelve months, with anxiety and depression being et al., 2019), as well as common factors underlying both conditions (e.g.
the most prevalent diagnoses (Auerbach et al., 2016). Symptoms of negative affect and information processing difficulties) (Garber &
depression and anxiety impose a considerable burden on students and Weersing, 2010), a transdiagnostic approach is potentially more clini­
society due to their association with strained interpersonal relation­ cally relevant than disorder-specific approaches. Further, because
ships, poor functioning, and increased risk for premature mortality “one-size doesn’t fit all”, it is important to address the specific needs of
(Auerbach et al., 2018; Buchanan, 2012). Adding to the personal patients through individually tailored interventions. Research in this
suffering of affected students, mental health problems jeopardize aca­ area has demonstrated that transdiagnostic and individually tailored
demic progress, thereby reducing the students’ employment prospects interventions have moderate to large effects particularly in reducing
(Wilks et al., 2020). adult depression (Păsărelu et al., 2017). Thus, such interventions may
To effectively manage symptoms of depression and anxiety in college offer a viable way in exploiting the full potential of iCBT among college
students, early interventions are warranted. Ample research suggests students.
that psychological interventions are effective in addressing symptoms of Day, McGrath, and Wojtowicz (2013) conducted a randomized
depression and anxiety (Bandelow et al., 2015; Carl et al., 2020; Cuijpers controlled trial to investigate the effects of an Internet-based guided
et al., 2016; Cuijpers, Karyotaki, de Wit, & Ebert, 2020; Cuijpers, Noma, transdiagnostic intervention compared to a waiting list in reducing
et al., 2020). Nevertheless, the majority of college students with mental symptoms of moderate depression, anxiety and/or stress among 66
health problems do not receive psychological care despite the available college students (Day et al., 2013). The authors found moderate to large
counseling services at many colleges (Auerbach et al., 2016; Blanco effects for anxiety and depression (Day et al., 2013). Similarly, Mullin
et al., 2008; Bruffaerts et al., 2019). We should note that such services, et al. (2015) examined the effects of transdiagnostic iCBT against
however, are often limited, focused on study-related issues (e.g., exam waiting list in a sample of 55 college students with depression and
anxiety), and include long waiting lists (Bolinski et al., 2020; Cuijpers, anxiety (Mullin et al., 2015). The intervention resulted in significantly
Auerbach, et al., 2019). Next to the limited availability of counseling lower symptoms of anxiety at post-treatment and 3 months follow-up.
services, many students are reluctant to seek help from counsellors due However, there were no significant differences between the interven­
to the stigma associated with depression and anxiety (Czyz, Horwitz, tion and the control group in reducing depressive symptoms (Mullin
Eisenberg, Kramer, & King, 2013; Farrer, Christensen, Griffiths, & et al., 2015), which contradicts the respective findings of Day et al.
Mackinnon, 2011; Reichert, 2012). Other commonly reported barriers (2013). Based on these conflicting findings, it remains unclear whether
include the preference for self-management of psychological problems, transdiagnostic internet-based interventions effectively reduce depres­
lack of time, and the perception that treatment is unnecessary (Czyz sive symptoms. Moreover, both trials used a waiting list as a comparison
et al., 2013; Ebert et al., 2019). To address some of the help-seeking condition, which may have artificially inflated the intervention’s out­
barriers, exploring alternative modes of psychological interventions comes (Cunningham, Kypri, & McCambridge, 2013; Furukawa et al.,
delivery is needed. 2014). Therefore, it is essential to examine further the effects of trans­
Actively reaching out to students and offering them an easily diagnostic and individually tailored approaches for college students
accessible psychological intervention via the internet may be a prom­ with anxiety and depression against other control conditions that reflect
ising solution in expanding psychological services in colleges while the existing practices better.
overcoming many treatment barriers. The main advantage of actively Thus, in the present study, we aimed to evaluate the effects of a new
reaching out to students (e.g., by offering all college students an online guided Internet-based transdiagnostic individually tailored intervention
screening) is that common mental health problems can be recognized called “ICare Prevent”. This new program is based on established iCBT
and treated at an earlier stage. Further, advantages of Internet-based components (e.g., problem-solving, behavioral activation, and cognitive
interventions include patient empowerment, increased treatment restructuring) that have been proven effective in previous RCTs in the
accessibility, and low cost (Andersson & Titov, 2014; Samoocha, general population (Furukawa et al., 2021). Several randomized trials
Bruinvels, Elbers, Anema, & van der Beek, 2010). Moreover, are currently examining the effectiveness of the ICare Prevent, but all of
internet-based interventions can substantially reduce stigma fears and them are still ongoing; thus, the effects of this program remain unclear.
counsellors’ time, allowing students to follow treatment at their own Nevertheless, ICare Prevent had promising outcomes in a recent pilot
pace in any place and time (Titov, 2011). In this fast-growing field, study examining its feasibility and acceptability in Indonesian college
Cognitive Behavioral Therapy (iCBT) is the most widely studied type of students with anxiety and/or depression (Rahmadiana et al., 2021).
intervention (Cuijpers, Kleiboer, Karyotaki, & Riper, 2017). Previous Moreover, similar promising findings were observed in a recent German
research has shown that iCBT has moderate effects in reducing symp­ mixed method study aimed at assessing the feasibility of the ICare Pre­
toms of adult depression and anxiety when delivered with some form of vent intervention (Weisel et al., 2020). In a total sample of 49 adults
therapeutic support (Karyotaki et al., 2018; Karyotaki et al., 2021; with a diagnosis of anxiety disorders, the authors found that the inter­
Pauley, Cuijpers, Papola, Miguel, & Karyotaki, 2021). Nevertheless, vention was feasible and potentially effective in reducing symptoms of
recent meta-analytic evidence suggests that these effects are much anxiety and depression (Weisel et al., 2020). Finally, a smaller pilot by

2
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

Gericke, Ebert, Breet, Auerbach, and Bantjes (2021) found that this social media, study website, colleges websites, and a mental health
intervention facilitated self-disclosure, emotional expression, awareness campaign). Moreover, study advisors, students’ mentors and
self-awareness, and skill acquisition in a sample of 22 South African student ambassadors informed potential participants about the study
first-year college students with depression (Gericke et al., 2021). Such and provided them with useful links in case they wished to participate in
preliminary evidence highlights the need to explore further this in­ the e-survey. The participation was voluntary. After completing the
tervention’s effectiveness in a randomized controlled trial. e-survey, students who were eligible for the RCT, due to their mild to
This study aimed to investigate the effectiveness of ICare Prevent moderate symptoms on the GAD-7 and/or PHQ-9, received an infor­
against treatment as usual in reducing mild to moderate depression and/ mation letter about the study and an informed consent form. Those who
or anxiety symptoms among college students. We hypothesized that signed the latter were invited to a diagnostic interview by phone and
iCBT would be more effective than treatment as usual (TAU). further assessed against the study’s eligibility criteria. Eligible partici­
pants were randomized to the trial’s arms and completed the baseline
2. Methods questionnaires.

2.1. Design 2.4. Randomization, treatment allocation, and blinding

This study was embedded within the World Mental Health college The randomization was conducted by two independent researchers
surveys initiative (WHO WMH-ICS: http://www.hcp.med.harvard. who were not involved in the study using a computer random sequence
edu/wmh/college_student_survey.php), which aims to assess the prev­ generator. Participants were randomized at an individual-level (1:1
alence and correlates of mental health problems in college students ratio) and were stratified by recruitment location (VU and UvA). Block
around the world (Cuijpers, Auerbach, et al., 2019). The present study randomization was applied with randomly varied block sizes (6–12 al­
employed a two-arm randomized control superiority trial design to locations per block). Allocation was concealed from all researchers
compare a guided transdiagnostic and individually tailored iCBT inter­ involved in this study. It was not possible to mask personnel and par­
vention to TAU in college students with mild to moderate symptoms of ticipants due to the nature of the intervention. However, the diagnostic
depression and/or anxiety. After eligibility screening, measures were interviews at 12 months were performed by interviewers who were
administered at baseline, post-treatment, six months, and twelve months blind to the allocation assignment.
post-randomization. More details about study design and methods are
provided in our study protocol (Karyotaki et al., 2019). 2.5. Sample size calculation

2.2. Study population The sample size calculation was based on depressive symptoms. We
have decided to base our sample calculation on the effects of iCBT on
This study was carried out in two Dutch Universities, namely the depression because Internet-based interventions have overall higher
Vrije Universiteit (VU) and Universiteit van Amsterdam (UvA). Partici­ effects on anxiety than depression (Andrews, Cuijpers, Craske, McEvoy,
pants were recruited from March 2018 through July 2019 and included & Titov, 2010). We initially anticipated a moderate effect of Cohen’s d
in the RCT based on the following criteria: (a) 18 years of age or older, = 0.70 based on the findings of previous meta-analyses on the effec­
(b) enrolment as a bachelor’s or master’s student in a Dutch college, (c) tiveness of psychological interventions in treating depressive symptoms
fluency in Dutch or English, (d) mild to moderate symptoms of depres­ among college students (Cuijpers et al., 2016; Davies, Morriss, & Gla­
sion defined by scoring above the cut-off score of 4 on the Patient Health zebrook, 2014). Such a moderate effect was also in line with the results
Questionnaire (PHQ-9) (Kroenke, Spitzer, & Williams, 2001) and/or of the two former RCTs on transdiagnostic internet-based interventions
anxiety symptoms as defining by scoring above the cut-off score of 4 on for depression and/or anxiety among college students (Day et al., 2013;
the Generalized Anxiety Disorder scale – 7 items (GAD – 7) (Spitzer, Mullin et al., 2015). Thus, based on the available evidence by the time
Kroenke, Williams, & Löwe, 2006), and (e) provision of a written we designed our study, the present RCT was powered to detect a con­
informed consent before participation. We focused on mild to moderate servative estimate of Cohen’s d = 0.55. Approximately 100 participants
depression after consultation with university stakeholders who advised were needed to achieve a d = 0.55 when adjusting for a dropout rate of
encouraging students with more severe symptoms to seek high-intensity ~25% (power 1- β = 0.8 and α = 0.05).
care since the present intervention was new and, thus, untested in the
vulnerable group of students with more severe symptomatology. 2.6. Intervention & e-coaching
Participants were excluded if they met any of the following criteria:
(a) diagnosis of bipolar disorder according to the MINI International The guided iCBT used in this study is a transdiagnostic individually
Neuropsychiatric Interview (MINI) (Sheehan et al., 1998), (b) moder­ tailored intervention called ‘ICare Prevent’ (Weisel et al., 2019; Weisel
ately severe/severe depressive symptoms as defined by scoring above et al., 2018). The content of this intervention was tailored to college
the cut-off score of 14 on the PHQ-9 and/or moderately severe/severe student needs through focus group discussions (Bolinski et al., 2018).
anxiety symptoms as defined by scoring above the cut-off score of 14 on The ICare Prevent strategies were based mainly on CBT and were
the GAD-7 scale, (c) receiving psychological treatment for depression delivered in 7 weekly online sessions: (1) introduction, (2) identification
and/or anxiety in the past 12 months, and (e) or no Internet connection. of problems and behavioral activation, (3) psychoeducation, (4) cogni­
The exclusion criterion (c) was chosen to eliminate possible confounding tive restructuring, (5 & 6) problem solving or exposure in daily life, and
effects of face-to-face treatment. (7) plan for the future. Four weeks after completion of the seventh
session, participants were invited to a booster session, which aimed at
2.3. Procedures reflecting on goal achievement and learned experiences. Further, in
sessions 2 through 7, participants were free to choose elective modules
The study protocol was approved by the Medical Ethical Committee based on their needs (i.e., worry and rumination, acceptance of unful­
of the VU Medical Centre (VUMC; nr 2016-538) and it was prospectively filled needs, relaxation, alcohol consumption as emotion regulator,
registered at the Netherlands Trial Registry (NTR6797). The recruitment self-worth, perfectionism, appreciation and gratitude, and sleep hy­
was conducted through the epidemiological e-survey of the WHO WMH- giene). The intervention is defined as transdiagnostic because it tackles
ICS (More information on the e-survey can be found in our study pro­ common underlying mechanisms of anxiety and depression in sessions
tocol, Karyotaki, et al., 2019). In this e-survey, we recruited participants 1–4, 7 and 8 next to the two individually tailored sessions (5th and 6th
through emails and advertisements (i.e., flyers, faculty newsletters, sessions). Also, all the optional modules tackle problems that are

3
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

common in both anxiety and depression. The “individually tailored” psychometrics properties (Dear et al., 2011). Diagnoses of mental health
term was used because participants could focus either on depression or disorders were established based on the MINI (version 5.0) conducted
anxiety first based on their preferences and needs and received modules via the telephone. The MINI is a brief structured diagnostic interview
that were tailored to the primary condition (problem-solving vs. expo­ based on the Diagnostic and Statistical Manual of Mental disorders
sure). Further, in sessions 2 through 7, participants could choose be­ fourth edition (DSM-IV) and has good psychometric properties
tween a range of optional modules based on their goals, needs, and (Lecrubier et al., 1997).
preferences. Therefore, each participant could follow an individually Secondary outcome measures included EuroQol - 5 Dimensions (EQ-
tailored path throughout the intervention, meaning that participants did 5D) (Group, 1990; König et al., 2010; van Agt, Essink-Bot, Krabbe, &
not follow the same sessions and in the exact same sequence from the Bonsel, 1994), Client satisfaction with treatment – 8 items (CSQ-8)
beginning to the end. Such flexibility contradicts the standardized (Attkisson & Greenfield, 1996, p. 120), university dropout, and educa­
intervention approaches, and it is more well-suited to individually tional achievement. The EQ-5D is a self-report questionnaire assessing
tailored approaches.” The intervention included text, testimonials, quality of life, which consists of five dimensions (i.e., mobility, self-care,
(homework) exercises, audio–visual components, diaries, downloadable ordinary activities, discomfort, and mood state, related to anxiety or
information sheets, a mood graph, and a messaging system that allowed depression), and has shown to have adequate validity (Group, 1990;
participants to contact their online coach. The intervention was avail­ König et al., 2010; van Agt et al., 1994). The CSQ-8 is a self-report
able in both Dutch and English. measure that consists of 8 items and assesses client satisfaction related
Each session required between 45 and 60 min, which was self-paced. to the treatment. Item responses are on a 1–4 scale and total scores range
The sessions were delivered with manualized asynchronous support from 8 to 32, with higher scores of CSQ-8 indicating higher treatment
provided by trained psychology master’s students (e-coaches) who were satisfaction. The CSQ-8 has shown to have high internal consistency
supervised by a senior PhD-level researcher (details about the e-coach (Boβ et al., 2016). Educational achievement was measured using the
training can be found in the protocol - (Karyotaki et al., 2019). More Presenteeism Scale for Students (PSS), which measures academic
specifically, the e-coaches followed standard templates for providing impairment by addressing questions like “In the past 4 weeks, how often
feedback to participants, and they could tailor these templates to par­ has your primary health condition affected your academic work?” The
ticipants individual needs. At the beginning of the sessions, the total score shows academic impairment due to presenteeism in per­
e-coaches received feedback from a senior researcher who monitored centages. PSS has shown to be a valid and reliable measure in college
the participants’ progress. After the first sessions, the senior researcher students (Matsushita et al., 2011). Students were also asked about the
randomly checked the e-coaches’ responses to ensure fidelity to the number of European Credit Transfer System (ECTs) achieved during a
feedback templates and provided additional feedback to the e-coaches given study period, while university dropout was monitored through
whenever necessary. Finally, the e-coaches were advised to seek the self-report questions.
support of the senior research staff at any time needed (e.g., presence of Finally, treatment adherence was defined as the total number of
suicidal ideation during the trial). The e-coaches were advised to spend online sessions completed divided by the total number of intervention
less than 30 min per feedback and reply within a maximum of two sessions. Per protocol, adherence was also defined as completion of the
working days. The asynchronous support was given via the messaging 4th module in which the core component of iCBT (i.e., behavioral
function of the intervention platform. The reader is referred to our activation and cognitive restructuring) was delivered.
protocol for a more detailed description of our intervention (Karyotaki
et al., 2019).
2.9. Analyses
2.7. Treatment as usual (TAU)
All analyses were performed in STATA version 16.0. Baseline dif­
Participants in the TAU group received detailed information about ferences in demographic and clinical characteristics were examined with
the available regular care services in the community (i.e., primary and chi-square and t-tests. The results of the MINI interview were summa­
secondary mental health services delivered by the student counseling rized by descriptive statistics. Primary analyses were based on the
services/general practitioners/psychologists/psychiatrists). This is in intention-to-treat (ITT) principle. Missing data were handled by multi­
line with existing routine care practices in the Dutch universities, where ple imputation. The effects of the iCBT intervention on depression and
students with mental health problems are advised to seek help through anxiety were analyzed using mixed effects linear regression with par­
community services. In contrast, university counseling services are ticipants nested within the recruiting universities (VU and UvA). The
meant to manage study-related issues (e.g., exam anxiety). Thus, the post-treatment depression and anxiety scores were used as a dependent
present control reflected existing routine care practices in the univer­ variable and trial arm condition as an independent variable while
sities where the study was conducted. It should be noted that info about adjusting for baseline symptom severity (OUT0ij) and major depression
the available services was also provided to participants in the inter­ diagnosis (MDD0ij). The main model used for the present analysis for
vention group. However, students in the TAU group were strongly continuous outcomes can be described as follows:
advised to seek support, but they were free to decide whether they
OUTij = αi + βiOUT0ij + βiMDD0ij + θixij + eij
would follow this advice or not. We recorded the use of such services in
both the intervention and control group throughout the trial θi = θ + ui
assessments.
ui ~ N (0, τ2)
2.8. Measures
eij ~ N (0, σ2i)

Depressive symptoms were assessed by the PHQ-9 (Kroenke et al., In this expression, under the random effects model (random treat­
2001). Item responses are on a 0–3 scale and total scores ranging from ment effects), the jth participant provides their OUT (i.e., treatment
0 to 27 with higher scores indicating more severe depression. The PHQ-9 outcome: depressive/anxiety symptoms) after treatment, OUTij, and
has shown to have good psychometric properties (Wittkampf, Naeije, their OUT at baseline OUT0ij. The subscript, i, denotes that a separate
Schene, Huyser, & van Weert, 2007). Anxiety symptoms were measured parameter is estimated per each recruitment center. For instance, αi
by the GAD-7 (Spitzer et al., 2006). Items are each scored on a 0–3 scale denotes that a separate intercept term is estimated per each recruitment
and total score range is 0–21, with higher scores indicating more severe center (clustering of participants within recruitment centers: VU and
anxiety symptoms. The GAD-7 scale has shown to have good UvA). Similarly, βi and σ2i denote a different adjustment term for

4
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

baseline values and a distinct residual variance per recruitment centers, 3. Results
respectively. Effect size Cohen’s d was calculated by subtracting the
average score on primary outcome measures (PHQ-9 and GAD-7 scales) 3.1. Participant characteristics and flow
of the iCBT group from the average scores of the TAU group at the post-
treatment and dividing the results by the pooled SD. Accordingly, sec­ Fig. 1 shows the flow of participants throughout the study. Out of
ondary outcomes were analyzed using mixed effects logistic or linear 3879 students who were assessed for eligibility, 3779 were excluded
regression depending on whether the outcome was continuous or based on our eligibility criteria (n = 3758), because they declined to
dichotomous. Finally, we performed sensitivity analysis with complete participate (n = 2) or for other reasons (n = 19). A total of 100 partic­
cases to test the robustness of our findings. ipants were randomized into the iCBT (n = 48) or the TAU (n = 52)
groups. Study dropout was 24%, 26%, and 18% at post-test, 6- and 12-
month follow-ups, respectively. Study dropout was well-balanced be­
tween the intervention and the control group (see Fig. 1). Participants
who dropped out did not differ significantly from the participants who
completed the assessment in most baseline characteristics apart from

Fig. 1. Participant flow through study.

5
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

gender, with females being more likely to complete the post-treatment Table 1
assessment than males (Cramer’s V = .325, p = .003). Socio-demographic characteristics and baseline scores.
Table 1 shows the participant characteristics at baseline. The ma­ Whole iCBT TAU p
jority of participants were female (81%), full-time (99%), bachelor sample (n (n = (n =
students (68%). The sample had a mean age of 21.91 (SD = 2.61). Most = 100) 48) 52)
of the participants were Dutch (62%) while other prevalent nationalities M (SD) M (SD) M (SD) t r
were German (7%), Italian (5%) and Chinese (4%). According to the Age (years) 21.91 21.75 22.06 0.59 .06 .558
MINI diagnostic interview, 28% of participants had a major depressive (2.61) (2.70) (2.53)
episode in their lifetime, while 38% experienced a current major N n (%) n (%) χ2 V
depression and 39% experience a current generalized anxiety disorder. Gender 0.20 0.05 .800
Male 19 10 9
Other diagnoses included: moderate suicidal risk (18%), current panic
(20.8) (17.3)
disorder (10%), lifetime panic disorder (21%), lifetime panic attacks Female 81 38 43
(5%), current agoraphobia (20%), and current social phobia (22%). At (79.2) (82.7)
12-months, based on MINI, out of 79 respondents 12 had major Type of programme 0.93 0.10 .999
depression (iCBT: 7/40; TAU: 5/39), 13 had generalized anxiety disor­ Full-time degree 99 48 51
(100) (98.1)
der (iCBT: 5/40; TAU 8/39), 3 had panic disorder (iCBT: 0/40; TAU: 3/ Part-time degree 1 0 (0) 1 (1.9)
39) and 9 had agoraphobia (iCBT: 4/40; TAU: 5/39). University 3.07 .18 .239
Participants had a mean score of depression and anxiety of 8.23 (SD VU 70 35 35
= 2.93) and 6.77 (SD = 2.82), respectively. Participants had a mean (72.9) (67.3)
UvA 18 10 8
quality of life of 0.78 (SD = 0.13) at the baseline and reported having a
(20.8) (15.4)
GPA average of 6.2/10 (SD = 17.5, note: Dutch grades ranging from 1 to Windesheim 12 3 (6.3) 9
10). Students had on average 49% impaired academic performance due (17.3)
to depression and/or anxiety symptoms, indicated by the results of PSS. Faculty
We did not find any outliers in the sample after examining the normality Behavioural, 31 20 11 6.19 .25 .417
Social or (41.7) (21.2)
of the variable distribution. As can be observed from the table, there Movement
were no significant differences between the conditions on any of the Sciences
socio-demographic variables at baseline, with the exception of higher Humanities 9 3 (6.3) 6
number of participants with current major depressive disorder in the (11.5)
Law 9 4 (8.3) 5 (9.6)
intervention group (iCBT = 24/48 & TAU = 14/52; t = 5.6, p = .02; see
Medicine 5 3 (6.3) 2 (3.8)
Table 1). Thus, all subsequent analyses have been adjusted for this Business or 7 3 (6.3) 4 (7.7)
baseline imbalance. Economics
Science 19 8 11
3.2. Treatment adherence (16.7) (21.2)
No information 20 7 13 (25)
(14.6)
Among the 48 participants allocated to the intervention, 26 (54.2%) Level of 0.50 .07 .525
completed the core modules of the intervention (all the sessions pre­ programme
senting the core techniques of cognitive restructuring and behavioral Bachelor 68 31 37
(64.6) (71.2)
activation, i.e., at least 4 modules). Reasons stated for not completing
Master 32 17 15
the treatment included the following: 5 participants did not respond to (35.4) (28.8)
repeated reminders. Three participants indicated in the post-assessment Nationality 1.03 .10 .392
that they believed they needed different help, 6 indicated to have lost Dutch 68 35 33
their interest, 7 did not to have time and 1 unexpectedly had no internet (72.9) (63.5)
International 32 13 19
access during the intervention period. On average, the participants (27.1) (36.5)
completed 4.50 (SD = 3.04) sessions, 19 participants (39.6%) completed Ethnicity 0.01 .01 .999
all 7 sessions of the intervention, and 15 participants (31%) also Dutch 62 30 32
completed the booster session. On average, participants completed 2.67 (62.5) (61.5)
Other 38 18 20
(SD = 2.14) optional modules and 36 participants (75%) completed at
(37.5) (38.5)
least 1 optional module. Diagnosesa
As per our protocol, every e-coach spent approximately 2.48 (SD = Current major 38 24 (50) 14 5.64 .24 .023*
2.10) hour per participant. Finally, there were rare cases of direct cor­ depressive (26.9)
respondence between the e-coaches and participants. As per our proto­ disorder
Dysthymia 5 2 (4.2) 3 (5.8) 0.14 .04 .999
col, feedback was mainly motivational, and e-coaches did not start a Current panic 10 3 (6.3) 7 1.44 .12 .322
conversation with the participants. In some rare cases, participants disorder (13.5)
responded to reminder emails stating their willingness to continue with Current 39 20 19 0.28 .05 .683
the intervention. We should note that the e-coaches did not send feed­ generalized (41.7) (36.5)
anxiety disorder
back via a conversation function but through a feedback window at the
M (SD) M (SD) M (SD) t r
end of each completed session. Therefore, the participants were not Scores on questionnairesb
prompted to reply. Depressive 8.23 8.52 7.96 − 0.95 .10 .342
symptoms (2.93) (2.87) (2.98)
3.3. Use of mental health services Anxiety 6.77 6.81 6.73 − 0.14 .01 .886
symptoms (2.82) (2.71) (2.95)
Quality of life .78 (.13) .77 .79 0.59 .06 .557
Table 2 presents the use of services in each of the two arms. During (.12) (.15)
the 2 months before filling in the post-treatment questionnaire, the most
Note. Abbreviations (alphabetical): χ2: Statistic of χ2-test; M: Mean; n: Number of
common services used by the iCBT and the TAU group, respectively,
participants; p: p-value; r: Pearson’s r, SD: Standard deviation; t: Statistic of
were the service of a general practitioner (11 and 14 participants, independent t-test; UvA: University of Amsterdam; V: Cramer’s V; VU: Vrije
respectively), study advisor (5 and 11 participants, respectively), and

6
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

Universiteit Amsterdam. Table 2

*p < .05. Use of healthcare services during the follow-up perioda.
a
based on the MINI-international neuropsychiatric interview.
b iCBT (n = 48) TAU (n = 52)
based on the Patient Health Questionnaire-9, Generalized Anxiety Disorder-
7 and EuroQol – 5 dimensions (expressed in utility scores). Post-treatment
General practitioner 11 (22.9%) 14 (26.9%)
Study advisor 5 (10.4%) 11 (21.2%)
the services of a psychiatrist, psychologist, or psychotherapist (3 and 6 Student counsellor 1 (2.1%) 3 (5.8%)
participants, respectively). These services remained the most used in Student psychologist 2 (4.2%) 3 (5.8%)
both the intervention and the control condition during the 2 months Psychiatrist/ 3 (6.3%) 6 (11.5%)
psychologist/
before the 6-month follow-up period (general practitioner: 18 and 7
psychotherapist
participants, respectively; study advisor: 9 and 7; psychiatrist/psychol­ Medical specialist 2 (4.2%) 1 (1.9%)
ogist/psychotherapist: 10 and 6, respectively) as well as during the 2 Paramedical healthcare 2 (4.2%) 3 (5.8%)
months before the 12-month follow-up period (general practitioner: 12 Alternative medicine 1 (2.1%) 0 (0.0%)
and 13 participants, respectively; study advisor: 9 and 6; psychiatrist/ Self-help groups 0 (0.0%) 2 (3.8%)
Other healthcare servicesb 0 (0.0%) Mindfulness practices (1;
psychologist/psychotherapist: 15 and 11, respectively). 1.9%)
In the TAU group, one participant reported the use of benzodiaze­ Mental health care nurse
pines, ADHD medication and cognitive enhancers at post-treatment, (1; 1.9%)
cognitive enhancers at 6-month follow-up and sleep medication or Mental health Cannabis oil (1; 2.1%) Benzodiazepines (1;
medicationb 1.9%)
supplements at 12-month follow-up. In the iCBT group, one participant
ADHD Medication (1;
reported using cannabis oil at post-treatment and antidepressants and 1.9%)
ADHD medication at 6-month follow-up. At the 12-month follow-up, 2 Cognitive enhancers (1;
participants from the intervention group reported the use of antide­ 1.9%)
pressants and ADHD medication, while one reported the use of benzo­ 6-month follow-up
General practitioner 18 (37.5%) 7 (13.5%)
diazepines. In both groups and at all time-points, standard drugs were Study advisor 9 (18.8%) 7 (13.5%)
reported besides the medication described above, such as antihista­ Student counsellor 0 (0.0%) 2 (3.8%)
mines, anti-asthmatics, antibiotics, pain killers, and contraceptives. Student psychologist 2 (4.2%) 3 (5.8%)
Overall, at post-test, the iCBT and TAU groups received similar ser­ Psychiatrist/ 10 (20.8%) 6 (11.5%)
psychologist/
vices. Although a greater number of participants in the TAU than the
psychotherapist
iCBT group used these services, there was no significant difference in Medical specialist 3 (6.3%) 1 (1.9%)
service use between the two groups at post-intervention. On the con­ Paramedical healthcare 2 (4.2%) 0 (0.0%)
trary, at 6-month post-randomization, participants of the intervention Alternative medicine 2 (4.2%) 0 (0.0%)
group were significantly accessing help from more health care providers Self-help groups 1 (2.1%) 3 (5.8%)
Other healthcare servicesb Meditation group (1; Dietologist (1; 1.9%)
on average (mean = 1.04, SD = 1.18; p = .033) compared to the control 2.1%)
group. Reasons behind this significant difference remain largely un­ Social support (1; 2.1%)
known, but it could be attributed to possible increased awareness about Medicationb Antidepressants (1; Cognitive enhancers (1;
the benefits of psychological help among the intervention participants or 2.1%) 1.9%)
ADHD medication (1;
to the need for more substantial care after the end of the intervention.
2.1%)
The use of services was more balanced between the iCBT and TAU 12-month follow-up
groups at 12-months follow-up, with no significant differences between General practitioner 12 (25%) 13 (25%)
the groups in terms of service use. Study advisor 9 (18.8%) 6 (11.5%)
Student counsellor 3 (6.3%) 1 (1.9%)
Student psychologist 1 (2.1%) 3 (5.8%)
3.4. Main outcomes
Psychiatrist/ 15 (31.3%) 11 (21.2%)
psychologist/
When including the full sample (intention-to-treat analysis), we psychotherapist
found no significant differences between the intervention and the con­ Medical specialist 2 (4.2%) 3 (5.8%)
Paramedical healthcare 4 (8.3%) 2 (3.8%)
trol in symptoms of depression and anxiety at post-test (depression: β =
Alternative medicine 1 (2.1%) 1 (1.9%)
− 0.50; anxiety: β = − 0.46; p > .05). Similar outcomes were observed at Self-help groups 1 (2.1%) 1 (1.9%)
6- (depression: β = 0.30; anxiety: β = 0.13; p > .05) and 12-month Other healthcare servicesb Creative therapy (1; 0 (0.0%)
follow-ups (depression: β = − 0.19; anxiety: β = − 0.61; p > .05), and 2.1%)
in analyses including only the complete cases (see Table 3). In iCBT, Psycho-motor therapy
(1; 2.1%)
within group effect sizes on depression ranged from small to moderate
Social support (3; 6.3%)
(d = .19 at post-test, d = 0.52 at 6-months, d = 39 at 12-months). Similar Medicationb Antidepressants (2; Sleep medication (1;
within group effects were observed for TAU, albeit slightly smaller in 4.2%) 1.9%)
magnitude compared to within group effects of iCBT at post-test and 6 ADHD medication (2; Sleep supplement (1;
4.2%) 1.9%)
months (d = 0.06 at post-test, d = 0.47 at 6-months, d = 39 at 12-
Benzodiazepines (1;
months). We observed much smaller in magnitude within group ef­ 2.1%)
fects on anxiety in both iCBT (d = 0.18 at post-test, d = 0.22 at 6-
months, d = 0.26 at 12-months) and TAU (d = 0.10 at post-test, d = Note. Medical specialist = e.g., cardiologist, rheumatologist, or neurologist.
Paramedical healthcare provider = e.g., physiotherapist, speech therapist,
0.16 at 6-months, d = 0.25 at 12-months).
occupational therapist. Alternative medicine practice = e.g., homeopath,
At 12 months, there was no significant difference in the rates of
acupuncturist, Reiki practitioner. Self-help group = e.g., Alcoholics Anonymous,
depression and anxiety current diagnoses based on MINI between the patient association.
intervention and the control (MDD: β = 0.12, SE = 0.64; GAD: β = a
Details are provided when some participants used the given service more
− 0.62, SE = 1.0.; panic disorder: β = − 0.41, SE = 1.5; Agoraphobia: β = than 3 times. If no details are provided, participants used the service 1–3 times
− 0.22, SE = 0.73; p > .05). only.
b
The numbers in brackets relate to the number of participants who reported
the use of the service or medication.

7
 E. Karyotaki et al.
Table 3
Effects of iCBT compared to TAU for college students with depressive and/or anxiety symptoms at post-treatment, 6- and 1-month follow-up.
Post-treatment 6-month follow up 12-month follow-up

M (SE), n d (95% CI) β (95% CI) p M (SE), n d (95% CI) β (95%CI) p M (SE), n d (95% CI) β (95%CI) p

PHQ-9
ITT analysis
iCBT 7.66 (.83), 48 .003 (− .38 to .39) -.50 (− 2.67 to 1.66) .65 6.63 (.65), 48 -.11 (− .50 to .29) .30 (− 1.46 to 2.06) .74 6.73 (.85), 48 -.06 (− .45 to .33) -.19 (− 2.46 to 2.08) .87
TAU 7.68 (.75), 52 6.17 (.59), 52 6.39 (.68), 52
Complete Cases
iCBT 7.37 (.78), 35 .08 (− .36 to .54) -.69 (− 2.73 to 1.36) .51 6.60 (.70), 38 -.14 (− .59 to .32) .29 (− 1.50 to 2.08) .75 6.83 (.84), 41 -.06 (− .49 to .37) -.12 (− 2.12 to 1.87) .90
TAU 7.78 (.75), 41 6.08 (.54), 36 6.54 (.64), 41
GAD-7
ITT analysis
iCBT 6.15 (.64), 48 .04 (− .35 to .43) -.46 (− 2.13 to 1.22) .52 5.96 (.67), 48 .008 (− .38 to .40) .13 (− 1.77 to 2.04) .89 5.65 (.76), 48 .04 (− .35 to .43) -.61 (− 2.81 to 1.58) .58
TAU 6.33 (.64), 52 6.00 (.76), 52 5.84 (.57), 52
Complete Cases
iCBT 6.00 (.63), 35 .14 (− .32 to .59) -.63 (− 2.2 to .95) .43 5.89 (.73), 38 .08 (− .38 to .54) -.02 (− 1.93 to 1.88) .98 5.63 (.76), 41 .04 (− .39 to .47) -.32 (− 2.07 to 1.43) .72
TAU 6.50 (.59), 40 6.23 (.67), 35 5.80 (.57), 41
EQ-5D
ITT analysis
8

iCBT .75 (.03), 48 .09 (− .30 to .49) -.005 (− .11 to .10) .92 .78 (.02), 48 .00 (− .04 to .39) .01 (− .06 to .09) .69 .79 (.03), 48 .05 (− .34 to .44) .003 (− .08 to .09) .93
TAU .77 (.03), 52 .78 (.03), 52 .80 (.03), 52
Complete Cases
iCBT .77 (.02), 35 .00 (− .45 to .45) -.0004 (− .12 to .81) .99 .77 (.02), 33 -.15 (− .67 to .36) .03 (− .05 to .10) .47 .79 (.03), 41 .06 (− .37 to .49) -.002 (− .08 to .075) .96
TAU .77 (.03), 40 .75 (.03), 26 .80 (.02), 41
PSS
ITT analysis
iCBT 47 (2.56), 48 .05 (− .34 to .44) .08 (− 7.50 to 7.66) .98 40 (3.39), 48 .19 (− .19 to .59) − 4.73 (− 14.6 to 5.14) .34 39 (3.10), 48 .09 (− .31 to .48) − 2.80 (− 12.9 to 7.34) .59
TAU 48 (3.05), 52 45 (3.85), 52 41 (3.45), 52
Complete Cases
iCBT 47 (2.54), 33 -.06 (− .5 to .40) -.36 (− 8.05 to 7.32) .92 40 (2.77), 34 .33 (− .16 to .83) − 5.68 (− 14.1 to 2.70) .18 40 (3.17), 38 .11 (− .35 to .56) − 2.61 (− 11.0 to 5.81) .54
TAU 46 (2.75), 38 46 (3.81), 29 42 (3.08), 36
Average grade
ITT analysis

Behaviour Research and Therapy 150 (2022) 104028

iCBT 7.0 (.12), 48 .00 (− .39 to .39) .081 (− .26 to .42) .64 7.2 (.08), 48 -.16 (− .55 to .24) .13 (− .20 to .47) .43 6.9 (.95), 48 .22 (− .17 to .61) − 2.19 (− 6.45 to 2.08) .31
TAU 7.0 (.11), 52 7.1 (.10), 52 9.3 (1.9), 52
Complete Cases
iCBT 7.1 (.13), 33 -.15 (− .62 to .31) .10 (− .20 to .41) .50 7.2 (.07), 35 -.45 (.95–.04) .17 (− .07 to .42) .17 7.0 (.10), 38 .25 (− .21 to .70) − 1.86 (− 6.09 to 2.37) .39
TAU 7.0 (.09), 39 7.0 (.09), 29 9.2 (2.1), 37

Abbreviations: d = Cohen’s d; EQ-5D = EuroQol - 5 Dimensions; GAD-7 = Generalized Anxiety Disorder 7-itmes; iCBT = Internet based Cognitive Behavioral Therapy; M = mean score; p = p-value; PHQ-9 = Patient Health
Questionnaires 9-items; PSS = Presenteeism Scale for Students (PSS); SE = Standard error; TAU = Treatment as Usual; β = betta coefficient adjusted for diagnosis of Major Depression at baseline.
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

3.5. Secondary outcomes usual at post-treatment, 6- and 12-month follow-up assessments (Kar­
yotaki et al., 2021).
We found no evidence of a difference between iCBT and TAU in However, the present findings on depression contrast with a previous
quality of life at post-treatment (β = − 0.005; p > .05) and follow-up trial on a transdiagnostic internet-based intervention for college stu­
assessments (6 months: β = 0.01; 12 months: β = 0.003; p > .05). In dents that found large effects on depression compared to a waiting list
addition, we found no differences in academic role impairment between group (Day et al., 2013). Several reasons may explain these inconsistent
the conditions at post-treatment (β = 0.8; p > .05) and follow-ups (6 findings. First, it is well known that waiting list controls may artificially
months: β = − 4.73; 12 months: β = − 2.80; p > .05). Similarly, no sig­ inflate the outcomes (Cunningham et al., 2013; Furukawa et al., 2014).
nificant differences were observed in the GPA at post-treatment (β = Based on previous literature findings, a waiting list control group may
0.081; p > .05) and follow-up assessments (6 months: β = 0.13; 12- inflate the effects sizes of the intervention because participants are
months: β = − 2.19; p > .05). All outcomes have been replicated in the actively discouraged from seeking alternative help, which is reinforced
complete case analyses and results are presented in Table 3. Regarding by the expectation of receiving treatment in the future (Cuijpers, Kar­
college dropout, in total 2 participants dropped out from the university yotaki, Reijnders, & Ebert, 2019). It has also been suggested that a
at the post-treatment (iCBT = 1; TAU = 1), 6 at 6-month follow-up waiting list control may decrease the willingness to be engaged in pos­
(iCBT = 2; TAU = 4) and 7 at 12 months follow up (iCBT = 3; TAU = itive activities, thereby acting as a nocebo (Furukawa et al., 2014).
4). University dropout rates did not differ significantly between the Therefore, the waiting list differs substantially from no treatment or
conditions (post-treatment: β = 0.17, SE = 1.31; 6-months: β = − 0.72, TAU, where participants are actively encouraged to seek help. Thus, the
SE = 0.91; 12-months: β = − 0.47, SE = 0.78; p > .05. Overall, partici­ discrepancy in our findings and the results of the trial by Day et al.
pants reported a 72% (SD = 7.6%) rate of satisfaction with the inter­ (2013) may partly be explained by the different control conditions.
vention. On average, participants completed approximately half of the Further, another plausible explanation may be the differences in base­
main 7 sessions of the iCBT intervention (55%). Treatment adherence line symptom severity. The trial of Day et al. (2013) had participants
and treatment (i.e., number of completed modules) satisfaction were not with moderate depression symptoms while in the present study, we
significantly associated with the effect size within the intervention included participants experiencing mild to moderate symptoms. Thus,
group at post-treatment, 6- and 12-month follow-up. Nevertheless, the the room for improvement was much smaller given the mild symp­
sample size of the iCBT group was small (n = 48 participants). So, we tomatology of our sample. Finally, Day and colleagues administered a
cannot rule out the possibility of an association between treatment brief intervention (n = 5 core sessions) and weekly support via phone or
adherence/satisfaction and effect sizes that we could not detect due to email. Shorter programs and synchronous support may be more bene­
the limited statistical power. ficial for college students than lengthier interventions with asynchro­
nous support, but such a hypothesis should be examined by future
4. Discussion studies. Our results, however, replicated the findings of Mullin and
colleges (2015) who reported no significant effects for transdiagnostic
In this study, we examined the effects of a guided transdiagnostic iCBT on depressive symptoms (Mullin et al., 2015).
individually tailored iCBT compared to TAU in reducing symptoms of The present results on anxiety were in line with previous meta-
depression and anxiety among college students. In contrast with our analytic findings. Harrer and colleagues found no evidence of a differ­
hypothesis, we found no evidence of a difference between the inter­ ence in the effects of internet-based interventions compared to controls
vention and the control condition in any of the examined outcomes (i.e., for college students with anxiety after adjusting for publication bias.
depression, anxiety, quality of life, educational achievement, and col­ Nevertheless, we did not replicate the conclusions of previous trials in
lege dropout) across the post-treatment and follow-up assessments. this field that showed moderate to large effects in favor of trans­
Overall, participants reported good satisfaction with the intervention, diagnostic iCBT (Day et al., 2013; Mullin et al., 2015). Similarly to the
and more than half of the participants completed the core modules of the above, possible explanations of this discrepancy include differences in
intervention, which is comparable to the findings of a previous meta- the control condition, baseline symptom severity, intervention length
analysis on iCBT for depression in the general adult population (Van and type of support. Finally, our secondary outcomes related to the
Ballegooijen et al., 2014). More specifically, van Ballegooijen and col­ quality of life, academic achievement, and college dropout are in
leagues (2014) found that the percentage of completers of iCBT is on accordance with previous literature findings. Recent meta-analyses have
average 65.1%, which is somewhat higher than what we observed in the shown that internet-based interventions do not significantly improve
present trial. Nevertheless, we should bear in mind that the intervention quality of life and academic performance among college students with
type and target group differ between our study and van Ballegooijen and common mental disorders (Bolinski et al., 2020; Harrer et al., 2019).
colleagues (2014) meta-analysis. Indicatively, a previous trial in the The present results should be interpreted cautiously due to several
same field found a 43% intervention completion rate, which is in line limitations. First, although our sample was sufficiently powered to
with our findings (Mullin et al., 2015). We should note that the effects of detect a moderate effect on symptoms of depression and anxiety, a
the intervention were not significantly associated with treatment bigger sample (>500 participants) would be required to detect a smaller
adherence and satisfaction. but clinically relevant effect of d = 0.24 (Cuijpers, Turner, Koole, Van
The null findings found by the present study are in accordance with Dijke, & Smit, 2014). Thus, future studies should include a much bigger
the results of previous literature on digital interventions for college sample to investigate small effects of iCBT on mild-to-moderate anxiety
students with depression. A recent meta-analysis by Harrer et al. (2019) and depression. To increase the sample size, future studies should
showed a small but significant effect (g = 0.18) of internet-based in­ consider lowering the recruitment threshold. For instance, the admin­
terventions on depressive symptoms among college students (Harrer istration of MINI may have served as a barrier to participation for some
et al., 2019). Nevertheless, the authors reported that prediction intervals students because they had no time to take such interview next to their
crossed zero (g = − 0.26 to 0.62), indicating that results of future trials busy schedules, or due to stigma around formal mental health diagnoses.
would probably range from negative to moderate (Harrer et al., 2019). A Moreover, many potential participants were excluded because they did
similar conclusion can be drawn using the personalized estimates of a not sign the informed consent form. Plausible explanations for not
recent individual patient data meta-analysis on iCBT for depression completing the consent form include the lack of treatment motivation in
(Karyotaki et al., 2021). Setting the parameters to the mean age (22) and general, and lack of interest in participating in iCBT or the clinical trial.
PHQ-9 score (8) of our target group (students) at the baseline (see htt Next, another probable reason is that the informed consent type was
ps://bit.ly/3faSRdV), it can be seen that guided iCBT results in a another barrier to participation. Based on the medical ethics committee
small, but non-significant effect on depression compared to treatment as regulations, participants had to print, sign, and mail through regular

9
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

post their informed consent form to participate in the trial. It is very well of focus group discussions before the adaption of the original interven­
possible that a digital informed consent would have increased partici­ tion to the student sample, these focus groups were generic (e.g., dis­
pation. Nevertheless, despite our systematic efforts, we either could not cussing possible topics that the intervention should cover). If we were to
reach participants who did not complete the informed consent forms, or re-do these focus groups, it would seem essential to ask end-users to
they did not provide us with specific explanations. Thus, the actual closely evaluate all intervention sessions and give concrete feedback on
reasons behind not completing the forms remain unknown. what to alter. Given that lack of time was reported as a reason for
Second, we could not examine students’ educational achievement in dropout, it would seem important to evaluate whether reducing the text
terms of the number of European Credit Transfer System (ECTs) ach­ and adding more audiovisual components would minimize the time
ieved during a given study period. This resulted from differences in the needed to complete the intervention.
number of ECTs/study periods among the participating universities and To conclude, based on the present findings, the transdiagnostic and
misinterpretations of the related questions (e.g., some students reported individually tailored iCBT does not appear to lead to moderate effects in
the ECTs achieved throughout the study years instead of a given period). college students with mild to moderate depression and/or anxiety
Thus, we excluded these data from our analysis as unreliable. In addi­ symptoms. Future trials in this field should include a larger sample to
tion, the most reliable way to measure education achievement would be detect possible small effects of the iCBT in the given target group.
through academic records. However, access to such records was not Moreover, future research is needed to test whether the effects of the
permitted due to ethical restrictions. Third, despite our continuous effort intervention would be improved if it is administered to students who
to approach participants who dropped out, we could not reach 23% of actively seek help. Finally, given that actively reaching out to students
them. Thus, our overview of dropout reasons is limited. Finally, we used offers key advantages (e.g., early detection and treatment), we need to
only the Client Satisfaction Questionnaire for measuring intervention explore ways to improve the effects of iCBT when it is delivered to
satisfaction. Employing qualitative measures (e.g., in-depth interviews) students who are not actively seek help.
may be more informative regarding participant satisfaction with the
several aspects of the iCBT program ranging from coaching to the con­ CRediT authorship contribution statement
tent and the interface.
Overall, it is unclear why we did not identify evidence of difference Eirini Karyotaki: Conceptualization, Methodology, Formal analysis,
between the intervention and the control group. In the present study, we Supervision, Project administration, Writing – review & editing. Anke
have included participants through systematic screening for mental M. Klein: Conceptualization, Supervision, Project administration,
health problems during the college years. Previous literature has sug­ Writing – review & editing. Marketa Ciharova: Formal analysis, Project
gested that psychological interventions do not result in significant im­ administration, Writing – review & editing. Felix Bolinski: Conceptu­
provements in depression if patients are recruited through systematic alization, Methodology, Project administration, Writing – review &
screening (Cuijpers, van Straten, van Schaik, & Andersson, 2009). editing. Lisa Krijnen: Project administration, Writing – review & edit­
Possibly, patients who are identified through systematic screening do ing. Lisa de Koning: Project administration, Writing – review & editing.
not actively seek treatment. Thus, they are not enough motivated to be Leonore de Wit: Conceptualization, Methodology, Supervision, Project
engaged in the therapeutic process (Cuijpers et al., 2009). Such lack of administration, Writing – review & editing. Claudia M. van der Heijde:
motivation is even more challenging in the case of self-help in­ Project administration, Writing – review & editing. David D. Ebert:
terventions that rely solely on the motivational readiness of the users to Conceptualization, Writing – review & editing. Heleen Riper: Concep­
adapt the intervention strategies in their everyday lives. tualization, Methodology, Supervision, Funding acquisition, Writing –
Further, our sample was mildly impaired, suggesting that the overall review & editing. Neeltje Batelaan: Conceptualization, Methodology,
room for improvement was much smaller than other treatment studies. Supervision, Writing – review & editing. Peter Vonk: Project adminis­
The mild symptomatology of our sample is more comparable to what we tration, Writing – review & editing. Randy P. Auerbach: Conceptuali­
see in indicative prevention studies that usually include much larger zation, Writing – review & editing. Ronald C. Kessler:
numbers of participants to detect small effects. However, we should note Conceptualization, Writing – review & editing. Ronny Bruffaerts:
that our study was not indicative prevention (e.g., 38% of our sample Conceptualization, Writing – review & editing. Sascha Struijs: Project
met criteria for current major depressive disorder at baseline). Never­ administration, Writing – review & editing. Reinout W. Wiers:
theless, there is a possibility that focusing on participants with mild Conceptualization, Methodology, Supervision, Funding acquisition,
symptomatology makes the trial more susceptible to floor effects. Next, Writing – review & editing. Pim Cuijpers: Conceptualization, Method­
many individuals with mild concerns may remit spontaneously and thus, ology, Supervision, Funding acquisition, Writing – review & editing.
they may not necessarily need to follow an intervention. Future trials
should consider including participants with more severe symptom­ Declaration of competing interest
atology as recent meta-analytic evidence suggests that guided in­
terventions result in larger effects in moderately severe and severe In the past years, Dr. Kessler received support for his epidemiological
depression in the general population (Karyotaki et al., 2021). Another studies from Sanofi Aventis; was a consultant for Johnson & Johnson
point of interest is that we strongly advised the participants in the Wellness and Prevention, Sage Pharmaceuticals, Shire, Takeda; and
control group to seek help in the community. Thus, although we cannot served on an advisory board for the Johnson & Johnson Services Inc.
be certain, it is possible that some participants followed this advice and Lake Nona Life Project. Kessler is a co-owner of DataStat, Inc., a market
sought help for their symptoms. Thus, in our study TAU is probably more research firm that carries out healthcare research.
than what students would typically do under different circumstances.
This is also evident from the small-moderate within group effects on Acknowledgments
depression and anxiety that we observed in both iCBT and TAU condi­
tions. Finally, since we used a new intervention program, some com­ This trial was funded by ZonMw, Research Program GGz, grant
ponents or other aspects may have been suboptimal. We should note that number 636110005. We thank all student representatives as well as the
the present intervention was adapted from its original version to meet university stakeholders of the Vrije Universiteit (VU) of Amsterdam and
college student needs. Thus, we cannot rule out the possibility that the Universiteit van Amsterdam (UvA) for their great help during the
during this process important elements of the intervention were omitted. development of our study. Moreover, we thank the e-coaches and
Nevertheless, such omissions are very improbable given that the research assistants who worked hard throughout this trial.
adapted intervention content were reviewed by a licensed psychologist
with ample experience in CBT. Further, although we performed a series

10
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

Appendix A. Supplementary data Cuijpers, P., Turner, E. H., Koole, S. L., Van Dijke, A., & Smit, F. (2014). What is the
threshold for a clinically relevant effect? The case of major depressive disorders.
Depression and Anxiety, 31, 374–378.
Supplementary data to this article can be found online at https://doi. Cuijpers, P., van Straten, A., van Schaik, A., & Andersson, G. (2009). Psychological
org/10.1016/j.brat.2021.104028. treatment of depression in primary care: A meta-analysis. British Journal of General
Practice, 59, e51–e60.
Cunningham, J. A., Kypri, K., & McCambridge, J. (2013). Exploratory randomized
References controlled trial evaluating the impact of a waiting list control design. BMC Medical
Research Methodology, 13, 1–7.
van Agt, H. M., Essink-Bot, M.-L., Krabbe, P. F., & Bonsel, G. J. (1994). Test-retest Czyz, E. K., Horwitz, A. G., Eisenberg, D., Kramer, A., & King, C. A. (2013). Self-reported
reliability of health state valuations collected with the EuroQol questionnaire. Social barriers to professional help seeking among college students at elevated risk for
Science & Medicine, 39, 1537–1544. suicide. Journal of American College Health, 61, 398–406.
Andersson, G., & Titov, N. (2014). Advantages and limitations of Internet-based Davies, E. B., Morriss, R., & Glazebrook, C. (2014). Computer-delivered and web-based
interventions for common mental disorders. World Psychiatry, 13, 4–11. interventions to improve depression, anxiety, and psychological well-being of
Andrews, G., Cuijpers, P., Craske, M. G., McEvoy, P., & Titov, N. (2010). Computer university students: A systematic review and meta-analysis. Journal of Medical
therapy for the anxiety and depressive disorders is effective, acceptable and practical Internet Research, 16, Article e130.
health care: A meta-analysis. PLoS One, 5, Article e13196. Day, V., McGrath, P. J., & Wojtowicz, M. (2013). Internet-based guided self-help for
Arnett, J. J. (2000). Emerging adulthood: A theory of development from the late teens university students with anxiety, depression and stress: A randomized controlled
through the twenties. American Psychologist, 55, 469. clinical trial. Behaviour Research and Therapy, 51, 344–351.
Attkisson, C. C., & Greenfield, T. K. (1996). The client satisfaction questionnaire (CSQ) Dear, B. F., Titov, N., Sunderland, M., McMillan, D., Anderson, T., Lorian, C., et al.
scales and the service satisfaction scale-30 (SSS-30). Outcomes assessment in clinical (2011). Psychometric comparison of the generalized anxiety disorder scale-7 and the
practice. Penn State Worry Questionnaire for measuring response during treatment of
Auerbach, R. P., Alonso, J., Axinn, W. G., Cuijpers, P., Ebert, D. D., Green, J. G., et al. generalised anxiety disorder. Cognitive Behaviour Therapy, 40, 216–227.
(2016). Mental disorders among college students in the World Health Organization Ebert, D. D., Mortier, P., Kaehlke, F., Bruffaerts, R., Baumeister, H., Auerbach, R. P., et al.
world mental health surveys. Psychological Medicine, 46, 2955–2970. (2019). Barriers of mental health treatment utilization among first-year college
Auerbach, R. P., Mortier, P., Bruffaerts, R., Alonso, J., Benjet, C., Cuijpers, P., et al. students: First cross-national results from the WHO World Mental Health
(2018). WHO world mental health surveys international college student Project: International College Student Initiative. International Journal of Methods in Psychiatric
Prevalence and distribution of mental disorders. Journal of Abnormal Psychology, Research, 28, Article e1782.
127, 623. Farrer, L., Christensen, H., Griffiths, K. M., & Mackinnon, A. (2011). Internet-based CBT
Auerbach, R. P., Mortier, P., Bruffaerts, R., Alonso, J., Benjet, C., Cuijpers, P., et al. for depression with and without telephone tracking in a national helpline:
(2019). Mental disorder comorbidity and suicidal thoughts and behaviors in the Randomised controlled trial. PLoS One, 6.
world health organization world mental health surveys international college student Furukawa, T., Noma, H., Caldwell, D., Honyashiki, M., Shinohara, K., Imai, H., et al.
initiative. International Journal of Methods in Psychiatric Research, 28, Article e1752. (2014). Waiting list may be a nocebo condition in psychotherapy trials: A
Baghurst, T., & Kelley, B. C. (2014). An examination of stress in college students over the contribution from network meta-analysis. Acta Psychiatrica Scandinavica, 130,
course of a semester. Health Promotion Practice, 15, 438–447. 181–192.
Bandelow, B., Reitt, M., Röver, C., Michaelis, S., Görlich, Y., & Wedekind, D. (2015). Furukawa, T., Suganuma, A., Ostinelli, E., Andersson, G., Beevers, C., Shumake, J., et al.
Efficacy of treatments for anxiety disorders: A meta-analysis. International Clinical (2021). Dismantling, optimising, and personalising internet cognitive behavioural
Psychopharmacology, 30, 183–192. therapy for depression: A systematic review and component network meta-analysis
Blanco, C., Okuda, M., Wright, C., Hasin, D. S., Grant, B. F., Liu, S.-M., et al. (2008). using individual participant data. The Lancet Psychiatry, 8, 500–511.
Mental health of college students and their non–college-attending peers: Results Garber, J., & Weersing, V. R. (2010). Comorbidity of anxiety and depression in youth:
from the national epidemiologic study on alcohol and related conditions. Archives of Implications for treatment and prevention. Clinical Psychology: Science and Practice,
General Psychiatry, 65, 1429–1437. 17, 293–306.
Bolinski, F., Boumparis, N., Kleiboer, A., Cuijpers, P., Ebert, D. D., & Riper, H. (2020). Gericke, F., Ebert, D. D., Breet, E., Auerbach, R. P., & Bantjes, J. (2021). A qualitative
The effect of e-mental health interventions on academic performance in university and study of university students’ experience of Internet-based CBT for depression. Counselling
college students: A meta-analysis of randomized controlled trials, 100321, Internet and Psychotherapy Research.
Interventions. Group, T. E. (1990). EuroQol-a new facility for the measurement of health-related quality
Bolinski, F., Kleiboer, A., Karyotaki, E., Bosmans, J. E., Zarski, A.-C., Weisel, K. K., et al. of life. Health Policy, 16, 199–208.
(2018). Effectiveness of a transdiagnostic individually tailored internet-based and Harrer, M., Adam, S. H., Baumeister, H., Cuijpers, P., Karyotaki, E., Auerbach, R. P., et al.
mobile-supported intervention for the indicated prevention of depression and (2019). Internet interventions for mental health in university students: A systematic
anxiety (ICare prevent) in Dutch college students: Study protocol for a randomised review and meta-analysis. International Journal of Methods in Psychiatric Research, 28,
controlled trial. Trials, 19, 1–13. e1759.
Boß, L., Lehr, D., Reis, D., Vis, C., Riper, H., Berking, M., et al. (2016). Reliability and Karyotaki, E., Cuijpers, P., Albor, Y., Alonso, J., Auerbach, R. P., Bantjes, J., et al. (2020).
validity of assessing user satisfaction with web-based health interventions. Journal of Sources of stress and their associations with mental disorders among college
Medical Internet Research, 18, e234. students: Results of the world health organization world mental health surveys
Bruffaerts, R., Mortier, P., Auerbach, R. P., Alonso, J., Hermosillo De la Torre, A. E., international college student initiative. Frontiers in Psychology, 11, 1759.
Cuijpers, P., et al. (2019). Lifetime and 12-month treatment for mental disorders and Karyotaki, E., Ebert, D. D., Donkin, L., Riper, H., Twisk, J., Burger, S., et al. (2018). Do
suicidal thoughts and behaviors among first year college students. International guided internet-based interventions result in clinically relevant changes for patients
Journal of Methods in Psychiatric Research, 28, Article e1764. with depression? An individual participant data meta-analysis. Clinical Psychology
Buchanan, J. L. (2012). Prevention of depression in the college student population: A Review, 63, 80–92.
review of the literature. Archives of Psychiatric Nursing, 26, 21–42. Karyotaki, E., Efthimiou, O., Miguel, C., Bermpohl, F. M. G., Furukawa, T. A.,
Carl, E., Witcraft, S. M., Kauffman, B. Y., Gillespie, E. M., Becker, E. S., Cuijpers, P., et al. Cuijpers, P., et al. (2021). Internet-based cognitive behavioral therapy for
(2020). Psychological and pharmacological treatments for generalized anxiety depression: A systematic review and individual patient data network meta-analysis.
disorder (GAD): A meta-analysis of randomized controlled trials. Cognitive Behaviour JAMA Psychiatry, 78(4), 361–371.
Therapy, 49, 1–21. Karyotaki, E., Klein, A. M., Riper, H., De Wit, L., Krijnen, L., Bol, E., et al. (2019).
Craske, M. G. (2012). Transdiagnostic treatment for anxiety and depression. Depression and Examining the effectiveness of a web-based intervention for symptoms of depression
anxiety. and anxiety in college students: Study protocol of a randomised controlled trial. BMJ
Cuijpers, P., Auerbach, R. P., Benjet, C., Bruffaerts, R., Ebert, D., Karyotaki, E., et al. Open, 9, Article e028739.
(2019). The world health organization world mental health international college König, H.-H., Heider, D., Lehnert, T., Riedel-Heller, S. G., Angermeyer, M. C.,
student initiative: An overview. International Journal of Methods in Psychiatric Matschinger, H., et al. (2010). Health status of the advanced elderly in six European
Research, 28, Article e1761. countries: Results from a representative survey using EQ-5D and SF-12. Health and
Cuijpers, P., Cristea, I. A., Ebert, D. D., Koot, H. M., Auerbach, R. P., Bruffaerts, R., et al. Quality of Life Outcomes, 8, 1–11.
(2016). Psychological treatment of depression in college students: A metaanalysis. Kroenke, K., Spitzer, R. L., & Williams, J. B. (2001). The PHQ-9: Validity of a brief
Depression and Anxiety, 33, 400–414. depression severity measure. Journal of General Internal Medicine, 16, 606–613.
Cuijpers, P., Karyotaki, E., de Wit, L., & Ebert, D. D. (2020). The effects of fifteen Lecrubier, Y., Sheehan, D. V., Weiller, E., Amorim, P., Bonora, I., Sheehan, K. H., et al.
evidence-supported therapies for adult depression: A meta-analytic review. (1997). The mini international neuropsychiatric interview (MINI). A short diagnostic
Psychotherapy Research, 30, 279–293. structured interview: Reliability and validity according to the CIDI. European
Cuijpers, P., Karyotaki, E., Reijnders, M., & Ebert, D. D. (2019). Is psychotherapy Psychiatry, 12, 224–231.
effective? Pretending everything is fine will not help the field forward. Epidemiology Matsushita, M., Adachi, H., Arakida, M., Namura, I., Takahashi, Y., Miyata, M., et al.
and Psychiatric Sciences, 28, 356–357. (2011). Presenteeism in college students: Reliability and validity of the presenteeism
Cuijpers, P., Kleiboer, A., Karyotaki, E., & Riper, H. (2017). Internet and mobile scale for students. Quality of Life Research, 20, 439–446.
interventions for depression: Opportunities and challenges. Depression and Anxiety, Mullin, A., Dear, B. F., Karin, E., Wootton, B. M., Staples, L. G., Johnston, L., et al. (2015).
34, 596–602. The UniWellbeing course: A randomised controlled trial of a transdiagnostic
Cuijpers, P., Noma, H., Karyotaki, E., Vinkers, C. H., Cipriani, A., & Furukawa, T. A. internet-delivered cognitive behavioural therapy (CBT) programme for university
(2020). A network meta-analysis of the effects of psychotherapies, students with symptoms of anxiety and depression. Internet Interventions, 2, 128–136.
pharmacotherapies and their combination in the treatment of adult depression. Păsărelu, C. R., Andersson, G., Bergman Nordgren, L., & Dobrean, A. (2017). Internet-
World Psychiatry, 19, 92–107. delivered transdiagnostic and tailored cognitive behavioral therapy for anxiety and

11
E. Karyotaki et al. Behaviour Research and Therapy 150 (2022) 104028

depression: A systematic review and meta-analysis of randomized controlled trials. Van Ballegooijen, W., Cuijpers, P., Van Straten, A., Karyotaki, E., Andersson, G.,
Cognitive Behaviour Therapy, 46, 1–28. Smit, J. H., et al. (2014). Adherence to internet-based and face-to-face cognitive
Pauley, D., Cuijpers, P., Papola, D., Miguel, C., & Karyotaki, E. (2021). Two decades of behavioural therapy for depression: A meta-analysis. PLoS One, 9, Article e100674.
digital interventions for anxiety disorders: A systematic review and meta-analysis of Weisel, K. K., Zarski, A.-C., Berger, T., Krieger, T., Moser, C. T., Schaub, M. P., et al.
treatment effectiveness. Psychological Medicine, 1–13. (2020). User experience and effects of an individually tailored transdiagnostic
Pedrelli, P., Nyer, M., Yeung, A., Zulauf, C., & Wilens, T. (2015). College students: Mental internet-based and mobile-supported intervention for anxiety disorders: Mixed-
health problems and treatment considerations. Academic Psychiatry, 39, 503–511. methods study. Journal of Medical Internet Research, 22, Article e16450.
Rahmadiana, M., Karyotaki, E., Schulte, M., Ebert, D. D., Passchier, J., Cuijpers, P., et al. Weisel, K. K., Zarski, A.-C., Berger, T., Krieger, T., Schaub, M. P., Moser, C. T., et al.
(2021). Transdiagnostic internet intervention for Indonesian university students (2019). Efficacy and cost-effectiveness of guided and unguided internet-and mobile-
with depression and anxiety: Evaluation of feasibility and acceptability. JMIR Ment based indicated transdiagnostic prevention of depression and anxiety (ICare
Health, 8, Article e20036. prevent): A three-armed randomized controlled trial in four European countries.
Reichert, E. (2012). Reducing stigma barriers to help-seeking behaviors among college Internet Interventions, 16, 52–64.
students. Psychology, 3, 892. Weisel, K. K., Zarski, A.-C., Berger, T., Schaub, M. P., Krieger, T., Moser, C. T., et al.
Samoocha, D., Bruinvels, D. J., Elbers, N. A., Anema, J. R., & van der Beek, A. J. (2010). (2018). Transdiagnostic tailored internet-and mobile-based guided treatment for
Effectiveness of web-based interventions on patient empowerment: A systematic major depressive disorder and comorbid anxiety: Study protocol of a randomized
review and meta-analysis. Journal of Medical Internet Research, 12, e23. controlled trial. Frontiers in Psychiatry, 9, 274.
Sheehan, D. V., Lecrubier, Y., Sheehan, K. H., Amorim, P., Janavs, J., Weiller, E., et al. Wilks, C. R., Auerbach, R. P., Alonso, J., Benjet, C., Bruffaerts, R., Cuijpers, P., et al.
(1998). The mini-international neuropsychiatric interview (MINI): The development (2020). The importance of physical and mental health in explaining health-related
and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD- academic role impairment among college students. Journal of Psychiatric Research,
10. Journal of Clinical Psychiatry, 59, 22–33. 123, 54–61.
Spitzer, R. L., Kroenke, K., Williams, J. B., & Löwe, B. (2006). A brief measure for Wittkampf, K. A., Naeije, L., Schene, A. H., Huyser, J., & van Weert, H. C. (2007).
assessing generalized anxiety disorder: The GAD-7. Archives of Internal Medicine, 166, Diagnostic accuracy of the mood module of the patient health questionnaire: A
1092–1097. systematic review. General Hospital Psychiatry, 29, 388–395.
Titov, N. (2011). Internet-delivered psychotherapy for depression in adults. Current
Opinion in Psychiatry, 24, 18–23.

12