International Journal of

Environmental Research
and Public Health

Review
Sex Education in the Spotlight: What Is Working?
Systematic Review
María Lameiras-Fernández * , Rosana Martínez-Román , María Victoria Carrera-Fernández
and Yolanda Rodríguez-Castro

Faculty of Education and Social Work, University of Vigo, 32004 Ourense, Spain; [email protected] (R.M.-R.);
[email protected] (M.V.C.-F.); [email protected] (Y.R.-C.)
* Correspondence: [email protected]; Tel.: +34-988-387-121

Abstract: Adolescence, a period of physical, social, cognitive and emotional development, represents
a target population for sexual health promotion and education when it comes to achieving the 2030
Agenda goals for sustainable and equitable societies. The aim of this study is to provide an overview
of what is known about the dissemination and effectiveness of sex education programs and thereby
to inform better public policy making in this area. Methodology: We carried out a systematic review
based on international scientific literature, in which only peer-reviewed papers were included. To
identify reviews, we carried out an electronic search of the Cochrane Database Reviews, ERIC, Web
of Science, PubMed, Medline, Scopus and PsycINFO. This paper provides a narrative review of
reviews of the literature from 2015 to 2020. Results: 20 reviews met the inclusion criteria (10 in
school settings, 9 using digital platforms and 1 blended learning program): they focused mainly





 on reducing risk behaviors (e.g., VIH/STIs and unwanted pregnancies), whilst obviating themes
such as desire and pleasure, which were not included in outcome evaluations. The reviews with the
Citation: Lameiras-Fernández, M.;
lowest risk of bias are those carried out in school settings and are the ones that most question the
Martínez-Román, R.;
effectiveness of sex education programs. Whilst the reviews of digital platforms and blended learning
Carrera-Fernández, M.V.;
Rodríguez-Castro, Y. Sex Education in
show greater effectiveness in terms of promoting sexual and reproductive health in adolescents
the Spotlight: What Is Working? (ASRH), they nevertheless also include greater risks of bias. Conclusion: A more rigorous assessment
Systematic Review. Int. J. Environ. of the effectiveness of sexual education programs is necessary, especially regarding the opportunities
Res. Public Health 2021, 18, 2555. offered by new technologies, which may lead to more cost-effective interventions than with in-person
https://doi.org/10.3390/ programs. Moreover, blended learning programs offer a promising way forward, as they combine the
ijerph18052555 best of face-to-face and digital interventions, and may provide an excellent tool in the new context of
the COVID-19 pandemic.
Academic Editor: Paul B. Tchounwou

Keywords: adolescents; sexual education; sexual and reproductive health; review of reviews; school
Received: 21 January 2021
setting; digital platforms; blended learning
Accepted: 2 March 2021
Published: 4 March 2021

Publisher’s Note: MDPI stays neutral

1. Introduction
with regard to jurisdictional claims in
published maps and institutional affil- Adolescence is a period of transition, growth, exploration and opportunities that
iations. the World Health Organization defines as referring to individuals between 10 years and
19 years of age [1]. During this life phase, adolescents undergo physical, psychological
and sexual maturation and tend to develop an increased interest in sex and relationships,
with positive relationships becoming strongly linked to sexual and reproductive health as
Copyright: © 2021 by the authors.
well as overall wellbeing [2]. Sexual health is understood as a state of wellness comprising
Licensee MDPI, Basel, Switzerland.
physical, emotional, mental, and social dimensions [3]: it represents one of the necessary
This article is an open access article
requirements to achieve the general objective of sustainable and equitable societies in terms
distributed under the terms and of the 2030 Agenda [4], which advocates the need for a sexual education that is anchored
conditions of the Creative Commons in a gender- and human rights-oriented perspective.
Attribution (CC BY) license (https:// In high-income countries, sexual debut usually occurs during adolescence [5], though
creativecommons.org/licenses/by/ research suggests that sexual initiation is increasingly occurring at earlier ages [6]. Adoles-
4.0/). cents have to deal with the results of unhealthy sexual behaviors, including unplanned

Int. J. Environ. Res. Public Health 2021, 18, 2555. https://doi.org/10.3390/ijerph18052555 https://www.mdpi.com/journal/ijerph
Int. J. Environ. Res. Public Health 2021, 18, 2555 2 of 31

pregnancies and sexually transmitted infections [7], as well as experiences of sexual vio-
lence [8,9]. Adolescents are aware that they need more knowledge in order to enjoy healthy
relationships [10], yet do not receive enough of the kind of information from parents
or other formal sources that would allow them to develop a more positive, respectful
experience of sexuality and sexual relationships [11].
Sexual education can be defined as any combination of learning experiences aimed
at facilitating voluntary behavior conducive to sexual health. Sex education during ado-
lescence has centered on the delivery of content (abstinence-only vs. comprehensive
instruction) by teachers, parents, health professionals or community educators, and on the
context (within school and beyond) of such delivery [12]. As regards content, the propo-
nents of abstinence-only programs aim to help young adults avoid unintended pregnancies
and sexually transmitted diseases (STDs), working on the assumption that while contra-
ceptive use merely reduces the risk, abstinence will eliminate it entirely [13]. Nevertheless,
an overwhelming majority of studies in this field have shown that programs advocating
abstinence-only-until-marriage (AOUM) are neither effective in delaying sexual debut
nor in changing other sexual risk behaviors [14,15], and participants in abstinence-only
sex education programs consider that these had only a low impact in their lives [16]. On
the other hand, holistic and comprehensive approaches to sex education go beyond risk
behaviors and acknowledge other important aspects, as for example love, relationships,
pleasure, sexuality, desire, gender diversity and rights, in accordance with internationally
established guidelines [17], and with the 2030 Agenda [4]. Comprehensive Sexuality Edu-
cation (CSE) “plays a central role in the preparation of young people for a safe, productive,
fulfilling life” (p. 12) [17] and adolescents who receive comprehensive sex education are
more likely to delay their sexual debut, as well as to use contraception during sexual initia-
tion [18]. Comprehensive sexual education initiatives thereby promote sexual health in a
way that involves not only the biological aspects of sexuality but also its psychological and
emotional aspects, allowing young people to have enjoyable and safe sexual experiences.
With regard to context, sexual education may occur in different settings. School
settings are key sites for implementing sexual education and for promoting adolescent
sexual health [19], but today internet is becoming an increasingly important source of
information and advice on these topics [20]. Access to the internet by adolescents is almost
universal in high-income countries. The ubiquity and accessibility of digital platforms
result in adolescents spending a great deal of time on the internet, and the search for
information is the primary purpose of health-related internet use [21]. At the same time,
this widespread use of technology by young people offers interesting possibilities for
sexual health education programs, given the ease of access, availability, low cost, and the
possibility of participating remotely [22]. The topics that young people search for online
include information on everyday health-related issues, physical well-being and sexual
health [23]. The majority of internet users of all ages in the US (80%) search online for health
information including sexual health information [24], and among adolescents social media
platforms are the most frequent means of obtaining information about health, especially
regarding sexuality [25].
Thanks to the ubiquity and popularity of technologies, digital media interventions
for sexual education offer a promising way forward, both via the internet (eHealth) and
via mobile phones (mHealth, a specific way of promoting eHealth), given the privacy
and anonymity they afford, especially for young people. Digital interventions in school—
both inside and outside the classroom—offer interesting possibilities, because of their
greater flexibility with regard to a variety of learning needs and benefits in comparison
with traditional, face-to-face interventions, and because they offer ample opportunities
for customization, interactivity as well as a safe, controlled, and familiar environment
for transmitting sexual health knowledge and skills [26]. As Garzón-Orjuela et al. [27]
argues, contemporary adolescents’ needs are mediated by their digital and technological
environment, making it important to adapt interventions in the light of these realities.
Online searches for sexual health information are likely to become increasingly important
Int. J. Environ. Res. Public Health 2021, 18, 2555 3 of 31

for young people with diminishing access to information from schools or health care
providers in the midst of the lockdowns and widespread school closures during the
COVID-19 pandemic [28], with more than two million deaths and 94 million people
infected around the world [29]. Specifically, blended learning programs, consisting of
internet-based educational interventions complemented by face-to-face interventions, may
prove a significant addition to regular secondary school sex education programs [30,31].
Blended learning programs can be especially helpful in promoting sexual and reproductive
health in the context of the COVID-19 pandemic, which is challenging the way we have
so far approached formal education, with its focus on face to face interventions, given the
need, now more than ever, to “develop and disseminate online sex education curricula,
and ensure the availability of both in-person and online instruction in response to school
closures caused by the pandemic” [28].
The present study sets out to research the dissemination and effectiveness in different
settings (school, digital and blended learning) of sex education programs that promote
healthy and positive relationships and the reduction of risk behaviors, so as to make current
sexual health interventions more effective [32]. Numerous researchers have carried out
trials and systematic reviews so as to evaluate the effectiveness of school-based sexual
health and relationship education [19,27,33–35], as well as that of digital platform pro-
grams [36–39]. However, there has not been a review that is representative of the literature
as a whole. Furthermore, in the reviews that have been carried out, differing aims and
inclusion criteria have led to differences in the sampling of available primary studies [19].
As Garzón-Orjuela et al. [27] asserts, the field of adolescent sex education is continuously
evolving and in need of evaluation and improvement. Better assessments are necessary
in order to clarify whether they offer a viable and effective strategy for influencing ado-
lescents, especially with respect to improved ASRH behaviors. Hence, given the need for
an up-to-date revision so as to consider more recent emerging evidence in this field, in
this study we carry out a review of reviews that includes reviews of interventions both in
school settings and via digital platforms, as well as, for the first time, those that combine
both formats (blended learning).
The decision to conduct a review of reviews (RoR), assessing the quality and sum-
marizing the findings of existing systematic reviews, rather than working directly with
primary intervention studies, addresses the need to include as wide a range of topics
covered within the field of sex education as possible [40]. As Schackleton et al. [35] (p. 383)
point out, in order to provide overviews of research evidence that are relevant to policy
making, it is important “to bring together evidence on different forms of intervention and
on different outcomes because it is useful for policy makers to know what is the range of
approaches previously evaluated and whether these have consistent effects across different
outcomes.” Carrying out and publicly sharing reviews of reviews such as the present study
constitutes one way of better providing practitioners with evidence they can then carry
over into their interventions [32].

2. Methodology
2.1. Aims
(1) To systematically review existing reviews of Sex Education (SE) of school-based
(face-to-face), digital platforms and blended learning programs for adolescent populations
in high-income countries.
(2) To summarize evidence relating to effectiveness.

2.2. Methods
The review is structured in accordance with the PRISMA checklist (Preferred Reporting
Items for Systematic Reviews and Meta-Analysis) (see Figure A1), and the systematic
review protocol has previously been published on the PROSPERO International Prospective
Registry of Technical Reviews (CRD42021224537).
Int. J. Environ. Res. Public Health 2021, 18, 2555 4 of 31

2.3. Search Strategy

This systematic review is based on international scientific literature and only peer-
reviewed papers have been included. Only meta-analyses (publications that combine
results from different studies) and systematic reviews (literature reviews that synthesize
high-quality research evidence) were used for this review. Findings from reviews of
reviews were not analyzed. To identify reviews, we electronically searched the Cochrane
Database Reviews, ERIC, Web of Science, PubMed, Medline, Scopus and PsycINFO. After
the list was completed the duplicated papers were automatically removed. Two reviewers
working independently applied inclusion criteria in screening citations by titles, abstracts,
and keywords to identify records for full-text review. A third reviewer reconciled any
disagreement. The same procedure was carried out in screening the full text of studies
selected after the title and abstract screening phase. Two reviewers then examined the
full text of each article to determine which satisfied inclusion criteria. Data extraction
was carried out independently by the first and second reviewer. The extracted data
included specific details about the interventions, populations, study methods and outcomes
significant to the review question and objective. Any discrepancies were discussed until
consensus was reached. Search terms are included in Table A1.
This RoR included the reviews published since 2015, when the United Nations decided
on new Global Sustainable Development Goals, until December 2020. The 2030 Agenda for
Sustainable Development [4] takes into account the relevance of Sexual Health to achieve
peace and prosperity.

2.4. Inclusion Criteria

We extracted data using a “Population, Intervention, Comparison, Outcome” structure,
PICO [41].
Population: Reviews of interventions targeting adolescents (aged 10–19 years), school-
setting, digital platforms or blended learning education were eligible for inclusion. Reviews
in which studies of interventions targeted youth and adults were eligible if the primary
studies included people between the ages of 10–19 years.
Intervention: Reviews of interventions developed in school-setting (school-based),
digital (digital platforms) or blended learning programs were included. Interventions
based on multiple settings or targeted multiple health-related issues were only considered
for inclusion if any primary studies were linked to school-based, digital or blended learning
interventions, as well as targeting Sexual and Reproductive Health (SRH).
Comparison groups: Randomized controlled trials (RCTs) and studies using a quasi-
experimental design (including non-randomized trials—nRCTs). Single group, pre- and
post-test research designs, group exposed to sexual education (SE) program (school-based,
digital platforms or blended learning) compared with non-exposed control group or an-
other intervention.
Outcomes: Primary outcomes: (1) Sexual behavior and (2) Health and social outcomes
related to sexual health. Secondary outcomes: (1) Knowledge and understanding of sexual
health and relationship issues and (2) Attitudes, values and skills.

2.5. Exclusion Criteria

Reviews were excluded if:
• Their primary focus was adult people and adolescents were not included.
• Their primary focus was sexual-health screening, sexual abuse or assault or prevention
of sexual abuse or rape.
• The studies targeted specific populations (e.g., pre-pubertal children, children with
developmental disorders, migrant and refugee, or sexual minorities).
• The interventions focused on low- and middle-income countries or if high income
countries were not included in the study.
• Recipients were professionals, teachers, parents or a combination of the latter.
Int. J. Environ. Res. Public Health 2021, 18, 2555 5 of 31

2.6. Risk of Bias and Assessment of Study Quality

Review quality was assessed by the first author using the AMSTAR II checklist [42].
This is an updating and adaptation of AMSTAR [43,44] which allows a more detailed
assessment of systematic reviews that include randomized or non-randomized studies of
healthcare interventions, or both. It consists of a 16-item tool (including 5 critical domains)
assessing the quality of a review’s design, its search strategy, inclusion and exclusion
criteria, quality assessment of included studies, methods used to combine the findings,
likelihood of publication bias and statements of conflict of interest. The maximum quality
score is 16.

2.7. Data Synthesis

After manually coding the papers and extracting relevant data, we used a narra-
tive/descriptive approach for data synthesis to summarize characteristics of the studies
included. Considering the heterogeneity of outcomes, their measures and research de-
signs, meta-analysis of all the studies included was not carried out. Two researchers were
involved in data synthesis. Discrepancies were resolved through discussion, and a third
researcher was consulted to resolve any remaining discrepancies. For the classification
of the information and presentation of the effects of the interventions reported, data was
separated (school setting, digital platforms or blended learning) and structured around
population, intervention, comparison, and outcome. To address the main review questions,
data was synthesized in two phases. Phase 1 addressed the first question, the description
of sex education/sexual health interventions. Phase 2 addressed the second question,
the effectiveness and benefit of the interventions; studies with a low risk of bias were
highlighted, so as to strengthen the reliability of findings (AMSTAR II) [42].

3. Results
3.1. Results of Search
Our searches yielded 1476 unique citations. After excluding 776 records based on
title and abstract screening, we reviewed 217 full-text articles for eligibility, of which 20
ultimately met inclusion criteria, and proceeded to data extraction. Of the 197 studies
that we excluded after full-text review, 82 were carried out in low- and middle-income
countries, 47 targeted exclusively adults, 56 dealt with minority groups, and 12 targeted
exclusively pre-teen students.

3.2. Risk of Bias in Included Studies

According to the AMSTAR II quality assessment tool’s developers [42] scores may
range from 1 to 16: in this case only 2 reviews scored 16 out of 16: 1 in a school setting [45],
and 1 on a digital platform [46]. 6 of the 20 systematic reviews were of high quality: 5 in
school settings [45,47–50], and 1 in digital platforms [46]; there was one study of medium
quality in a school setting [51]. The remaining studies were of low or very low quality
(N = 13). It is possible that low quality reviews may not provide reliable evidence, so those
scoring in low and critically low quality should be regarded skeptically.

3.3. Reviews Included

Key information regarding the 20 reviews included is shown in Tables A2 and A3.

3.3.1. Setting
Ten studies (50%) dealt with school-based interventions [45,47–55], 9 (45%) referred to
online interventions [46,56–63] and 1 (5%) was a review of blended learning programs [64].
In total 491 studies were included in the 20 reviews covered by the present RoR. The
10 reviews of school setting interventions include a total of 266 studies (54%), the 9 reviews
of online interventions cover a total of 216 (44%) studies, and the only review of blended
learning interventions includes a total of 9 studies (2%). All studies were conducted in
high-income economies following the World Bank classification [65], including US samples
Int. J. Environ. Res. Public Health 2021, 18, 2555 6 of 31

in 16 of the 20 studies, although there are two studies in which the country of the sample is
not identified [51,52]. Most of the studies evaluating interventions in school settings also
include developing countries (low- and middle-income economies) [45,47,50,52,53,55], as
is also the case in three reviews of online interventions [46,61,62] (see Table A2).

3.3.2. Population
The targeted age for reviews in school settings, as shown in Table A2, is the period of
adolescence, from 10 to 19 years of age, though one of the studies covers ages from 7 to
19 years [53]. All the online studies also include young adults (20–24 years old), alongside
the adolescent sample [46,56,57,59–63], whilst the review by DeSmet et al. [58] extends the
upper limit to 29 years of age. Along with the sample of adolescents and young adults, the
blended learning studies review also incorporates adults of over 25 years of age [64].

3.3.3. Interventions/Types of Study

All the studies included in this review of reviews used randomized controlled trials
(RCTs), non-randomized controlled trials (non-RCT), and a quasi-experimental design or a
pre-test/post-test design to examine program effects.

3.3.4. Outcomes
The term “sexual outcomes” refers to the attitudes, behaviors, and experiences of
adolescents consequent to their sex education [14] (p. 1), and an extensive range of variables
was included (see Table A2): knowledge (e.g., knowledge of contraceptive effectiveness
or effective method use); attitudes (e.g., about sex and reproductive health); beliefs (e.g.,
self-efficacy); skills (e.g., condom skills); intentions/motivation (e.g., use of birth control
methods; condom use); behaviors (e.g., sexual debut; condom use; contraception use;
intercourse; initiation of sexual activity) and; other outcomes related to sexual behavior
(e.g., pregnancy prevalence; number of partners; rates of sexually transmissible infections
(STIs); cervical screening; appreciation of sexual diversity; dating and intimate partner
violence prevention; sexual violence).

3.3.5. Country of Review

Of the 10 reviews of interventions in school settings, the authors are from the USA in
7 reviews [47–50,53–55], from the United Kingdom in 1 [45], from Australia in 1 [51], and
from Thailand in 1 [52]. Of the 9 reviews of interventions in digital settings, the authors are
from the United States in 3 reviews [59,60,63], from the United Kingdom in 2 [46,56], from
Australia in 1 [62], from Belgium in 1 [58], from France in 1 [61] and from Turkey in 1 [57].
The authors of the blended learning review are from the USA [64].

3.3.6. Year of Last Paper Included

The studies cited in the reviews that met the inclusion criteria for this review were
published over a wide range of years (between 1981–2019), although only one [61], with
articles published up to and including 2019 was published later than 2017. Of these, 3 were
carried out in school settings [49,51,53], and 1 on digital platforms [46].

3.3.7. Search Tools

All reviews include more than 2 tools to carry out the search, in a range of 3–12, and
in 7 of them the review of gray literature was included.

3.3.8. Multicenter Studies and Number of Studies Included

All reviews from school settings are multicenter, except that of Mirzazadeh et al. [49],
which includes only one North American sample. The same is true for the blended learning
review [64] and for the reviews of digital platforms, except for the reviews by Bailey
et al. [56], L´Engle et al. [60], and Widman et al. [63]. Regarding the number of countries
included in the reviews, the range in the school-setting reviews is from 1 to 11, in digital
Int. J. Environ. Res. Public Health 2021, 18, 2555 7 of 31

platforms reviews from 1 to 16, and in the only review of blended learning, 3. As for the
range of studies included, in the reviews in school setting the range is between 8 and 80, in
digital platforms, between 5 and 60, and in the only review of reviews of blended learning
9 studies were included.

3.3.9. Number of Reviews Covered That Include Meta-Analysis

As for the number of reviews that carry out a meta-analysis, there are 8 in total: 4
in school settings [45,48,49,55] and 4 on digital platforms [43,46,56,58], while in the only
review of blended learning there is no meta- analysis.

3.4. Effectiveness
3.4.1. School Settings
Half of the reviews conclude that interventions are not effective in promoting healthy
sexual behaviors and/or reducing risks [45,47–50]. These reviews are of high quality and
with a reduced risk of bias (see Table A4), so that the results are highly reliable, even
though in most of the studies cited the risk of bias was judged to be high and the quality
of evidence was low or very low. These reviews include those of the Marseille et al. [48]
and Mirzazadeh et al. [49] team, who in two studies—each led by one of the two authors—
analyze, on the one hand, the effectiveness of school-based teen pregnancy prevention
programs [48], and, on the other hand, the effectiveness of school-based programs prevent
HIV and other sexually transmitted infections in North America [49]. The results of the
studies question the usefulness of interventions carried out in schools to prevent both
unwanted pregnancies and the incidence of HIV and other sexual transmitted infections
in adolescents in North America. In addition to these results, those of Lopez et al. [47]
focus on analyzing the effectiveness of programs implemented in schools to promote the
use of contraceptive methods and conclude that many trials reported contraceptive use
as an outcome but did not take into consideration whether contraceptive methods and
their relative effectiveness were part of the content. For its part, the review by Mason-Jones
et al. [45] also concludes that the educational programs covered had no significant effect
as regards the prevalence of HIV or other STIs (herpes simplex virus, moderate evidence
and syphilis, low evidence), nor was there any apparent effect in terms of the number of
pregnancies at the end of the trial (moderate evidence). Finally, the review by Oringanje
et al. [50] finds only limited evidence for program effects on biological measures, and
inconsistent results for behavioral (secondary) outcomes across trials and concludes that
it was only the interventions which combined education and contraception promotion
(multiple interventions) that led to a significant reduction in unintended pregnancies over
the medium- and long-term follow-up period.
In contrast to these negative results in terms of the effectiveness of the programs
implemented in the school environment (identified in 5 of the 10 reviews included), 3 of
the 10 reviews concluded that the programs evaluated were mostly effective in promoting
knowledge, attitudes and/or in reducing risk behaviors [51–53] whilst programs were effec-
tive in terms of some of the primary outcomes in the reviews by Haberland et al., [54], and
Peterson et al. [55]. However, these data must be taken with caution since the level of bias
in these reviews—excepting that of Kedzior et al. [51] with a medium quality level—is at a
low or critically low-quality level. In the review by Chokprajakchad et al. [52], 22 programs
reviewed were effective in changing targeted adolescent psychosocial and/or behavioral
outcomes, in 12 of 17 studies evaluating delay in the initiation of sexual intercourse, the
programs were effective and many of the reviewed studies demonstrated impacts on
short-term outcomes, such as knowledge, attitudes, perception and intention. The review
by Goldfarb et al. [53] identifies changes in appreciation of sexual diversity, dating and
intimate partner violence prevention, healthy relationships, child sex abuse prevention and
additional outcomes. According to the review by Kedzior et al. [51], focused on studies
promoting social connectedness with regard to sexual and reproductive sexual health, the
programs reviewed improved condom use, delayed initiation of sex, and reduced preg-
Int. J. Environ. Res. Public Health 2021, 18, 2555 8 of 31

nancy rates. Additionally, in this review, program effectiveness was influenced by ethnicity
and gender: greater improvements in condom use were often reported among African
American students. For its part, in the study by Peterson et al. [55] the meta-analysis of
three randomized trials provided some evidence that school-environment interventions
may contribute to a later sexual debut while their narrative synthesis of other outcomes
offered only mixed results.
Finally, the review by Haberland et al. [54], which focused on studies analyzing
whether addressing gender and power in sexuality education curricula is associated
with better outcomes, concluded that where interventions addressed gender or power
(N = 10/22) there was a fivefold greater likelihood of effectiveness than in those that
did not.

3.4.2. Online Platforms

The reviews included show a very diverse panorama of digital platforms used to carry
out educational interventions (e.g., websites, social media, gaming, apps or text messaging
and mailing), which makes it difficult to compare the results. Of the 9 reviews of studies
included, only one—in which the effects of TCCMD (Targeted Client Communication
delivered via Mobile Devices) are evaluated [46]—meets the quality criteria according to
the AMSTAR II quality assessment tool [42] (see Table A4); the rest include biases that
limit the reliability of the results so that these must be taken with caution. In the studies
reviewed by Palmer et al. [46] among adolescents nine programs were delivered only
via text messages; four programs used text messages in combination with other media
(for example, emails, multimedia messaging, or voice calls); and one program used only
voice calls.
When compared with more conventional approaches, interventions that use TCCMD
may increase sexual health knowledge (low certainty evidence), and may modestly increase
contraception use (low certainty evidence) while the effect on condom use remains unclear
given the very low certainty evidence. Additionally, when compared with digital non-
targeted communication, the effects TCCMD on sexual health knowledge, condom and
contraceptive use are also unclear, again given the very low-certainty evidence. The review
finds evidence of a modest beneficial intervention effect on contraceptive use among
adolescent (and adult) populations, but that there was insufficient evidence to demonstrate
that this translated into a reduction in contraception.
Most of the reviews included refer to changes to a greater or lesser extent [56,57,59,60,62,63],
while no changes determined by the intervention were identified in the study by DeSmet
et al. [58]. Finally, the review by Martin et al. [61] does not include details about changes as
a result of the programs.
The review by L´Engle et al. [60] assesses mHealth mobile phone interventions for
ASRH (almost all of which were carried out via SMS platforms, with the notable exception
of only four of the programs covered which used other media formats instead of or as
well as SMS). The interventions reviewed set out to foster positive and preventive SRH
behaviors, augment take-up and continued use of contraception, support medication
adherence for HIV-positive young people, support teenage parents, and encourage use of
health screening and treatment services. Results from the studies covered in the review
offer support for diverse uses of mobile phones in order to help further ASRH. The health
promotion programs that made use of text messaging demonstrated robust acceptability
and relevance for young people globally and contributed to improved SRH awareness, less
unprotected sex, and more testing for STIs. However, the review also found that improved
reporting on essential mHealth criteria is necessary in order to understand, replicate,
and scale up mHealth interventions. Holstrom’s [59] review, focused on evaluations of
internet-based sexual health interventions, finds that these were associated with greater
sexual health knowledge and awareness, lower rates of unprotected sex and higher rates
of condom use, as well as increased STI testing. Moreover, the review explores young
people’s continuing use of and trust in internet as a source of information about sexual
Int. J. Environ. Res. Public Health 2021, 18, 2555 9 of 31

health, as well as the particular themes that interest them. Specifically, the study finds that
young people want to know not only about STIs, but also about sexual pleasure, about
how to talk with partners about their sexual desires, as well as about techniques to better
pleasure their partners.
The review by Widman et al. [63] reveals a significant weighted mean effect of
technology-based interventions on condom use and abstinence, the effects of which were
not affected by age, gender, country, intervention, dose, interactivity, or program tailoring.
The effects were more significant when evaluated with short-term (one to five months)
follow-ups than with longer term (over six months) ones. Moreover, digital programs were
more effective than control programs in contributing to sexual health knowledge and safer
sex norms and attitudes. This meta-analysis, drawing on fifteen years of research into
youth-oriented digital interventions, is clear evidence of their ability to contribute to safer
sex behavior and awareness. In the review by Wadham et al. [62] the majority of studies
used a web-based platform for their programs (16 out of 25). These web-based programs
varied between complex, bespoke multimedia interventions to more simplified educational
modules. Five studies employed SMS platforms both via mobile phone messaging and
web-based instant message services. Three of the programs used social networking sites, ei-
ther for live chat purposes or alongside a web-based platform. Several studies showed that
variety in terms of media and platforms was associated with stronger positive responses
among participants and improved outcomes. Eleven of the twenty-five studies focused
specifically on HIV prevention, with seven finding a statistically significant effect of the
program with regard to knowledge levels about prevention of HIV and other STIs, as well
as about general sexual health knowledge. However, only twenty percent of the programs
that assessed intended use of condoms reported significant effects due to the intervention.
The review by Bailey et al. [56] (p. 5) assesses interactive digital interventions (IDIs),
defined as “digital media programs that provide health information and tailored decision
support, behavioral-change support and/or emotional support” and focuses on the sexual
well-being of young people between the ages of thirteen and twenty four in the United
Kingdom. IDIs have significant though small effects on self-efficacy and sexual behavior,
although there is not sufficient evidence to ascertain the effects on biological outcomes or
other longer-term impacts. When comparing IDIs with in-person sexual health programs,
the former demonstrate significant, moderate positive effects on sexual health knowledge,
significant small effects on intention but no demonstrable effects on self-efficacy. The review
by Celik et al. [57] looks at digital programs (the majority internet- and computer-based
with only six making use of mobile phone-based applications) and sets out to understand
their effectiveness in changing adolescents’ health behaviors. Findings from the studies
(n = 9) suggest that the digital interventions carried out with the adolescents generally
had a positive effect on health-promoting behaviors. However, in another study focused
on fostering HIV prevention [66], there was a statistically significant increase in health-
promoting behavior in only one of the four studies reviewed.
In the review by DeSmet et al. [58], no significant behavioral changes as a result
of the interventions for sexual health promotion using serious digital games are iden-
tified, although the interventions did have significant though small positive effects on
outcomes. The fact that so few studies both met the inclusion criteria and also analyzed
behavioral effects suggests the need to further investigate the effectiveness of this kind of
game-based approach.
Finally, in the review by Martin et al. [61] 60 studies were covered, detailing a total of
37 interventions, though only 23 of the reviews included effectiveness results. A majority
of the interventions were delivered via websites (n = 20) while online social networks were
the second most favored medium (n = 13), mostly via Facebook (n = 8). The programs
under review favored online interaction, principally amongst peers (n = 23) but also with
professionals (n = 16). The review concludes that ASHR programs promoting these kinds
of online participation interventions have demonstrated feasibility, practical interest, and
Int. J. Environ. Res. Public Health 2021, 18, 2555 10 of 31

attractiveness, though their effectiveness has yet to be determined, given that they are still
in the early stages of design and evaluation.

3.4.3. Blended Learning

In the only blended learning review included in our study [64], the authors conclude
that blended learning approaches are being successfully applied in ASHR interventions,
including in school-based programs, and have led to positive behavioral and psychosocial
changes. However, these results should be treated with caution as the review does not
follow the guidelines recommended in the AMSTAR II quality assessment tool [44] (see
Table A4) and only includes nine studies.

4. Discussion
The present review of reviews assesses, for the first time jointly to our knowledge,
the effectiveness of sexual education programs for the adolescent population (ASRH) de-
veloped in school settings, digital platforms and blended learning. Of the twenty reviews
included (comprising a total of 491 programs, mostly from the USA), ten correspond to
reviews of programs implemented in school settings, nine to those dealing with interven-
tions via digital platforms and only one deals with studies relating to blended learning.
Twelve (60%) of the reviews included (6 out of 10 in school settings, 5 out of 9 on digital
platforms, and the only blended learning review) have been published in the last 3 years
(between 2018 and 2020). Thus, the present study constitutes the most up-to-date and
recent review of reviews incorporating several contemporary studies not covered by earlier
reviews [19,27,33,35–39].

4.1. Interventions Reviewed

The interventions included in the reviews covered by our study were largely focused
on reducing risk behaviors (e.g., VIH/STIs and unwanted pregnancies), and envisaging
sex as a problem behavior. Programs reviewed often focused on the physical and biological
aspects of sex, including pregnancy, STIs, frequency of sexual intercourse, use of condom,
and reducing adolescents´ number of sexual partners. One exception is Golfard’s et al. [53]
review about comprehensive sex education, which is centered on healthy relationships
and sexual diversity, though it also makes reference to prevention of violence (dating
and intimate partner violence prevention and sex abuse prevention). However, Golfard’s
et al.’s [53] rejects more than 80% of the studies initially reviewed because they were
focused solely on pregnancy and disease prevention. In the reviews of interventions on
digital platforms and via blended learning all the outcomes focused on behaviors related
to sexual health (focused on the prevention of risk behaviors), and in several cases also
addressed perceived satisfaction and usability. These results are in line with other studies
that confirm the over-attention given to risk behaviors, to the detriment of other more
positive aspects of sexuality [67,68]. Teachers continue to perceive their responsibility as
combating sexual risk, whilst viewing young people as immature and oversexualized [69],
even as adolescents themselves express a preference for sex education with less emphasis
on strictly negative sexual outcomes [16], and more emphasis on peer education [70].
As for more positive views of sexuality, only on rare occasions do interventions address
issues such as sexual pleasure, desire and healthy relationships. Desire and pleasure were
not included in the outcome evaluations for school settings, nor for digital and blended
learning programs included in this review: again this is in line with the position of other
authors cited in the present study, who advocate the need to also embrace the more positive
aspects of sexuality [53,56]. Specifically, Bailey and colleagues [56] (p. 73) suggest as
“optimal outcomes” social and emotional well-being in sexual health. Young people want
to know about more than STIs, they also “want information about sexual pleasure, how
to communicate with partners about what they want sexually and specific techniques to
better pleasure their partners” [59] (p. 282). Similarly, Kedzior et al. [51] also argue for the
Int. J. Environ. Res. Public Health 2021, 18, 2555 11 of 31

need to move beyond a risk-aversion approach and towards one that places more emphasis
on positive adolescent sexual and reproductive health.
Pleasure and desire are largely absent within sex and relationship education [71] and,
when they are included, they are often proposed as part of a discourse on safe practice,
where pleasure continues to be equated with danger [72]. The persistent absence of a
“discourse of desire” in sex education [73,74] is especially problematic for women, for whom
desire is still mediated by (positive) male attention, and for whom pleasure is derived from
being found desirable and not from sexual self-expression or from their own desires [75].
Receiving sexualized attention from men makes women “feel good” by increasing their
self-esteem and self-confidence [76]. However, it is still men who decide what is sexy
and what is not, based on the attention they pay to women “girl watching”, [77] (p. 386),
which leads the latter to self-objectify [78] with all the attendant negative consequences for
their overall and sexual health [79]. In fact, women experience “pushes” and “pulls” [80]
(p.393) with regard to sexualized culture. In one sense, the sexualization of culture has
placed women in the position of subjects who desire, not just that of subjects who are
desired, but at the same time it becomes a form of regulation in which young women are
forced to assume the current sexualized ideal [81,82] in order to position themselves as
“modern, liberated and feminine,” and avoid being seen as “outdated or prudish” [83] (p.
16). Koepsel [84] provides a holistic definition of pleasure as well as clear recommendations
for how educators can overcome these deficits by incorporating pleasure into their existing
curricula. At present, sexual education is still largely centered on questions of public
health, and there is as yet no consensus on criteria for defining sexual well-being and
other aspects of positive sexuality [85]. Patterson et al. [86] argue for the need to mandate
“comprehensive, positive, inclusive and skills-based learning” to enhance people´s ability
to develop healthy positive relationships throughout their lives.
The absence of desire and pleasure in the outcomes of the evaluated reviews is con-
nected with the absence of gender-related outcomes. Only one of the reviews addresses
the issue of gender and power in sexuality programs [54], illustrating how their inclu-
sion can bring about a five-fold increase in the effectiveness of risk behavior prevention.
Nonetheless, men are far less likely than women to sign up for a sexuality course, and as a
result of masculine ideologies many young males experience negative attitudes towards
sex education [87]. To date we still have little idea as to what are the “active ingredients”
that can contribute to successfully encouraging men to challenge gender inequalities, male
privilege and harmful or restrictive masculinities so as to help improve sexual and repro-
ductive health for all [88] (p.16). Schmidt et al.’s [89] review looks at 10 evidence-based
sexual education programs in schools: the majority discuss sexually transmitted diseases
and unplanned pregnancy, abstinence, and contraceptive use, while very few address
components related to healthy dating relationships, discussion of interpersonal violence or
an understanding of gender roles.
The International Guidance on Sexuality Education [90], and the International Techni-
cal Guidance on Sexuality Education [17] promote the delivery of sexual education within
a framework of human rights and gender equality to support children and adolescents
in questioning social and cultural norms. The year 2020 marked the anniversaries of sev-
eral path breaking policies, laws and events for women’s rights: the 100th anniversary
of women´s suffrage in the United States; the 25th anniversary of the Beijing Platform
for Action, a global roadmap for women´s empowerment; and, the 20th anniversary of
the United Nations Security Council Resolution for a Women, Peace and Security agenda.
Although there have been important advances in recent years in research relating to the
inclusion of gender equality and human rights interventions in ASRH policies and pro-
gramming still “fundamental gaps remain” [40] (p. 14). Gender equality, and to an even
greater extent human rights, have had very little presence in sexual and reproductive
health programs and policies, and there is a pressing need to do more to address these
issues systematically. Specifically, issues such as abortion and female genital mutilation,
Int. J. Environ. Res. Public Health 2021, 18, 2555 12 of 31

with clear repercussions in terms of gender equality and human rights, are rarely dealt
with [40].
Furthermore, sexual education that privileges heterosexuality reinforces hegemonic
attributes of femininity and masculinity, and ignores identities that distance themselves
from these patterns. Our collective heteronormative legacy marginalizes and harms LGB
families [91] and LGBTQ+-related information about healthy relationships is largely absent
from sexual and reproductive health programs [92]. Students want a more LGBTQ+
inclusive curriculum [92]: in the present RoR one review [53] addresses the issue of non-
heteronormative identity in sexuality programs with significant results; and other authors
are exploring promising initiatives which are also challenging this lack of inclusivity [93]
and rectifying heterosexual bias [94]. However, unfortunately, the underlying neoliberal
focus of the majority of contemporary sexuality education militates to assimilate LGBTQ+
people into existing economic and social normative frameworks rather than helping disrupt
them [95].

4.2. Effectiveness
This present review of reviews shows a variety of types of sexual health promotion
initiatives across the three settings (school-based, digital and blended learning), with
inconsistent results. The reviews with lower risk of bias are those carried out in school
settings and those that are most critical regarding the effectiveness of programs promoting
ASRH, both in the prevention of pregnancies and of HIV/STIs. Reviews dealing with
digital platforms and blended learning show greater effectiveness in terms of promoting
adolescent sexual health: however, these are also the studies that incorporate the highest
risks of bias. Specifically, in digital platforms programs the great variety of alternatives
makes comparability difficult. Moreover, these programs, along with blended learning, are
in a more incipient state of evaluation, compared to school-setting evaluations, and present
greater risks of lower quality than reviews in school settings.
The results of the present RoR are in line with those of previous RoRs [19,32]. The
review of reviews by Denford et al.s´ [19] RoR covered 37 reviews up to 2016 and sum-
marized 224 primary randomized controlled trials: whilst it concludes that school-based
programs addressing risky sexual behavior can be effective, its reviews of exclusively
school-based studies offer mixed results as to effectiveness in relation to attitudes, skills
and behavioral change. Some of those studies report positive effects while others find there
are no effects, if not even negative effects, in terms of the aforementioned outcomes [19]. As
regards pregnancy, programs appear to be effective at increasing awareness regarding STIs
and contraception but overall the findings suggest that the impact of these interventions on
attitudes, behaviors and skills variables are mixed, with some studies leading to improve-
ments whilst others show no change. Moreover, the fact that community-based programs
were also taken into consideration might have led to the effectiveness of school-based
programs being exaggerated [19].
However, although in our RoR the higher quality/lower bias studies—in keeping with
the findings of previous reviews [19,33]—fail to show a clear pattern of effectiveness, the in-
terventions could nevertheless be generating changes as Denford et al. [19] suggest, though
not in the measured outcomes, bearing in mind the low incidence of sexual intercourse
and pregnancy in school-going adolescents.
With regard to school settings, Peterson et al. [55] conclude that further, more rigorous
evidence is necessary to evaluate the extent to which interventions addressing school-
related factors are effective and to help better understand the mechanisms by which they
may contribute to improving adolescent sexual health. With regard to digital platform
programs, Wadham et al. [62] (p. 101) argue that “although new media has the capacity to
expand efficiencies and coverage, the technology itself does not guarantee success.” An
interesting observation in their review was that interventions which were either web-based
adaptations of prior prevention programs, or were theory-based or had been developed
from models of behavioral change appeared effective independently of the chosen dig-
Int. J. Environ. Res. Public Health 2021, 18, 2555 13 of 31

ital media mode. However, digital programs are still in the early stages of design and
evaluation, especially in terms of the effects of peer interaction and often diverge from
existing theoretical models [61] (p. 13). The expert opinion-based proposal of the European
Society for Sexual Medicine [96] argues that e-sexual health education can contribute to
improving the sexual health of the population it seems the future of CSHE is moving
towards smartphone apps [97].
However, “despite clear and compelling evidence for the benefits of high-quality
curriculum-based CSE, few children and young people receive preparation for their lives
that empowers them to take control and make informed decisions about their sexuality and
relationships freely and responsibly” [17] (p. 12), and during “the current public health
crisis, the sexual and reproductive health of adolescents and young adults must not be
overlooked, as it is integral to both their and the larger society’s well-being” [28] (p. 9). In
the light of these challenges, Coyle et al.’s [64] suggestion that the blended learning model
may end up achieving a far more dominant role in the future of sexual education acquires
even more relevance.

4.3. Limitations
This study represents the first review of reviews, as far as we are aware, in which
the effectiveness of sex education programs in different settings (school-based, digital
and blended learning) is evaluated, using a rich methodology and providing interesting
conclusions. However, the present review of reviews is not without its limitations.
While systematic reviews and reviews of reviews can offer a way synthesizing large
amounts of data, the great heterogeneity and diversity of measured outcomes make it
difficult to establish a synthesis of the results, even more so in cases where it is not possible
to apply meta-analysis. Furthermore, the quality of reviews of reviews is limited by that of
the reviews they include and RoRs do not necessarily represent the leading edge research
in the field.
In addition, although we searched for a wide range of keywords on the most com-
monly used databases in the field of health (namely ERIC, Web of Science, PubMed, and
PsycINFO) to identify relevant papers, it is possible that the choice of keywords and
database may have resulted in our omitting some relevant studies. Moreover, our review
has focused on articles in international journals published in English, allowing us access to
the most rigorous peer-reviewed studies and to those with greater international diffusion,
given that English is the most frequently used language in the scientific environment:
notwithstanding, this has also limited the scope of our review by precluding research
published in other languages and contexts. Nor have documents that could have been
found in the gray literature been included, given that only peer-reviewed studies have
been considered for inclusion.
It is worth remembering moreover that most of the data on the outcomes of the studies
included are self-reported, with mention of only occasional biological outcomes, which
may limit the reliability of the effectiveness results. This represents another interesting
reflection on the way in which the evaluation of the effectiveness of programs on sexual
education is being carried out, and alerts us to the need for change.
Finally, it should be noted that this review of reviews is focused on adolescents from
high-income countries, and our results show that studies carried out in the United States are
largely overrepresented, since it is the country that provides the highest number of samples,
especially in school settings: this may give rise to bias when it comes to generalizing from
these results. Once again, this raises another necessary reflection on the capitalization that
studies focused on American samples are having in the construction of the body of scientific
knowledge on sexual and reproductive behavior, when in reality sexuality is conditioned by
socio-economic variables that require a far-more multicultural and world-centric approach.
Int. J. Environ. Res. Public Health 2021, 18, 2555 14 of 31

5. Conclusions
This review of reviews is the first to assess jointly the effectiveness of school-based,
digital and blended learning interventions in ASRH in high-income countries. The ef-
fectiveness of the sex education programs reviewed mostly focused on the reduction of
risky behaviors (e.g., STI or unwanted pregnancies) as public health outcomes; however,
pleasure, desire and healthy relationships are outcomes that are mostly conspicuous by
their absence in the reviews we have covered. Nonetheless, the broad range of studies
included in this RoR, with their diversity of settings and methods, populations and objec-
tives, precludes any easily drawn comparisons or conclusions. The inconsistent results and
the high risk of bias reduce the conclusiveness of this review, so a more rigorous assessment
of the effectivity of sexual education programs is pending and action needs to be taken
to guarantee better and more rigorous evaluations, with sufficient human and financial
resources. Schools and organizations need technical assistance to build the capacity for
rigorous program planning, implementation and evaluation [98]. To this end, there are
already examples of interesting proposals, such as that of the Working to Institutionalize
Sex Education (WISE) Initiative, a privately funded effort to help public school districts
develop and deliver comprehensive sexuality programs in the USA [99].
The extent of the risks of bias identified in the reviews and studies covered by this
RoR points to an important conclusion, allowing us to highlight the precariousness that
characterizes the evaluation of sexual education programs and the consequent undermining
of public policy oriented to promoting ASRH. Public policies that promote ASRH are of vital
importance when it comes to minimizing risks related to sexual behavior, and maximizing
healthy relations and sexual well-being for the youngest members of our society.
Above all it is important to recognize the opportunities afforded by new technologies,
so ubiquitous in the lives of young people, since they allow for programs that are far
more cost-effective than traditional, in-person interventions. Finally, blended learning
programs are perhaps even more promising, given their combination of the best of face-to-
face and digital interventions, meaning they provide an excellent educative tool in the new
context of the COVID-19 pandemic, and may even become the dominant teaching model
in the future.

Author Contributions: Conceptualization, M.L.-F. and R.M.-R.; methodology, M.L.-F.; R.M.-R.;

Y.R.-C. and M.V.C.-F.; formal analysis, M.L.-F.; R.M.-R.; Y.R.-C. and M.V.C.-F.; investigation, M.L.-F.;
R.M.-R.; Y.R.-C. and M.V.C.-F.; writing—original draft preparation, M.L.-F. and R.M.-R.; writing—
review and editing, M.L.-F.; R.M.-R., and Y.R.-C. and.; supervision, M.L.-F.; R.M.-R.; Y.R.-C. and
M.V.C.-F. All authors have read and agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: The data presented in this study are available from the corresponding
author on reasonable request.
Conflicts of Interest: The authors declare that they have no conflicts of interest.
 Int.
Int. J. Environ.
J. Environ. Res.
Res. Public
Public Health
Health 18,18,
2021,
2021, 2555
2555 1515
ofof
2931

Appendix
AppendixAA

Figure A1. Flow diagram Preferred reporting items for systematic reviews and meta-analysis, PRISMA).
Figure A1. Flow diagram Preferred reporting items for systematic reviews and meta-analysis, PRISMA).
Int. J. Environ. Res. Public Health 2021, 18, 2555 16 of 31

Table A1. Search Terms Used.

Characteristic Search Terms

“sex education” OR “sexuality education” OR “sex education program” OR
“sexuality education program” OR “reproductive education” OR “Sexual
Sex education
health education” OR “reproductive health education” OR “sexual and
reproductive health” OR “sexual health”
“adolescent” OR “adolescents” OR “teenagers” OR “young people” OR
Study population (adolescents) “young person” OR “primary students” OR “Secondary Students” OR
“student”
“internet” OR “online” OR “offline” OR “virtual” OR “digital” OR “computer”
OR “computer-technology” OR “technology” OR “computerized” OR
“internet-based intervention” OR “computer based approach” OR
Setting (school, online, blended learning)
“computer-assisted education” OR “school” OR “school-based” OR “K-12
setting” OR “school based programs” or “school setting” OR “blended
learning”
“evaluation” OR “assessment” OR “impact” OR “intervention” OR “impact
evaluation” OR “outcome evaluation” OR “process evaluation” OR
Evaluation (review of reviews)
“comparative effectiveness research” OR “review” OR “review of reviews” OR
“systematic reviews” OR “narrative reviews”

Table A2. Description of studies.

School
Year of last Country of the
Country of Last Paper
Authors/Year Title Search Tools Cover Period Paper Studies Synthesis
the Review Included
Included Included
PubMed,
Sexual Health
CINAHL,
Interventions
Scopus,
Chokprajakchad Among Early
Thailand Science Direct, 2006–2017 2016 33 studies International. Narrative
et al. (2018) Adolescents:
Web of Science,
An Integrative
Thaijo and
Review.
TCI.
USA (n = 55),
Israel (n = 1),
Canada (n = 6),
Australia (n = 3),
New Zealand
(n = 1),
The Netherlands
Three Decades
(n = 2)
of Research:
ERIC, Psycinfo Kenya (n = 1),
Goldfarb et al. The Case for
USA and 1990–2017 2017 80 studies Mexico (n = 2), Narrative
(2020) Comprehen-
MEDLINE. South Africa
sive Sex
(n = 1),
Education.
Ireland (n = 2),
South Korea
(n = 1),
China (n = 1),
Holland (n = 1)
U.K (n = 1),
Europe (n = 2).
The Case for
Addressing
PubMed, USA (n = 14).
Gender and
ERIC, High income Meta-
Power in
Cochrane countries other analysis
Sexuality and
Haberland et al. Central than the United (one
HIV USA 1990–2012 2011 22 studies
(2016) Register of States (n = 2). outcome)
Education: A
Controlled Low or middle and
Comprehen-
Trials and income country Narrative
sive Review of
Eldis. (n = 6).
Evaluation
Studies.
Int. J. Environ. Res. Public Health 2021, 18, 2555 17 of 31

Table A2. Cont.

A Systematic
Review of
School-Based
Programs to
Improve PubMed,
Adolescent CINAHL,
Kedzior et al.
Sexual and Australia Embase, July 2019 2017 18 studies International. Narrative
(2020)
Reproductive Psycinfo, ERIC
Health: and SCOPUS.
Considering
The Role of
Social Con-
nectedness.
School-Based
PubMed, USA (n = 6). U.K
Interventions
CENTRAL, (n = 1). Mexico
Lopez et al. for Improving
USA ERIC, Web of 1981–2016 2014 11 studies (n = 3). Narrative
(2016) Contraceptive
Science and South Africa
Use in
POPLINE. (n = 1).
Adolescents.
Effectiveness
of
School-Based Cochrane
Teen Central, ERIC,
Pregnancy PubMed,
Marseille et al. Prevention Psycinfo, USA (n = 14). Meta-
USA 1985–2017 2016 21 studies
(2018) Programs in Scopus, Web Canada (n = 4). analysis
The USA: A of Science and
Systematic The Gray
Review and Literature.
Meta-
Analysis.
Sub-Saharan
Africa:
School-Based (South Africa,
Interventions MEDLINE, Tanzania
for Preventing CENTRAL, Zimbabwe,
Mason-Jones HIV, Sexually United OMS, AIDS, Malawi Meta-
1990–2016 2015 8 studies
et al. (2016) Transmitted Kingdom AEGIS, CDC, Kenya) n = 5, analysis
Infections, and and Europe:
Pregnancy in ONUSIDA. (England and
Adolescents. Scotland) n = 2,
Latin America
(n = 1).
Do
School-Based PubMed,
Programs Cochrane
Prevent HIV Central
and Other Register of
Sexually Controlled
Mirzazadeh et al. Meta-
Transmitted USA Trials, ERIC, May 2017 2017 9 studies USA (n = 9).
(2018) analysis
Infections in Psycinfo,
Adolescents? Scopus, Web
A Systematic ofScience
Review and andThe Gray
Meta- Literature.
Analysis.
CENTRAL,
The Cochrane
Library,
MEDLINE,
EMBASE,
LILACS,
USA (n = 41),
Social Science
England (n = 2),
Interventions Citation Index
Scotland (n = 2),
for Preventing and Science
Canada (n = 1),
Oringanje et al. Unintended Citation Index,
USA 1994–2015 2015 53 studies Italy (n = 1), Narrative
(2016) Pregnancies Dissertations
Mexico (n = 2),
Among Abstracts
Low and middle
Adolescents Online,
income countries
Network,
(n = 4).
HealthStar,
Psycinfo,
CINAHL,
POPLINE and
The Gray
Literature
Int. J. Environ. Res. Public Health 2021, 18, 2555 18 of 31

Table A2. Cont.

Effects of
Interventions
Addressing BiblioMap,
School CINAHL Plus, Australia and
Environments ERIC, IBSS, USA (n = 5),
or Educational Open Grey, South Africa and Meta-
Peterson et al.
Assets on USA ProQuest, 1999–2016 2016 11 studies Kenya (n = 4), analysis and
(2019)
Adolescent Psycinfo, Malawi and narrative
Sexual Health: Medline and Zimbabwe
Systematic Web of (n = 2).
Review and Science.
Meta-
Analysis.
Online
Year of last
Country of Last paper Country of the
Authors/Year Title Search tools Cover period paper Synthesis
the review included included studies
included
CENTRAL,
Sexual Health
DARE,
Promotion for
MEDLINE, Meta-
Young People
Bailey et al. United EMBASE, United Kingdom analysis
Delivered Via 1989–2013 2013 19 studies
(2015) Kingdom CINAHL, BNI, (n = 19). andNarra-
Digital Media:
Psycinfo and tive
A Scoping
The Gray
Review.
Literature.
The Effect of
Technology-
Based Canada (n = 2),
Programmes New Zealand
PubMeb and
On Changing (n = 1), Australia
Celik et al. (2020) Turkey Science direct 2011–2016 2016 16 studies Narrative
Health (n = 3), Norway
databases.
Behaviours of (n = 1),
Adolescents: USA (n = 9).
Systematic
Review.
A Systematic
Review and
Meta-Analysis
of
PubMed, Web
Interventions USA (n = 6),
Desmet et al. of Science, Meta-
for Sexual Belgium July 2013 2012 7 studies United Kingdom
(2015) CINAHL and analysis
Health (n = 1).
Psycinfo.
Promotion
Involving
Serious Digital
Games.
Sexuality
Education
Medline,
Goes Viral:
EBSCO, USA (n = 3),
What We
Holstrom (2015) USA ERIC and 2004–2014 2012 5 studies Australia (n = 1), Narrative
Know About
PubMed. The Europe (n = 1).
Online Sexual
EBSCO.
Health
Information.
PubMed,
Mobile Phone Embase,
Interventions Global Health,
for Adolescent Psycinfo,
L’Engle et al. Sexual and Popline,
USA 2000–2014 2014 35 studies USA (n = 35). Narrative
(2016) Reproductive Cochrane
Health: A Library, Web
Systematic of Science and
Review. The Gray
Literature.
USA (n = 38),
Participatory
Canada (n = 1),
Interventions
United Kingdom
for Sexual
(n = 4),
Health PubMeb,
Netherlands
Promotion for Aurore
Martin et al. (n = 1),
Adolescents France database and 2006–2019 2019 60 studies Narrative
(2020) Europe (n = 2).
and Young The Gray
Australia (n = 3),
Adults on The Literature.
Uganda (n = 4),
Internet:
Brazil (n = 2),
Systematic
Chile (n = 2),
Review.
Asia (n = 3),
Int. J. Environ. Res. Public Health 2021, 18, 2555 19 of 31

Table A2. Cont.

Cochrane
Targeted Central
Colombia (n = 1),
Client Com- Register of
China (n = 2),
munication Controlled
Australia (n = 2), Meta-
Via Mobile Trials,
Palmer et al. United USA (n = 9), U.K. analysis
Devices for MEDLINE, July 2019 2017 33 studies
(2020) Kingdom (n = 2), Peru AndNarra-
Improving POPLINE,
(n = 1), Lower tive
Sexual and WHO Global
middle income
Reproductive Health Library
(n = 16).
Health. and The Gray
Literature.
USA (n = 16),
New Digital Canada (n = 1),
Media Netherlands
CINAHL,
Interventions (n = 2),
Medline,
for Sexual Australia (n = 2),
Psycinfo,
Wadham et al. Health African
Australia Socindex, 2010–2017 2016 25 studies Narrative
(2019) Promotion American
Informit,
Among Young communities
PubMed and
People: A (n = 1), Chile
Scopus.
Systematic (n = 1), Uganda
Review. (n = 1),
Thailand (n = 1).
Technology-
Based
Interventions
Medline,
to Reduce
Psycinfo and
Widman et al. Sexually Meta-
USA Communica- May 2017 2015 16 studies USA (n = 16).
(2018) Transmitted analysis
tion
Infections and
Source.
Unintended
Pregnancy
Among Youth.
Blended Learning
Year of last Country of the
Country of Last Paper
Authors/Year Title Search Tools Cover Period Paper Included Synthesis
the Review Included
Included Studies
Blended
Learning for Google
Sexual Health Scholar,
Education: PubMed and USA (n = 6), U.K
Coyle et al.
Evidence Base, USA the 2000–2017 2015 9 studies (n = 2), Europe Narrative
(2019)
Promising Cumulative (n = 1).
Practices, and Index of
Potential Nursing.
Challenges.

Table A3. Characteristics and main results of the studies included.

School
Authors/Year Objective Participants Type of Study Outcomes Results

• A total of 14 studies measured only

adolescent psychosocial outcomes
related to sexual behavior.
• A total of 17 studies measured the
outcomes of sexual initiation, while 18
PRIMARY studies measured other sexual risk
To describe and behaviors such as recent sexual activity
(a) Adolescent sexual
analyze (six studies), a number of sexual
behavior.
methodological partners (three studies) and
14 studies used (b) Initiation of sexual
and substantive contraception and/or condom use
randomized activity.
features of research (nine studies).
controlled trials (c) Condom use and
on interventions to • In total, 22 programs reviewed were
Chokprajakchad (RCTs), 16 used other. Contraceptive
delay the initiation 10–13 years effective in changing targeted
et al. (2018) quasi-experimental use.
of sexual adolescent psychosocial and/or
designs and three SECONDARY
intercourse and behavioral outcomes.
used a pre-test, (a) Adolescents’
prevent other • Many of the studies reviewed
post-test design. attitudes.
sexual risk demonstrated impacts on short-term
(b) Self-efficacy.
behaviors among outcomes, such as knowledge,
(c) Intentions related to
early adolescents. attitudes, perception and intention.
sexual behavior.
• Delay in the initiation of sexual
intercourse, the sexual behavior most
commonly measured by studies in this
review, was seen in 12 of 17 studies
evaluating this outcome.
Int. J. Environ. Res. Public Health 2021, 18, 2555 20 of 31

Table A3. Cont.

PRIMARY
(a) Appreciation of sexual
diversity:
Homophobia,
homophobic bullying,
understanding of
gender/gender norms, • Appreciation of Sexual Diversity:
recognition of gender lower homophobia, reduced
equity, rights, and social homophobic bullying, expanded
justice. understanding of gender/gender
(b) Dating and intimate norms, recognition of gender equity,
partner violence rights, and social justice.
prevention: • Dating and Intimate Partner Violence
Knowledge and prevention: improved knowledge and
attitudes about, and attitudes about, and reporting of, DV
To find evidence reporting of, DV and and IPV, decreased DV and IPV
Randomized
for the IPV; DV and IPV perpetration and victimization,
controlled trial
effectiveness of perpetration and increased bystander intentions and
Goldfarb et al. (RCTs),
comprehensive sex 3–18 years victimization; bystander, behaviors.
(2020) quasi-experimental,
education in intentions and • Healthy relationships: increased
and pre- and
school-based behaviors. knowledge, attitudes and skills,
post-test.
programs. (c) Healthy Relationships. improved communication skills and
Knowledge, attitudes, intentions.
and skills and • Child Sex Abuse prevention: improved
intentions. knowledge, attitudes, skills and
(d) Child sex abuse social-emotional outcomes related to
prevention: personal safety and touch, improved
Knowledge, attitudes, disclosure and behaviors).
skills and • Additional outcomes: social-emotional
social-emotional learning and media literacy.
outcomes related to
personal safety and
touch.
(e) Additional outcomes
Social emotional
learning.
Media literacy.

• Of the 22 interventions that met the

inclusion criteria, 10 addressed gender
Evaluation of or power, and 12 did not.
behavior-change • The programs that addressed gender
interventions to or power were five times as likely to be
prevent HIV, STIs effective (positive effects on sexual and
or unintended PRIMARY reproductive health—including
Randomized
pregnancy to (a) STIs. knowledge, attitudes, reported
Haberland et al. Adolescents Controlled Trials
analyze whether (b) HIV. behavior change and health outcomes)
(2016) under 19 years (RCTs) or quasi-
addressing gender (c) Pregnancy. as those that did not; in all 80% of
experimental.
and power in (d) Childbearing. them were associated with a
sexuality education significantly lower rate of STIs or
curricula is unintended pregnancy. In contrast,
associated with among the programs that did not
better outcomes. address gender or power, only 17%
had such an association.

Randomized
controlled trials,
non-randomized PRIMARY
controlled trials (a) Contraception use. • Improved condom use, delayed
(including quasi), (b) Intercourse initiation of sex, and reduced
controlled (frequency or another pregnancy rates.
Determine the
before-after outcome as defined by • Program effectiveness was influenced
impact of
(pre-/post-) authors). by ethnicity and gender: greater
school-based
interrupted time (c) Risk of adolescent improvements in condom use were
programs that
Kedzior et al. series, and pregnancy and birth. often reported among African
promote social 10–19 years
(2020) program (d) Rates of sexually American students.
connectedness on
evaluations. transmissible infections • Programs that were most effective
adolescent sexual
Program (STIs). incorporated multiple constructs of
and reproductive
evaluation without (e) Attitudes, beliefs and social connectedness, included social
health.
a control group knowledge about sex skill-building and had a sustained
were eligible if they and reproductive health. intensity.
reported on (f) Autonomy.
outcomes pre- and (g) Connectedness.
post- program
implementation.
Int. J. Environ. Res. Public Health 2021, 18, 2555 21 of 31

Table A3. Cont.

• Of the trials included, most compared

PRIMARY the new programs to ‘usual’ sex
(a) Pregnancy (six education.
months or more after • Many trials assessed contraceptive use
the intervention began). as an outcome but did not report
(b) Contraceptive use whether the content included
(three months or more contraceptive methods and their
To identify Randomized relative effectiveness.
after the intervention
school-based controlled trials • Since most trials aimed to prevent
began).
interventions that 19 years or (RCTs). (Of 11 STI/HIV and pregnancy, they focused
Lopez et al. (2016) SECONDARY
improved younger trials, 10 were on condom use. However, several
(a) Knowledge of
contraceptive use cluster studies covered a variety of birth
contraceptive
among adolescents. randomized). control methods.
effectiveness or effective
method use. • The overall quality of results was low:
(b) Attitude about some trials lacked information on how
contraception or a their programs worked, many
specific contraceptive analyzed subsamples rather than all
method. students in the study, and most had
high losses.

• Regarding primary outcomes: 30

unique pooled comparisons for
pregnancy were included, of which 24
were not statistically significant and 6
showed statistically significant
Randomized changes in pregnancy rates (two with
controlled trials increased risk and four with decreased
To evaluate the (RCTs) (10 studies) PRIMARY risk)
effectiveness of and non-RCTs (11 Pregnancy. • Regarding the secondary outcomes:
school-based teen studies) with SECONDARY the majority of the pooled risk
Marseille et al. reduction ratios were not statistically
pregnancy 10–19 years comparator groups (a) Sexual Initiation.
(2018) significant. No consistent evidence of
prevention were eligible (b) Condom Use.
programs in the yielded 30 unique (c) Oral Contraception increasing condom or OCP use, or
USA. pooled Pill Use. delaying sexual initiation were found.
comparisons for The six that were statistically
pregnancy. significant for sexual initiation showed
a reduced risk of sexual initiation as
did the four for no condom use.
• All studies were at high risk of bias
and the quality of evidence was low or
very low.

• The educational programs evaluated

had no demonstrable effect on the
prevalence of HIV (low certainty
evidence), or other sexually
PRIMARY transmitted infections (Herpes Simplex
To evaluate the Clinical/biological virus prevalence: moderate certainty
effects of outcomes: evidence; Syphilis prevalence: low
school-based (a) HIV prevalence. certainty evidence). There was also no
sexual and Randomized (b) STI prevalence. apparent effect on the number of
reproductive health Controlled Trials (c) Pregnancy young women who were pregnant at
programs on (RCTs) (both prevalence. the end of the trial (moderate certainty
Mason-Jones et al. sexually individually Behavioral self-reported evidence).
10–19 years • Combined educational and
(2016) transmitted randomized and outcomes:
infections (such as cluster-randomized (a) Use of male incentive-based programs herpes
HIV, herpes included 8 condoms at first sex. simplex virus infection was reduced,
simplex virus, and cluster-RCTs). (b) Use of male predominantly in young women, but
syphilis), and condoms at most recent no effect was detected for HIV or
pregnancy among (last) sex. pregnancy (low certainty evidence).
adolescents. (c) Initiation (sexual • It was not possible to show
debut). effectiveness for educational
curriculum-based interventions on
biologically measured adolescent
sexual and reproductive health
outcomes.
Int. J. Environ. Res. Public Health 2021, 18, 2555 22 of 31

Table A3. Cont.

• Of the eight studies reviewed, only

two studies published from one
intervention that had an effect on the
primary outcome of interest, STI
incidence, and none that reported HIV
incidence.
• No studies that assessed the effect of
school-based prevention programs on
HIV incidence among adolescents
PRIMARY were found. The only effective
(a) HIV/STI incidence intervention seemed to be one that
To evaluate the or prevalence. covered multiple years, started early,
effectiveness of (b) HIV/STI testing. and had multiple components.
school-based SECONDARY • The quality of evidence for all
programs prevent Three RCTs and six (a) Frequency of outcomes was very low. Studies,
Mirzazadeh et al. HIV and other non-RCTs intercourse. including the RCTs, were of low
10–19 years
(2018) sexually describing seven (b) Number of partners. methodological quality and had mixed
Transmitted interventions. (c) Initiation of sexual findings, thus offering no persuasive
Infections in intercourse. evidence for the effectiveness of
adolescents in (d) Sex without a school-based programs.
the USA. condom. • While some positive effects on changes
(e) HIV/STI knowledge, in STI-related knowledge and attitudes
attitude, and behavior. were found, there was little evidence
that these changes decrease STI.
• The variability in the interventions,
study populations, settings, and
outcomes reviewed make it difficult to
identify the specific aspects of an
intervention that may be most effective
at reducing STIs and HIV among
young people.

PRIMARY • Only interventions involving a

(a) Unintended combination of education and
pregnancy. contraception promotion (multiple
SECONDARY interventions) were seen to
(a) Reported changes in significantly reduce unintended
knowledge and pregnancy over the medium-term and
attitudes about the risk long-term follow-up period.
To assess the effects • Evidence for program effects on
of unintended
of primary 53 Randomized biological measures is limited.
pregnancies.
prevention Controlled Trials • Results for behavioral (secondary)
(b) Initiation of sexual
interventions (RCTs) comparing outcomes were inconsistent across
intercourse.
(school-based, these interventions trials.
Oringanje et al. (c) Use of birth control
community/home- 10–19 years to various control • The variability in study populations,
(2016) methods.
based, clinic-based, groups (mostly interventions and outcomes of
(d) Abortion.
and faith-based) on usual standard sex included trials, and the paucity of
(e) Childbirth.
unintended education offered studies directly comparing different
(f) Morbidity related to
pregnancies among by schools). interventions preclude a definitive
pregnancy, abortion or
adolescents. conclusion regarding which type of
child birth.
(g) Mortality related to intervention is most effective.
pregnancy, abortion or • Limitations include reliance on
childbirth. program participants to report their
(h) Sexually transmitted behaviors accurately and
infections (including methodological weaknesses in the
HIV). trials.

Randomized trial
or quasi
experimental
design, in which • The meta-analysis of three randomized
To examine trials provided some evidence that
control groups
whether school-environment interventions may
received usual PRIMARY Interventions
interventions, delay sexual debut (pooled odds ratio,
treatment or a designed specifically to
addressing 0.5).
comparison improve:
school-level • Narrative synthesis of the remaining
Peterson et al. intervention, and (a) Knowledge.
environment or 10–19 years outcomes found mixed results, but
(2019) they must have (b) Attitudes.
student-level suggests that interventions addressing
reported at least (c) Skills.
educational assets, school-level environment may delay
one sexual health (d) Services related to
can promote young sexual debut and that those addressing
outcome, such as sexual health.
people’s sexual student-level educational assets may
pregnancy, STDs or
health. reduce risk of pregnancy and STDs.
sexual behaviors
associated with
increased risk of
pregnancy or STDs.
Int. J. Environ. Res. Public Health 2021, 18, 2555 23 of 31

Table A3. Cont.

Online
Authors/Year Objective Participants Type of Study Outcomes Results

• Interactive digital interventions are

effective tools for learning about
To summarize sexual health.
evidence on • Interactive digital interventions have
effectiveness, small but significant effects on
cost-effectiveness self-efficacy, and sexual behavior.
and mechanism of • There is not enough evidence to be
action of interactive sure of the effects on biological
digital outcomes or to be sure of longer-term
interventions (IDIs) impacts.
PRIMARY
for sexual health;
(a) Sexual health Effectiveness of interactive digital interventions
optimal practice for
knowledge. effective compared with minimal interventions.
intervention Randomized
(b) Self-efficacy.
Bailey et al. (2015) development; 12–19 years controlled trials • Significant, moderate effect on sexual
(c)
contexts for (RCTs). health knowledge.
Intention/motivation.
successful • A small but significant effect on
(d) Sexual behavior and
implementation; self-efficacy.
biological. • No demonstrable effect on sexual
research methods
for digital behavior and on STI diagnoses.
intervention Effectiveness of interactive digital interventions
evaluation; and the compared to face-to-face sexual health
future potential of interventions.
sexual health
promotion via • Significant, moderate positive effect on
digital media. sexual health knowledge.
• No demonstrable effect on self-efficacy.
• Small, significant effect on intention.

PRIMARY
Adolescents’
health-promoting • A statistically significant increase was
To determine the
behaviors: pregnancy, determined in health-promoting
effect of
HIV/disease-related behavior in one (Marsch et al., 2011) of
technology-based
Randomized knowledge, condom four studies on sexual health.
Celik et al. (2020) programmes in 10–24 years
control group. use, condom intentions, • In 56.25% of the studies, the
changing
condom skills, development in the studied health
adolescent health
self-efficacy, and related behaviors was found to be significant.
behaviors.
infectious diseases risk
behavior.

PRIMARY • Interventions for sexual health

Behavior, knowledge, promotion using serious games have
behavioral intention, significant positive effects for
To analyze the
perceived determinants, albeit rather small.
effectiveness of
Randomized environmental • The effects on behavior, measured in
interventions for
control group, and constraints, skills, only two studies, were not significant.
Desmet et al. (2015) sexual health 13–29 years
randomized on an attitudes, subjective • Most games did not use many
promotion that use
individual. norm, and self-efficacy. immersive game features. Instead,
serious digital
SECONDARY there was a strong reliance on pure
games.
Clinical effects (e.g., gamification features such as reward
rates of sexually and feedback.
transmitted infections).

• Intervention exposure was associated

with increased sexual health
knowledge and awareness, lower rates
of unprotected sex and higher rates of
To draw a more condom use, and greater STI testing.
comprehensive PRIMARY • First, it is worth trying to replicate and
picture of how (a) Sexual Health continue evaluating the interventions
Randomized
online sexual information. that yielded modest results. Second,
controlled trials
health (b) What topics they online sexual health education is
Holstrom (2015) 10–24 years (RCTs), and focus
interventions do want to know about. lacking consensus on what is a
groups
and do not align (c) Evaluations of successful outcome, how to measure it,
participants.
with real world Internet-based sexual or what theoretical foundations should
habits and interests health interventions. be used to build interventions. Third,
of adolescents. the evaluated interventions do not
echo some primary components of
what we know adolescents want from
a sexual health website.
Int. J. Environ. Res. Public Health 2021, 18, 2555 24 of 31

Table A3. Cont.

• Evidence on mobile phone

interventions for ASRH published in
peer-reviewed journals reflects a high
degree of quality in methods and
reporting.
• Improved SRH knowledge, less
unprotected sex and more STI testing.
PRIMARY • Leveraging mobile phones to increase
(a) Promote positive youth contact for STI screening and
and preventive SRH follow-up yielded higher rates of
To assess strategies, behaviors. screening and recall and more timely
Randomized
findings, and (b) Increase adoption and complete STI treatment and
controlled trials
quality of evidence and continuation of vaccination.
(RCTs),
on using mobile contraception. • Increased adolescent patient adherence
L’Engle et al. (2016) 13–24 years quasi-experimental,
phones to improve (c) Support medication to medication (oral contraceptive pills
observational, or
adolescent sexual adherence for and SRT) in USA.
descriptive
and reproductive HIV-positive young • Using mobile phone calls to provide
research.
health (ASRH). people. adolescent patient counseling was
(d) Encourage use of ineffective, except for 1 small study.
health screening and • Mobile phones were used to increase
treatment services. health program reach to adolescents
and ethnic and minority subgroups, to
increase confidentiality in providing
sensitive SRH information to young
people, and to provide a supportive
“friend in your pocket” who reminds
and encourages good health.

PRIMARY
Process outcomes
evaluated:
Acceptability,
Attractiveness,
Feasibility, Satisfaction
and Implementation.
Outcomes evaluation
conducted:
Behaviors.
Condom use, condom
use intention, • Effectiveness results (n = 23)
self-efficacy toward • Online peer interaction, the major
condom use, and participatory component, is not
16 Randomized attitude toward condom sufficiently conceptualized and
To describe existing Controlled Trial use attitudes. defined as a determinant of change or
published studies (RCT), 15 Control Communication. theoretical model component.
on online group (NI = 2), 4 Knowledge. • Still in the early stages of design and
participatory Information-only Behavioral skills. evaluation, particularly as regards the
intervention control website, 7 Self-efficacy. effect of peer interaction, and do not
Martin et al. (2020) 10–24 years
methods used to Before-after study Contraception use. always adhere to existing theoretical
promote the sexual (no RCT), 3 History of sexually models.
health of Cross-sectional transmitted infections. • Participatory online interventions for
adolescents and study, 8 other HIV stigma. young people’s sexual health have
young adults. design, 3 HIV test history (date shown their feasibility, practical
Unspecified. and result of the last interest, and attractiveness, but their
test). effectiveness has not yet been
Incidence of sexually sufficiently evaluated.
transmitted infections.
Intentions related to
risky sexual activity.
Internalized
homophobia.
Intimate partner
violence.
Motivation.
Pubertal development.
Sexual abstinence.
Waiting before
having sex.
Int. J. Environ. Res. Public Health 2021, 18, 2555 25 of 31

Table A3. Cont.

PRIMARY
Health behavior change: • TCCMD (Targeted client
• STI/HIV prevention. communication (TCC) delivered via
• STI/HIV treatment. mobile devices (MD)) versus standard
• Contraception/family care TCC may increase sexual health
planning. knowledge (risk ratio (RR) 1.45, 95%
• Pre-conception care. confidence interval (CI) 1.23 to 1.71;
• Partner violence. low-certainty evidence). TCCMD may
Service utilization: modestly increase contraception use
• STI/HIV (RR 1.19, 95% CI 1.05 to 1.35;
prevention/treatment. low-certainty evidence). The effects on
• Contraception/family condom use, antiretroviral therapy
planning. (ART) adherence, and health service
• HPV vaccination. use are uncertain due to very
• Cervical screening. low-certainty evidence. The effects on
• Pre-conception care. abortion and STI rates are unknown
Partner violence: due to lack of studies.
• Use of services • TCCMD versus non-digital TCC (e.g.,
designed for those who pamphlets) The effects of TCCMD on
To assess the effects behavior (contraception use, condom
have experienced
of targeted client use, ART adherence), service use,
partner violence.
communication via health and wellbeing (abortion and STI
Health status and
delivered via rates) are unknown due to lack of
well-being:
mobile devices on studies for this comparison.
• STI/HIV prevention.
adolescents’ • TCCMD versus digital non-targeted
Randomized • STI/HIV treatment.
knowledge, and on communication The effects on sexual
Palmer et al. (2020) 10 -24 years controlled trials • Contraception/family
adolescents’ and health knowledge, condom and
(RCTs). planning.
adults’ sexual and contraceptive use are uncertain due to
• Partner violence.
reproductive health very low-certainty evidence.
• Well-being.
behavior, health Interventions may increase health
Any measure of knowledge
service use, and service use (attendance for STI/HIV
or attitudes relating to the
health and testing, RR 1.61, 95% CI 1.08 to 2.40;
following:
well-being. low-certainty evidence). The
• STI prevention and/or
treatment. intervention may be beneficial for
• Contraception/family reducing STI rates (RR 0.61, 95% CI
planning. 0.28 to 1.33; low-certainty evidence),
• Cervical cancer but the confidence interval
screening. encompasses both benefit and harm.
• Sexual violence. The effects on abortion rates and on
• HPV vaccination. ART adherence are unknown due to
• Puberty. lack of studies. We are uncertain
whether TCCMD results in unintended
SECONDARY consequences due to lack of evidence.
•Patient/client • There was evidence of a modest
acceptability and beneficial intervention effect on
satisfaction with the contraceptive use among adolescent
intervention. and adult populations, but there was
•Resource use, not sufficient evidence to demonstrate
including cost to the that this translated into a reduction in
system and unintended contraception.
consequences.

• A large proportion of studies (11/25)

PRIMARY specifically focused on HIV
(a) Behavior (number of prevention.
sexual partners, number • Three interventions reported
of unprotected sexual non-significant effect in condom use,
acts, frequency of two interventions reported an increase
condom use, in condom use and another study
negotiation skills for reported a significant increase in
condom use, sex under self-efficacy related to condom usage.
Randomized to a
the influence of alcohol • Seven studies found a statistically
control group and
and other drugs, testing significant effect of the intervention on
pre-/post-test
seeking behavior). knowledge levels regarding the
To assess the evaluation design,
(b) Self-efficacy prevention HIV and other STI, as well
effectiveness of uncontrolled
(condom use). as general sexual health knowledge,
sexual health longitudinal
Wadham et al. (c) Skills and Abilities but only one-fifth of interventions
interventions 12–24 years studies and the
(2019) (sexual communication evaluating intentions to use condoms
delivered via new remaining studies
and risk assessment). reported significant effects due to the
digital media to comprised a
(d) Intentions (to use intervention.
young people. mixture of
condoms). • Of the 12 studies evaluating
qualitative cohort,
(e) Attitudes. knowledge-based outcomes, seven
observational and
(f) Knowledge (HIV, STI, found a significant effect.
mixed methods.
general sexual health). • Of the four studies that evaluated
(g) Efficacy of the sexual communication, only one
Intervention (feasibility, reported a significant effect.
acceptability, usability, • The broad range of studies included in
satisfaction). this review, with their diversity of
(h) Well-being (mental methods, populations and objectives,
health, sexuality, precludes any easily drawn
self-acceptance). comparisons or conclusions.
Int. J. Environ. Res. Public Health 2021, 18, 2555 26 of 31

Table A3. Cont.

• There was a significant weighted mean

effect of technology-based
interventions on condom use (d = 0.23,
PRIMARY 95% confidence interval [CI] [0.12,
To synthesize the (a) Condom use 0.34], p < 0.001) and abstinence (d =
technology-based (b) Abstinence. 0.21, 95% CI [0.02, 0.40], p = 0.027).
sexual health SECONDARY • Effects did not differ by age, gender,
interventions Randomized to a (a) Safer sex attitudes. country, intervention dose,
among youth control group and (b) Social norms for interactivity, or program tailoring.
Widman et al. • Effects were stronger when assessed
people to 13–24 years experimental or safer sexual activity.
(2018) with short-term (1–5 months) than
determine their quasi-experimental (c) self-efficacy.
overall efficacy on design. (d) Behavioral with longer term (greater than 6
two key behavioral intentions to practice months) follow-ups.
outcomes: condom safer sex. • Compared with control programs,
use and abstinence. (e) Sexual health technology-based interventions were
knowledge. also more effective in increasing sexual
health knowledge (d = 0.40, p < 0.001),
safer sex norms (d = 0.15, p = 0.022),
and attitudes (d = 0.12, p = 0.016)

Blended Learning
Authors/Year Objective Participants Type of Study Outcomes Results
PRIMARY
(a) Initiation of sexual
intercourse (vaginal,
oral or anal intercourse).
(b) Other sexual risk
behaviors (condom use,
communication,
condom use skills,
frequency of sex, • Blended learning approaches are being
unprotected sex, used successfully in sexual health
number of partners education programs, including
To identify sexual with whom had sex school-based programs, and have
health education without protection, yielded positive behavioral and
studies using frequency of using psychosocial changes.
Randomized
blended learning to 13–24 years, and alcohol and or other • Blended learning approaches are
Coyle et al. (2019) Controlled Trials
summarize the best adults of over 25 substances during sex). viable for sexual health education and
(RCTs).
practices and (c) Sexual coercion or offer numerous advantages over
potential dating violence (sexual group-based only programs, such as
challenges. coercion, dating confidential personalization and an
violence). instructional approach that is familiar
(d) Sexuality-related and engaging for participants.
psychosocial factors
(attitudes, beliefs,
perceptions regarding
abstinence, and
protection).
(e) Perceived
satisfaction and
usability (of blended
learning).

Table A4. Evaluation of the studies included (AMSTAR II).

School

1 Overall
Authors 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Rating 2
Chokprajakchad
Y N Y Y N N N Y N N NM NM N Y NM N CL
et al. (2018)
Goldfarb et al. Partial
Y Y N Y Y Y Y N N NM NM N Y NM Y CL
(2020) Y
Haberland et al. Partial
Y Y Y Y N N N N N NM NM N Y NM N CL
(2016) Y
Kedzior et al. Partial
Y Y Y Y Y Y Y Y N NM NM Y Y NM Y M
(2020) Y
Lopez et al.
Y Y Y Y Y Y Y Y Y Y NM NM Y Y NM Y H
(2016)
Marseille et al.
Y Y Y Y Y Y Y Y Y N Y Y Y Y Y Y H
(2018)
Mason-Jones
Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y H
et al. (2016)
Mirzazadeh et al.
Y Y Y Y Y Y Y Y Y N Y Y Y Y Y Y H
(2018)
Oringanje et al.
Y Y Y Y Y Y Y Y Y N NM NM Y Y NM Y H
(2016)
Peterson et al.
Y Y Y Y Y Y N Y Y N Y Y Y Y Y N L
(2019)
Int. J. Environ. Res. Public Health 2021, 18, 2555 27 of 31

Table A4. Cont.

Online
Bailey et al.
Y Y Y Y Y Y N Y Y N Y Y Y Y Y Y L
(2015)
Celik et al. (2020) Y Y Y N N N Y Y N N NM NM N Y NM Y CL
DeSmet et al. Partial Partial
Y Y Y Y Y N Y N Y Y Y Y N Y CL
(2015) Y Y
Holstrom (2015) N N N Y N N N Y N N NM NM N N NM N CL
L´Engle et al. Partial Partial
Y Y Y Y Y Y N Y NM NM N Y NM Y CL
(2016) Y Y
Martin et al.
Y Y Y Y Y Y Y Y N N NM NM N Y NM Y CL
(2020)
Palmer et al.
Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y H
(2020)
Wadham et al. Partial Partial
N Y Y Y N Y N N NM NM N N NM Y CL
(2019) Y Y
Widman et al. Partial Partial
Y Y Y Y Y Y Y N Y Y N Y Y Y L
(2018) Y Y
Blended Learning
Coyle et al.
Y N N Y N N N Y N N NM NM N Y NM N CL
(2019)
1 1. Did the research questions and inclusion criteria for the review include the components of PCIO?; 2. Did the report of the review
contain an explicit statement that the review methods were established prior to the conduct of the review and did the report justify any
significant deviations from the protocol?; 3. Did the review authors explain their selection of the study designs for inclusion in the review?;
4. Did the review authors use a comprehensive literature search strategy?; 5. Did the review authors perform study selection in duplicate?;
6. Did the review authors perform data extraction in duplicate?; 7. Did the review authors provide a list of excluded studies and justify
the exclusions?; 8. Did the review authors describe the included studies in adequate detail?; 9. Did the review authors use a satisfactory
technique for assessing the risk of bias (RoB) in individual studies that were included in the review?; 10. Did the review authors report on
the sources of funding for the studies included in the review?; 11. If meta-analysis was performed, did the review authors use appropriate
methods for statistical combination of results?; 12. If meta-analysis was performed, did the review authors assess the potential impact of
RoB in individual studies on the results of the meta-analysis or other evidence synthesis?; 13. Did the review authors account for RoB in
primary studies when interpreting/discussing the results of the review?; 14. Did the review authors provide a satisfactory explanation for,
and discussion of, any heterogeneity observed in the results of the review?; 15. If they performed quantitative synthesis did the review
authors carry out an adequate investigation of publication bias (small study bias) and discuss its likely impact on the results of the review?;
16. Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the
review? 2 H = Hight; M = Media; C = Low; CL = Critically Low. N = No; Y = Yes.

References
1. WHO. The Health of Youth. Geneva 1989. Available online: https://www.who.int/maternal_child_adolescent/documents/
pdfs/9241591269_op_handout.pdf (accessed on 12 November 2020).
2. Anderson, R.M. Positive sexuality and its impact on overall well-being. Bundesgesundheitsbl 2013, 56, 208–214. [CrossRef]
3. WHO. Defining Sexual Health Report of a Technical Consultation on Sexual Health, Geneva 2006. Available online: https:
//www.who.int/reproductivehealth/topics/gender_rights/defining_sexual_health.pdf (accessed on 15 November 2020).
4. UN. Agenda 2030 para el Desarrollo Sostenible. 2015. Available online: https://www.un.org/sustainabledevelopment/es/2015
/09/la-asamblea-general-adopta-la-agenda-2030-para-el-desarrollo-sostenible/ (accessed on 9 April 2020).
5. Blanc, M.A.; Rojas, T.A. Condom use, number of partners and sexual debut in young people in penile-vaginal intercourse, oral
sex and anal sex. Rev. Int. 2018, 16, 8.
6. Magnusson, B.M.; Crandall, A.; Evans, K. Early sexual debut and risky sex in young adults: The role of low self-control. BMC
Public Health 2019, 19, 1483. [CrossRef] [PubMed]
7. Williams, E.A.; Jensen, R.E. Conflicted identification in the sex education classroom: Balancing professional values with
organizational mandates. Qual. Health Res. 2016, 26, 1574–1586. [CrossRef]
8. Ybarra, M.L.; Thompson, R.E. Predicting the emergence of sexual violence in adolescence. Prev. Sci. 2018, 19, 403–415. [CrossRef]
9. Rodríguez-Castro, Y.; Alonso, P.; Lameiras, M.; Faílde, J.M. From sexting to cybercontrol among dating teens in Spain: An analysis
of their arguments. Rev. Lat. Am. Psicol. 2018, 50, 170–178.
10. Helmer, J.; Senior, K.; Davison, B.; Vodic, A. Improving sexual health for young people: Making sexuality education a priority.
Sex Educ. 2015, 15, 158–171. [CrossRef]
11. Lindberg, L.D.; Maddow-Zimet, I.; Boonstra, H. Changes in adolescents’ receipt of sex education, 2006–2013. J. Adolesc. Health
2016, 58, 621–627. [CrossRef] [PubMed]
12. Garcia, L.; Fields, J. Renewed commitments in a time of vigilance: Sexuality education in the USA. Sex Educ. 2017, 17,
471–481. [CrossRef]
13. Kirby, D.B. The Impact of Abstinence and Comprehensive Sex and STD/HIV Education Programs on Adolescent Sexual Behavior.
Sex. Res. Soc. Policy 2008, 5, 18–27. [CrossRef]
14. Heels, S.W. The Impact of Abstinence-Only Sex Education Programs in the United States on Adolescent Sexual Outcomes.
Perspectives 2019, 11, 3.
Int. J. Environ. Res. Public Health 2021, 18, 2555 28 of 31

15. Santelli, J.S.; Kantor, L.M.; Grilo, S.A.; Speizer, I.S.; Lindberg, L.D.; Heitel, J.; Ott, M.A. Abstinence-only-until-marriage: An
updated review of US policies and programs and their impact. J. Adolesc. Health 2017, 61, 273–280. [CrossRef]
16. Gardner, E.A. Abstinence-only sex education: College students’ evaluations and responses. Am. J. Sex. Educ. 2015, 10,
125–139. [CrossRef]
17. UNESCO. International Technical Guidance on Sexuality Education. 2018. Available online: file:///C:/Users/Profesor/Desktop/
260770eng.pdf (accessed on 14 October 2020).
18. Bourke, A.; Boduszek, D.; Kelleher, C.; McBride, O.; Morgan, K. Sex education, first sex and sexual health outcomes in adulthood:
Findings from a nationally representative sexual health survey. Sex Educ. 2014, 14, 299–309. [CrossRef]
19. Denford, S.; Abraham, C.; Campbell, R.; Busse, H. A comprehensive review of reviews of school-based interventions to improve
sexual-health. Health Psychol. Rev. 2017, 11, 33–52. [CrossRef]
20. Gray, B.J.; Jones, A.T.; Couzens, Z.; Sagar, T.; Jones, D. University students’ behaviours towards accessing sexual health
information and treatment. Int. J. Std Aids 2019, 30, 671–679. [CrossRef]
21. WHO. Guideline on Digital Interventions for Health System Strengthening. 2019. Available online: file:///C:/Users/Profesor/
Desktop/9789241550505-eng.pdf (accessed on 23 September 2020).
22. Madden, M.; Lenhart, A.; Duggan, M.; Cortesi, S.; Gasser, U. Teens and Technology. Pew Research Center. 2013. Available
online: http://www.pewinternet.org/files/oldmedia//Files/Reports/2013/PIPTeensandTechnology2013.pdf (accessed on 18
November 2020).
23. Park, E.; Kwon, M. Health-related internet use by children and adolescents: Systematic review. J. Med Internet Res. 2018, 20,
e120. [CrossRef]
24. Eleuteri, S.; Rossi, R.; Tripodi, F.; Fabrizi, A.; Simonelli, C. Sexual health in your hands: How the smartphone apps can improve
your sexual wellbeing? Sexologies 2018, 27, e57–e60. [CrossRef]
25. Lim, M.S.; Vella, A.; Sacks-Davis, R.; Hellard, M.E. Young people’s comfort receiving sexual health information via social media
and other sources. Int. J. Std. Aids 2014, 25, 1003–1008. [CrossRef]
26. Widman, L.; Kamke, K.; Evans, R.; Stewart, J.L.; Choukas-Bradley, S.; Golin, C.E. Feasibility, acceptability, and preliminary efficacy
of a brief online sexual health program for adolescents. J. Sex Res. 2020, 57, 145–154. [CrossRef]
27. Garzón-Orjuela, N.; Samacá-Samacá, D.; Moreno-Chaparro, J.; Ballesteros-Cabrera, M.D.P.; Eslava-Schmalbach, J. Effectiveness of
sex education interventions in adolescents: An overview. Compr. Child Adolesc. Nurs. 2020, 1–34. [CrossRef]
28. Lindberg, L.D.; Bell, D.L.; Kantor, L.M. The Sexual and Reproductive Health of Adolescents and Young Adults During the
COVID-19 Pandemic. Perspect. Sex. Reprod. Health 2020. [CrossRef]
29. Johns Hopkins Coronavirus Resource Center. COVID-19 Map. 2021. Available online: https://coronavirus.jhu.edu/map.html
(accessed on 14 January 2021).
30. Decker, M.J.; Gutmann-Gonzalez, A.; Price, M.; Romero, J.; Sheoran, B.; Yarger, J. Evaluating the effectiveness of an intervention
integrating technology and in-person sexual health education for adolescents (In the Know): Protocol for a cluster randomized
controlled trial. JMIR Res. Protoc. 2020, 9, e18060. [CrossRef]
31. Doubova, S.V.; Martinez-Vega, I.P.; Infante-Castañeda, C.; Pérez-Cuevas, R. Effects of an internet-based educational intervention
to prevent high-risk sexual behavior in Mexican adolescents. Health Educ. Res. 2017, 32, 487–498. [CrossRef]
32. Bowring, A.L.; Wright, C.J.; Douglass, C.; Gold, J.; Lim, M.S. Features of successful sexual health promotion programs for young
people: Findings from a review of systematic reviews. Health Promot. J. Aust. 2018, 29, 46–57. [CrossRef]
33. Bowring, A.L.; Veronese, V.; Doyle, J.S.; Stoove, M.; Hellard, M. HIV and sexual risk among men who have sex with men and
women in Asia: A systematic review and meta-analysis. Aids Behav. 2016, 20, 2243–2265. [CrossRef] [PubMed]
34. Kirby, D.B.; Laris, B.A.; Rolleri, L.A. Sex and HIV education programs: Their impact on sexual behaviors of young people
throughout the world. J. Adolesc. Health 2007, 40, 206–217. [CrossRef]
35. Shackleton, N.; Jamal, F.; Viner, R.M.; Dickson, K.; Patton, G.; Bonell, C. School-based interventions going beyond health education
to promote adolescent health: Systematic review of reviews. J. Adolesc. Health 2016, 58, 382–396. [CrossRef]
36. Collins, R.L.; Martino, S.C.; Shaw, R. Influence of New Media on Adolescent Sexual Health: Evidence and Opportunities (Working
Paper). Rand Health. 2011. Available online: http://aspe.hhs.gov/hsp/11/AdolescentSexualActivity/NewMediaLitRev/index.
pdf (accessed on 26 November 2020).
37. Döring, N.M. The Internet’s impact on sexuality: A critical review of 15 years of research. Comput. Hum. Behav. 2009, 25,
1089–1101. [CrossRef]
38. Simon, L.; Daneback, K. Adolescents’ use of the internet for sex education: A thematic and critical review of the literature. Int. J.
Sex. Health 2013, 25, 305–319. [CrossRef]
39. Talukdar, J. The prospects of a virtual sex education: A review. Am. J. Sex. Educ. 2013, 8, 104–115. [CrossRef]
40. Hartmann, M.; Khosla, R.; Krishnan, S.; George, A.; Gruskin, S.; Amin, A. How are gender equality and human rights interventions
included in sexual and reproductive health programmes and policies: A systematic review of existing research foci and gaps.
PLoS ONE 2016, 11, e0167542. [CrossRef]
41. Schardt, C.; Adams, M.B.; Owens, T.; Keitz, S.; Fontelo, P. Utilization of the PICO framework to improve searching PubMed for
clinical questions. BMC Med Inf. Decis. Mak. 2007, 7, 16. [CrossRef]
Int. J. Environ. Res. Public Health 2021, 18, 2555 29 of 31

42. Shea, B.J.; Reeves, B.C.; Wells, G.; Thuku, M.; Hamel, C.; Moran, J.; Moher, D.; Tugwell, P.; Welch, G.; Thuku, M.; et al. AMSTAR 2:
A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions,
or both. BMJ 2017, 358, j4008. [CrossRef]
43. Shea, B.J.; Grimshaw, J.M.; Wells, G.A.; Boers, M.; Andersson, N.; Hamel, C.; Porter, A.C.; Tugwell, P.; Moher, D.; Bouter, L.M.
Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Med. Res.
Methodol. 2007, 7, 10. [CrossRef]
44. Shea, B.J.; Bouter, L.M.; Peterson, J.; Boers, M.; Andersson, N.; Ortiz, Z.; Ramsay, T.; Bai, A.; Shukla, V.K.; Grimshaw, J.M. External
validation of a measurement tool to assess systematic reviews (AMSTAR). PLoS ONE 2007, 2, e1350. [CrossRef] [PubMed]
45. Mason-Jones, A.J.; Sinclair, D.; Mathews, C.; Kagee, A.; Hillman, A.; Lombard, C. School-based interventions for preventing HIV,
sexually transmitted infections, and pregnancy in adolescents. Cochrane Database Syst. Rev. 2016, 11, 1–72. [CrossRef]
46. Palmer, M.J.; Henschke, N.; Villanueva, G.; Maayan, N.; Bergman, H.; Glenton, C.; Free, C. Targeted client communication via
mobile devices for improving sexual and reproductive health. Cochrane Database Syst. Rev. 2020, 8, 1–84.
47. Lopez, L.M.; Bernholc, A.; Chen, M.; Tolley, E.E. School-based interventions for improving contraceptive use in adolescents.
Cochrane Database Syst. Rev. 2016, 6, 1–66. [CrossRef]
48. Marseille, E.; Mirzazadeh, A.; Biggs, M.A.; Miller, A.P.; Horvath, H.; Lightfoot, M.; Kahn, J.G. Effectiveness of school-based
teen pregnancy prevention programs in the USA: A systematic review and meta-analysis. Prev. Sci. 2018, 19, 468–489.
[CrossRef] [PubMed]
49. Mirzazadeh, A.; Biggs, M.A.; Viitanen, A.; Horvath, H.; Wang, L.Y.; Dunville, R.; Marseille, E. Do school-based programs prevent
HIV and other sexually transmitted infections in adolescents? A systematic review and meta-analysis. Prev. Sci. 2018, 19, 490–506.
[CrossRef] [PubMed]
50. Oringanje, C.; Meremikwu, M.; Eko, H.; Esu, E.; Meremikwu, A.; Ehiri, J.E. Interventions for preventing unintended pregnancies
among adolescents. Cochrane Database Syst. Rev. 2016, 2, 1–105. [CrossRef]
51. Kedzior, S.G.; Lassi, Z.S.; Oswald, T.K.; Moore, V.M.; Marino, J.L.; Rumbold, A.R. A Systematic Review of School-based Programs
to Improve Adolescent Sexual and Reproductive Health: Considering the Role of Social Connectedness. Adolesc. Res. Rev. 2020,
1–29. [CrossRef]
52. Chokprajakchad, M.; Phuphaibul, R.; Sieving, R.E. Sexual health interventions among early adolescents: An integrative review. J.
Health Res. 2018, 32, 467–477. [CrossRef]
53. Goldfarb, E.S.; Lieberman, L.D. Three Decades of Research: The Case for Comprehensive Sex Education. J. Adolesc. Health 2020,
68, 7–8. [CrossRef]
54. Haberland, N.A. The case for addressing gender and power in sexuality and HIV education: A comprehensive review of
evaluation studies. Int. Perspect. Sex. Reprod. Health 2015, 41, 31–42. [CrossRef] [PubMed]
55. Peterson, A.J.; Donze, M.; Allen, E.; Bonell, C. Effects of Interventions Addressing School Environments or Educational Assets on
Adolescent Sexual Health: Systematic Review and Meta-analysis. Perspect. Sex. Reprod. Health 2019, 51, 91–107. [CrossRef]
56. Bailey, J.; Mann, S.; Wayal, S.; Hunter, R.; Free, C.; Abraham, C.; Murray, E. Sexual health promotion for young people delivered
via digital media: A scoping review. Public Health Res. 2015, 3, 1–119. [CrossRef]
57. Celik, R.; Toruner, E.K. The Effect of Technology-Based Programmes on Changing Health Behaviours of Adolescents: Systematic
Review. Compr. Child Adolesc. Nurs. 2020, 43, 92–110. [CrossRef]
58. DeSmet, A.; Shegog, R.; Van Ryckeghem, D.; Crombez, G.; De Bourdeaudhuij, I. A systematic review and meta-analysis of
interventions for sexual health promotion involving serious digital games. Games Health J. 2015, 4, 78–90. [CrossRef]
59. Holstrom, A.M. Sexuality education goes viral: What we know about online sexual health information. Am. J. Sex. Educ. 2015, 10,
277–294. [CrossRef]
60. L’Engle, K.L.; Mangone, E.R.; Parcesepe, A.M.; Agarwal, S.; Ippoliti, N.B. Mobile phone interventions for adolescent sexual and
reproductive health: A systematic review. Pediatrics 2016, 138. [CrossRef]
61. Martin, P.; Cousin, L.; Gottot, S.; Bourmaud, A.; Rochebrochard, E.; Alberti, C. Participatory interventions for sexual health
promotion for adolescents and young adults on the internet: Systematic review. J. Med Internet Res. 2020, 22, e15378. [CrossRef]
62. Wadham, E.; Green, C.; Debattista, J.; Somerset, S.; Sav, A. New digital media interventions for sexual health promotion among
young people: A systematic review. Sex. Health 2019, 16, 101–123. [CrossRef]
63. Widman, L.; Nesi, J.; Kamke, K.; Choukas-Bradley, S.; Stewart, J.L. Technology-based interventions to reduce sexually transmitted
infections and unintended pregnancy among youth. J. Adolesc. Health 2018, 62, 651–660. [CrossRef]
64. Coyle, K.K.; Chambers, B.D.; Anderson, P.M.; Firpo-Triplett, R.; Waterman, E.A. Blended Learning for Sexual Health Education:
Evidence Base, Promising Practices, and Potential Challenges. J. Sch. Health 2019, 89, 847–859. [CrossRef] [PubMed]
65. The World Bank. 2020. Available online: https://datatopics.worldbank.org/world-development-indicators/stories/the-
classification-of-countries-by-income.html (accessed on 23 December 2020).
66. Marsch, L.A.; Grabinski, M.J.; Bickel, W.K.; Desrosiers, A.; Guarino, H.; Muehlbach, B.; Acosta, M. Computer-assisted HIV
prevention for youth with substance use disorders. Subst. Use Misuse 2011, 46, 46–56. [CrossRef]
67. Janssens, A.; Blake, S.; Allwood, M.; Ewing, J.; Barlow, A. Exploring the content and delivery of relationship skills education
programmes for adolescents: A systematic review. Sex Educ. 2020, 20, 494–516. [CrossRef]
68. Todaro, E.; Silvaggi, M.; Aversa, F.; Rossi, V.; Nimbi, F.M.; Rossi, R.; Simonelli, C. Are Social Media a problem or a tool? New
strategies for sexual education. Sexologies 2018, 27, e67–e70. [CrossRef]
Int. J. Environ. Res. Public Health 2021, 18, 2555 30 of 31

69. Preston, M. Very Very Risky: Sexuality Education Teachers’ Definition of Sexuality and Teaching and Learning Responsibilities.
Am. J. Sex. Educ. 2013, 8, 18–35. [CrossRef]
70. Layzer, C.J.; Rosapep, L.; Barr, S. A Peer education program: Delivering highly reliable sexual health promotion messages in
schools. J. Adolesc. Health 2014, 54, 70–77. [CrossRef] [PubMed]
71. Ketting, E.; Brockschmidt, L.; Ivanova, O. Investigating the C’in CSE: Implementation and effectiveness of comprehensive
sexuality education in the WHO European region. Sex Educ. 2020, 1–15. [CrossRef]
72. Lamb, S.; Lustig, K.; Graling, K. The use and misuse of pleasure in sex education curricula. Sex Educ. 2013, 13, 305–318. [CrossRef]
73. Fine, M. Sexuality, schooling, and adolescent females: The missing discourse of desire. Harv. Educ. Rev. 1988, 58, 29–54. [CrossRef]
74. Fine, M.; McClelland, S. Sexuality education and desire: Still missing after all these years. Harv. Educ. Rev. 2006, 76,
297–338. [CrossRef]
75. Gill, R. The sexualisation of culture? Soc. Personal. Psychol. Compass 2012, 6, 483–498. [CrossRef]
76. Liss, M.; Erchull, M.J.; Ramsey, L.R. Empowering or oppressing? Development and exploration of the Enjoyment of Sexualization
Scale. Personal. Soc. Psychol. Bull. 2011, 37, 55–68. [CrossRef]
77. Quinn, B.A. Sexual harassment and masculinity: The power and meaning of “girl watching”. Gend. Soc. 2002, 16,
386–402. [CrossRef]
78. Fredrickson, B.L.; Roberts, T.A. Objectification theory: Toward understanding women’s lived experience and mental health risks.
Psychol. Women Q. 1997, 21, 173–206. [CrossRef]
79. Calogero, R.M.; Herbozo, S.; Thompson, J.K. Complimentary weightism: The potential costs of appearancerelated commentary
for women’s self-objectification. Psychol. Women Q. 2009, 33, 120–132. [CrossRef]
80. Renold, E.; Ringrose, J. Schizoid subjectivities? Re-theorizing teen girls sexual cultures in an era of sexualization. J. Sociol. 2011,
47, 389–409. [CrossRef]
81. Gill, R.C. Critical Respect: The difficulties and dilemmas of agency and ‘choice’ for feminism: A reply to Duits and van Zoonen.
Eur. J. Womens Stud. 2007, 14, 69–80. [CrossRef]
82. Gill, R. Empowerment/sexism Figuring female sexual agency in contemporary advertising. Fem. Psychol. 2008, 18,
35–60. [CrossRef]
83. Thompson, L. Desiring to be Desired: A Discursive Analysis of Women’s Responses to the ‘Raunch Culture’ Debates. Ph.D.
Thesis, Murdoch University, Murdoch, Australia, 2012.
84. Koepsel, E.R. The power in pleasure: Practical implementation of pleasure in sex education classrooms. Am. J. Sex. Educ. 2016, 11,
205–265. [CrossRef]
85. Ketting, E.; Friele, M.; Micielsen, K. European Expert Group on Sexuality Education. Evaluation of Holistic Sexuality Education:
A European Expert Group Consensus Agreement. Eur. J. Contracept. Reprod. Health Care 2015, 21, 68–80. [CrossRef]
86. Patterson, S.; Hilton, P.; Flowers, S.; McDaid, L. What are the Barriers and Challenges Faced by Adolescents When Searching for
Sexual Health Information on the Internet? Implications for Policy and Practice from a Qualitative Study. Sex. Transm. Infect.
2019, 95, 462–467. [CrossRef]
87. King, B.M.; Burke, S.R.; Gates, T.M. Is there a gender difference in US college students’ desire for school-based sexuality education?
Sex Educ. 2020, 20, 350–359. [CrossRef]
88. Ruane-McAteer, E.; Gillespie, K.; Amin, A.; Aventin, A.; Robinson, M.; Hanratty, J.; Lohan, M. Gender-transformative program-
ming with men and boys to improve sexual and reproductive health and rights: A systematic review of intervention studies. BMJ
Glob. Health 2020, 5, e002997. [CrossRef] [PubMed]
89. Schmidt, S.C.; Wandersman, A.; Hills, K.J. Evidence-Based Sexuality Education Programs in Schools: Do They Align with the
National Sexuality Education Standards? Am. J. Sex. Educ. 2015, 10, 177–195. [CrossRef]
90. European Expert Group on Sexuality Education. Sexuality education—What is it? Sex Educ. 2016, 16, 427–431. [CrossRef]
91. McCarty-Caplan, D. Sex education and support of LGB families: A family impact analysis of the personal responsibility education
program. Sex. Res. Soc. Policy 2015, 12, 213–223. [CrossRef]
92. Jarpe-Ratner, E. How can we make LGBTQ+-inclusive sex education programmes truly inclusive? A case study of Chicago Public
Schools’ policy and curriculum. Sex Educ. 2020, 20, 283–299. [CrossRef]
93. Mustanski, B.; Greene, G.J.; Ryan, D.; Whitton, S.W. Feasibility, acceptability, and initial efficacy of an online sexual health
promotion program for LGBT youth: The Queer Sex Ed intervention. J. Sex Res. 2015, 52, 220–230. [CrossRef]
94. Haley, S.G.; Tordoff, D.M.; Kantor, A.Z.; Crouch, J.M.; Ahrens, K.R. Sex Education for Transgender and Non-Binary Youth:
Previous Experiences and Recommended Content. J. Sex. Med. 2019, 16, 1834–1848. [CrossRef] [PubMed]
95. Shannon, B. Comprehensive for who? Neoliberal directives in Australian ‘comprehensive’sexuality education and the erasure of
GLBTIQ identity. Sex Educ. 2016, 16, 573–585. [CrossRef]
96. Kirana, P.S.; Gudeloglu, A.; Sansone, A.; Fode, M.; Reisman, Y.; Corona, G.; Burri, A. E-Sexual Health: A Position Statement of the
European Society for Sexual Medicine. J. Sex. Med. 2020, 17, 1246–1253. [CrossRef] [PubMed]
97. Brayboy, L.M.; McCoy, K.; Thamotharan, S.; Zhu, E.; Gil, G.; Houck, C. The use of technology in the sexual health education
especially among minority adolescent girls in the United States. Curr. Opin. Obstet. Gynecol. 2018, 30, 305. [CrossRef]
Int. J. Environ. Res. Public Health 2021, 18, 2555 31 of 31

98. Cushman, N.; Kantor, L.M.; Schroeder, E.; Eicher, L.; Gambone, G. Sexuality education: Findings and recommendations from an
analysis of 10 United States programmes. Sex Educ. 2014, 14, 481–496. [CrossRef]
99. Butler, R.S.; Sorace, D.; Beach, K.H. Institutionalizing sex education in diverse US school districts. J. Adolesc. Health 2018, 62,
149–156. [CrossRef] [PubMed]