1058374

research-article2021
JHLXXX10.1177/08903344211058374Journal of Human LactationHaile

About Research
Journal of Human Lactation

Critical Appraisal Tools and Reporting

2022, Vol. 38(1) 21­–27
© The Author(s) 2021
Article reuse guidelines:
Guidelines sagepub.com/journals-permissions
DOI: 10.1177/08903344211058374
https://doi.org/10.1177/08903344211058374
journals.sagepub.com/home/jhl

Zelalem T. Haile1

Keywords
breastfeeding, critical appraisal, public health, reporting guidelines

Critical appraisal tools and reporting guidelines are the valid critical appraisal tool. The lack of a set of “gold stan-
two most important instruments available to researchers dard” tools means that researchers and practitioners must be
and practitioners involved in research, evidence-based careful in selecting tools appropriate for the evidence
practice, and policymaking. Each of these instruments has reviewed (Buccheri & Sharifi, 2017). Some critical appraisal
unique characteristics, and both instruments play an essen- tools are generic, whereas others are study-design specific
tial role in evidence-based practice and decision-making. (Katrak et al., 2004). Since each tool has strengths and limi-
One must select the right set of tools to report findings tations, researchers and practitioners must be cautious when
accurately and to appraise available evidence adequately. selecting critical appraisal tools. Reviewing the documenta-
The main aims for this paper are to (a) describe steps tion and guidelines about how to use these tools is highly
involved in selecting appropriate critical appraisal tools recommended. Most importantly, selecting an appropriate
and research evidence reporting guidelines; and (b) iden- critical appraisal tool requires having a sufficient under-
tify a list of commonly used critical appraisal tools and standing of the context of the review, including the study
reporting guidelines in health research. The paper also design, the quality of the process involved, and that the evi-
includes information about the authors of these instru- dence generated can be verified (Crowe & Sheppard, 2011).
ments and specific examples of applying these tools and Although there are no specific recommendations on how
guidelines in lactation research. to initiate a critical appraisal, a typical sequence of selecting
an appropriate appraisal tool involves identifying the type of
evidence to be appraised (for example, individual studies vs.
Critical Appraisal Tools systematic reviews) followed by choosing a suitable tool for
Research and evidence-based practice require having a sys- the evidence to be appraised and reviewing documentation
tematic approach to assessing, evaluating, and synthesizing associated with the tool and completing checklists specific to
the quality, reliability, validity, methodological rigor, and the evidence type reviewed (Al-Jundi & Sakka, 2017;
transparency of available evidence. In healthcare settings, Buccheri & Sharifi, 2017; Crowe & Sheppard, 2011).
evidence-based practice is considered a problem-solving It is important to note that critical appraisal tools have dif-
approach providers use in clinical decision-making (Dang et ferent metrics to assess the quality of a study, including
al., 2021). Critical appraisal is a crucial component in con- design and risk of bias. These include ranking and comparing
ducting research and evidence-based clinical practice. One the summary scores evaluating various components of the
dictionary of epidemiology defines a critical appraisal as the manuscript, comparing component-specific scores, weighing
“application of rules of evidence to a study to assess the different components, or appraising research qualitatively by
validity of the data, completeness of reporting, methods and describing the processes involved in generating the pub-
procedures, conclusions, compliance with ethical standards, lished results (Crowe & Sheppard, 2011; Higgins et al.,
etc.” (Porta, 2014, p. 56). If performed appropriately, a criti- 2019). Each of these methods is known to have strengths and
cal appraisal reduces information overload because the
researchers systematically identify credible, unbiased, and
1
relevant evidence to improve clinical practice and policy Department of Social Medicine, Ohio University Heritage College of
(Al-Jundi & Sakka, 2017; Melnyk & Fineout-Overholt, Osteopathic Medicine, Dublin, OH, USA
2015). Date submitted: October 15, 2021; Date accepted: October 21, 2021.
A wide range of critical appraisal tools is available for Corresponding Author:
researchers and practitioners. In an era in which there is a Zelalem T. Haile, Ohio University Heritage College of Osteopathic
continuous proliferation of these tools, the first step often Medicine, 6775 Bobcat Way MEB-1 Rm-450, Dublin, OH 43016, USA.
involves the identification of a well-established, reliable, and Email: [email protected]
22 Journal of Human Lactation 38(1)

weaknesses. For example, a single summary score may not agencies to promote good reporting practices (Simera et al.,
appropriately capture weak evidence, whereas a rating of 2009).
articles using an ordinal scale may lack rationale for the defi- There is no specific rule of thumb for selecting appropri-
nition of the ordinal scale (Crowe & Sheppard, 2011; Heller ate reporting guidelines for dissemination. However, it is
et al., 2008). Likewise, open-ended qualitative appraisals can essential to note that reporting guidelines vary by study type.
be difficult to analyze (Long et al., 2020). Applying a weight- Therefore, a good understanding of the type of evidence to
ing factor to different sections of the research has been pro- be disseminated, including the study design, is one of the
posed as an alternative approach to address issues related to basic requirements of selecting an appropriate reporting
assigning a single summary score (Vali et al., 2021). guideline. Once the applicable reporting guideline is identi-
However, some researchers have shown that weighting fac- fied, researchers and practitioners can locate, use, and cite it
tors in a critical appraisal tool did not affect article ranking as recommended. In 2015 the EQUATOR Network released
(Valentine & Cooper, 2008). Authors should follow instruc- a reporting guideline decision tree and an easy-to-use online
tions and other synthesis manuals as recommended for the wizard of the decision tree to assist authors in selecting
selected appraisal tool. appropriate checklists. These tools are freely available on the
Several organizations and researchers have been involved EQUATOR Network website (The EQUATOR Network,
in developing well-established and often study design-spe- n.d.b). Table 2 presents a description and application of the
cific critical appraisal tools used in the critical appraisal of the frequently used reporting guidelines. Because different jour-
literature. One must note that different critical appraisal tools nals have different requirements, researchers should use
have different functions. Whereas some tools are used to reporting guidelines with the journal-specific author guide-
appraise the quality of a study, others, such as the Grading of lines. This approach will help authors avoid unnecessary
Recommendations Assessment, Development, and Evaluation delay or rejection of manuscripts.
(GRADE), are used to make practice recommendations. Finally, it is crucial to understand the differences and rela-
Others, such as are the Cochrane Risk of Bias Tool, are used tionships between critical appraisal tools and reporting
to assess the risk of bias in randomized trials. Table 1 includes guidelines. Whereas critical appraisal tools are used to assess
a description and application of some of the most used critical the quality of existing evidence, the primary function of
appraisal tools. The table also includes information related to reporting guidelines is to provide authors with guidance on
the sources or authors of these tools and a list of examples in the minimum reporting requirements of findings from stud-
human lactation literature of researchers who used these tools ies conducted using specific study designs. Both tools are
to appraise available evidence. Researchers and practitioners related to one another because poorly reported studies will
can quickly locate and use these resources when appraising receive poor critical appraisal scores and are less likely to be
available literature. used by researchers, healthcare providers, and healthcare
policymakers. Studies with poor or inadequate reporting are
more likely to be excluded from subsequent studies that syn-
Research Reporting Guidelines thesize existing literature, including systematic review and
Poor or inadequate reporting of evidence reduces the value meta-analysis. By using appropriate reporting guidelines,
of the evidence to researchers, practitioners, policymakers, authors can include sufficient detail on essential elements of
and even patients (Brett et al., 2017). Researchers and prac- the research study, which allows readers to adequately
titioners involved in generating and disseminating evidence appraise the quality and relevance of the findings to practice
must use appropriate reporting guidelines to avoid poor qual- and policy using standard critical appraisal tools.
ity reporting of evidence. The main purpose of reporting
guidelines is to systematically evaluate the quality and
Conclusion
improve transparency in published literature.
The genesis of reporting guidelines goes back to 2006 In summary, critical appraisal is a fundamental aspect of
when an international initiative, the Enhancing the Quality evaluating the quality, value, and relevance of available evi-
and Transparency of health Research (EQUATOR) Network, dence. It is a systematic way of assessing the evidence that
was established to address the problem of poor reporting. provides the basis for clinical practice and policy. Authors
This global initiative was formed to “improve the reliability must have adequate knowledge of the different types of stud-
and value of published health research literature by promot- ies to adequately appraise the methodological rigor and qual-
ing transparent and accurate reporting and wider use of ity of the evidence using appropriate critical appraisal tools
robust reporting guidelines” (The EQUATOR Network, deemed reliable and valid. Similarly, because reporting
n.d.a, para. 1). Since its establishment, the initiative has pro- guidelines are study design-specific, researchers should first
moted accurate and transparent reporting by developing and consider the type of evidence to be reported to select the cor-
disseminating a wide range of reporting guidelines. rect reporting guidelines. Finally, authors should use report-
Furthermore, the EQUATOR team works closely with jour- ing guidelines together with journal-specific instructions for
nals, reporting guideline developers, and research funding authors when preparing a manuscript for publication.
 Table 1. Examples of Widely Used Critical Appraisal Tools.

Example of application in
Tool Authors/Organization Applicability/Study design lactation research
Critical Appraisal Skills Programme (CASP) CASP Systematic Reviews, Randomized Controlled (Channell Doig et al., 2020)
Trials, Cohort Studies, Case Control Studies,
Economic Evaluations, Diagnostic Studies,
Qualitative Studies, and Clinical Prediction Rule
Appraisal of Guidelines for Research & AGREE Collaboration Guideline Quality (Boss et al., 2021)
Evaluation II (AGREE II)
Johns Hopkins Research Evidence Appraisal The Johns Hopkins Hospital/Johns Quantitative, Qualitative and Mixed Methods (Robinson et al., 2018)
Tool Hopkins University Studies
Johns Hopkins Non-Research Evidence The Johns Hopkins Hospital/Johns Clinical Practice Guidelines, Consensus, or (Cvach, 2012)1
Appraisal Tool Hopkins University Position Statements; Organizational Experience;
Case Report, Integrative Review, Literature
Review, Expert Opinion, Community Standard,
Clinician Experience, Consumer Preference
Grading of Recommendations Assessment, GRADE Working Group Certainty in Evidence and Strength of (McFadden et al., 2019)
Development and Evaluation (GRADE) Recommendations
Cochrane Risk of Bias Tool (RoB 2) (Higgins et al., 2011) Risk of Bias in Randomized Trials and other (Huda et al., 2021)
Systematic Reviews
Joanna Briggs Institute Critical Appraisal Joanna Briggs Institute Analytical Cross-sectional Studies, Case-Control (Dall’Oglio et al., 2020)
Tools (JBI) Studies, Case Reports, Case Series, Cohort
Studies, Diagnostic Test Accuracy Studies,
Economic Evaluations, Prevalence Studies,
Qualitative Research, Quasi-Experimental
Studies, Randomized Controlled Trials,
Systematic Reviews and Text and Opinion
Rapid Critical Appraisal Checklists (Melnyk & Fineout-Overholt, Descriptive Studies, Evidence-Based Practice (Hubschman-Shahar, 2021)
2015) Implementation or Quality Improvement
Projects,
Cohort Studies, Randomized Controlled Trials,
Systematic Reviews of Clinical
Interventions/Treatments, Qualitative Evidence,
And Evidence-Based Guidelines
Centre for Evidence-Based Medicine The Oxford Centre for Evidence- Systematic Reviews, Diagnostics, Prognosis, (Johnson et al., 2020)
Critical Appraisal Tools (CEBM) Based Medicine (CEBM) Randomized Controlled Trials, Qualitative
Reviews, IPD Reviews in multiple languages
A Measurement Tool to Assess Systematic (Shea et al., 2017) Systematic Reviews of Randomized or Non- (Fair et al., 2021)
Reviews 2 (AMSTAR 2) randomized Studies
Quality Assessment of Diagnostic Accuracy (Whiting et al., 2011) Diagnostic Accuracy Studies (Raskovalova et al., 2015)
Studies 2
(QUADAS 2)

Note. 1Not lactation study.

23
24
Table 2. Examples of Commonly Used Research Reporting Guidelines.

Example of application in lactation

Study Design Reporting Guideline Authors/Organization research
Systematic Reviews/Meta-analyses Preferred Reporting Items for Systematic Reviews and (Page et al., 2021) (Liu et al., 2021)
Meta-analysis (PRISMA)
Meta-analysis of observational Meta-analysis Of Observational Studies in Epidemiology (Stroup et al., 2000) (Chetwynd et al., 2019)
studies in epidemiology (MOOSE)
Synthesis of qualitative research Enhancing Transparency in Reporting the synthesis of (Tong et al., 2012) (Bazzano et al., 2017)
Qualitative Research (ENTREQ)
Experimental Studies
Randomized trials1 Consolidated Standards of Reporting Trials (CONSORT) (Schulz et al., 2010) (Khorana et al., 2021)
Intervention evaluation studies Transparent Reporting of Evaluations with Nonrandomized (Des Jarlais et al., (Schondelmeyer et al., 2021)2
using nonrandomized designs Designs (TREND) 2004)
Observational Studies
Cohort studies Strengthening the Reporting of Observational Studies in (von Elm et al., 2008) (Stuebe et al., 2019)
Case-control studies Epidemiology (STROBE)
Cross-sectional studies
Observational studies using Reporting of studies Conducted using Observational (Benchimol et al., (Whitford et al., 2018)
routinely collected health data Routinely-collected Data (RECORD) 2016)
Case Report CAse Report (CARE) (Gagnier et al., 2013) (Lee et al., 2019)
Web-based surveys Checklist for Reporting Results of Internet E-Surveys (Eysenbach, 2004) (Gallegos et al., 2020)
(CHERRIES)
Qualitative Research Consolidated Criteria for Reporting Qualitative Research (Tong et al., 2007) (Iglesias-Rosado & Leon-Larios,
(COREQ) 2021)
Standards for Reporting Qualitative Research (SRQR) (O’Brien et al., 2014) (Aşcı et al., 2021)
Mixed-Methods Studies Good Reporting of a Mixed Methods Study (GRAMMS) (O’Cathain et al., (Emary et al., 2021)2
2008)
Economic Evaluations of Consolidated Health Economic Evaluation Reporting (Husereau et al., 2013) (Zanganeh et al., 2021)
Interventions Standards (CHEERS)
Reliability and Agreement Studies Guidelines for Reporting Reliability and Agreement Studies (Kottner et al., 2011) (Zhang et al., 2012)
(GRRAS)
Study Protocols Standard Protocol Items: Recommendations for (Chan et al., 2013) (Savitri et al., 2016)
Interventional Trials (SPIRIT)
Systematic Review and Meta- PRISMA for Systematic Review Protocols (PRISMA-P) (Moher et al., 2015) (Chesnel et al., 2021)
analysis Protocols
Diagnostic Accuracy Studies Standards for Reporting of Diagnostic Accuracy (STARD) (Bossuyt et al., 2003) (Chiesa et al., 2015)
Development, Validation, or Transparent Reporting of a multivariable prediction model (Moons et al., 2015) (Hassen et al., 2020)
Update of a Prediction Model for Individual Prognosis Or Diagnosis (TRIPOD)

Note. 1Design-specific CONSORT statement should be used based on the type of randomized trial to be reported. 2Not a lactation study.
Haile 25

Disclosures and Conflicts of Interest experienced childhood maltreatment: A review. Journal

of Human Lactation, 36(4), 710–722. https://doi.org/10.
Zelalem Haile is the Statistical Editor of the Journal of Human
1177/0890334420950257
Lactation. He has no other disclosures or conflicts of interests.
Chesnel, M. J., Healy, M., & McNeill, J. (2021). The experiences
of trained breastfeeding support providers that influence how
ORCID iD breastfeeding support is practised: A protocol for a qualitative
Zelalem T. Haile https://orcid.org/0000-0002-2912-8564 evidence synthesis. PLOS ONE, 16(7), e0254445. https://doi.
org/10.1371/journal.pone.0254445
References Chetwynd, E. M., Wasser, H. M., & Poole, C. (2019). Breastfeeding
support interventions by International Board Certified
Al-Jundi, A., & Sakka, S. (2017). Critical appraisal of linical Lactation Consultants: A systemic review and meta-analysis.
research. Journal of Clinical and Diagnostic Research, 11(5), Journal of Human Lactation, 35(3), 424–440. https://doi.
JE01–JE05. https://doi.org/10.7860/JCDR/2017/26047.9942 org/10.1177/0890334419851482
Aşcı, Ö., Demirgöz Bal, M., & Ergin, A. (2021). The breastfeed- Chiesa, C., Pacifico, L., Osborn, J. F., Bonci, E., Hofer, N., &
ing experiences of COVID-19-positive women: A qualitative Resch, B. (2015). Early-onset neonatal sepsis: Still room
study in Turkey. Japan Journal of Nursing Science, e12453. for improvement in procalcitonin diagnostic accuracy stud-
https://doi.org/10.1111/jjns.12453 ies. Medicine, 94(30), e1230. https://doi.org/10.1097/
Bazzano, A. N., Kaji, A., Felker-Kantor, E., Bazzano, L. A., & MD.0000000000001230
Potts, K. S. (2017). Qualitative studies of infant and young Crowe, M., & Sheppard, L. (2011). A review of critical appraisal
child feeding in lower-income countries: A systematic review tools show they lack rigor: Alternative tool structure is pro-
and synthesis of dietary patterns. Nutrients, 9(10), 1140. https:// posed. Journal of Clinical Epidemiology, 64(1), 79–89. https://
doi.org/10.3390/nu9101140 doi.org/10.1016/j.jclinepi.2010.02.008
Benchimol, E. I., Smeeth, L., Guttmann, A., Harron, K., Hemkens, Cvach, M. (2012). Monitor alarm fatigue: An integrative review.
L. G., Moher, D., Petersen, I., Sørensen, H. T., von Elm, E., Biomedical Instrumentation & Technology, 46(4), 268–277.
Langan, S. M., & RECORD Working Committee. (2016). [The https://doi.org/10.2345/0899-8205-46.4.268
REporting of studies Conducted using Observational Routinely- Dall’Oglio, I., Marchetti, F., Mascolo, R., Amadio, P., Gawronski,
collected health Data (RECORD) statement]. Zeitschrift Fur O., Clemente, M., Dotta, A., Ferro, F., Garofalo, A.,
Evidenz, Fortbildung Und Qualitat Im Gesundheitswesen, Salvatori, G., Tarantino, A., Tiozzo, E., & Giusti, A. (2020).
115–116, 33–48. https://doi.org/10.1016/j.zefq.2016.07.010 Breastfeeding protection, promotion, and support in humani-
Boss, M., Turner, J., Boss, P., Hartmann, P., Pritchard, D., & tarian emergencies: A systematic review of literature.
Clifford, R. (2021). Integrating approaches for quality guide- Journal of Human Lactation, 36(4), 687–698. https://doi.
line development in LactaMap, an online lactation care support org/10.1177/0890334419900151
system. BMC Pregnancy and Childbirth, 21(1), 322. https:// Dang, D., Dearholt, S. L., Bissett, K., Ascenzi, J., & Whalen, M.
doi.org/10.1186/s12884-021-03775-9 (2021). Johns Hopkins evidence-based practice for nurses
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., and healthcare professionals: Model and guidelines. (4th ed.).
Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, Sigma Theta Tau International.
D., & de Vet, H. C. W. (2003). Towards complete and accurate Des Jarlais, D. C., Lyles, C., Crepaz, N., & TREND Group. (2004).
reporting of studies of diagnostic accuracy: The STARD ini- Improving the reporting quality of nonrandomized evaluations
tiative. British Medical Journal, 326(7379), 41–44. https://doi. of behavioral and public health interventions: The TREND
org/10.1136/bmj.326.7379.41 statement. American Journal of Public Health, 94(3), 361–366.
Brett, J., Staniszewska, S., Simera, I., Seers, K., Mockford, C., https://doi.org/10.2105/ajph.94.3.361
Goodlad, S., Altman, D., Moher, D., Barber, R., Denegri, Emary, P. C., Stuber, K. J., Mbuagbaw, L., Oremus, M., Nolet,
S., Entwistle, A. R., Littlejohns, P., Morris, C., Suleman, R., P. S., Nash, J. V., Bauman, C. A., Ciraco, C., Couban, R. J.,
Thomas, V., & Tysall, C. (2017). Reaching consensus on & Busse, J. W. (2021). Quality of reporting in chiropractic
reporting patient and public involvement (PPI) in research: mixed methods research: A methodological review proto-
Methods and lessons learned from the development of report- col. Chiropractic & Manual Therapies, 29(1), 35. https://doi.
ing guidelines. BMJ Open, 7(10), e016948. https://doi. org/10.1186/s12998-021-00395-0
org/10.1136/bmjopen-2017-016948 Eysenbach, G. (2004). Improving the quality of web surveys:
Buccheri, R. K., & Sharifi, C. (2017). Critical appraisal tools and The Checklist for Reporting Results of Internet E-Surveys
reporting guidelines for evidence-based practice. Worldviews (CHERRIES). Journal of Medical Internet Research, 6(3),
on Evidence-Based Nursing, 14(6), 463–472. https://doi. e34. https://doi.org/10.2196/jmir.6.3.e34
org/10.1111/wvn.12258 Fair, F. J., Morrison, A., & Soltani, H. (2021). The impact of Baby-
Chan, A.-W., Tetzlaff, J. M., Gøtzsche, P. C., Altman, D. G., Mann, Friendly Initiative accreditation: An overview of systematic
H., Berlin, J. A., Dickersin, K., Hróbjartsson, A., Schulz, K. F., reviews. Maternal & Child Nutrition, 17(4), e13216. https://
Parulekar, W. R., Krleza-Jeric, K., Laupacis, A., & Moher, D. doi.org/10.1111/mcn.13216
(2013). SPIRIT 2013 explanation and elaboration: Guidance Gagnier, J. J., Kienle, G., Altman, D. G., Moher, D., Sox, H.,
for protocols of clinical trials. BMJ, 346, e7586. https://doi. Riley, D., & CARE Group. (2013). The CARE guidelines:
org/10.1136/bmj.e7586 Consensus-based clinical case report guideline development.
Channell Doig, A., Jasczynski, M., Fleishman, J. L., & Aparicio, Journal of Dietary Supplements, 10(4), 381–390. https://doi.
E. M. (2020). Breastfeeding among mothers who have org/10.3109/19390211.2013.830679
26 Journal of Human Lactation 38(1)

Gallegos, D., Parkinson, J., Duane, S., Domegan, C., Jansen, E., studies (GRRAS). Journal of Clinical Epidemiology, 64(1),
& Russell-Bennett, R. (2020). Understanding breastfeeding 96–106. https://doi.org/10.1016/j.jclinepi.2010.03.002
behaviours: A cross-sectional analysis of associated factors Lee, A. H. X., Wen, B., Hocaloski, S., Sandholdt, N., Hultling,
in Ireland, the United Kingdom and Australia. International C., Elliott, S. L., & Krassioukov, A. V. (2019). Breastfeeding
Breastfeeding Journal, 15(1), 103. https://doi.org/10.1186/ before and after spinal cord injury: A case report of a mother
s13006-020-00344-2 with C6 tetraplegia. Journal of Human Lactation, 35(4), 742–
Hassen, H. Y., Gebreyesus, S. H., Endris, B. S., Roro, M. A., & Van 747. https://doi.org/10.1177/0890334419844234
Geertruyden, J.-P. (2020). Development and validation of a Liu, F., Chen, X., Nie, P., Lin, S., Guo, J., Chen, J., & Yu, L.
risk score to predict low birthweight using characteristics of the (2021). Can tai chi improve cognitive function? A systematic
mother: Analysis from BUNMAP Cohort in Ethiopia. Journal review and meta-analysis of randomized controlled trials. The
of Clinical Medicine, 9(5), E1587. https://doi.org/10.3390/ Journal of Alternative and Complementary Medicine, https://
jcm9051587 doi.org/10.1089/acm.2021.0084
Heller, R. F., Verma, A., Gemmell, I., Harrison, R., Hart, J., & Long, H. A., French, D. P., & Brooks, J. M. (2020). Optimizing the
Edwards, R. (2008). Critical appraisal for public health: A value of the critical appraisal skills programme (CASP) tool for
new checklist. Public Health, 122(1), 92–98. https://doi. quality appraisal in qualitative evidence synthesis. Research
org/10.1016/j.puhe.2007.04.012 Methods in Medicine & Health Sciences, 1(1), 31–42. https://
Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, doi.org/10.1177/2632084320947559
D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, McFadden, A., Siebelt, L., Marshall, J. L., Gavine, A., Girard,
J. A. C., Cochrane Bias Methods Group, & Cochrane Statistical L.-C., Symon, A., & MacGillivray, S. (2019). Counselling
Methods Group. (2011). The Cochrane Collaboration’s tool for interventions to enable women to initiate and continue breast-
assessing risk of bias in randomised trials. BMJ, 343, d5928. feeding: A systematic review and meta-analysis. International
https://doi.org/10.1136/bmj.d5928 Breastfeeding Journal, 14, 42. https://doi.org/10.1186/s13006-
Higgins, J. P. T., Thomas, J., Chandler, J., Cumpston, M., Li, T., 019-0235-8
Page, M., & Welch, V. (2019). Cochrane handbook for system- Melnyk, B. M., & Fineout-Overholt, E. (2015). Evidence-based
atic reviews of Iinterventions. Wiley-Blackwell. practice in nursing & healthcare: A guide to best practice (3rd
Hubschman-Shahar, L. E. (2021). Lactation telehealth in primary ed.). Wolters Kluwer.
care: A systematic review. Breastfeeding Medicine. https://doi. Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A.,
org/10.1089/bfm.2021.0105 Petticrew, M., Shekelle, P., Stewart, L. A., & PRISMA-P
Huda, M. H., Chipojola, R., Lin, Y. M., Lee, G. T., Shyu, M.-L., & Group. (2015). Preferred reporting items for systematic review
Kuo, S.-Y. (2021). The influence of breastfeeding educational and meta-analysis protocols (PRISMA-P) 2015 statement.
interventions on breast engorgement and exclusive breastfeed- Systematic Reviews, 4, 1. https://doi.org/10.1186/2046-4053-
ing: A systematic review and meta-analysis. Journal of Human 4-1
Lactation. https://doi.org/10.1177/08903344211029279 Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P.
Husereau, D., Drummond, M., Petrou, S., Carswell, C., Moher, D., A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff,
Greenberg, D., Augustovski, F., Briggs, A. H., Mauskopf, J., D. F., & Collins, G. S. (2015). Transparent reporting of a
Loder, E., & CHEERS Task Force. (2013). Consolidated health multivariable prediction model for Individual Prognosis or
economic evaluation reporting standards (CHEERS) statement. Diagnosis (TRIPOD): Explanation and elaboration. Annals of
BMJ), 346, f1049. https://doi.org/10.1136/bmj.f1049 Internal Medicine, 162(1), W1–73. https://doi.org/10.7326/
Iglesias-Rosado, B., & Leon-Larios, F. (2021). Breastfeeding M14-0698
experiences of Latina migrants living in Spain: A qualitative O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook,
descriptive study. International Breastfeeding Journal, 16(1), D. A. (2014). Standards for reporting qualitative research: A
76. https://doi.org/10.1186/s13006-021-00423-y synthesis of recommendations. Academic Medicine, 89(9),
Johnson, H. M., Eglash, A., Mitchell, K. B., Leeper, K., Smillie, C. 1245–1251. https://doi.org/10.1097/ACM.0000000000000388
M., Moore-Ostby, L., Manson, N., Simon, L., & Academy of O’Cathain, A., Murphy, E., & Nicholl, J. (2008). The quality of
Breastfeeding Medicine. (2020). ABM Clinical Protocol #32: mixed methods studies in health services research. Journal of
Management of hyperlactation. Breastfeeding Medicine. 15(3), Health Services Research & Policy, 13(2), 92–98. https://doi.
129–134. https://doi.org/10.1089/bfm.2019.29141.hmj org/10.1258/jhsrp.2007.007074
Katrak, P., Bialocerkowski, A. E., Massy-Westropp, N., Kumar, S., Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann,
& Grimmer, K. A. (2004). A systematic review of the content of T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E.
critical appraisal tools. BMC Medical Research Methodology, A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M.,
4, 22. https://doi.org/10.1186/1471-2288-4-22 Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-
Khorana, M., Wongsin, P., Torbunsupachai, R., & Kanjanapattanakul, Wilson, E., McDonald, S., . . . Moher, D. (2021). The PRISMA
W. (2021). Effect of domperidone on breast milk production in 2020 statement: An updated guideline for reporting systematic
mothers of sick neonates: A randomized, double-blinded, pla- reviews. Journal of Clinical Epidemiology, 134, 178–189.
cebo-controlled trial. Breastfeeding Medicine, 16(3), 245–250. https://doi.org/10.1016/j.jclinepi.2021.03.001
https://doi.org/10.1089/bfm.2020.0234 Porta, M. (Ed.). (2014). Dictionary of Epidemiology. (5th Ed.).
Kottner, J., Audigé, L., Brorson, S., Donner, A., Gajewski, B. J., Oxford University Press. https://www.oxfordreference.com/
Hróbjartsson, A., Roberts, C., Shoukri, M., & Streiner, D. view/10.1093/acref/9780195314496.001.0001/acref-
L. (2011). Guidelines for reporting reliability and agreement 9780195314496
Haile 27

Raskovalova, T., Teasley, S. L., Gelbert-Baudino, N., Mauri, P. Breastfeeding Medicine, 14(8), 551–559. https://doi.org/10.
A., Schelstraete, C., Massoutier, M., Berger, M., François, 1089/bfm.2019.0079
P., & Labarère, J. (2015). Breastfeeding Assessment Score: The EQUATOR Network. (n.d.a). The EQUATOR Network.
Systematic review and meta-analysis. Pediatrics, 135(5), Retrieved October 14, 2021, from https://www.equator-net-
e1276–e1285. https://doi.org/10.1542/peds.2014-3072 work.org/about-us/
Robinson, K., Garnier-Villarreal, M., & Hanson, L. (2018). The EQUATOR Network. (n.d.b). Toolkits | The EQUATOR
Effectiveness of centering pregnancy on breastfeeding initia- Network. Retrieved October 14, 2021, from https://www.equa-
tion among African Americans: A systematic review and meta- tor-network.org/toolkits/
analysis. The Journal of Perinatal & Neonatal Nursing, 32(2), Tong, A., Flemming, K., McInnes, E., Oliver, S., & Craig, J. (2012).
116–126. https://doi.org/10.1097/JPN.0000000000000307 Enhancing transparency in reporting the synthesis of qualitative
Savitri, A. I., Idris, N. S., Indawati, W., Saldi, S. R. F., Amelia, research: ENTREQ. BMC Medical Research Methodology, 12,
D., Baharuddin, M., Sastroasmoro, S., Grobbee, D. E., & 181. https://doi.org/10.1186/1471-2288-12-181
Uiterwaal, C. S. P. M. (2016). BReastfeeding Attitude and Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria
Volume Optimization (BRAVO) trial: Study protocol for a for reporting qualitative research (COREQ): A 32-item check-
randomized controlled trial. Trials, 17(1), 271. https://doi. list for interviews and focus groups. International Journal
org/10.1186/s13063-016-1397-y for Quality in Health Care, 19(6), 349–357. https://doi.
Schondelmeyer, A. C., Bettencourt, A. P., Xiao, R., Beidas, R. org/10.1093/intqhc/mzm042
S., Wolk, C. B., Landrigan, C. P., Brady, P. W., Brent, C. R., Valentine, J. C., & Cooper, H. (2008). A systematic and trans-
Parthasarathy, P., Kern-Goldberger, A. S., Sergay, N., Lee, V., parent approach for assessing the methodological quality
Russell, C. J., Prasto, J., Zaman, S., McQuistion, K., Lucey, of intervention effectiveness research: The Study Design
K., Solomon, C., & Garcia, M., . . . Pediatric Research in and Implementation Assessment Device (Study DIAD).
Inpatient Settings (PRIS) Network. (2021). Evaluation of an Psychological Methods, 13(2), 130–149. https://doi.
educational outreach and audit and feedback program to reduce org/10.1037/1082-989X.13.2.130
continuous pulse oximetry use in hospitalized infants with Vali, Y., Leeflang, M. M. G., & Bossuyt, P. M. M. (2021).
stable bronchiolitis: A nonrandomized clinical trial. JAMA Application of weighting methods for presenting risk-of-bias
Network Open, 4(9), e2122826. https://doi.org/10.1001/jama- assessments in systematic reviews of diagnostic test accu-
networkopen.2021.22826 racy studies. Systematic Reviews, 10(1), 191. https://doi.
Schulz, K. F., Altman, D. G., & Moher, D. CONSORT Group. org/10.1186/s13643-021-01744-z
(2010). CONSORT 2010 Statement: Updated guidelines for von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche,
reporting parallel group randomised trials. Trials, 11, 32. P. C., & Vandenbroucke, J. P., & STROBE Initiative. (2008).
https://doi.org/10.1186/1745-6215-11-32 The Strengthening the Reporting of Observational Studies in
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, Epidemiology (STROBE) statement: Guidelines for reporting
J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., & observational studies. Journal of Clinical Epidemiology, 61(4),
Henry, D. A. (2017). AMSTAR 2: A critical appraisal tool for 344–349. https://doi.org/10.1016/j.jclinepi.2007.11.008
systematic reviews that include randomised or non-randomised Whitford, H., Hoddinott, P., Amir, L. H., Chamberlain, C., East,
studies of healthcare interventions, or both. BMJ, 358, j4008. C. E., Jones, L., & Renfrew, M. J. (2018). Routinely collected
https://doi.org/10.1136/bmj.j4008 infant feeding data: Time for global action. Maternal & Child
Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. Nutrition, 14(4), e12616. https://doi.org/10.1111/mcn.12616
(2009). The EQUATOR Network and reporting guidelines: Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S.,
Helping to achieve high standards in reporting health research Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C.,
studies. Maturitas, 63(1), 4–6. https://doi.org/10.1016/j. & Bossuyt, P. M. M. QUADAS-2 Group. (2011). QUADAS-2:
maturitas.2009.03.011 A revised tool for the quality assessment of diagnostic accu-
Stroup, D. F., Berlin, J. A., Morton, S. C., Olkin, I., Williamson, racy studies. Annals of Internal Medicine, 155(8), 529–536.
G. D., Rennie, D., Moher, D., Becker, B. J., Sipe, T. A., & https://doi.org/10.7326/0003-4819-155-8-201110180-00009
Thacker, S. B. (2000). Meta-analysis of observational stud- Zanganeh, M., Jordan, M., & Mistry, H. (2021). A systematic
ies in epidemiology: A proposal for reporting. Meta-analysis review of economic evaluations for donor human milk versus
of Observational Studies in Epidemiology (MOOSE) standard feeding in infants. Maternal & Child Nutrition, 17(2),
group. JAMA, 283(15), 2008–2012. https://doi.org/10.1001/ e13151. https://doi.org/10.1111/mcn.13151
jama.283.15.2008 Zhang, S., Wu, Q., van Velthoven, M. H., Chen, L., Car, J., Rudan,
Stuebe, A. M., Meltzer-Brody, S., Propper, C., Pearson, B., Beiler, P., I., Zhang, Y., Li, Y., & Scherpbier, R. W. (2012). Smartphone
Elam, M., Walker, C., Mills-Koonce, R., & Grewen, K. (2019). versus pen-and-paper data collection of infant feeding prac-
The mood, mother, and infant study: Associations between tices in rural China. Journal of Medical Internet Research,
maternal mood in pregnancy and breastfeeding outcome. 14(5), e119. https://doi.org/10.2196/jmir.2183