Journal of the International Neuropsychological Society (2021), 1–19

Copyright © INS. Published by Cambridge University Press, 2021. This is an Open Access article, distributed under the terms of the Creative Commons
Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided
the original work is properly cited.
doi:10.1017/S1355617721000862

CRITICAL REVIEW

Assessment of Neurocognitive Functions, Olfaction, Taste, Mental,

and Psychosocial Health in COVID-19 in Adults: Recommendations
for Harmonization of Research and Implications for Clinical Practice

Lucette A. Cysique1,19,20,* , Emilia Łojek2,*, Theodore Ching-Kong Cheung3,† , Breda Cullen4,†, Anna Rita Egbert5,† ,
Jonathan Evans4,†, Maite Garolera6,†, Natalia Gawron7,†, Hetta Gouse8,†, Karolina Hansen2,†, Paweł Holas2,†,
Sylwia Hyniewska9,†, Ewa Malinowska2,†, Bernice A. Marcopulos10,11,†, Tricia L. Merkley12,†, Jose A. Muñoz-Moreno13,†,
Clare Ramsden14,†, Christian Salas15,†, Sietske A.M. Sikkes16,†, Ana Rita Silva17,†, Imane Zouhar18,† and the NeuroCOVID
International Neuropsychology Taskforce
1
Psychology Department, Faculty of Sciences, The University of New South Wales, Sydney, NSW, Australia
2
Department of Clinical Neuropsychology and Psychotherapy, Faculty of Psychology, University of Warsaw, Warsaw, Poland
3
Department of Psychology, University of Toronto; Centre for Neuropsychology and Emotional Wellness, Markham, ON, Canada
4
Institute of Health and Wellbeing, University of Glasgow, Glasgow, UK
5
Faculty of Medicine, The University of British Columbia, Vancouver, BC, Canada
6
Neuropsychology Unit, Consorci Sanitari de Terrassa, Barcelona, Spain
7
Department of Adult Clinical Psychology, Institute of Psychology, The Maria Grzegorzewska University, Warsaw, Poland
8
Department of Psychiatry & Mental Health, University of Cape Town, Cape Town, South Africa
9
Division of Psychology and Language Sciences, University College London, London, UK
10
Department of Graduate Psychology, James Madison University, Harrisonburg, USA
11
Department of Psychiatry and Neurobehavioral Sciences, University of Virginia School of Medicine, Charlottesville, USA
12
Department of Psychology and Neuroscience Center, Brigham Young University, Brigham, USA
13
Infectious Diseases Department, Lluita contra la SIDA Foundation, Germans Trias i Pujol Hospital, Barcelona, Spain
14
Tasmanian Health Service, Hobart, Australia
15
Faculty of Psychology, Diego Portales University, Santiago, Chile
16
Faculty of Behavioural and Movement Sciences, Department of Clinical, Neuro and Developmental Psychology, Amsterdam University Medical Centers &
VU University, Amsterdam, the Netherlands
17
Center for Research in Neuropsychology and Cognitive Behavioral Intervention – CINEICC, University of Coimbra, Coimbra, Portugal
18
Department of Psychology, University of Toronto, Toronto, ON, Canada
19
St. Vincent’s Applied Medical Research Centre, Peter Duncan Neuroscience Unit, Darlinghurst, Australia
20
MAP Centre for Urban Health Solutions, St. Michael’s Hospital, Toronto, ON, Canada

(RECEIVED March 2, 2021; FINAL REVISION May 7, 2021; ACCEPTED May 13, 2021)

Abstract
Objective: To propose a set of internationally harmonized procedures and methods for assessing neurocognitive
functions, smell, taste, mental, and psychosocial health, and other factors in adults formally diagnosed with COVID-19
(confirmed as SARS-CoV-2 þ WHO definition). Methods: We formed an international and cross-disciplinary
NeuroCOVID Neuropsychology Taskforce in April 2020. Seven criteria were used to guide the selection of the
recommendations’ methods and procedures: (i) Relevance to all COVID-19 illness stages and longitudinal study design;
(ii) Standard, cross-culturally valid or widely available instruments; (iii) Coverage of both direct and indirect causes of
COVID-19-associated neurological and psychiatric symptoms; (iv) Control of factors specifically pertinent to
COVID-19 that may affect neuropsychological performance; (v) Flexibility of administration (telehealth, computerized,
remote/online, face to face); (vi) Harmonization for facilitating international research; (vii) Ease of translation to clinical
practice. Results: The three proposed levels of harmonization include a screening strategy with telehealth option, a
medium-size computerized assessment with an online/remote option, and a comprehensive evaluation with flexible

Correspondence and reprint requests to: Lucette A. Cysique, Ph.D.,

Department of Psychology, Faculty of Science, The University of New
South Wales, Sydney, NSW, 2052 Australia. Office Ph: þ61 (2) 9385 0999.
E-mail: [email protected]
*First co-authors.
†
Co-authors contributed similarly and are listed alphabetically.

1
Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 2 L.A. Cysique, E. Łojek et al.

administration. The context in which each harmonization level might be used is described. Issues of assessment
timelines, guidance for home/remote assessment to support data fidelity and telehealth considerations, cross-cultural
adequacy, norms, and impairment definitions are also described. Conclusions: The proposed recommendations provide
rationale and methodological guidance for neuropsychological research studies and clinical assessment in adults with
COVID-19. We expect that the use of the recommendations will facilitate data harmonization and global research.
Research implementing the recommendations will be crucial to determine their acceptability, usability, and validity.
Keywords: COVID-19, Neuropsychological functions, Assessment, Guidelines

Since December 2019, the world has been grappling with Xiang et al., 2020). While such high prevalence may be asso-
escalating cases of Severe Acute Respiratory Syndrome ciated with pandemic stress and higher anxio-depressive symp-
Coronavirus 2 (SARS-CoV-2) infections leading to previ- toms across the community (Ettman et al., 2020), the
ously unknown Coronavirus Disease – COVID-19. By possibility of immune-related or direct SARS-CoV-2 brain
April 15, 2021 – there were nearly 140 million confirmed impact cannot be excluded at this stage (Troyer, Kohn, &
cases of infection with the SARS-CoV-2 and almost 3 million Hong, 2020). PTSD is known to occur in patient groups
deaths from COVID-19 (for up-to-date data, see World who undergo severe and critical illness, especially ICU survi-
Health Organization, WHO 2020). vors, those who are intubated and mechanically ventilated, and
Within a few months of the initial SARS-CoV-2 infections ultimately those that experience delirium (Marra,
detected in Wuhan, physicians in charge of ill patients Pandharipande, & Patel, 2017). An association between delir-
observed that the disease involved multiple organs besides ium and PTSD has been described recently in COVID-19
the lungs, including the heart, liver, gut, peripheral nerves, (Kaseda & Levine, 2020). Depression, anxiety, and PTSD
and the brain (Yang et al., 2020). A retrospective observatio- can be associated with various neuropsychological deficits
nal case series of 214 consecutive hospitalized patients with (Marcopulos, 2018), which will complicate the differential
laboratory-confirmed diagnosis of SARS-CoV-2 (Mao et al., diagnosis of long-term neurocognitive effects of COVID-19
2020) showed that neurological involvement was frequent (in (Kaseda, & Levine, 2020). Finally, the rate and extent of recov-
36% of 214 patients and in 45% of those with severe disease ery (chronic effects of COVID-19 on the CNS and the newly
vs. 30% in those with non-severe disease). Various cerebro- recognized “Long-COVID” also newly known as Post-Acute
vascular events (e.g., ischemic stroke, intracerebral hemor- Sequelae of SARS-CoV-2 infection (PASC)), and potential
rhage, cerebral venous sinus thrombosis) are described as increased risk for long-term neurodegenerative effects and
the most prominent COVID-19-associated neurological neuropsychological sequelae are yet to be investigated (De
symptoms. This is followed by inflammatory CNS syn- Felice, Tovar-Moll, Moll, Munoz, & Ferreira, 2020; Wilson
dromes (e.g., encephalitis, encephalomyelitis). Peripheral & Jack, 2020).
neurological disorders (e.g., Guillain–Barré) and variants SARS-CoV-2 neuropathogenic mechanisms are thought
are less common (Frontera et al., 2020; Paterson et al., to be multifactorial, including possible direct and indirect
2020; Varatharaj et al., 2020). SARS-Cov-2 may change effects of the virus in the CNS (Frontera et al., 2020;
the risk of stroke through an enhanced systemic inflammatory Koralnik & Tyler, 2020). Evidence for the presence of
response, hypercoagulable state, and endothelial damage in SARS-CoV-2 RNA in the CNS and associated morphologi-
the cerebrovascular system (Abootalebi et al., 2020). cal changes (such as thromboembolic ischemic infarction of
Frequent but typically less severe neurological symptoms the CNS), specifically in the brain stem, has been shown
include headache, dizziness, anosmia, and ageusia (Meinhardt et al., 2021). Viral load of 5.0–59.4 copies per
(Frontera et al., 2020; Helms et al., 2020). Anosmia and ageu- cubic millimeter was also reported in the brain sections from
sia are reported even in patients whose presentation is not the medulla oblongata, the frontal lobes, and olfactory nerves,
severe enough to warrant hospital admission or who are oth- which is obtained from 16 patients who died with COVID-19
erwise asymptomatic (Gane, Kelly, & Hopkins, 2020). In (Solomon et al., 2020). Inconsistencies in the detection of
some cases, the involvement of the nasal epithelium may only SARS-CoV-2 in the CNS remain. This may be due to the
reflect local inflammation. However, trafficking of viral par- dynamics of the infection in relation to when samples were
ticles and protein, in addition to SARS-CoV-2 RNA to the obtained, and/or the fact that viral load and neural infectivity
CNS cannot be excluded (Meinhardt et al., 2021). have a nonlinear relationship (Yi et al., 2020).
Across the pool of retrospective studies on COVID-19, In the acute phase, progressive respiratory involvement
new-onset psychosis, affective disorders, altered mental status can lead to Acute Respiratory Distress Syndrome (ARDS),
including agitation, and dysexecutive symptoms have also which is itself associated with a high risk of hypoxia and con-
been reported (Helms et al., 2020). Some of these neuropsychi- comitant cognitive and psychiatric sequelae; this represents
atric symptoms were linked to premorbid status (e.g., demen- as one of the main indirect pathways to brain damage in
tia), while others represented de novo symptoms (Varatharaj COVID-19 (Ellul et al., 2020; von Weyhern, Kaufmann,
et al., 2020). Among the emerging prospective studies of Neff, & Kremer, 2020; Wu et al., 2020). Acute hypoxic inju-
COVID-19, one key finding is the relatively high prevalence ries were detected in the cerebrum and cerebellum in 18
of PTSD, depressive, and anxiety symptoms (Bo et al., 2020; patients who died with COVID-19, with loss of neurons in

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 3

the cerebral cortex, hippocampus, and cerebellar Purkinje cell speed, executive functions, and perceptual abilities) in 29
layer (Solomon et al., 2020). hospitalized patients who recovered from COVID-19 and
Severe forms of COVID-19 illness requiring intensive 29 closely matched controls. They found impairment of sus-
care unit (ICU), intubation, and ventilation may be associated tained attention in the clinical group and a significant rela-
with further immune, inflammatory, and vascular brain dam- tionship between reaction time and inflammatory level as
age. Secondary effects such as ICU delirium and possible indicated by C-reactive protein.
long-term cognitive disorders are further observed and may Almeria, Cejudo, Sotoca, Deus, and Krupinski (2020)
be related to CNS invasion, inflammation, other organ failure, described cognitive disorders in 35 patients (aged 20–60)
and induction of sedatives (Kotfis et al., 2020). with confirmed COVID-19, without any previous neurologi-
The picture, course, and long-term consequences of cal or psychiatric diseases. The patients were examined in-
COVID-19 are modified by many factors. Serious health com- person, for 10–31 days after hospital discharge, using a set
plications and the death toll from infection are greater among of standardized neuropsychological tests. Individuals pre-
older individuals (>60 years), those with underlying medical senting with headache, anosmia, dysgeusia, diarrhea, and
conditions (including hypertension, obesity, chronic lung dis- those who required oxygen therapy had lower scores in
ease, diabetes, and cardiovascular disease). COVID-19 may memory, attention, and executive function tests as compared
also have a distinct course and impact in patients with preex- to asymptomatic patients. Marked disorders (scores 2 SD
isting neurological, psychiatric, and immune conditions below appropriate norms, controlling for age and education)
including schizophrenia (Fonseca et al., 2020; Kozloff, were noted in the domains of memory, attention, and seman-
Mulsant, Stergiopoulos, & Voineskos, 2020), mild cognitive tic fluency [in 2 patients (5.7%)], in working memory and
impairment, Alzheimer’s disease, Parkinson’s disease, multi- mental flexibility [3 (8.6%)], and phonetic fluency
ple sclerosis (Matías-Guiu et al., 2020), and HIV-associated [4 (11.4%)]. Anxiety and depression indicators were signifi-
neurocognitive disorder (Levine, Sacktor, & Becker, 2020). cantly related to subjective cognitive complaints.
Poverty, living in densely populated neighborhoods of lower Finally, an Australian study (Darley et al., 2020) con-
socioeconomic status, a higher prevalence of comorbid dis- ducted in a community sample (only 10% hospitalized) found
eases, and poor accessibility to healthcare facilities and ser- a low rate of neurocognitive impairment (9%) 2 months after
vices are further risk factors for contracting the virus, as recovering from COVID-19 illness on the Cogstate Test
well as negative health outcomes (Bialek et al., 2020; Battery measuring visual learning, speed of processing, atten-
Laurencin & McClinton, 2020; Public Health England, tion/working memory, and executive functions. However,
2020; Raifman & Raifman, 2020). 24% showed impairment on the NIH Toolbox Odor
The above data indicate that as a result of many pathologi- Identification test, and this was associated with neurocogni-
cal factors and mechanisms associated with COVID-19, peo- tive impairment. Further, there was an association between
ple recovering from that disease may experience cognitive, moderate-to-severe initial neurological symptoms and con-
emotional, and behavioral problems that require a referral tinued subtle neurocognitive changes. Because this is a pro-
to neuropsychology and/or neuropsychiatry services. It is spective study, it will be important to assess how these results
not known how long these problems may persist, but for a evolve on longitudinal testing.
certain number of COVID-19 survivors, it may even be a life- In response to the urgent needs associated with possible
long impairment, significantly influencing everyday life. neuropsychological consequences of COVID-19, we formed
Neuropsychologists have already signaled urgent needs the NeuroCOVID International Neuropsychology Taskforce
for developing research as well as clinical practice services in April 2020, with the goal of developing recommendations
for COVID-19 survivors (Postal et al., 2021; Sozzi et al., for harmonized standard neuropsychological methods and
2020; Wilson, Betteridge, & Fish, 2020). These studies procedures/protocols to determine the prevalence, pattern,
and mounting evidence from neurological studies (Taquet and incidence of neurological and neuropsychological symp-
et al., 2021) support the hypothesis that COVID-19 may lead toms associated with COVID-19 in adults. The use of similar,
to neurocognitive disorders. One study included a sample of harmonized assessment methods will help to combine data on
over 84,000 individuals who were coincidently participating COVID-19 from different sources. As of April 2021, the
in another study amid the COVID-19 pandemic (Hampshire group has 107 members from 19 countries (see Figure 1).
et al., 2020). This UK study revealed that individuals who Neuropsychological knowledge and methods can play a
recovered from suspected or confirmed COVID-19 per- key role in understanding the prevalence, profile, and nature
formed significantly worse on tests in multiple cognitive of COVID-19 neurological and psychiatric symptoms. They
domains compared to people who did not suffer from may also contribute to the development of clinical manage-
COVID-19. This deficit was evident in hospitalized ment and facilitate the development of rehabilitation guide-
COVID-19 survivors, but also among individuals who did lines for patients with COVID-19-related neurological
not receive hospital treatment. However, the study had sig- disorders worldwide. There are currently no definitive stan-
nificant methodological limitations in determining what dards for neuropsychological (i.e., cognition, motor func-
may have been due to COVID-19 versus any other causes tions, global-, mental-, and psychosocial health, olfaction,
of impairment. Zhou et al. (2020) in China, Wuhan, examined and taste) assessment of patients with COVID-19. A lack
cognitive functions (i.e., attention, memory, processing of standard will lead to disparate results, which will be

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 4 L.A. Cysique, E. Łojek et al.

Fig. 1. The Taskforce international representation.

Taskforce includes 107 members from the following countries: USA (52 Members), Australia (15), Poland (7), Canada (5), Netherlands (5),
South Africa (4), UK (4), Spain (2), Belgium (2), Norway (2), Chile (1), Finland (1), Germany (1), Greece (1), Israel (1), Malaysia (1), Mexico
(1), Zambia (1), Portugal (1) [numbers correct as of February–April 27, 2021].

difficult to interpret as the methods and procedures will not be COVID-19 infection and potential translation to clinical prac-
comparable and have unreliable associations with disease tice, we apply the following selection criteria:
processes and biomarkers. This could result in inconsistent A) Methods appropriate for measuring the consequences of
management guidelines, inadequate policies, and poor out- COVID-19, in order to:
comes for patients. • Measure the range and severity of COVID-19-associated
COVID-19 is a new disease. It is complex as different neuropsychological dysfunctions (i.e., direct and indirect
(both direct infection and indirect) mechanisms, may be causes of COVID-19-associated neurological and psychiat-
responsible for neuropsychological dysfunctions. The range ric symptoms).
and severity of neurological symptoms are varied and poten- • Differentiate neuropsychological impairment from psycho-
tially affect the entire neuraxis (Paterson et al., 2020). logical distress.
Developing research protocols that appreciate this complex- • Measure consequences at different phases of disease
ity will have important clinical repercussions. The social (acute/infectious, subacute, chronic) that fit the require-
ments of longitudinal study design.
lockdowns make standard in-person neuropsychological
• Consider premorbid and comorbid effects, performance
assessment practice difficult or impossible, even in countries validity, and other factors that may affect neuropsychologi-
with developed neuropsychological services. While awaiting cal performance in a manner specific to patients with
a global vaccine and its rollout, neuropsychologists adapted COVID-19.
to the COVID-19 pandemic by modifying their services and B) Methods and procedures adaptable to the pandemic social lock-
adapting their assessments using telehealth – audio or video down, and patients’ quarantine status, or patient’s hospitali-
conferencing technologies (Bilder et al., 2020; Matchanova zation and alertness status (e.g., ICU vs. ambulatory):
et al., 2020; Postal et al., 2021). This adaptation also neces- • Telehealth, computerized, remote/online, pen, and pencils
sitates a shift in standard methods of neuropsychological assessments options.
research of patients infected with COVID-19. • Screening strategies, medium-size evaluation, comprehen-
Since COVID-19 is a global pandemic, we must develop sive assessment options.
harmonized methods and procedures that are globally rel- C) Methods and procedures appropriate for international
evant and promote health equity just as we strived to do purposes:
for HIV infection. Our recommendations must be applicable • Selection of tests with evidence for cross-cultural validity
or widely available instruments.
across various settings and work in low-middle and high-
• Guidelines or other considerations to promote valid cross-
income countries. Building capacity to address such diverse cultural test translation/adaption, as well as data fidelity.
objectives is fully embraced as one of the major goals of these
recommendations. To facilitate the implementation of the recommendations,
To provide standard and harmonized neuropsychological the context in which each harmonization level could be used
methods and procedures for research in patients with is described. Issues pertinent to required training level for

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 5

administration and scoring, assessment timeline, guidance to Time of testing for hospitalized patients
support (remote) data fidelity, norms, and impairment defini-
Assessment of cognition should be completed around the
tions are also described.
time of discharge, ideally before.
To address our aims, we propose three levels of harmoni-
zation of neuropsychological examination methods and pro-
cedures in COVID-19. Each level of harmonization covers a Quarantine status
different level (from minimal, medium to comprehensive) of Can be in quarantine or no quarantine.
neurocognitive, mental and psychosocial functions, and other
important factors for describing medical and demographic
characteristics. Harmonization Level 3 (HL 3) was designed Patient’s alertness status
to represent a close equivalent to clinical practice. All proce- Test should only be completed when the patient is fully able
dures recommended in the current work involve human par- to do the testing via a brief assessment of CNS symptoms.
ticipants and thus they should be conducted in accordance
with the ethical standards of the relevant institutional and/
or national research committee and with the 1964 Helsinki
Setting
Declaration and its later amendments or comparable ethical Telehealth, in-person with PPE. Considering pandemic-
standards. related limitations in research and clinical activities, the
HL1 protocol can avoid in-person face-to-face contact
through the use of remote assessment methods. Thus, HL1
RECOMMENDATIONS FOR facilitates studying participants in the infectious phase who
HARMONIZATION LEVEL 1 are (self-)quarantined, isolated, or hospitalized.

General Assumptions Testing type

Harmonization Level 1 (HL1) is focused on research and
Brief/screen.
clinical contexts requiring brief screening either remotely
or in-person, adaptable to various health settings and the
health/infectious status of the patient. It is also based on tools Level of required training for administration and
that have global applications and are inexpensive for cogni- scoring
tion, sensation, and mental health; with administration requir- Minimal.
ing minimum training. HL1 is designed to fit a baseline
assessment to a potential prospective longitudinal observatio- Control group
nal study; it can also serve as a stand-alone cross-sectional
study design. The recommendations of an exact set of mea- SARS-CoV-2-negative individuals can be recruited as the
sures and variables will facilitate data merging for control group. Control group should be matched on demo-
international comparisons and global epidemiological data. graphics, health characteristics, quarantine, and hospitali-
zation setting.
Table 1 summarizes the HL1 protocol.
Recommendations for Application
Patient’s infectious status Recommended Measurement Methods
Eligible participants include SARS-CoV-2 seropositive Demographic inventory and medical history
patients (see WHO case definition at WHO 2021) in the early questionnaires
phase of the disease, including asymptomatic individuals), as
Refer to the material provided in Supplementary Material 1
well as more advanced stages of the disease including patients
either via link access or copy of the material when authorized.
presenting progressive respiratory involvement and focal/
We recommend the use of the Case Report Form (CRF)
systemic inflammation. In these phases, it is very important
developed by the COVID-Neuro Network (access to the
to take into account the medical history to assess whether
CRF requires a registration at Brain Infectious Global
remote or bedside testing should be conducted at all. HL1
COVID-Neuro Network, 2021). We strongly recommend
should only be conducted when a patient is fully able to par-
completing all the demographic and medical data sections
ticipate in testing.
of the CRF. The laboratory data sections are optional. This
CRF includes CNS symptoms using the Glasgow Coma
Scale (GCS) and the Modified Rankin Score, which is pro-
ICU status
vided in Supplementary Material 1. We recommend docu-
Can be in ICU, any exams require personal protective equip- menting acute confusion states using the Confusion
ment (PPE) in ICU settings and depend on local capacity to Assessment Method (CAM), which is also included in
handle exams in ICU. Supplementary Material 1.

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 6 L.A. Cysique, E. Łojek et al.

Table 1. Harmonization Level 1 protocol

Completion time
# Domain Format (min) Mode of administration

1 Demographic inventory CRF 5–10 Self

Can be aided by informant
Use COVID-19 Neuro Network CRF Questionnaire In-person or telehealth*
2 Medical history questionnaires includes CNS CRF 10–20 Email or secure mailing of questionnaires
symptoms Questionnaire may be used*
Use COVID-19 Neuro Network CRF and Use medical records
added scales Use information from treating doctors
3 Neurocognitive screens Standard screening 10–20 In-person or telehealth
test**
4 Six items Smell and taste questionnaire Questionnaire 1–2 In-person or telehealth*
5 Cognitive symptoms questionnaire Questionnaire 4–5 Email or secure mailing of questionnaires
may be used*
6 Mental and Psychosocial health Questionnaire 5–8 Self
Can be aided by informant

NB: the protocol material is available in Supplementary Material 1.

*Some information may be filled out by participants/patients and their informants at their convenience within 3 days of the screen exam.
** We also recommend the Brief Test of Adult Cognition Telephone (BTACT; Tun & Lachman, 2006) as a potential alternative to the MOCA-5/T-MoCA as the
standard screening test. The BTACT is composed of the Rey Auditory Verbal Learning Test (RAVLT), Wechsler Adult Intelligence Scale, Third Edition
(WAIS-III) Digit Span Backwards, Category Fluency test, Red/Green test, Number series, and Backward counting. The BTACT was originally developed to
monitor the effects of aging on cognition; thus, it assesses wider neurocognitive function than dementia screening tools (Bodien et al., 2018; Dams-O’Connor,
Landau, Hoffman, & St De Lore, 2018). The administration time is 15–20 min. The BTACT has high validity with other pen & paper tests and has good
reliability, hence the in-person test version can be alternatively used when possible. The testing procedure includes accuracy checks and time of completion.
There are four alternate versions (1 original þ 3 alternatives) to minimize practice effects on repeated assessments. The subtests examine episodic memory,
working memory, reasoning, verbal fluency, and executive function and there is an option to calculate a composite score. The English, Spanish, and French
versions of the BTACT have been normed (Lachman, Agrigoroaei, Tun, & Weaver, 2013). Importantly, the BTACT can be accessed and used for free with
permission from the developer (versions A–B contact Dr. Lachman; versions C–D contact Dr. Silverberg).

Neurocognitive screens While the MoCA-5 could be a preferred choice for quick
screening in-person or in teleneuropsychology, domains such
Montreal Cognitive Assessment 5-Minute Protocol
as attention and executive functions are abbreviated. A solu-
(MoCA-5, Wong et al., 2015) is the short form of the
tion would be to consider the 22-point telephone Montreal
Montreal Cognitive Assessment (MoCA), which was origi-
Cognitive Assessment (T-MoCA, suggested cutoff = 18/19;
nally developed to screen for vascular cognitive impairment
Pendlebury et al., 2012), sometimes referred to as the
and dementia (Nasreddine et al., 2005; O’Driscoll &
“Blind MoCA” (Wittich et al., 2010). It essentially removes
Shaikh, 2017; Wong et al., 2015), but later research covers
the visually related items from the full MoCA, and thus could
various other neurological conditions (Hebert, Day,
cover the rest of the cognitive domains in all the languages in
Steriade, Tang-Wai, & Wennberg, 2017; Phabphal &
which MoCA has been validated. Its limitations, as pointed
Kanjanasatien, 2011; Rodrigues, Gouveia, & Bentes,
out by the test co-developers, are the lack of published vali-
2020). The four items of the shortened protocol cover atten-
dations with remote testing and norms for key groups of inter-
tion, verbal learning, and memory (with delayed recall),
est (Phillips et al., 2020). However, recent evidence supports
executive functions/language, and orientation. The advan-
the validity of remote administration of the TMoCA in an eth-
tage of the test is that it could be used in teleneuropsychol-
nically and economically diverse US community cohort
ogy. A shortcoming is that visuospatial abilities would not
(Katz et al., 2021).
be assessed. The full form has been translated and validated
Alternatively, we recommend The Brief Test of Adult
in 27 languages with most of them having norms provided
Cognition Telephone (BTACT; Tun & Lachman, 2006), though
(Mast & Gerstenecker, 2010). MoCA-5 is also available
it is important to note that this tool has been recommended only
with alternative versions in English, French, Italian, and
for research. See legend of Table 1 for further details.
Chinese. Its cultural sensitivity among racial and ethnic
minorities has been researched (Milani, Marsiske, Cottler,
Chen, & Striley, 2018; Milani, Marsiske, & Striley, 2019;
O’Driscoll & Shaikh, 2017). The test is freely accessible,
Cognitive symptoms
though test users are recommended to complete an official The Patient’s Assessment of Own Functioning (PAOFI) is a
online training and certification in order to administer and well-validated self-report questionnaire (Chelune, Heaton, &
interpret the MoCA and its various short forms. Lehman, 1986). The PAOFI covers cognitive domains such

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 7

as memory function, language, and communication, sensory, Quarantine status

and perceptual function, use of hands, and also provides a
Quarantine or no quarantine
summary score. The PAOFI has been translated into multiple
languages (HIV Neurobehavioral Research Program, 2020
see also Supplementary Material 1). Patient’s antibody status
Documented if possible
Smell/taste questionnaire
This is a very brief set of questions adapted from The Smell Patient’s alertness status
and Taste component of the National Health and Nutrition
Examination Survey (NHANES) 2013–2014, which can be As for a standard neuropsychological assessment
easily adapted/translated. The questionnaire is provided in
Supplementary Material 1.
Setting
Telehealth by video call, remote online testing, in-person/in-
Mental and psychological health questionnaires clinic (PPE), maximizing ventilation (e.g., open windows).
This step includes Global health assessment with the widely
used – MOS 36-Item Short-Form Health Survey (SF-36) and
the assessment of psychological health using the Depression, Testing type
Anxiety, and Stress (DASS-21) short form. Both instruments 5–20 min screens; 3–10 min questionnaires
have been translated and adapted in many languages.

Level of required training for administration and

RECOMMENDATION FOR HARMONIZATION scoring
LEVEL 2
Closely follow the available manual guidelines and use super-
General Assumptions vised administration training when indicated
Harmonization Level 2 (HL2) can be used as a first follow-up
assessment for post-acute infection. The set of measurement Control group
methods proposed at this level enables a more in-depth
SARS-CoV-2-negative individuals (no history of a positive
examination equivalent to a medium-size research battery,
test) can be recruited as the control group. Control group
which would also enable collaborative projects. Clinically,
should be matched on demographics, health characteristics,
it could also serve as a more in-depth screen. This harmoni-
quarantine, and hospitalization setting.
zation level also incorporates some flexibility for the tests’
Table 2 summarizes the HL2 protocol.
administration mode (telehealth and in-person) and attempts
to minimize the testing duration. At this level, harmonization
is achieved by recommending a set of selected tools, and rec- Recommended Measurement Methods
ommending the coverage of specific cognitive, sensory,
global, and psychosocial domain areas. Additionally, the The aim of HL2 is to examine the effects of COVID-19 on
availability of adaptations/translation and cross-cultural val- neurocognition, olfaction, taste, and psychological well-
idity is documented. At this level, objective olfaction testing being in greater detail. HL2 can assist in providing a more
is also recommended. robust estimate of the potential disease-related neurocogni-
tive impairment prevalence, but it cannot be considered a
comprehensive assessment. HL2 is a medium-size assess-
Recommendations for Application ment, with remote options (although with some caveats for
Patient’s infectious status cognitive computerized testing). Where a longitudinal study
has used HL1 as a study screen, HL2 outcome scores may be
Negative (HL2 testing is deferred until the patient has recov- adjusted for performance on HL1. For participants who are
ered). Eligible participants are no longer infectious as proven unable to perform computerized neurocognitive testing
with a SARS-CoV-2-negative result. HL2 should only be (e.g., because of lack of appropriate hardware), HL1 assess-
conducted when the patient is fully able to participate in ment protocol is recommended. At HL2, options for remote
testing. completion of questionnaires are also proposed. We recom-
mend clearly documenting the role of any informant in assist-
ing questionnaire completion. We also recommend that the
Time of testing for hospitalized patients
examiner dedicates some time with the participant/patient
Assessment of cognition should be completed close to the over the phone or face to face to clarify any responses on these
time of discharge. questionnaires as appropriate. Finally, at this level, we

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 8 L.A. Cysique, E. Łojek et al.

Table 2. Harmonization Level 2 protocol

Completion
# Domain Format time (min) Mode of administration

1 Demographic inventory CRF 5–10 Self

Can be aided by informant
Use COVID-19 Neuro Network CRF Questionnaire In-person or telehealth*
Use medical records
Use information from treating doctors
2 Medical history questionnaire includes CNS CRF 10–20
symptoms Questionnaire
Use COVID-19 Neuro Network CRF and
added scales
3 Neurocognitive testing Computerized standard 10–15 In-person
test:
Four options Telehealth is possible if carefully adapted, but
guidance from test developers is strongly
recommended
See also dedicated sections below
4 Smell and taste questionnaire Questionnaire 5–10 In-person or telehealth*
5 Cognitive symptoms questionnaire Questionnaire 4–15 Email or secure mailing of questionnaires may
be used*
6 Mental and Psychosocial health Questionnaire 5–20 Self
Can be aided by informant
7 Objective olfaction/taste testing Standard test: 3–5 In-person
Four options Can be adapted for telehealth
8 Literacy, quality of education, and premor- Standard test In-person
bid ability and additional neuropsycho- Can be adapted for telehealth
logical measures
9 Everyday activities Questionnaire 5 In-person or telehealth*
Email or secure mailing of questionnaires may
be used*
Self
Can be aided by informant
10 Performance validity Standard test 3–10 In-person or telehealth*

NB: the protocol material is available in Supplementary Material 2.

*Some information may be filled out by participants/patients and their informants at their convenience within 3 days of the neuropsychological exam.

recommend the inclusion of performance validity tests (see capture potential direct COVID-19 effects on the brain as
Supplementary File 5 for further guidance). COVID-19 is a well as potential indirect effects: (1) Attention/working
widespread condition affecting a wide range of people. memory; (2) Executive function; (3) Motor function; (4)
Ensuring that measurements of cognitive performance are Processing speed; and (5) Learning and memory. HL2 remote
valid is therefore essential. testing is possible through online self-administration, but we
recommend checking with the test providers whether this will
fit your study population. In case of conflict with national
Demographic inventory and medical history health guidelines on telehealth, or wide variability of
questionnaires Internet access and hardware suitability in your study popu-
lation, we advise that you conduct the neurocognitive testing
We recommend using the same protocol as for HL1 and com-
in person. The other option is to repeat the HL1 protocol via
plementing the basic demographic data with a more extensive
telephone, and the rest of the HL2 protocol using telehealth or
testing of premorbid abilities. See also Supplementary
in-person assessment. Thus, using some flexibility in your
Material 2.
protocol, you may be able to conduct a minority of tests/ques-
tionnaires in person and use telehealth for the remainder. The
rationale for the neurocognitive test selection includes (i)
Neurocognitive testing tools that are widely used with well-developed training man-
The cognitive domains of interest include those affected in a uals; (ii) tools used internationally that have several language
wide range of neurological and psychiatric conditions to versions with evidence of cross-cultural validity and for some

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 9

use in resource-limited settings; (iii) and tools that have good evidence of cross-cultural adaptation. Test details and access
criterion validity and test–retest reliability. Construct validity are described in Supplementary Material 2. The quickest
for standard neuropsychological tests was not retained as a olfaction tests may be adapted to remote online testing using
selection criterion, but is documented in Supplementary a webcam, plus mailing of the scratch and sniff cards.
Material 2. The computerized format was primarily chosen
to facilitate test administration (including by trained nonspe-
cialists), integrated data capture, and automatic scoring. The Mental and psychosocial health questionnaires
computerized format also facilitates multi-site studies. Lastly,
we considered the availability of large normative datasets for We recommend using the HL1 protocol and, time permit-
optimal interpretation of performance. Supplementary ting, adding a wider array of mental and psychosocial
Material 2 includes detailed information about the four neuro- health questionnaires (see Supplementary Material 2 for
cognitive computerized tests, all available on tablets/iPad: details). As per current mental health literature in
Test My Brain (TMB); Cogstate Computerized Battery; COVID-19, symptoms of PTSD, anxiety, depression,
NeuroScreen; and the NIH Toolbox Cognition and Motor and fatigue may be the most important to screen. Careful
Batteries. consideration of mental health risk is needed if sending
psychological questionnaires remotely; the scoring should
be immediately interpreted using remote technologies to
Literacy, quality of education, and premorbid ability flag and follow-up with patients at high risk of distress.
and additional neuropsychological measures
Literacy, quality of education, and premorbid abilities can
be documented via a demographic interview to which stan- Activities of Daily Living (functional) assessment
dard tests of reading or reasoning may be added. Careful (ADL)
consideration of the person’s native language and level
It may be useful to assess Instrumental Activities of Daily
of education is needed to interpret test performance. The
Living (IADL), particularly for hospitalized cohorts, which
study scope might require additional neuropsychological
typically have more severe COVID-associated neurological
tests, which we have also documented in Supplementary
symptoms. Indeed, it is important to document the everyday
Material 2. The Grooved Pegboard Test could be used
functioning relevance of any acquired neurocognitive
for motor functions or, alternatively, the 9-hole
impairment. This also represents as the first step toward
Pegboard Test is part of the recommended NIH Toolbox –
rehabilitation strategies when needed. Traditional tools
Motor.
for IADL assessment are based on a set of predetermined
activities, which may not be relevant to some individuals,
Cognitive symptoms depending on their gender, age, educational status, and spe-
cific activity engagement (Sikkes, de Lange-de Klerk,
Use HL1 protocol or consider other options provided in Pijnenburg, Scheltens, & Uitdehaag, 2009). Traditional
Supplementary Material 2. Specific consideration should IADL measures also have low cross-cultural validity and
be given to the timelines covered by these questionnaires, poor psychometric properties for both criterion validity of
which may not fit the timeline of an acute infection with a IADL impairment and detection of decline upon repeated
range of recovery such as COVID-19. testing (Sikkes et al., 2009). We, therefore, recommend
the use of recent instruments, which have addressed some
Smell/taste questionnaire of these challenges (see Supplementary Material 2). These
new instruments also have screening versions and several
We recommend the longer version questionnaire adapted language versions and offer methods for developing
from the Taste and Smell component of the NHANES cross-culturally validated versions (Dubbelman et al.,
2013–2014, which can be easily adapted/translated. This is 2020; Jutten et al., 2018).
provided in Supplementary Material 2.

Objective smell/taste testing RECOMMENDATIONS FOR

Olfactory disturbances are commonly observed in HARMONIZATION LEVEL 3
COVID-19. Therefore, at HL2, we recommend the objective
General Assumptions
testing of olfaction because it is well established that self-
report is not reliable, although this may not be the case for HL3 is akin to a standard, in-person comprehensive
an acute infection such as COVID-19. We recommend the neuropsychological assessment for which we recommend
use of standard tests, which were selected based on their val- a set of standard neuropsychological tests including per-
idity to determine anosmia and ageusia at various levels of formance validity tests. Objective olfaction and taste test-
granularity, the availability of good normative data, and some ing is also recommended.

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 10 L.A. Cysique, E. Łojek et al.

Recommendations for Application Recommended Measurement Methods

Patient’s infectious status The aim of HL3 is to examine in more detail the permanent,
long-term, and transient characteristics of COVID-19
HL3 testing is deferred until the patient recovers. Eligible par-
effects on neurocognitive functions. Such a comprehensive
ticipants are no longer infectious as proven with a SARS-
assessment is critical to establish a solid rehabilitation
CoV-2-negative result.
strategy in patients with moderate-to-severe neurologi-
cal/neuropsychological symptoms. HL3-in-person assess-
ment comprises a selection of well-known standard
Time of testing for hospitalized patients neuropsychological tests, in addition to the olfactory and
Assessment of cognition should be completed close to the taste tests described in HL2. The primary cognitive
time of discharge. domains of interest at HL3 are common to HL2, and so
HL2 and HL3 may be combined when desirable. HL3
remote testing options represent as a more robust estimate
Quarantine status of the potential disease-related neurocognitive impairment
prevalence at this stage of the disease than HL2 testing. If a
No quarantine. study has used HL1 as a screen or even HL2, HL3 outcome
scores can be adjusted for previous performance. For par-
ticipants who are unable to undergo computerized neuro-
Patient’s antibody status cognitive testing, the HL1 over-the-phone assessment
Should be documented if possible. protocol can be repeated. At HL3, depending on the level
of physical and possible cognitive difficulties a participant/
patient may experience, some or all questionnaires may be
Patient’s alertness status done at home, but we recommend clearly documenting the
role of any informant in assisting their completion. We also
As for a standard neuropsychological assessment. recommend that the examiner dedicates time with the par-
ticipant/patient face to face to clarify any responses on
these questionnaires as appropriate. For this more exten-
Setting sive assessment, we strongly recommend the inclusion
In-patient, in-clinic face-to-face (no/partial PPE), telehealth of performance validity tests (see Supplementary File 5).
may be used for parts of the assessment. For greater ecological validity, we also recommend the
use of prospective memory tests which can have a closer
relationship to functional changes compared to other cog-
Testing type nitive domains (Woods et al., 2008, 2012).

Comprehensive, 2–4 h sessions with breaks as appropriate.

Demographic inventory and medical history
questionnaires
Level of required training for administration and
scoring In line with the harmonization aim of our recommendations,
we advise using HL1/2 protocols and supplementing as
Clinical Neuropsychology training, psychometricians. appropriate (e.g., depending on your study/patient popula-
tion) with a more extensive assessment of demographics,
socioeconomic and cultural factors. Please consult
Control group Supplementary Material 2, where you will also find sugges-
SARS-CoV-2-negative individuals (no history of a positive tions on the assessment of premorbid abilities.
test) can be recruited as the control group. Control group
should be matched on demographics, health characteristics,
quarantine, and hospitalization setting. Neurocognitive testing
Supplementary Material 3 presents a detailed description of a
standard neuropsychological battery. This covers core
Sanitary considerations for an in-person examination
domains for HL1 and HL2 and goes well beyond to cover
Mask and gloves should be used when appropriate in a dedi- the complex neurological syndromes that have been described
cated room which would be disinfected after each patient. whether directly due to COVID-19 or due to associated and
Test materials would also need to be disinfected (see underlying comorbidities. Addition of specific tests is war-
Postal et al., 2021). ranted for other patient populations who have also been diag-
Table 3 summarizes the HL3 protocol. nosed with COVID-19 (e.g., Parkinson’s disease).

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 11

Table 3. Harmonization Level 3 protocol

Completion
# Domain Format time (min) Mode of administration

1 Demographic inventory CRF 10–30 Self

Always consider informant
Use COVID-19 Neuro Network CRF Questionnaire In-person
Telehealth may be used*
Adapt/include socioeconomic and Use medical records
cultural factors Use information from treating doctors
2 Medical history questionnaire CRF 10–60
includes CNS symptoms Questionnaire
Use COVID-19 Neuro Network CRF
and added scales
Supplement with history taking
3 Neurocognitive testing Standard neuropsychological 60–90 In-person
test battery Part of the assessment may be adapted for tele-
Combine with HL1 and HL2 health, but guidance from test developers is
as appropriate strongly recommended
(see also dedicated section on telehealth)
4 Smell and taste questionnaire Questionnaire 5–10 In-person
Telehealth may be used*
5 Cognitive symptoms questionnaire Questionnaire 5–20 Self
Can be aided by informant
Supplement with history taking Email or secure mailing of questionnaires may
be used*
6 Mental and Psychosocial health Questionnaire 5–30
Supplement with history taking to
target key information and select
more questionnaires.
7 Objective olfaction/taste testing Standard test: 5–20 In-person
Four options Can be adapted for telehealth for some tests
8 Literacy, quality of education & pre- Standard test 5–10 In-person
morbid ability Can be adapted for telehealth for some tests
9 ADL Questionnaire 5–30 In-person
Supplement with history taking to þInformant Telehealth may be used*
target key information and select Email or secure mailing of questionnaires may
more questionnaires. be used*
Self þ Should be aided by informant
10 Performance validity Standard test 3–10 In-person or telehealth*

NB: the protocol material is available in Supplementary Materials 2 and 3.*Some information may be filled out by participants/patients and their informants at
their convenience within 3 days of the neuropsychological exam.

Literacy, quality of education, and premorbid ability Objective smell/taste testing

Use HL2 protocol and see Supplementary Material 2. Use HL2 protocol. The short or long version of the proposed
assessments could be used, depending on your study ques-
tions (e.g., focusing on perception rather than cognition), time
Cognitive symptoms constraints, and participant/patient engagement and fatigue.
Use HL1 protocol and consider other options provided in
Supplementary Material 2. The history taken prior to the
comprehensive assessment is important to consider and to Mental and psychosocial health questionnaires
allow a nuanced interpretation of responses to the question- We recommend including HL1 and HL2 protocols (see
naires (particularly with regard to symptom timelines). Supplementary Material 2). Depending on study-related fac-
tors (e.g., focusing on mental health more than cognition; dif-
ferent study populations) or patient-related factors (e.g., time
Smell/taste questionnaire constraints, engagement, and fatigue), you may select more
Use HL2 protocol. targeted mental and psychosocial health questionnaires.

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 12 L.A. Cysique, E. Łojek et al.

IADL complete online assessments. These issues must be care-

fully considered during study design. Availability and
For HL3, ADL assessment is strongly recommended. Consult
affordability of study measures vary across countries and
HL2 information above and Supplementary Material 2.
so, if possible, tests that are in the public domain should
be used. With regard to test use and adaptation, if the rec-
HARMONIZATION LEVELS: CROSS- ommended tools have not been adapted, existing original
or adapted tests that measure the same construct should
CULTURAL AND DISPARITIES ISSUES
be used. If no adapted/validated measure is available, best
The recommendations attempt to deal a priori with the practice guidelines for test adaptation and translation
international aspect of the epidemic. In this section, we, should be followed (e.g., a committee/team translation
therefore, provide guidance on how best to use the recom- or forward- and back translation (Harkness & Schoua-
mended tests across diverse populations. Although our rec- Glusberg, 1998; Vallejo-Medina et al., 2017). Test pub-
ommended tests are used internationally, cross-cultural lishers must be contacted timeously for permission to
appropriateness and availability of tests are crucial. translate tests. All measures (e.g., mental health, medical
Within local contexts, tests should be selected and adminis- history, etc.) must be translated and adapted to reflect
tered considering the background characteristics of the tar- the regional language and cultural practices.
get population to avoid violating fairness in testing The recommended neuropsychological measures
(Aghvinian et al., 2020; International Test Commission, included in HL 1–3 to a large degree reflect the neuro-
2019). Cultural and sociodemographic factors (sex, age, psychological test battery widely used in HIV studies, both
education, ethnicity, socioeconomic status) impact neuro- in high-income countries (HIC) and LMIC (Kabuba, Anitha
psychological test performance (Brickman, Cabo, & Menon, Franklin, Heaton, & Hestad, 2017; Nyamayaro,
Manly, 2006). Neuropsychological tests must therefore be Chibanda, Robbins, Hakim, & Gouse, 2019), suggesting
culturally appropriate with regard to language use and test the applicability of tests across diverse settings. There
stimulus items. Age- and education-appropriate norms are are, however, some considerations to keep in mind when
necessary to determine whether a person’s performance falls selecting tests for the purpose of describing the neurocogni-
outside the normal range (Fernandez, 2019; Mitrushina, tive presentation associated with COVID-19. In particular,
Boone, Razani, & D’Elia, 2005). Where such normative tests that measure cognitive constructs and global levels of
data are not available, a well-matched control sample is functional capacity must be culturally valid. Below, we pro-
required (Casaletto & Heaton, 2017). These issues are par- vide further comments on particular measures from HL 1–3
ticularly pertinent in low- and middle-income countries that may require cross-cultural adaptation in some settings
(LMIC) where few neuropsychological measures have been (Table 4); for computerized tests, see also Supplementary
adapted and validated, and normative data are scarce. Material 2.
Key issues for the implementation of the current recom-
mendations across settings include access to human resour-
ces and expertise, technological and socioeconomic HARMONIZATION LEVELS: NORMS,
considerations, and availability and adaptation of study IMPAIRMENTS RATINGS, AND REPEATED
measures. Access to human resources and expertise varies NEUROPSYCHOLOGICAL TESTING
between and within countries. Where there is a lack of Practitioners should carefully follow the standard scoring
expert- and human resources (e.g., trained neuropsycholo- instructions and guidance for interpretation of all the tests,
gists) in LMIC, clinical, or general psychologists may be using standard materials obtained from accredited test provid-
involved. Laypeople can also be trained to do assessments ers. Practitioners are responsible for determining whether the
under supervision by a psychologist, allowing the delega- tools we have recommended are either in the public domain
tion to less specialized health care- or lay-workers, particu- and thus free to use and reproduce, or whether the tools need
larly when combined with automated, easy-to-use tests that to be purchased from accredited providers, some of which
can be performed on a phone or a tablet (e.g., NeuroScreen; require specific qualifications to access.
Magidson et al., 2017; Robbins et al., 2018). Mental health
screening must similarly be supervised by a clinical/neuro-
psychologist or psychiatrist familiar with the local setting.
Distance supervision applies in locations without direct
Norms
access to specialists, in line with the current Taskforce
guidelines. We recommend the use of published nationally representative
Access to technology and connectivity also varies normative data appropriate to your study sample age, educa-
across settings and the use of mobile health applications tion/SES, and sex, in addition to race/ethnicity and rural/
must be viewed in light of available resources. In LMIC urban living when possible. When nationally representative
access to computers or tablets may be limited, for example, normative data are not available, we recommend using a dem-
but access to smartphones is ubiquitous. The high cost of ographically and geographically comparable control sample
mobile data in some settings may limit the ability to and, if capacity and expertise permits, developing norms.

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 13

Table 4. Cross-cultural considerations and recommendations for instruments proposed in HL 1–3

Domains Tools Considerations and recommendations

Cognitive screeners (HL 1–2) MOCA-5/T-MoCA þ More culturally appropriate telephonic cognitive screening measure.
− Recommended over BTACT.
BTACT þ Relies partially on instruments with cross-cultural validity testing (e.g.,
RAVLT; Digit backwards).
− Number Series task compromised validity for some LMIC populations
(social and formal educational differences).
− Must adjust the BTACT cutoff score when some tasks are not suitable for
inclusion (e.g., Number Series task).
Functional Screeners (HL 1–2) Activities of Daily Living þ ADLs are culturally bound and vary significantly between and within settings.
(ADL) Scales þ Local/national tools are more suitable than global measures, if available
(Pashmdarfard & Azad, 2020).
þ Lawton Instrumental Activities of Daily Living Scale (IADL) has several
cultural adaptations (Dubbelman et al., 2020; Ng, Niti, Chiam, & Kua,
2006; Siriwardhana, Walters, Rait, Bazo-Alvarez, & Weerasinghe, 2018;
Stone et al., 2018).
− If not available, develop and validate tools using Siestke et al. methods.
Standard neuropsychological BVMT-R (visuospatial − Some subjectivity associated with scoring with an inter-rater agreement of
tests (HL 2–3) learning and memory) ∼60% (Caneda, Cuervo, Marinho, & Vecino, 2018).
− Must adhere strictly to standard scoring guidelines to reduce bias.
HVLT-R (verbal learning − A culturally and linguistically appropriate version of the HVLT-R must be
and memory) used
− In the absence of an appropriate version of the HVLT-R, a culturally suit-
able substitute e.g., Rey Auditory Verbal Learning Test (RAVLT),
California Verbal Learning Test (CVLT), or Free and Cued Selective
Reminding Test (FCSRT) (Lim et al., 2009) can be used.
Category fluency þ More normative data exist for the Animal category than the Fruits and
Vegetables category. The former is therefore recommended.
Premorbid ability − LMIC lacks normative data to reliably estimate premorbid intelligence.
− Within countries, disparities in terms of socioeconomic status must be taken
into account (Shuttleworth-Edwards, 2016) in the validation processes of
these measures.
− Premorbid IQ should only be assessed if appropriate normative data are
available accounting for age, education, and socio-economic status (SES)
status.
Performance Validity − See Supplementary File 5.
− Collection of data in appropriate control group will be needed in many
locations as cross-cultural versions of such tests are lacking

Controls literature using those tests to determine standard levels of def-

icits. Reporting the level of “neurocognitive impairment” in
Data collection in a local, demographically representative,
controls is advised for transparency and better interpretation
and healthy control group is recommended. The published
of the burden of the disease in COVID-19 samples.
norms should be checked in your local sample to assess if
they “work”, that is, whether they correct for demographic
effects. Depending on the study question, controls may also Smell/taste impairment levels
be from a clinical comparison group, for example, patients
who have been through ICU. The current recommendations include both questionnaires
and smell/taste tests. The suggested measures are commonly
used in general as well as clinical populations. They can be
Neurocognitive impairment levels
used to quantify smell loss or describe the severity of alter-
In research studies, methods to determine levels of cognitive ation caused by COVID-19. While we selected tests with
impairment (e.g., as cutoff scores on screens, or normative available norms in several countries, it is still possible that
standard scores on one or more neuropsychological tests) these norms may not be appropriate for your population. In
should be clearly described and linked to a well-established this case, we recommend that you compare results from your
nomenclature of performance levels. Extra attention should SARS-CoV-2-positive sample to a demographically compa-
be given to computerized cognitive tests, and associated rable SARS-CoV-2-negative or asymptomatic control group,

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 14 L.A. Cysique, E. Łojek et al.

and/or longitudinally in order to track within-patient changes describe the potential impact of the administration procedure,
across the stages of the disease. Diagnosing impairment and its alterations, on the proposed diagnosis and (if appli-
should be done with caution and follow the standard impair- cable) recommended treatment.
ment grading of the original norms.

GUIDELINES FOR HOME/REMOTE

Repeated Neuropsychological testing
ASSESSMENT TO SUPPORT DATA FIDELITY
COVID is an evolving condition that starts with an acute AND TELEHEALTH CONSIDERATIONS
infection phase, whether symptomatic or not. It is likely that
a large part of the forthcoming research will be longitudinal to Although there are no formal published standards for remote
assess disease recovery on several occasions. Because of this, assessment and telehealth in neuropsychology, several
longitudinal data analysis and considerations of issues asso- national organizations have issued professional practice guide-
ciated with neuropsychological repeated testing will be criti- lines in recent months (American Psychological Association,
cal to characterize the neurocognitive complications of 2020a; Interorganizational Practice Committee, 2020; The
COVID-19. Please consult Supplementary File 6 for further British Psychological Society, 2020). Key points from these
guidance. guidelines are summarized in Supplementary Material 4.
The APA has published a useful (though US-focused) check-
list to help practitioners prepare for clinical sessions with these
Consideration of contributing, confounding, and considerations in mind (American Psychological Association,
incidental medical, psychological, lifestyle, and 2020b). Practitioners must adhere to test publisher rules
demographic factors (beyond the norms corrections) regarding copyright and sharing of materials (e.g., Pearson
Assessment, 2020).
We advise carefully documenting any preexisting (e.g., sys-
A meta-analysis of teleneuropsychology administration
temic, immune, neurological, or psychiatric) or comorbid
compared with in-person (Brearly et al., 2017) found that
(e.g., stroke, hypoxia, lung disease) conditions to determine
the difference between videoconference and in-person perfor-
to what extent they may impact on neurocognitive, sensori-
mance was very small (Hedges g = −0.03), and not sta-
motor, and psychosocial health. Besides traditional demo-
tistically or clinically significant. Results were less
graphics, it is important to note whether the participant is
consistent in patients aged over 75 and in situations with
literate or may have been diagnosed with a learning disability.
slower Internet connection speed. The authors concluded that
These various factors should be carefully documented, and
videoconference administration of verbally mediated tasks by
their effects tested as appropriate.
qualified professionals using existing norms was supported,
and the use of visually dependent tasks may also be consid-
Feedback reports for research ered, but motor-based tasks require further investigation.
Lastly of interest, is a newly proposed telehealth battery,
It is advised to produce individual feedback reports when
which greatly overlaps with our HL3 recommendations,
conducting a research study. Such reports should ideally be
but also include testing of prospective memory
sent to the participant’s doctor of choice so that the informa-
(Matchanova et al., 2020).
tion is interpreted in the relevant clinical context. Reports
should provide a detailed description of tests/questionnaires
administered, modality of testing (in-person; remote: over-
CONCLUDING REMARKS AND FUTURE
the-phone/computer-based), and involvement of the
DIRECTIONS
research/clinical personnel (personnel present in-person/
remotely; or self-administered by the participant/informant). The long-term impact for COVID-19 survivors in the months
For remote assessment, reports should additionally provide and years post-recovery is, as yet, unknown; however, there
information on the testing platform and describe nonstandard are suggestions that based on the prevalence of critical illness
administration procedures and related limitations (e.g., lim- alone, post-COVID-19 long-term cognitive impairment will
ited understanding of participant’s vision, hearing or level be significant in some patients (Needham, Chou, Coles, &
of familiarization with testing devices). Research reports Menon, 2020). Neuropsychologists will benefit from
employing (at least a part of) the currently recommended approaching assessment and rehabilitation of individuals
neuropsychological protocols are welcome to include a cita- after COVID-19 from a holistic point of view, considering
tion of this paper. However, the description of the testing pro- cognition, emotional functioning, behavior, and potential so-
tocol should still be provided in order to enable comparisons cioeconomic pandemic impact as interacting variables that
with other sites. Additionally, references can be made directly impact functional independence, quality of life, and emo-
to the Harmonization Levels 1–3 (basic/full; in-person/ tional well-being. It is with this framework in mind that
remote) and selected measures. Citation will further enhance the current recommendations have been prepared.
the visibility of the original research papers and support The NeuroCOVID Neuropsychology International
building comparable databases for future meta-analysis and Taskforce will promote these recommendations through
between-site data sharing. Clinical reports should further our research and collaborations. The group anticipates that

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 15

the recommendations will facilitate multi-site and • Mr. Theodore C.K. Cheung is supported by the Ontario
international collaborations and we encourage colleagues Graduate Scholarship, Canada.
from HIC to develop studies that assist research in LMIC • Dr. Gouse is supported by the Fogarty International Center
when appropriate. Implementation research regarding the 1K43TW010361–01.
• Dr. Silva (A.R.) is supported by the Portuguese Agency of
acceptability, usability, and validity of the recommendations
Science and Technology.
will be critical to their uptake and the Taskforce welcomes
feedback on potential improvements and adjustments to
inform refinement of the recommendations. It is important CONFLICTS OF INTEREST
to note that these recommendations apply only to adult The authors have nothing to disclose.
research and practice; analogous recommendations for neuro-
psychological research with children infected with SARS-
CoV-2 are urgently needed. References
The Taskforce will start the development of a minimum Abootalebi, S., Aertker, B.M., Andalibi, M.S., Asdaghi, N., Aykac,
dataset and associated code book protocol, including a pro- O., Azarpazhooh, M.R., : : : Zand, R. (2020). Call to Action:
posal for protocol registration. This minimum dataset will SARS-CoV-2 and Cerebrovascular DisordErs (CASCADE).
first be based on the lowest common denominator as devel- Journal of Stroke and Cerebrovascular Diseases, 104938. doi:
oped in the recommendations (i.e., HL 1) and we hope to then 10.1016/j.jstrokecerebrovasdis.2020.104938
Aghvinian, M., Santoro, A.F., Gouse, H., Joska, J.A., Linda, T.,
include other harmonization levels. This effort will include
Thomas, K.G.F., & Robbins, R.N. (2020). Taking the test: A
secure online data storage and good practice guidelines for qualitative analysis of cultural and contextual factors impacting
the participating sites. It is planned that individual researchers neuropsychological assessment of Xhosa-Speaking South
will access the database after contacting the coordinators and Africans. Archives of Clinical Neuropsychology. doi: 10.1093/
proposing the analysis to be conducted. Funding will be arclin/acaa115
sought for the development and maintenance of the database. Almeria, M., Cejudo, J.C., Sotoca, J., Deus, J., & Krupinski, J.
(2020). Cognitive profile following COVID-19 infection:
Clinical predictors leading to neuropsychological impairment.
SUPPLEMENTARY MATERIAL
Brain, Behavior, & Immunity - Health, 9, 100163. doi: 10.
To view supplementary material for this article, please visit 1016/j.bbih.2020.100163
https://doi.org/10.1017/S1355617721000862. American Psychological Association. (2020a). Guidelines for the
practice of telepsychology. Retrieved from https://www.apa.
org/practice/guidelines/telepsychology
ACKNOWLEDGMENTS American Psychological Association. (2020b). Office and technol-
ogy checklist for telepsychological services. Retrieved from
The authors would like to thank all members of the https://www.apa.org/practice/programs/dmhi/research-information/
NeuroCOVID International Taskforce. New members are wel- telepsychological-services-checklist
come, and membership enquiries can be sent to Dr. Cysique Bialek, S., Bowen, V., Chow, N., Curns, A., Gierke, R., Hall, A., : : :
([email protected]) and Dr. Łojek ([email protected]. Wen, J. (2020). Geographic differences in COVID-19 cases,
edu.pl) by email. The group works from the Slack platform deaths, and incidence — United States, February 12–April 7,
at: https://neurocovidinssig.slack.com/?redir=%2Fgantry% 2020. Morbidity and Mortality Weekly Report, 68(17). doi: 10.
2Fclient#/. The Taskforce is also represented at the 15585/mmwr.mm6915e4
International Neuropsychological Society (INS) as a Special Bilder, R.M., Postal, K.S., Barisa, M., Aase, D.M., Cullum, C.M.,
Interest Group (https://www.the-ins.org/sigs/). The statement Gillaspy, S.R., : : : Woodhouse, J. (2020). InterOrganizational
practice committee recommendations/guidance for teleneuropsy-
that the opinions/data/perspectives belong solely to the authors
chology in response to the COVID-19 pandemic. Archives of
and the NeuroCOVID International Taskforce and may not
Clinical Neuropsychology, 35(6), 647–659. doi: 10.1093/arclin/
necessarily represent the viewpoint of INS. We thank Dr. acaa046
Susan McPherson for bringing to our attention about the ques- Bo, H.-X., Li, W., Yang, Y., Wang, Y., Zhang, Q., Cheung, T., : : :
tion of performance validity tests. Xiang, Y.-T. (2020). Posttraumatic stress symptoms and attitude
toward crisis mental health services among clinically stable
patients with COVID-19 in China. Psychological Medicine.
FINANCIAL SUPPORT doi: 10.1017/S0033291720000999
Bodien, Y.G., McCrea, M., Dikmen, S., Temkin, N., Boase, K.,
• Dr. Cysique is supported for her NeuroCOVID research by Machamer, J., : : : Investigators, T.-T. (2018). Optimizing out-
the Peter Duncan Neuroscience Unit, St. Vincent’s Applied come assessment in multicenter TBI trials: Perspectives from
Medical Research Centre, Sydney, Australia, and the TRACK-TBI and the TBI Endpoints Development Initiative.
Alfred Hospital in Melbourne, Australia. The Journal of Head Trauma Rehabilitation, 33(3), 147–157.
• Drs. Łojek, Hansen, Holas, and Malinowska are supported doi: 10.1097/HTR.0000000000000367
by the Faculty of Psychology, Warsaw University, Poland. Brain Infection Global COVID-Neuro Network. (2021). Neurology
• Dr. Muñoz-Moreno is supported by the Fundació Lluita Case Report Form (CRF) access per registration. Retrieved from
contra la SIDA, Germans Trias i Pujol University https://braininfectionsglobal.tghn.org/covid-neuro-network/ accessed
Hospital, Barcelona, Catalonia, Spain. Februray 1st, 2021.

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 16 L.A. Cysique, E. Łojek et al.

Brearly, T.W., Shura, R.D., Martindale, S.L., Lazowski, R.A., Hampshire, A., Trender, W., Chamberlain, S.R., Jolly, A., Grant,
Luxton, D.D., Shenal, B.V., & Rowland, J.A. (2017). J.E., Patrick, F., : : : Mehta, M.A. (2020). Cognitive deficits in
Neuropsychological test administration by videoconference: A people who have recovered from COVID-19 relative to controls:
systematic review and meta-analysis. Neuropsychology Review, An N=84,285 online study. medRxiv. doi: 10.1101/2020.10.20.
27(2), 174–186. doi: 10.1007/s11065-017-9349-1 20215863
Brickman, A.M., Cabo, R., & Manly, J.J. (2006). Ethical issues in Harkness, J. and Schoua-Glusberg, A. (1998). Questionnaires in
cross-cultural neuropsychology. Applied Neuropsychology, translation. In J. Harkness (Ed.), Cross-cultural survey equiva-
13(2), 91–100. doi: 10.1207/s15324826an1302_4 lence (pp. 87–126). Mannheim: Zentrum für Umfragen,
Caneda, M.A.G.d., Cuervo, D.L.M., Marinho, N.E., Vecino, Methoden und Analysen -ZUMA-. Retrieved from https://nbn-
M.C.A.d. (2018). The reliability of the Brief Visuospatial resolving.org/urn:nbn:de:0168-ssoar-49733-1.
Memory Test - Revised in Brazilian multiple sclerosis patients. Hebert, J., Day, G.S., Steriade, C., Tang-Wai, D.F., & Wennberg, R.
Dementia & Neuropsychologia, 12, 205–211. (2017). Retrospective study of neurocognitive outcomes in auto-
Casaletto, K.B. and Heaton, R.K. (2017). Neuropsychological immune encephalitis. Neurology, 88(16 Supplement),
assessment: Past and future. Journal of the International P6.319–P316.319.
Neuropsychological Society, 23(9–10), 778–790. doi: 10.1017/ Helms, J., Kremer, S., Merdji, H., Clere-Jehl, R., Schenck, M.,
S1355617717001060 Kummerlen, C., : : : Meziani, F. (2020). Neurologic features in
Chelune, G.J., Heaton, R.K., & Lehman, R. . (1986). severe SARS-CoV-2 infection. New England Journal of
Neuropsychological and personality correlates of patients’ com- Medicine, 382(23), 2268–2270. doi: 10.1056/NEJMc2008597
plaints of disability. In: G. Goldstein & R.E. Tarter (Eds.), HIV Neurobehavioral Research Program. (2020). HIV
Advances in Clinical Neuropsychology (pp. 95–118). New Neurobehavioral Research Program. Retrieved from https://
York: Plenum Press. hnrp.hivresearch.ucsd.edu/
Dams-O’Connor, K., Landau, A., Hoffman, J., & St De Lore, J. International Test Commission (2019). ITC guidelines for the large-
(2018). Patient perspectives on quality and access to healthcare scale assessment of linguistically and culturally diverse popula-
after brain injury. Brain Injury, 32(4), 431–441. doi: 10.1080/ tions. International Journal of Testing, 19(4), 301–336. doi:
02699052.2018.1429024 10.1080/15305058.2019.1631024
Darley, D.R., Dore, G.J., Cysique, L., Wilhelm, K.A., Andresen, D., Interorganizational Practice Committee. (2020). Guidelines for the
Tonga, K., : : : Masters, J. (2020). High rate of persistent symp- practice of teleneuropsychology. Retrieved from https://iopc.
toms up to 4 months after community and hospital-managed online/teleneuropsychology-guidelines
SARS-CoV-2 infection. The Medical Journal of Australia. Jutten, R.J., Harrison, J., Lee Meeuw Kjoe, P.R., Opmeer, E.M.,
De Felice, F.G., Tovar-Moll, F., Moll, J., Munoz, D.P., & Ferreira, S.T. Schoonenboom, N.S. ., de Jong, F.J., : : : Sikkes, S.A.M.
(2020). Severe Acute Respiratory Syndrome Coronavirus 2 (SARS- (2018). A novel cognitive-functional composite measure to detect
CoV-2) and the central nervous system. Trends in Neuroscience, changes in early Alzheimer’s disease: Test-retest reliability and
43(6), 355–357. doi: 10.1016/j.tins.2020.04.004 feasibility. Alzheimer’s & Dementia, 10, 153–160. doi: 10.
Dubbelman, M.A., Verrijp, M., Facal, D., Sánchez-Benavides, G., 1016/j.dadm.2017.12.002
Brown, L.J.E., van der Flier, W.M., : : : Sikkes, S.A.M. Kabuba, N., Anitha Menon, J., Franklin, D.R., Heaton, R.K., &
(2020). The influence of diversity on the measurement of func- Hestad, K.A. (2017). Use of Western neuropsychological test bat-
tional impairment: An international validation of the tery in detecting HIV-Associated Neurocognitive Disorders
Amsterdam IADL Questionnaire in eight countries. Alzheimer’s (HAND) in Zambia. AIDS and Behavior, 21(6), 1717–1727.
& Dementia, 12(1), e12021. doi: 10.1002/dad2.1202 doi: 10.1007/s10461-016-1443-5
Ellul, M., Benjamin, L., Singh, B., Lant, S., Michael, B., Kneen, R., Kaseda, E.T. and Levine, A.J. (2020). Post-traumatic stress disorder:
: : : Solomon, T. (2020). Neurological associations of COVID- A differential diagnostic consideration for COVID-19 survivors.
19. Lancet Neurology, 19(9), 767–783. doi: 10.1016/S1474- The Clinical Neuropsychologist, 34(7–8), 1498–1514. doi.org/
4422(20)30221-0. 10.1080/13854046.2020.1811894
Ettman, C.K., Abdalla, S.M., Cohen, G.H., Sampson, L., Vivier, Katz, M.J., Wang, C., Nester, C.O., Derby, C.A., Zimmerman, M.E.,
P.M., & Galea, S. (2020). Prevalence of depression symptoms Lipton, R.B., : : : Rabin, L.A. (2021). T-MoCA: A valid phone
in US adults before and during the COVID-19 pandemic. screen for cognitive impairment in diverse community samples.
JAMA Network Open, 3(9), e2019686. doi: 10.1001/ Alzheimers Dement (Amst), 13(1), e12144. doi: 10.1002/dad2.
jamanetworkopen.2020.19686 12144
Fernandez, A.L. (2019). Modern neuropsychological tests for a Koralnik, I.J. and Tyler, K.L. (2020). COVID-19: A global threat to
diversity of cultural contexts. The Clinical Neuropsychologist, the nervous system. Annals of Neurology, 88(1), 1–11. doi: 10.
33(2), 438–445. doi: 10.1080/13854046.2018.1560501 1002/ana.25807
Fonseca, L., Diniz, E., Mendonça, G., Malinowski, F., Mari, J., & Kotfis, K., Williams Roberson, S., Wilson, J.E., Dabrowski, W.,
Gadelha, A. (2020). Schizophrenia and COVID-19: Risks and Pun, B.T., & Ely, E. W. (2020). COVID-19: ICU delirium man-
recommendations. Brazilian Journal of Psychiatry, 42, 236–238. agement during SARS-CoV-2 pandemic. Critical Care, 24(1).
Frontera, J., Mainali, S., Fink, E.L., Robertson, C.L., Schober, M., doi: 10.1186/s13054-020-02882-x
Ziai, W., : : : Study, G.C.-N. (2020). Global Consortium Study of Kozloff, N., Mulsant, B.H., Stergiopoulos, V., & Voineskos,
Neurological Dysfunction in COVID-19 (GCS-NeuroCOVID): A.N. (2020). The COVID-19 global pandemic: Implications
Study design and rationale. Neurocritical Care, 33(1), 25–34. for people with schizophrenia and related disorders.
doi: 10.1007/s12028-020-00995-3 Schizophrenia Bulletin, 46(4), 752–757. doi: 10.1093/
Gane, S.B., Kelly, C., & Hopkins, C. (2020). Isolated sudden onset schbul/sbaa051
anosmia in COVID-19 infection. A novel syndrome? Rhinology, Lachman, M.E., Agrigoroaei, S., Tun, P.A., & Weaver, S.L. (2013).
58(3), 299–301. doi: 10.4193/Rhin20.114 Monitoring cognitive functioning: Psychometric properties of the

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 17

Brief Test of Adult Cognition by Telephone. Assessment, 21(4), Mitrushina, M., Boone, K.B., Razani, J., & D’Elia, L.F. (2005).
404–417. doi: 10.1177/1073191113508807 Handbook of normative data for neuropsychological assessment.
Laurencin, C.T. and McClinton, A. (2020). The COVID-19 pan- Oxford, UK: Oxford University Press.
demic: A call to action to identify and address racial and ethnic Nasreddine, Z.S., Phillips, N.A., Bedirian, V.r., Charbonneau, S.,
disparities. Journal of Racial and Ethnic Health Disparities, Whitehead, V., Collin, I., & Chertkow, H. (2005). The Montreal
7(3), 398–402. doi: 10.1007/s40615-020-00756-0 Cognitive Assessment, MoCA: A brief screening tool for mild
Levine, A., Sacktor, N., & Becker, J. T. (2020). Studying the neuro- cognitive impairment. Journal of the American Geriatrics Society,
psychological sequelae of SARS-CoV-2: Lessons learned from 53(4), 695–699. doi: 10.1111/j.1532-5415.2005.53221.x
35 years of neuroHIV research. Journal of Neurovirology. doi: Needham, E.J., Chou, S.H.-Y., Coles, A.J., & Menon, D.K. (2020).
10.1007/s13365-020-00897-2 Neurological implications of COVID-19 infections. Neurocritical
Lim, Y.Y., Prang, K.H., Cysique, L., Pietrzak, R.H., Snyder, P.J., & Care, 32(3), 667–671. doi: 10.1007/s12028-020-00978-4.
Maruff, P. (2009). A method for cross-cultural adaptation of a ver- Ng, T.-P., Niti, M., Chiam, P.-C., & Kua, E.-H. (2006). Physical and
bal memory assessment. Behavior Research Methods, 41(4), cognitive domains of the Instrumental Activities of Daily Living:
1190–1200. doi: 10.3758/BRM.41.4.1190 Validation in a multiethnic population of Asian older adults. The
Magidson, J.F., Gouse, H., Psaros, C., Remmert, J.E., O’Cleirigh, Journals of Gerontology: Series A, 61(7), 726–735. doi: 10.1093/
C., & Safren, S.A. (2017). Task shifting and delivery of behav- gerona/61.7.726
ioral medicine interventions in resource-poor global settings: Nyamayaro, P., Chibanda, D., Robbins, R.N., Hakim, J., & Gouse,
HIV/AIDS treatment in sub-Saharan Africa. In The H. (2019). Assessment of neurocognitive deficits in people living
Massachusetts General Hospital Handbook of Behavioral with HIV in Sub Saharan Africa: A systematic review. The
Medicine (pp. 297–320). Cham: Humana Press. Clinical Neuropsychologist, 33(sup1), 1–26. doi: 10.1080/
Mao, L., Jin, H., Wang, M., Hu, Y., Chen, S., He, Q., : : : Hu, B. 13854046.2019.1606284
(2020). Neurologic manifestations of hospitalized patients with O’Driscoll, C. and Shaikh, M. (2017). Cross-cultural applicability of
coronavirus disease 2019 in Wuhan, China. JAMA Neurology, the Montreal Cognitive Assessment (MoCA): A systematic
77(6), 683–690. doi: 10.1001/jamaneurol.2020.1127 review. Journal of Alzheimer’s Disease, 58(3), 789–801. doi:
Marcopulos, B.A. (2018). Neuropsychological functioning 10.3233/JAD-161042
in affective and anxiety-spectrum disorders in adults and Pashmdarfard, M. and Azad, A. (2020). Assessment tools to evalu-
children. In J. Morgan & J. Ricker (Eds.), Textbook of clinical ate Activities of Daily Living (ADL) and Instrumental Activities
neuropsychology (2nd ed.). New York: Taylor & Francis, of Daily Living (IADL) in older adults: A systematic review.
pp. 701–716. Medical Journal of The Islamic Republic of Iran, 34(1), 224–
Marra, A., Pandharipande, P.P., & Patel, M.B. (2017). Intensive 239. doi: 10.34171/mjiri.34.33
Care Unit Delirium and Intensive Care Unit-Related Paterson, R.W., Brown, R.L., Benjamin, L., Nortley, R., Wiethoff, S.,
Posttraumatic Stress Disorder. The Surgical Clinics of North Bharucha, T., : : : Zandi, M.S. (2020). The emerging spectrum of
America, 97(6), 1215–1235. doi: 10.1016/j.suc.2017.07.008 COVID-19 neurology: Clinical, radiological and laboratory find-
Mast, B.T. and Gerstenecker, A. (2010). In P.A. Lichtenberg (Ed.), ings. Brain, 143(10), 3104–3120. doi: 10.1093/brain/awaa240
Handbook of assessment in clinical gerontology (pp. 503–530). Pearson Assessments. (2020). Clinical and classroom assessment
Elsevier Academic Press. doi: 10.1016/B978-0-12-374961-1. products. Retrieved from https://www.pearsonassessments.com/
10019-3 professional-assessments.html
Matchanova, A., Babicz, M.A., Medina, L.D., Rahman, S., Johnson, Pendlebury, S.T., Welch, S.J.V., Cuthbertson, F.C., Mariz, J.,
B., Thompson, J.L., : : : Woods, S.P. (2020). Latent Structure of a Mehta, Z., & Rothwell, P.M. (2012). Telephone assessment of
Brief Clinical Battery of Neuropsychological Tests Administered cognition after transient ischemic attack and stroke: Modified
In-Home Via Telephone. Archives of Clinical Neuropsychology. telephone interview of cognitive status and telephone Montreal
doi: 10.1093/arclin/acaa111 Cognitive Assessment Versus face-to-face Montreal Cognitive
Matías-Guiu, J., Gomez-Pinedo, U., Montero-Escribano, P., Assessment and neuropsychological battery. Stroke, 44(1),
Gomez-Iglesias, P., Porta-Etessam, J., & Matias-Guiu, J.A. 227–229.
(2020). Should we expect neurological symptoms in the SARS- Phabphal, K. and Kanjanasatien, J. (2011). Montreal Cognitive
CoV-2 epidemic? Neurologia, 35(3), 170–175. doi: 10.1016/j. Assessment in cryptogenic epilepsy patients with normal Mini-
nrl.2020.03.001 Mental State Examination scores. Epileptic Disorders, 13(4),
Meinhardt, J., Radke, J., Dittmayer, C., Franz, J., Thomas, C., 375–381. doi: 10.1684/epd.2011.0469
Mothes, R., : : : Heppner, F.L. (2021). Olfactory transmucosal Phillips, N.A., Chertkow, H., Pichora-Fuller, M.K., & Wittich, W.
SARS-CoV-2 invasion as a port of central nervous system entry (2020). Special issues on using the Montreal Cognitive
in individuals with COVID-19. Nature Neuroscience. doi: 10. Asseessment for telemedicine assessment during COVID-19.
1038/s41593-020-00758-5 Journal of the American Geriatrics Society, 68(5), 942–944.
Milani, S.A., Marsiske, M., & Striley, C.W. (2019). Discriminative Postal, K.S., Bilder, R.M., Lanca, M., Aase, D.M., Barisa, M.,
ability of Montreal Cognitive Assessment subtests and items in Holland, A.A., : : : Salinas, C. (2021). InterOrganizational
racial and ethnic minority groups. Alzheimer Disease and practice committee guidance/recommendation for models of
Associated Disorders, 33(3), 226–232. doi: 10.1097/WAD. care during the novel coronavirus pandemic. The Clinical
0000000000000310 Neuropsychologist, 35(1), 81–98. doi: 10.1080/13854046.
Milani, S.A., Marsiske, M., Cottler, L.B., Chen, X., & Striley, C.W. 2020.1801847
(2018). Optimal cutoffs for the Montreal Cognitive Assessment Public Health England. (2020). Disparities in the risk and outcomes
vary by race and ethnicity. Alzheimer’s & Dementia, 10(1), of COVID-19 Retrieved from https://www.gov.uk/government/
773–781. doi: 10.1016/j.dadm.2018.09.003 publications/covid-19-review-of-disparities-in-risks-and-outcomes

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 18 L.A. Cysique, E. Łojek et al.

Raifman, M.A. and Raifman, J.R. (2020). Disparities in the popula- Varatharaj, A., Thomas, N., Ellul, M.A., Davies, N.W.S., Pollak,
tion at risk of severe illness from COVID-19 by race/ethnicity and T.A., Tenorio, E.L., : : : CoroNerve Study, G. (2020).
income. American Journal of Preventive Medicine, 59(1), 137– Neurological and neuropsychiatric complications of COVID-19
139. doi: 10.1016/j.amepre.2020.04.003 in 153 patients: A UK-wide surveillance study. Lancet
Robbins, R.N., Gouse, H., Brown, H.G., Ehlers, A., Scott, T.M., Psychiatry. doi: 10.1016/S2215-0366(20)30287-X
Leu, C.-S., : : : Joska, J.A. (2018). A mobile app to screen for neu- von Weyhern, C.H., Kaufmann, I., Neff, F., & Kremer, M. (2020).
rocognitive impairment: preliminary validation of NeuroScreen Early evidence of pronounced brain involvement in fatal COVID-
among HIV-infected South African adults. JMIR mHealth and 19 outcomes. The Lancet, 395(10241), e109. doi: 10.1016/s0140-
uHealth, 6(1), e5. 6736(20)31282-4
Rodrigues, S.G., Gouveia, R.G., & Bentes, C. (2020). MoCA as a Wilson, B.A., Betteridge, S., & Fish, J. (2020). Neuropsychological
cognitive assessment tool for absence status epilepticus. consequences of Covid-19. Neuropsychological Rehabilitation,
Epileptic Disorders, 22(2), 229–232. doi: 10.1684/epd.2020.1149 30(9), 1625–1628. doi: 10.1080/09602011.2020.1808483
Shuttleworth-Edwards, A.B. (2016). Generally representative is Wilson, M.P. and Jack, A.S. (2020). Coronavirus disease (COVID-
representative of none: commentary on the pitfalls of IQ test stand- 19) in neurology and neurosurgery: A scoping review of the early
ardization in multicultural settings. The Clinical Neuropsychologist, literature. Clinical Neurology and Neurosurgery, 193, 105866.
30(7), 975–998. doi: 10.1080/13854046.2016.1204011 doi: 10.1016/j.clineuro.2020.105866
Sikkes, S.A., de Lange-de Klerk, E.S., Pijnenburg, Y.A., Scheltens, Wittich, W., Phillips, N., Nasreddline, Z.S., & Chertkow, H. (2010).
P., & Uitdehaag, B.M. (2009). A systematic review of Sensitivity and specificity of the Montreal Cognitive Assessment
Instrumental Activities of Daily Living scales in dementia: room modified for individuals who are visually impaired. Journal of
for improvement. Journal of Neurology, Neurosurgery and Visual Impairment & Blindness, 104(6), 360–368.
Psychiatry, 80(1), 7–12. doi: 10.1136/jnnp.2008.155838 Wong, A., Nyenhuis, D., Black, S.E., Law, L.S.N., Lo, E.S.K.,
Siriwardhana, D.D., Walters, K., Rait, G., Bazo-Alvarez, J.C., & Kwan, P.W.L., : : : Mok, V. (2015). Montreal Cognitive
Weerasinghe, M.C. (2018). Cross-cultural adaptation and psycho- Assessment 5-Minute Protocol is a brief, valid, reliable, and fea-
metric evaluation of the Sinhala version of Lawton Instrumental sible cognitive screen for telephone administration. Stroke, 46(4),
Activities of Daily Living Scale. PLoS ONE, 13(6), e0199820. 1059–1064. doi: 10.1161/STROKEAHA.114.007253
doi: 10.1371/journal.pone.0199820 Woods, S.P., Iudicello, J.E., Moran, L.M., Carey, C.L., Dawson,
Solomon, I.H., Normandin, E., Bhattacharyya, S., Mukerji, S.S., M.S., & Grant, I. (2008). HIV-associated prospective memory
Keller, K., Ali, A.S., : : : Sabeti, P. (2020). Neuropathological impairment increases risk of dependence in everyday functioning.
features of Covid-19. New England Journal of Medicine, Neuropsychology, 22(1), 110–117. doi: 10.1037/0894-4105.22.1.
383(10), 989–992. doi: 10.1056/NEJMc2019373 110
Sozzi, M., Algeri, L., Corsano, M., Crivelli, D., Daga, M.A., Woods, S.P., Weinborn, M., Velnoweth, A., Rooney, A., & Bucks,
Fumagalli, F., : : : Balconi, M. (2020). Neuropsychology in the R.S. (2012). Memory for intentions is uniquely associated with
times of COVID-19. The role of the psychologist in taking charge instrumental activities of daily living in healthy older adults.
of patients with alterations of cognitive functions. Frontiers in Journal of the International Neuropsychological Society, 18(1),
Neurology, 11(1142). doi: 10.3389/fneur.2020.573207 134–138. doi: 10.1017/s1355617711001263
Stone, L., Heward, J., Paddick, S.-M., Dotchin, C.L., Walker, R.W., World Health Oraganization (WHO). (2021). WHO COVID-19 Case
Collingwood, C., : : : Gray, W.K. (2018). Screening for definition. Retrieved from https://www.who.int/publications/i/item/
Instrumental Activities of Daily Living in Sub-Saharan Africa: WHO-2019-nCoV-Surveillance_Case_Definition-2020.2 accessed
A balance between task shifting, simplicity, brevity, and training. Februray 1st, 2021.
Journal of Geriatric Psychiatry and Neurology, 31(5), 248–255. World Health Organization (WHO). (2020). WHO Int. Retrieved
doi: 10.1177/0891988718790400 from https://www.who.int/ accessed February 1st, 2021.
Taquet, M., Geddes, J.R., Husain, M., Luciano, S., & Harrison, P.J. Wu, Y., Xu, X., Chen, Z., Duan, J., Hashimoto, K., Yang, L., : : :
(2021). 6-month neurological and psychiatric outcomes in Yang, C. (2020). Nervous system involvement after infection
236379 survivors of COVID-19: a retrospective cohort study with COVID-19 and other coronaviruses. Brain, Behavior, and
using electronic health records. The Lancet Psychiatry, 8(5), Immunity, 87, 18–22. doi: 10.1016/j.bbi.2020.03.031
416–427. doi: 10.1016/S2215-0366(21)00084-5 Xiang, Y.T., Yang, Y., Li, W., Zhang, L., Zhang, Q., Cheung, T., &
The British Psychological Society. (2020). Coronavirus resources Ng, C.H. (2020). Timely mental health care for the 2019 novel
for professionals. Retrieved from https://www.bps.org.uk/ coronavirus outbreak is urgently needed. Lancet Psychiatry,
coronavirus-resources/professional 7(3), 228–229. doi: 10.1016/s2215-0366(20)30046-8
Troyer, E.A., Kohn, J.N., & Hong, S. (2020). Are we facing a crash- Yang, X., Yu, Y., Xu, J., Shu, H., Xia, J., Liu, H., : : : Shang, Y.
ing wave of neuropsychiatric sequelae of COVID-19? (2020). Clinical course and outcomes of critically ill patients with
Neuropsychiatric symptoms and potential immunologic mecha- SARS-CoV-2 pneumonia in Wuhan, China: A single-centered,
nisms. Brain, Behavior, and Immunity, 87, 34–39. doi: 10. retrospective, observational study. Lancet Respiratory
1016/j.bbi.2020.04.027 Medicine, 8(5), 475–481. doi: 10.1016/S2213-2600(20)30079-5
Tun, P.A. and Lachman, M.E. (2006). Telephone assessment of cog- Yi, S.A., Nam, K.H., Yun, J., Gim, D., Joe, D., Kim, Y.H., Kim,
nitive function in adulthood: the Brief Test of Adult Cognition by H.J., Han, J.W., & Lee, J. (2020). Infection of Brain Organoids
Telephone. Age and Ageing, 35(6), 629–632. doi: 10.1093/ and 2D Cortical Neurons with SARS-CoV-2 Pseudovirus.
ageing/afl095 Viruses, 12(9). doi: 10.3390/v12091004
Vallejo-Medina, P., Gómez-Lugo, M., Marchal-Bertrand, L., Zhou, P., Yang, X.-L., Wang, X.-G., Hu, B., Zhang, L., Zhang, W.,
Saavedra-Roa, A., Soler, F., & Morales, A. (2017). Developing : : : Huang, C.-L. (2020). A pneumonia outbreak associated with
guidelines for adapting questionnaires into the same language a new coronavirus of probable bat origin. Nature, 579, 270–273.
in another culture. Terapia Psicológica, 35(2), 181–194. doi: 10.1038/s41586-020-2012-7.

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862
 NeuroCOVID recommendations 19

APPENDIX 1. AUTHORS’ CONTRIBUTIONS

Name Affiliation Contribution

Dr. Cysique UNSW Psychology, St. Vincent’s Hospital Applied Co-chair of the NeuroCOVID International
Medical Research Centre, The Alfred Hospital Neuropsychological Taskforce, led, contributed,
and coordinated the manuscript writing and pro-
vided input on revisions and responses to
reviewers.
Dr. Łojek University of Warsaw Faculty of Psychology Co-chair of the NeuroCOVID International
Neuropsychological Taskforce SIG, led, contrib-
uted, and coordinated the manuscript writing and
provided input on revisions and responses to
reviewers.
Mr. Theodore C.K. Cheung Department of Psychology, University of Toronto; Contributed to manuscript writing and editing/
Centre for Neuropsychology and Emotional proofreading and provided input on revisions and
Wellness, Markham, ON, Canada responses to reviewers.
Dr. Breda Cullen Institute of Health and Wellbeing, University of Contributed to manuscript writing and editing/
Glasgow proofreading
Dr. Anna Egbert Faculty of Medicine, The University of British Contributed to manuscript writing and designed the
Columbia, Vancouver, BC, Canada figure
Dr. Jonathan Evans Institute of Health and Wellbeing, University of Contributed to manuscript writing and editing/
Glasgow proofreading and provided input on revisions and
responses to reviewers.
Dr. Natalia Gawron Academy of Special Pedagogics, Warsaw, Poland Contributed to manuscript writing
Dr. Hetta Gouse Department of Psychiatry & Mental HealthGroote Contributed to manuscript writing
Schuur Hospital, Cape Town, South Africa
Dr. Karolina Hansen University of Warsaw Faculty of Psychology Contributed to manuscript writing and organized
and prepared the references
Dr. Paweł Holas University of Warsaw Faculty of Psychology Contributed to manuscript writing
Dr. Sylwia Hyniewska University College London, Division of Contributed to manuscript writing
Psychology and Language Sciences, London, UK
Dr. Ewa Malinowska University of Warsaw Faculty of Psychology Contributed to manuscript writing
Dr. Bernice A. Marcopulos Department of Graduate Psychology, James Contributed to manuscript writing and provided
Madison University; Department of Psychiatry input on revisions and responses to reviewers.
and Neurobehavioral Sciences University of
Virginia School of Medicine
Dr. Tricia L. Merkley Department of Psychology and Neuroscience Contributed to manuscript writing, editing/proof-
Center, Brigham Young University reading and provided input on revisions and
responses to reviewers.
Dr. Jose A Muñoz-Moreno Lluita contra la SIDA Foundation Contributed to manuscript writing and input on
responses to reviewers.
Dr. Clare Ramsden Tasmanian Health Service Contributed to manuscript writing
Dr. Christian Salas Faculty of Psychology, Diego Portales University, Contributed to manuscript writing
Santiago, Chile
Dr. Ana Rita Silva Center for Neuroscience and Cell Biology, Contributed to manuscript writing and provided
University of Coimbra Portugal input on revisions and responses to reviewers.
Dr. Imane Zouhar Department of Psychology, University of Toronto, Contributed to manuscript writing
ON, Canada

The Taskforce is coordinated from a dedicated Slack platform. All members have access to the Welcome and Literature/Resources channels, and all are able to
share, propose, and access the material, including the manuscript as it evolved via a Slack-Google Drive link. The SIG co-chairs have administrative access to
Slack. SIG members are required to become INS members to join, as per INS policies. For the current recommendations, members were asked to self-nominate
for leading manuscript sections. More than one member could be a section author. The co-chairs led the synthesis from all the co-authors and all co-authors have
reviewed the paper, including two senior researchers who are native English speakers.

Downloaded from https://www.cambridge.org/core. IP address: 187.188.63.49, on 26 Nov 2021 at 05:07:17, subject to the Cambridge Core terms of use, available at https://www.cambridge.org/core/terms.
https://doi.org/10.1017/S1355617721000862