RESEARCH REPORT

Validation of the new DSM-5-TR criteria for prolonged grief disorder

and the PG-13-Revised (PG-13-R) scale
Holly G. Prigerson1,2, Paul A. Boelen3,4, Jiehui Xu1, Kirsten V. Smith5, Paul K. Maciejewski1,2,6
1
Cornell Center for Research on End-of-Live Care, Weill Cornell Medicine, New York, NY, USA; 2Division of Geriatrics and Palliative Medicine, Department of Medicine, Weill
Cornell Medicine, New York, NY, USA; 3Department of Clinical Psychology, Faculty of Social Sciences, Utrecht University, Utrecht, The Netherlands; 4ARQ Research, ARQ Na-
tional Psychotrauma Centre, Diemen, The Netherlands; 5Department of Experimental Psychology, University of Oxford, Oxford, UK; 6Department of Radiology, Weill Cornell
Medicine, New York, NY, USA

Although the concept of pathological grief dates back at least as far as Freud’s “Mourning and Melancholia”, there has been opposition to its
recognition as a distinct mental disorder. Resistance has been overcome by evidence demonstrating that distinctive symptoms of prolonged grief
disorder (PGD) – an attachment disturbance featuring yearning for the deceased, loss of meaning and identity disruption – can endure, prove
distressing and disabling, and require targeted treatment. In acknowledgement of this evidence, the American Psychiatric Association Assembly
has recently voted to include PGD as a new mental disorder in the DSM-5-TR. We tested the validity of the new DSM criteria for PGD and of an
adapted version of our PG-13 scale, the PG-13-Revised (PG-13-R), designed to map onto these criteria, using data from investigations conducted
at Yale University (N=270), Utrecht University (N=163) and Oxford University (N=239). Baseline assessments were performed at 12-24 months
post-loss; follow-up assessments took place 5.3-12.0 months later. Results indicated that the PG-13-R grief symptoms represent a unidimensional
construct, with high degrees of internal consistency (Cronbach’s alpha = 0.83, 0.90 and 0.93, for Yale, Utrecht and Oxford, respectively). The
DSM PGD diagnosis was distinct from post-traumatic stress disorder (phi=0.12), major depressive disorder (phi=0.25) and generalized anxiety
disorder (phi=0.26) at baseline. Temporal stability was remarkable for this diagnosis (r=0.86, p<0.001). Kappa agreement between a PG-13-R
threshold symptom summary score of 30 and the DSM symptom criterion for PGD was 0.70-0.89 across the datasets. Both the DSM PGD di-
agnosis and the PG-13-R symptom summary score at baseline were significantly associated (p<0.05) with symptoms and diagnoses of major
depressive disorder, post-traumatic stress disorder and/or generalized anxiety disorder, suicidal ideation, worse quality of life and functional
impairments at baseline and at follow-up, in the Yale, Utrecht and Oxford datasets. Overall, the DSM-5-TR criteria for PGD and the PG-13-R
both proved reliable and valid measures for the classification of bereaved individuals with maladaptive grief responses.

Key words: Prolonged grief disorder, DSM-5-TR, PG-13-R, ICD-11, pathological grief, bereavement, post-traumatic stress disorder

(World Psychiatry 2021;20:96–106)

Although the concept of pathological grief dates back at least but that they may respond only to specialist treatment. Specifi-
as far as Freud’s Mourning and Melancholia1, there has been cally, studies have documented that certain grief symptoms are
public and professional opposition to its recognition as a men- distinct from those of bereavement-related depression6-9, have
tal disorder2-5. For example, a 2015 international online survey of idiosyncratic neurobiological10 and clinical11-13 correlates, can
public attitudes revealed that approximately 25% of respondents persist unabated for months or even years8,14, prove distressing
did not endorse the position that grief could be a mental disor- and dysfunctional14-16, and may only respond to targeted inter-
der2. More recently, an online survey on public opinion in Chi- vention17,18. Thus, there exists a substantial and mounting body
na found that about 40% of participants did not agree that grief of evidence in support of a psychiatric syndrome of maladaptive
could be a mental disorder, even under circumstances such as grief.
threat of harm to self or others4. Concerns about “pathologizing” The ICD-11 Workgroup on Stress-Associated Disorders found
grief are reported to be rooted in the belief that all grief is normal the available evidence for prolonged grief disorder (PGD) suf-
and an expected response to the death of a loved one. Thus, a ficiently compelling to recommend its recognition as a new
diagnosis of pathological grief is considered to be tantamount to mental disorder19. The DSM-5 had included “persistent com-
stigmatizing, medicalizing and/or pathologizing love2,4. plex bereavement disorder” (PCBD) in Section III (i.e., among
Himself wary of pathologizing grief, Freud conceptualized “conditions for further study”). In response to the ICD’s inclu-
mourning (grief) as a normal, natural reaction to loss of a loved sion of PGD and the accumulated evidence, the DSM Steering
one, and even deemed working through grief as necessary to be- Committee convened a workshop in June 2019. An invited panel
reavement adjustment – the hard, often painful, work a mourner of researchers presented their data to the Committee, who con-
must do to withdraw emotional attachment to the deceased cluded that these data supported moving the disorder to Section
person. In fact, Freud considered medical interference in “grief II (i.e., among recognized mental disorders). A provisional PGD
work” to be “inadvisable if not even harmful”1. By contrast, he criteria set was then drafted, and the researchers were tasked
considered melancholia (i.e., depression) the pathological re- with using the best data available to inform the parameters of the
sponse to bereavement, and noted that this condition, not grief, PGD diagnostic algorithm, and then to evaluate that algorithm’s
posed a risk for suicide, and warranted medical attention. reliability and validity. The researchers submitted their reports,
Research over the past quarter century has shown not only which found the same PGD diagnostic algorithm to be optimal.
that a small but substantial proportion of grief reactions can The Steering Committee then posted that PGD algorithm online
be severe, disabling, and endure beyond normal expectations, on the American Psychiatric Association (APA)’s website and

96 World Psychiatry 20:1 - February 2021

opened a period for public commentary between April and May gender, relationship to the decedent, and time from loss in item
2020. After reviewing the research reports and submitted com- response theory-based item analysis, and which mapped onto
ments, the Steering Committee released the proposed criteria, our criteria for PGD proposed in 20098.
and on November 7, 2020, the APA Assembly approved the in- The present paper has two primary objectives. First, it aims to
clusion of PGD in the DSM-5-TR (see Table 1). introduce and validate the PG-13-R, a revised version of the PG-
In order to be sensitive to the concern expressed in the pub- 13 scale that corresponds to the new DSM-5-TR criteria for PGD.
lic commentary about pathologizing normal grieving and di- Second, it aims to validate these new DSM criteria for PGD. Data
agnosing a grief-related disorder “too soon” after the death, the from the US (the Yale Bereavement Study), the Netherlands (the
DSM-5-TR PGD criteria specify that 12 months must elapse since Utrecht Bereavement Study), and the UK (the Oxford Grief Study)
the death. This time frame contrasts with the ICD-11 diagnostic were used to evaluate the psychometric properties of the PG-13-R,
guidelines for PGD, requiring a period of 6 months20. Unlike the determine its agreement with the new DSM criteria for PGD, as-
PCBD criteria, the DSM-5-TR criteria for PGD acknowledge the sess the PG-13-R and DSM criteria’s predictive validity, and estab-
possibility of delayed onset of symptoms at or beyond 12 months lish a threshold PG-13-R score to identify syndromal level PGD.
post-loss. Furthermore, the PGD criteria require that three of eight
C criteria (compared to PCBD’s six of 12) be met for a diagnosis,
and focus more on “yearning for” and preoccupation with the METHODS
deceased person and less on “preoccupation with the circum-
stances of the death” – the latter of which could be captured by a Datasets and measures
post-traumatic stress disorder (PTSD) diagnosis. Lastly, the PGD
diagnosis allows for fewer combinations of symptoms to meet the Data to evaluate the performance of PG-13-R items and the
criteria compared to the PCBD diagnosis. An empirical analysis of new DSM criteria for PGD came from the Yale Bereavement
the performance of these new DSM criteria for PGD has not been Study, the Utrecht Bereavement Study, and the Oxford Grief
published, nor has the psychometric performance of a scale that Study. In the Yale Bereavement Study, community-based be-
maps onto these diagnostic criteria been evaluated. reaved individuals were recruited for a field trial of consensus
The PG-13 scale22 was introduced in the process of develop- criteria for PGD8. In the Utrecht Bereavement Study, commu-
ing PGD diagnostic criteria proposed for inclusion in the DSM- nity-based bereaved subjects were enrolled by mental health
5 and ICD-118. The scale contains 13 items that can be used for care providers to examine the role of cognitive behavioral fac-
the dual purposes of assessing grief intensity continuously on a tors in bereavement adjustment24. In the Oxford Grief Study, a
dimensional scale and of diagnosing PGD according to the pro- community-based bereaved sample was recruited to investigate
posed criteria. Items in the PG-13 are a subset of those in the In- loss-related memories, appraisals and coping strategies relevant
ventory of Complicated Grief - Revised23, which is a revision of to the development and maintenance of PGD25.
the Inventory of Complicated Grief7. Included items were those Across datasets, participants with at least one assessment
that we found to be informative and unbiased with respect to at 12-24 months post-loss were included. Participants without

Table 1  DSM-5-TR criteria for prolonged grief disorder

A. The death, at least 12 months ago, of a person who was close to the bereaved (for children and adolescents, at least 6 months ago).

B. Since the death, there has been a grief response characterized by one or both of the following, to a clinically significant degree, nearly every day or more
often for at least the last month:
1. Intense yearning/longing for the deceased person
2. Preoccupation with thoughts or memories of the deceased person (in children and adolescents, preoccupation may focus on the circumstances of the death)

C. As a result of the death, at least 3 of the following 8 symptoms have been experienced to a clinically significant degree since the death, including nearly
every day or more often for at least the last month:
1. Identity disruption (e.g., feeling as though part of oneself has died)
2. Marked sense of disbelief about the death
3. Avoidance of reminders that the person is dead (in children and adolescents, may be characterized by efforts to avoid reminders)
4. Intense emotional pain (e.g., anger, bitterness, sorrow) related to the death
5. Difficulty with reintegration into life after the death (e.g., problems engaging with friends, pursuing interests, planning for the future)
6. Emotional numbness (i.e., absence or marked reduction in the intensity of emotion, feeling stunned) as a result of the death
7. Feeling that life is meaningless as a result of the death
8. Intense loneliness (i.e., feeling alone or detached from others) as a result of the death

D. The disturbance causes clinically significant distress or impairment in social, occupational, or other important areas of functioning.

E. The duration and severity of the bereavement reaction clearly exceeds expected social, cultural, or religious norms for the individual’s culture and context.

F. The symptoms are not better explained by major depressive disorder, posttraumatic stress disorder, or another mental disorder, or attributable to the physi-
ological effects of a substance (e.g., medication, alcohol) or another medical condition.

©2020 American Psychiatric Association, all rights reserved. Reprinted with permission

World Psychiatry 20:1 - February 2021 97

Figure 1  PG-13-Revised (by H.G. Prigerson, J. Xu and P.K. Maciejewski)

complete responses to the new DSM PGD symptom items were was used for predictive external validity analysis, with a time lag
excluded (total missing rate ~5%), resulting in sample sizes of of 7.4±2.0, 12.0±0 (fixed by design), and 5.3±1.3 months after T1
N=270 (Yale), N=163 (Utrecht) and N=239 (Oxford), for a total of for Yale (N=48), Utrecht (N=90) and Oxford (N=35) subjects, re-
N=672. In participants with more than one assessment, the first spectively. All studies were approved by each university’s institu-
evaluation within the time frame was used for item evaluation tional review board.
and threshold sensitivity analysis. The average time post-loss for All three studies assessed the 10 symptom items included in
the first assessment (T1) was 16.7±2.6 months for the Yale study, both the new DSM criteria for PGD and the PG-13-R (yearning,
16.3±3.7 months for the Utrecht study, and 14.1±1.7 months for preoccupation, identity disruption, disbelief, avoidance, intense
the Oxford study. Participants’ next available assessment (T2) emotional pain, difficulty with reintegration, emotional numb-

98 World Psychiatry 20:1 - February 2021

Table 2  Sample characteristics for the three bereavement studies
Yale Study Utrecht Study Oxford Study
(N=270) (N=163) (N=239)

Age, years (mean±SD) 61.8±13.5 56.2±13.3 46.9±13.3

Time from loss, months (mean±SD) 16.7±2.6 16.3±3.7 14.1±1.7
Gender, N (%)
Male 67 (24.9) 44 (27.0) 50 (20.9)
Female 202 (75.1) 119 (73.0) 189 (79.1)
Highest education, N (%)
Primary/secondary school 103 (38.3) 102 (62.6) 55 (23.0)
College/university 166 (61.7) 61 (37.4) 184 (77.0)
Relationship to the deceased, N (%)
Partner/spouse 219 (83.6) 128 (78.5) 71 (29.7)
Other 43 (16.4) 35 (21.5) 168 (70.3)
Cause of death, N (%)
Natural 251 (94.0) 151 (92.6) 218 (91.2)
Unnatural 16 (6.0) 12 (7.4) 21 (8.8)

ness, feeling that life is meaningless, and intense loneliness). Cronbach’s alpha of the PG-13-R symptom items was used to
These items (questions Q3 through Q12 in the PG-13-R) were evaluate the internal consistency (reliability) of the scale.
rated using a 5-point Likert scale ranging from “1 = not at all” to A principal components factor analysis was conducted for
“5 = overwhelmingly”. In the PG-13-R, the symptom items are each dataset at T1 to evaluate the dimensionality of the grief
accompanied by three gatekeeper items exploring whether the symptoms (Q3-Q12) construct. In each dataset, the eigenvalues
respondent had lost a significant other (Q1), how long ago the obtained from actual PG-13-R symptom item data were com-
death occurred (Q2), and impairment associated with the above pared with those obtained from simulated random data (parallel
symptoms (Q13) (see Figure 1). analysis)34.
In the Yale study, the occurrence of PTSD, major depressive The external validity of the 10-item PG-13-R symptom score
disorder (MDD), generalized anxiety disorder (GAD) and panic at T1, not including the impairment item (Q13), was assessed by
disorder was further explored using the Structured Clinical In- its associations with other concurrent (T1, concurrent validity)
terview for DSM-IV Axis I Disorders (SCID-I)26; suicidal ideation and follow-up (T2, predictive validity) psychological and behav-
was assessed using the Yale Evaluation of Suicidality (YES)27; ioral health measures within each dataset, including measures of
and quality of life in eight domains (physical functioning, role- depression, post-traumatic stress, suicidality, quality of life and
physical, bodily pain, general health, vitality, social functioning, functional impairments. Associations with dichotomous vari-
role-emotional, and mental health) was evaluated using the SF- ables were estimated as odds ratios (ORs) using logistic regres-
12 Health Survey28. sion; associations with continuous variables were evaluated with
In the Utrecht study, PTSD symptoms were assessed using Pearson’s correlation coefficients.
the PTSD Symptom Scale Self-Report (PSS-SR)29, and depressive The summed PG-13-R score for the symptom items may
symptoms by the Beck Depression Inventory (BDI-II)30. In the range from 10 to 50. The optimal threshold was the symptom
Oxford study, mental health problems were assessed using the score that had the highest degree of agreement (kappa statistic)
Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5)31, the with fulfillment of B and C symptom criteria for PGD according
Patient Health Questionnaire (PHQ-9)32 and the Work and Social to DSM within each dataset. The median maximum-agreement
Adjustment Scale (WSAS)33. threshold score across the datasets was taken to be the overall
optimal PG-13-R symptom threshold score.
The associations between the dichotomous PG-13-R diagnos-
Statistical analysis tic threshold score plus the three gatekeeper criteria (i.e., loss,
timing, impairment) as well as the DSM PGD diagnosis with the
The item performance of the PG-13-R symptom items (Q3- mental and behavioral health outcomes at baseline and follow-
Q12) was evaluated within each dataset at T1. This included up were estimated as ORs using logistic regression.
inspection of item means and variances, percentage of syndrom­ Phi coefficients were used to determine associations between
al-level responses (score of 4 or 5), and item-total correlations. PGD and other diagnosed mental disorders (e.g., MDD, PTSD,

World Psychiatry 20:1 - February 2021 99

100
Table 3  PG-13-R item performance and scale internal consistency
Yale Study (N=270) Utrecht Study (N=163) Oxford Study (N=239)
Alpha=0.83 Alpha=0.90 Alpha=0.93
Corrected Corrected Corrected
Score Deleted item-total Score Deleted item-total Score Deleted item-total
PGD-13-R symptom item Rate (mean±SD) alpha correlation Rate (mean±SD) alpha correlation Rate (mean±SD) alpha correlation

Q3 Yearning 35.2% 2.9±1.3 0.81 0.59 68.1% 3.8±0.9 0.89 0.65 34.7% 3.1±1.2 0.92 0.75
Q4 Preoccupation 2.6% 1.3±0.8 0.82 0.53 26.4% 2.9±0.9 0.88 0.72 36.4% 3.2±1.2 0.92 0.74
Q5 Identity disruption 22.6% 2.2±1.4 0.81 0.58 42.3% 3.1±1.3 0.88 0.71 33.9% 2.7±1.4 0.92 0.76
Q6 Disbelief 6.3% 1.5±1.0 0.82 0.50 27.0% 2.9±1.2 0.89 0.56 33.9% 2.8±1.3 0.92 0.69
Q7 Avoidance 2.6% 1.3±0.7 0.84 0.25 5.5% 1.9±1.0 0.91 0.33 11.7% 1.8±1.2 0.93 0.52
Q8 Intense emotional pain 10.7% 2.1±1.0 0.82 0.51 49.7% 3.4±1.0 0.88 0.75 26.8% 3.0±1.1 0.92 0.74
Q9 Difficulty with reintegration 9.3% 1.8±1.1 0.82 0.52 26.4% 2.7±1.2 0.89 0.67 17.6% 2.1±1.3 0.92 0.76
Q10 Emotional numbness 7.4% 1.5±1.0 0.82 0.50 16.6% 2.4±1.1 0.88 0.70 21.8% 2.4±1.2 0.92 0.76
Q11 Life is meaningless 16.3% 2.0±1.2 0.81 0.61 39.3% 3.1±1.1 0.88 0.76 18.8% 2.1±1.3 0.92 0.80
Q12 Intense loneliness 33.3% 2.8±1.3 0.81 0.61 51.5% 3.4±1.1 0.89 0.65 26.4% 2.5±1.3 0.92 0.76

World Psychiatry 20:1 - February 2021

Figure 2  Eigenvalues from principal components factor analysis for PG-13-R symptom items and comparison to eigenvalues from parallel
analysis (median of 100 replications of simulated random data) for the three studies

GAD in the Yale data). Pearson’s correlation coefficients were Statistical analyses for the Yale, Utrecht and Oxford studies
used to determine stability of PGD and these other mental disor- were performed using SAS (version 9.4), R (version 3.6.2), and
ders between T1 and T2. SPSS (version 24), respectively.

Table 4  Concurrent and predictive validity of PG-13-R symptom score (excluding impairment)
PG-13-R symptom score (sum of 10 items) at T1
Concurrent (T1) outcome Predictive (T2) outcome
Yale Study N % OR p N % OR p

Post-traumatic stress disorder (PTSD) 270 1.5 1.23 0.007 48 2.1 n.e.
Major depressive disorder (MDD) 270 5.9 1.16 <0.001 48 4.2 n.e.
Generalized anxiety disorder (GAD) 270 3.3 1.24 <0.001 48 6.3 1.26 0.032
PTSD, MDD or GAD 270 8.1 1.18 <0.001 48 8.3 1.57 0.033
Yale Evaluation of Suicidality (YES): 269 17.5 1.18 <0.001 48 18.8 1.13 0.032
at least one positive response
Yale Study N mean±SD r p N mean±SD r p
SF-12: Physical functioning 269 5.1±1.3 –0.10 0.109 48 4.7±1.7 0.10 0.518
SF-12: Role-physical 270 3.5±0.8 –0.12 0.048 48 3.3±0.9 –0.05 0.715
SF-12: Bodily pain 270 4.5±0.9 –0.24 <0.001 48 4.4±1.0 –0.10 0.513
SF-12: General health 270 3.6±1.0 –0.25 <0.001 48 3.6±1.1 –0.21 0.162
SF-12: Vitality 270 2.6±1.3 –0.42 <0.001 48 2.4±1.3 –0.23 0.110
SF-12: Social functioning 270 4.3±1.0 –0.41 <0.001 48 4.4±1.0 –0.13 0.373
SF-12: Role-emotional 270 3.6±0.7 –0.45 <0.001 48 3.6±0.7 –0.42 0.003
SF-12: Mental health 270 7.4±2.0 –0.60 <0.001 48 7.3±2.1 –0.61 <0.001
Utrecht Study N mean±SD r p N mean±SD r p
PSS-SR 158 31.4±8.4 0.77 <0.001 85 26.3±6.5 0.68 <0.001
BDI-II 153 34.6±8.8 0.75 <0.001 82 31.1±7.8 0.53 <0.001
BDI-II: Suicidality (item 9) 161 1.2±0.4 0.34 <0.001 90 1.2±0.4 0.29 0.005
Oxford Study N mean±SD r p N mean±SD r p
PCL-5 239 23.5±17.8 0.78 <0.001 35 20.7±16.8 0.53 0.001
PHQ-9 239 8.9±7.1 0.68 <0.001 35 7.8±7.1 0.60 <0.001
PHQ-9: Suicidality (item 9) 239 0.4±0.8 0.52 <0.001 35 0.3±0.8 0.55 0.001
WSAS 237 12.8±9.4 0.77 <0.001 35 11.5±9.7 0.64 <0.001

OR – odds ratio, SF-12 – Medical Outcomes Short-Form-12, PSS-SR – PTSD Symptom Scale Self-Report, BDI-II – Beck Depression Inventory, PCL-5 – Post-
traumatic Stress Disorder Checklist for DSM-5, PHQ-9 – Patient Health Questionnaire-9, WSAS – Work and Social Adjustment Scale, n.e. – not estimated

World Psychiatry 20:1 - February 2021 101

Table 5  Concurrent and predictive validity of prolonged grief disorder (PGD) diagnosis using PG-13-R symptom threshold score of 30 and
including impairment
PG-13-R threshold score-based diagnosis of PGD at T1
Concurrent (T1) outcome Predictive (T2) outcome
Yale Study N OR p N OR p

Post-traumatic stress disorder (PTSD) 270 54.00 0.001 48 n.e.

Major depressive disorder (MDD) 270 18.98 <0.001 48 n.e.
Generalized anxiety disorder (GAD) 270 15.26 <0.001 48 28.00 0.014
PTSD, MDD or GAD 270 20.77 <0.001 48 63.00 0.002
Yale Evaluation of Suicidality (YES): 269 3.71 0.012 48 9.25 0.028
at least one positive response
Yale Study N r p N r p
SF-12: Physical functioning 269 –0.05 0.433 48 0.10 0.509
SF-12: Role-physical 270 –0.08 0.216 48 0.03 0.857
SF-12: Bodily pain 270 –0.24 <0.001 48 0.00 0.992
SF-12: General health 270 –0.17 0.006 48 –0.14 0.351
SF-12: Vitality 270 –0.29 <0.001 48 –0.20 0.183
SF-12: Social functioning 270 –0.34 <0.001 48 0.00 0.992
SF-12: Role-emotional 270 –0.38 <0.001 48 –0.31 0.034
SF-12: Mental health 270 –0.30 <0.001 48 –0.38 0.007
Utrecht Study N r p N r p
PSS-SR 158 0.48 <0.001 85 0.39 <0.001
BDI-II 153 0.47 <0.001 82 0.39 <0.001
BDI-II: Suicidality (item 9) 161 0.18 0.024 90 0.19 0.070
Oxford Study N r p N r p
PCL-5 239 0.51 <0.001 35 0.58 <0.001
PHQ-9 239 0.45 <0.001 35 0.59 <0.001
PHQ-9: Suicidality (item 9) 239 0.54 <0.001 35 0.79 <0.001
WSAS 237 0.49 <0.001 35 0.52 0.001

OR – odds ratio, SF-12 – Medical Outcomes Short-Form-12, PSS-SR – PTSD Symptom Scale Self-Report, BDI-II – Beck Depression Inventory, PCL-5 – Post-
traumatic Stress Disorder Checklist for DSM-5, PHQ-9 – Patient Health Questionnaire-9, WSAS – Work and Social Adjustment Scale, n.e. – not estimated

RESULTS were infrequent in the Yale study, where mean scores in general
were low. Variances for most items across the datasets were rea-
Table 2 summarizes the demographic characteristics of the sonably high, confirming the scale’s discriminating ability.
three study samples. The Yale sample was older (mean age: Across studies, the PG-13-R symptom items cohered well
61.8±13.5 years) than the Utrecht (mean age: 56.2±13.3 years) (Cronbach’s alpha = 0.83 for Yale, 0.90 for Utrecht, 0.93 for the
and Oxford (mean age: 46.9±13.3 years) ones. All three samples Oxford study) (see Table 3). This analysis revealed that the dele-
were primarily female (73.0 to 79.1%), and most survived a death tion of the avoidance item in each of the three datasets resulted
from natural causes (compared to unnatural causes such as sui- in either the same or an improved overall Cronbach’s alpha (de-
cide or homicide or accidental) (>90%). The Yale and Oxford leted alpha = 0.84, 0.91, 0.93 for the Yale, Utrecht and Oxford,
samples had higher levels of educational attainment (college or respectively). Similarly, while all the other items had high item-
above >60%) than the Utrecht sample (college or above <40%). total correlations (r ≥ 0.50, 0.56 and 0.69 for the three datasets,
The mean scores for each PG-13-R symptom item at T1 are respectively), the avoidance item was an exception, with lower
presented in Table 3. They ranged from 1.3 to 2.9 in the Yale item-total correlations (r=0.25, 0.33, 0.52, respectively).
study; from 1.9 to 3.8 in the Utrecht study; and from 1.8 to 3.2 As illustrated in Figure 2, principal components factor analysis
in the Oxford study. In general, most item means were located in combination with parallel analysis for each dataset supported
around the center of the range, which is an indication of desira- the conclusion that the PG-13-R grief symptoms represent a uni-
ble variability. The avoidance (Q7) and preoccupation (Q4) items dimensional construct. In fact, in each dataset, a single factor

102 World Psychiatry 20:1 - February 2021

Table 6  Concurrent and predictive validity of new DSM diagnostic criteria for prolonged grief disorder (PGD)
DSM diagnosis for PGD at T1
Concurrent (T1) outcome Predictive (T2) outcome
Yale Study N OR p N OR p

Post-traumatic stress disorder (PTSD) 270 7.73 0.087 48 n.e.

Major depressive disorder (MDD) 270 10.25 0.001 48 n.e.
Generalized anxiety disorder (GAD) 270 14.00 0.001 48 43.00 0.008
PTSD, MDD or GAD 270 10.13 <0.001 48 129.00 0.002
Yale Evaluation of Suicidality (YES): 269 1.61 0.486 48 19.00 0.017
at least one positive response
Yale Study N r p N r p
SF-12: Physical functioning 269 0.00 0.965 48 0.05 0.737
SF-12: Role-physical 270 –0.02 0.805 48 0.15 0.316
SF-12: Bodily pain 270 –0.14 0.024 48 0.03 0.828
SF-12: General health 270 –0.09 0.134 48 –0.25 0.086
SF-12: Vitality 270 –0.20 0.001 48 –0.31 0.032
SF-12: Social functioning 270 –0.32 <0.001 48 –0.05 0.760
SF-12: Role-emotional 270 –0.28 <0.001 48 –0.38 0.008
SF-12: Mental health 270 –0.19 0.002 48 –0.45 0.001
Utrecht Study N r p N r p
PSS-SR 158 0.48 <0.001 85 0.39 <0.001
BDI-II 153 0.47 <0.001 82 0.39 <0.001
BDI-II: Suicidality (item (9) 161 0.20 0.011 90 0.19 0.070
Oxford Study N r p N r p
PCL-5 239 0.48 <0.001 35 0.58 <0.001
PHQ-9 239 0.43 <0.001 35 0.59 <0.001
PHQ-9: Suicidality (item 9) 239 0.54 <0.001 35 0.79 <0.001
WSAS 237 0.48 <0.001 35 0.52 0.001

OR – odds ratio, SF-12 – Medical Outcomes Short-Form-12, PSS-SR – PTSD Symptom Scale Self-Report, BDI-II – Beck Depression Inventory, PCL-5 – Post-
traumatic Stress Disorder Checklist for DSM-5, PHQ-9 – Patient Health Questionnaire-9, WSAS – Work and Social Adjustment Scale, n.e. – not estimated

emerged whose eigenvalue was substantially larger than 1 and pa=0.89) study data, respectively. Overall, a symptom threshold
greater than would be expected by chance. This primary factor score of 30 optimized agreement with meeting DSM symptom
explained 40.3%, 53.5% and 61.8% of the variance in the Yale, criteria for PGD across the three datasets (kappa ≥0.70 across the
Utrecht and Oxford studies, respectively. datasets).
Results in Table 4 support the external validity of the PG-13-R Results in Table 5 illustrate that using a PG-13-R symptom
symptom score, not including the impairment item (Q13). PG- threshold score of 30 in combination with the impairment crite-
13-R symptom scores at T1 were significantly associated with rion demonstrated excellent external validity. The prevalence of
PTSD, MDD and/or GAD diagnoses or symptomatology and PGD using the PG-13-R score ≥30 at T1, including impairment,
suicidal ideation, both concurrently (p<0.001) and predictively was 6.3%, 16.6% and 11.3% for the Yale, Utrecht and Oxford
(p<0.05), in the Yale, Utrecht and Oxford data. PG-13-R symptom samples, respectively. The PG-13-R threshold-based diagnoses
scores were significantly associated with poorer role-emotional of PGD at T1 were significantly (p<0.05) associated with PTSD,
and mental health domains of quality of life both concurrently MDD and/or GAD diagnoses or symptomatology and suicidal-
and predictively in the Yale data (p<0.005), and with work and ity in the Yale, Utrecht and Oxford data, concurrently and pre-
social adjustment difficulties both concurrently and predictively dictively (except for suicidality in the Utrecht study, where the
in the Oxford data (p<0.001). association was significant only concurrently). PG-13-R thresh-
PG-13-R symptom threshold scores of 29, 32 and 30 maxi- old-based diagnoses of PGD were significantly associated with
mized agreement with meeting DSM symptom criteria for PGD poorer role-emotional and mental health domains of quality of
in the Yale (kappa=0.77), Utrecht (kappa=0.86), and Oxford (kap- life both concurrently and predictively in the Yale data (p<0.05),

World Psychiatry 20:1 - February 2021 103

and with work and social adjustment difficulties both concur- retained, revised or discarded.
rently and predictively in the Oxford data (p≤0.001). In accordance with the high internal consistency of the PG-
Results in Table 6 illustrate that the DSM diagnosis of PGD 13-R symptom items, factor analyses revealed that the scale is
demonstrated excellent external validity. The prevalence of unidimensional. These results are consistent with those re-
PGD using DSM criteria at T1 was 4.4%, 15.3% and 10.9% for the ported for the Inventory of Complicated Grief7 and its Dutch
Yale, Utrecht and Oxford samples, respectively. DSM diagnoses version35, and for the original PG-138 and its Swedish36, Chi-
of PGD at T1 were significantly (p<0.05) associated with PTSD, nese37, Portuguese38 and many other translated versionse.g.,39.
MDD and/or GAD diagnoses or symptomatology concurrently Though some studies have found multiple factors in this set of
and predictively in the Yale, Utrecht and Oxford data. Interest- grief symptoms40, these exceptions occurred only in highly co-
ingly, in the Yale sample, DSM diagnoses of PGD were signifi- morbid treatment-seeking and treatment-receiving samples and
cantly associated with suicidality predictively (at T2) but not a military family study, not in community-based samples. The
concurrently (at T1). DSM diagnoses of PGD were significantly preponderance of evidence supports the unidimensional nature
associated with poorer vitality, role-emotional and mental health of PGD symptomatology as found in the three studies examined
domains of quality of life both concurrently and predictively in here.
the Yale data (p<0.05), and with work and social adjustment dif- Because the Yale data alone included structured clinical in-
ficulties both concurrently and predictively in the Oxford data terviews that yielded diagnoses of mental disorders, only these
(p≤0.001). data could be used to assess PGD’s overlap with other disorders
In the Yale data (T1, N=270), the DSM PGD diagnosis was and to compare diagnostic stability over time. The results dem-
found to be distinct from PTSD (phi=0.12), MDD (phi=0.25) and onstrated minimal overlap between PGD and competing diag-
GAD (phi=0.26). Temporal stability (T1, T2 correlation; N=48) noses (i.e., PTSD, MDD and GAD) (phi=0.12-0.26), suggesting its
was greatest for DSM PGD (r=0.86, p<0.001), significant for MDD distinctness from mental disorders already included in Section
(r=0.31, p=0.030), and not significant for GAD (r=–0.07, p=0.653). II of the DSM. In addition, the PGD diagnosis proved remark-
We could not estimate the temporal stability for PTSD because ably stable between the T1 and T2 assessments approximately
no participants with T2 data met criteria for PTSD at T1 (and 7.4 months apart (r=0.86, p<0.001) and much more stable than
only one study participant met criteria for PTSD at T2). MDD (r=0.31, p=0.030) or GAD (r=–0.07, p=0.653). These results
suggest that PGD fills a diagnostic gap left open by other mental
disorders secondary to bereavement. Furthermore, they show
DISCUSSION that PGD is likely not to remit with the passage of time and to
require specialized treatment.
Results of analyses of data from independent Yale, Utrecht With respect to concurrent and predictive validity, we first
and Oxford bereavement studies suggest that both the PG-13-R sought to determine if the intensity of PGD symptoms alone
and the DSM-5-TR PGD diagnostic criteria possess desirable (excluding impairment, the DSM criterion D) would predict
performance characteristics. The symptoms were uniformly distress and dysfunction. The PG-13-R symptom score proved
higher in the Utrecht sample, which is unsurprising given that to be highly predictive of both concomitant and future distress
this sample was recruited via mental health professionals. Across and dysfunction, indicating that the severity of these symptoms
all three datasets, the preoccupation item was infrequently re- themselves is pathological even without “stacking the deck” by
ported at syndromal levels. This was most noticeable in the Yale requiring the fulfillment of an impairment criterion.
data, where syndromal level preoccupation was found in <3% Next, we sought to determine the threshold score of these
of the sample. Such low prevalence is an undesirable property symptoms that optimized agreement with meeting the B and C
for a “gatekeeper” item, which suggests that it might have been symptom criteria for PGD in the DSM. We found that the PG-
preferable to have only “yearning” in the B criterion for PGD in 13-R symptom score of 30 was the optimal threshold score across
the DSM. the three datasets. Finally, we sought to evaluate and compare
The weakest performing item across all the datasets was the concurrent and predictive validity of diagnoses for PGD us-
“avoidance of reminders that the deceased is dead”. Item-total ing the PG-13-R threshold diagnostic score, and, separately, us-
correlations for this item were the lowest of all items examined, ing the DSM criteria B and C, each in combination with meeting
and Cronbach’s alpha improved in the Yale and Utrecht datasets the impairment criterion. Results indicated that both performed
when the avoidance item was removed. It may be the case that extremely well in predicting substantial current and future mala-
avoidance is more a function of fear, with roots in psychological daptive behaviors and outcomes.
trauma, than a function of grief, with roots in an attachment dis- A strength of this study was the use of three independent
turbance. Alternately, there may be a need to revise the item to community-based bereavement cohort samples. A possible
focus on what aspect of the loss is avoided (e.g., avoidance of re- weakness was the fact that the wording for the PG-13-R ques-
minders of the death as an event may be more a traumatic stress tions was slightly different in the three studies. The Utrecht
response, while avoidance of reminders that the deceased is tru- sample was uniformly more distressed than the Yale and Oxford
ly gone may be the most relevant to disturbed grief). Future stud- samples, which is understandable given that Utrecht partici-
ies are needed to confirm whether the avoidance item should be pants were recruited via mental health care providers, who are

104 World Psychiatry 20:1 - February 2021

more likely to encounter distressed bereaved individuals. The in prolonged grief disorder (PGD): a systematic review. Psychiatry Res Neu-
roimaging 2020;303:111135.
Yale and Utrecht samples were predominantly comprised of 11. Boelen PA, Reijntjes A, Djelantik AAAMJ et al. Prolonged grief and depres-
widowed persons, which was not the case for the Oxford sam- sion after unnatural loss: latent class analyses and cognitive correlates. Psy-
ple (~80% to ~30%, respectively). With respect to ethnicity, all chiatry Res 2016;240:358-63.
12. Johnson JG, Zhang B, Greer JA et al. Parental control, partner dependency,
three samples nearly entirely consisted of people of Caucasian and complicated grief among widowed adults in the community. J Nerv
ethnicity. Ment Dis 2007;195:26-30.
In conclusion, three independent community-based sam- 13. Wright AA, Keating NL, Balboni TA et al. Place of death: correlations with
quality of life of patients with cancer and predictors of bereaved caregivers’
ples showed that the PG-13-R is a reliable tool for assessing grief mental health. J Clin Oncol 2010;28:4457-64.
symptoms on a dimensional scale. A PG-13-R symptom score 14. Maciejewski PK, Zhang B, Block SD et al. An empirical examination of the
of 30 or greater identifies syndromal-level PGD symptomatol- stage theory of grief. JAMA 2007;297:716-23.
15. Prigerson HG, Bierhals AJ, Kasl SV et al. Traumatic grief as a risk factor for
ogy. The dimensional PG-13-R symptom score, the diagnosis of mental and physical morbidity. Am J Psychiatry 1997;154:616-23.
PGD using the PG-13-R threshold symptom score of 30 plus the 16. Maciejewski PK, Maercker A, Boelen PA et al. “Prolonged grief disorder”
impairment criterion, and the diagnosis of PGD using the new and “persistent complex bereavement disorder”, but not “complicated grief”,
are one and the same diagnostic entity: an analysis of data from the Yale Be-
DSM-5-TR criteria all predict enduring distress and dysfunction. reavement Study. World Psychiatry 2016;15:266-75.
Thus, the PG-13-R and the new DSM-5-TR criteria for PGD ap- 17. Reynolds CF 3rd, Miller MD, Pasternak RE et al. Treatment of bereavement-
pear to be reliable and valid measures for the classification of related major depressive episodes in later life: a controlled study of acute and
continuation treatment with nortriptyline and interpersonal psychotherapy.
bereaved individuals with maladaptive grief responses. Future Am J Psychiatry 1999;156:202-8.
research is needed to confirm their psychometric performance 18. Shear K, Frank E, Houck PR et al. Treatment of complicated grief: a ran­
in more ethnically diverse samples. domized controlled trial. JAMA 2005;293:2601-8.
19. Maercker A, Brewin CR, Bryant RA et al. Diagnosis and classification of dis-
orders specifically associated with stress: proposals for ICD-11. World Psy-
chiatry 2013;12:198-206.
ACKNOWLEDGEMENTS 20. World Health Organization. ICD-11 guidelines. https://gcp.network/en/.
This work was supported by grants from the US National Cancer Institute (nos. 21. Boelen PA, Lenferink LIM. Comparison of six proposed diagnostic criteria
CA197730 and CA218313), the US National Institute of Minority Health and sets for disturbed grief. Psychiatry Res (in press).
Health Disparities (no. MD007652), the US National Institute of Nursing Re- 22. Prigerson HG, Maciejewski PK. Prolonged Grief Disorder (PG-13) scale. Bos-
search (no. NR018693), the US National Institute on Aging (no. AG049666), the ton: Dana-Farber Cancer Institute, 2008.
US National Institute of Mental Health (no. MH121886), the US National Center 23. Prigerson HG, Jacobs S. Traumatic grief as a distinct disorder: a ration-
for Advancing Translational Science (no. TR002384), the Wellcome Trust (no. ale, consensus criteria, and a preliminary empirical test. In: Stroebe MS,
200796), the National Institute for Health Research (NIHR) Biomedical Re- Hansson RO, Stroebe W et al (eds). Handbook of bereavement research:
search Centre, based at Oxford University Hospitals National Health System consequences, coping, and care. Washington: American Psychological As-
(NHS) Trust (no. NIHR-INF-0085), and the Oxford Health NIHR Biomedical sociation, 2001:613-45.
Research Centre. The views expressed are those of the authors and not neces- 24. Boelen PA, de Keijser J, Smid G. Cognitive-behavioral variables mediate
sarily those of the supporting institutions. The authors are grateful to A. Ehlers the impact of violent loss on post-loss psychopathology. Psychol Trauma
for her support and supervision of the Oxford Grief Study. A smart pdf version 2015;7:382-90.
of the PG-13-R is available at https://endoflife.weill.cornell.edu/sites/default/ 25. Smith KV, Ehlers A. Cognitive predictors of grief trajectories in the first
files/file_uploads/pg-13-r.pdf. months of loss: a latent growth mixture model. J Consult Clin Psychol
2020;88:93-105.
26. First MB, Spitzer RL, Gibbon M et al. Structured Clinical Interview for DSM-
IV Axis I Disorders, Non-Patient Version (SCID-V-NP). New York: Biometrics
REFERENCES
Research Department, New York State Psychiatric Institute, 1995.
1. Freud S. Mourning and melancholia. In: Rickman J (ed). A general selection 27. Latham AE, Prigerson HG. Suicidality and bereavement: complicated grief
from the works of Sigmund Freud. New York: Doubleday, 1957:124-40. as psychiatric disorder presenting greatest risk for suicidality. Suicide Life
2. Breen LJ, Penman EL, Prigerson HG et al. Can grief be a mental disorder?: An Threat Behav 2004;34:350-62.
exploration of public opinion. J Nerv Ment Dis 2015;203:569-73. 28. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-
3. Bambauer KZ, Prigerson HG. The Stigma Receptivity Scale and its asso- 36). I. Conceptual framework and item selection. Med Care 1992;30:473-
ciation with mental health service use among bereaved older adults. J Nerv 83.
Ment Dis 2006;194:139-41. 29. Engelhard IM, Arntz A, van den Hout MA. Low specificity of symptoms on
4. Tang S, Chow AYM, Breen LJ et al. Can grief be a mental disorder? An on- the post-traumatic stress disorder (PTSD) symptom scale: a comparison of
line survey on public opinion in mainland China. Death Stud 2020;44:152- individuals with PTSD, individuals with other anxiety disorders and indi-
9. viduals without psychopathology. Br J Clin Psychol 2007;46:449-56.
5. Johnson JG, First MB, Block S et al. Stigmatization and receptivity to mental 30. Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II.
health services among recently bereaved adults. Death Stud 2009;33:691- San Antonio: Psychological Corporation, 1996.
711. 31. Weathers F, Litz B, Keane T et al. The PTSD Checklist for DSM-5 (PCL-5).
6. Prigerson HG, Frank E, Kasl SV et al. Complicated grief and bereavement- www.ptsd.va.gov.
related depression as distinct disorders: preliminary empirical validation in 32. Kroencke K, Spitzer R, Williams J. The PHQ-9: validity of a brief depression
elderly bereaved spouses. Am J Psychiatry 1995;152:22-30. severity measure. J Gen Int Med 2001;16:606-13.
7. Prigerson HG, Maciejewski PK, Reynolds CF 3rd et al. Inventory of Compli- 33. Mundt JC, Marks IM, Shear MK et al. The Work and Social Adjustment
cated Grief: a scale to measure maladaptive symptoms of loss. Psychiatry Res Scale: a simple measure of impairment in functioning. Br J Psychiatry
1995;59:65-79. 2002;180:461-4.
8. Prigerson HG, Horowitz MJ, Jacobs SC et al. Prolonged grief disorder: psy- 34. Kabacoff R. Determining the dimensionality of data: a SAS Macro for parallel
chometric validation of criteria proposed for DSM-V and ICD-11. PLoS Med analysis. SAS Users Group International 28, Seattle, March 2003.
2009;6:e1000121. 35. Boelen PA, Van Den Bout J, De Keijser J et al. Reliability and validity of the
9. Boelen PA, van den Bout J. Complicated grief, depression, and anxiety as dis- Dutch version of the Inventory of Traumatic Grief (ITG). Death Studies
tinct postloss syndromes: a confirmatory factor analysis study. Am J Psychia- 2003;27:227-47.
try 2005;162:2175-7. 36. Pohlkamp L, Kreicbergs U, Prigerson HG et al. Psychometric properties of
10. Kakarala SE, Roberts KE, Rogers M et al. The neurobiological reward system the Prolonged Grief Disorder-13 (PG-13) in bereaved Swedish parents. Psy-

World Psychiatry 20:1 - February 2021 105

 chiatry Res 2018;267:560-5. and posttraumatic stress disorder among bereaved Kosovar civilian war sur-
37. He L, Tang S, Yu W et al. The prevalence, comorbidity and risks of prolonged vivors: a preliminary investigation. Int J Soc Psychiatry 2010;56:288-97.
grief disorder among bereaved Chinese adults. Psychiatry Res 2014;219:347- 40. Fisher JE, Mauro C, Cozza SJ et al. Examination of factor structure of the
52. inventory of complicated grief (ICG) in a sample of bereaved military fam-
38. Delalibera M, Coelho A, Barbosa A. Validação do instrumento de aval- ily members with persistent and elevated grief. Int J Methods Psychiatr Res
iação do luto prolongado para a população portuguesa. Acta Med Port 2017;26:e1571.
2011;24:935-42.
39. Morina N, Rudari V, Bleichhardt G et al. Prolonged grief disorder, depression, DOI:10.1002/wps.20823

106 World Psychiatry 20:1 - February 2021