Review

Reviewed Work(s): The Seductions of Quantification: Measuring Human Rights, Gender

Violence, and Sex Trafficking by Sally Engle Merry
Review by: David Cingranelli
Source: Human Rights Quarterly, Vol. 39, No. 1 (February 2017), pp. 239-246
Published by: The Johns Hopkins University Press
Stable URL: https://www.jstor.org/stable/44488982
Accessed: 03-10-2025 07:29 UTC

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide
range of content in a trusted digital archive. We use information technology and tools to increase productivity and
facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected].

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at
https://about.jstor.org/terms

The Johns Hopkins University Press is collaborating with JSTOR to digitize, preserve and
extend access to Human Rights Quarterly

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
2017 Book Reviews 239
eth century were understood by theorists century internationalist thinking on race.
and activists at the beginning of the cen- However, her general point about con-
tury as the engines of internationalism. tingency, contradictions, and tensions in
The key to overcoming the tendency to internationalism is a valid and important
study internationalism and nationalism one. Finally, her discussion of anarchist
as separate and incompatible movements and socialist internationalism, after her
and concepts, as well as the resulting claim about the formation of a liberal
errors, is, she reasserts, to overcome the consensus early in the book, is perhaps
false dichotomy of internationalism as too minimal. Nevertheless, the book is
Utopian and nationalism as realist, and a great contribution to the literature on
to recognize the mixture of idealist and internationalism and nationalism. It will
realist elements in both traditions. be indispensable to students and scholars
Sluga manages to provide a sweeping interested in the history of those move-
survey, a cluster of convincing arguments, ments and of human rights, race, empire
and a tremendous volume of evidence and social scientific theories thereon in
on the quantity and scope of twentieth- the long twentieth century.
century internationalist ideas, move-
Katherine Sorrels*
ments, and organizations in a concise
volume. Overall the book is clear and Assistant Professor of History

compelling, but there are a few points University of Cincinnati

that could be clearer. She intends to * Katherine Sorrels is Assistant Professor of

show the importance of internationalism History at the University of Cincinnati. She

through its sheer variety and magnitude, works in modern Central European cultural
and intellectual history focusing on themes
but in a few passages (particularly in from Jewish history, history of nationalism and
Chapter Two) this makes the thread of internationalism, history of the scientific study
her argument somewhat challenging to of race, and history empires and colonialism.
follow. Similarly, her discussion of the She is author of Cosmopolitan Outsiders : Im-
tensions and contradictions in interna- perial Inclusion, National Exclusion, and the
Pan-European Idea, 1900-1930 (New York:
tionalist thought on race and empire Palgrave Macmillan, 2016).
is sometimes a little difficult to follow.
In particular, in her effort to highlight
contestedness, she sometimes fails to
Sally Engle Merry, The Seductions of
do justice to points of coherence. When
viewed through the lens of intellectual
Quantification: Measuring Human
traditions such as social evolutionism, Rights/ Gender Violence , and Sex
which were pervasive, examples like Trafficking (University of Chicago
the favorable light in which W.E.B. Du Press 2016), ISBN 978-0-226-26128-
Bois and Pän-African activists viewed
7, 249 pages.
the Mandates System as a pathway to
social and economic improvement seem
Sally Engle Merry's The Seductions of
less contradictory. The same applies to
Quantification: Measuring Human Rights,
Sluga's discussion of the 1 91 1 Universal
Gender Violence, and Sex Trafficking
Races Congress, which at points takes
describes and critiques the process of
the coherence (however objectionable)
producing and analyzing numerical indi-
of evolutionist perspectives into account,
cators of human rights, gender violence,
but at others uses it as an example of the
and sex trafficking. She believes that the
tensions and contradictions of twentieth-

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
240 HUMAN RIGHTS QUARTERLY Vol. 39
process of quantification is highly politi- est in her subject and a bias against her
cal and ideological, and that quantitative argument.
measurement systems constitute a form I also read this book with great respect
of power, incorporating theories about for the views of this particular author. She
social change in their design. She empha- is an acknowledged expert on evaluating
sizes that measurement systems are de- the performance of respect for human
veloped by powerful organizations, and rights and other aspects of good gover-
the organizational value biases behind nance. She has written several previous
those systems are rarely explicit. Quan- books and articles on related subjects,
tification itself contributes to a "myth of and this book is filled with interesting
objectivity," meaning that the truth of examples drawn from her personal ex-
most things can be found in numbers. periences. She discusses her active par-
One of the main problems with ticipation helping intergovernmental and
quantification, she contends, is that the nongovernmental organizations develop
production of quantified measurements is creative ways to evaluate progress in
not as open and transparent as it should human rights, including reducing gender
be. If it were clearer, we would be able to violence and sex trafficking.
see that it is shaped by "ideology, inertia, Overall, I agree with the author's
social and political influence, inadequate diagnosis of the problems presented by
data, and the pragmatic compromises that quantification, and with her assessment
poor data require."1 In several chapters, that there is a trend towards produc-
she provides examples showing that what ing more numerical scores designed
is measured and how it is measured to capture complex human rights and
usually depends upon the preferences governance concepts. She accurately
of powerful organizations. As a result of describes the strong trend towards greater
this bias and other problems detailed in use and increasingly uncritical and naïve
her book, she concludes that "[n]ot all use of those scores by scholars and policy
that should be counted is counted, nor makers. She does a good job of explain-
does counting itself necessarily provide ing the reasons for this trend, or, as she
an accurate picture of a situation or its refers to it in her title, the reasons for "the

explanation."2 seductions of quantification."

In the interest of full disclosure, the I also found her book to be very read-
reader should know that I have spent able despite the fact that the discussion
most of my professional life quantifying sometimes addresses technical subjects
cross-national variations in government and provides detailed accounts of the
respect for a variety of human rights politics behind the development of
such as freedom of religion, freedom of numerical indicators. Her book is well
speech, and press. So I personally have organized and well written. It is com-
contributed and continue to contribute posed at a level appropriate for advanced
to the "seduction of quantification" the undergraduate college students. I will
author describes. Therefore, I began my probably assign the first two chapters and
reading of this work with a great inter- the conclusion to the book as required

1 . Sally Engle Merry, The Seductions of Quantification: Measuring Human Rights, Gender Violence,
and Sex Trafficking 220 (2016).
2. Id.

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
2017 Book Reviews 241
reading for my undergraduate seminar surable terms."3 She notes that numerical
on measuring human rights. scores, once constructed, convey an aura
The book contains an excellent, up-to- of objective truth and scientific authority
date review of the relevant literature. It is when they are really not very objective at
a must read for scholars, policy makers, all. She discusses the disparity between
and policy analysts who produce or use the kinds of information produced by
these types of quantitative indicators and qualitative, locally informed observations
for policy makers who want to learn more and the kinds of information produced
about the pitfalls of evidence-based gov- through a process designed to produce
ernance. Less sophisticated readers (those comparable, numerical scores for all
not directly involved in the knowledge countries of the world. She laments
production and consumption process) the fact that an "indicator culture" has
probably will not fully appreciate the emerged in which quantitative indica-
nuances of this account. tors dominate much of the scholarly and
In the remainder of this essay, I pro- policymaking discourse. Chapter One
vide a brief description of the organiza- also discusses some of the most success-
tion of the book and some highlights from ful, best known, widely used indicators,
several chapters. Here and there, I will such as the Human Development Index,
sprinkle in some of my own criticisms and why they are successful. She notes
of the author's argument. But my main that the most well-known and widely
purpose is to give the reader a sense of used quantitative indicators were de-
the contribution of the book, so they can veloped and used by leading, powerful
decide whether to read it themselves. institutions, such as the United Nations,
Personally, I am glad I read this book even Freedom House, and the World Bank.
though I criticize the author's too partisan In Chapter One, I would have
case against quantification. preferred that the author discuss the
The author begins by providing an strengths and weaknesses of quantitative
accurate, conceptual explanation of versus qualitative research for develop-
quantification as a way of knowing about ing theories, gathering information for
the world (Chapters One and Two). She policymaking, or for evaluating policies.
then provides detailed examples of the She does not actually claim that qualita-
serious, distinctive limitations of quan- tive research is better than quantitative
tifying the concepts of gender violence research, but she frequently suggests
(Chapters Three and Four), sex trafficking that qualitative evidence should be used
(Chapter Five and Six), and human rights along with quantitative indicators and
(Chapter Seven). She summarizes her ar- quantitative research. For example, the
gument against quantification in Chapter penultimate sentence in the book is " [t] he
Eight, the conclusion. Anyone with any narrative ethnographic account provides
interest in the subject of her book would an important complement to quantifica-
benefit from reading Chapters One, Two tion."4 I have heard similar exhortations
and Eight. over the past thirty years. However,
In Chapter One, she defines quantifi- no one, including this author, tells the
cation "as the use of numbers to describe reader how scholars and policy makers
social phenomena in countable and mea- should combine quantitative indicators

3. Id. at 1 .
4. Id. at 222.

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
242 HUMAN RIGHTS QUARTERLY Vol. 39
and qualitative evidence to reach their that they are not unduly influenced by
conclusions. In fact, the author never personal or organizational biases. For
actually defines qualitative research and example, the proper application of the
never explicitly compares and contrasts scientific method precludes choosing
the strengths and weaknesses of quantita- cases to prove a point. Instead, a random
tive and qualitative evidence. Here is the sample of cases or the full population of
short version of that comparison. all relevant cases is examined.
Quantitative research uses numerical The method of science is also ob-
indicators of concepts, and it uses statisti- jective in the sense it should be as
cal methods. Quantitative researchers use transparent as possible. Transparency
or develop indicators that are intended should extend to the description of how
to be valid in that they should represent quantitative indicators were constructed.
or capture the true level of a concept of The main purpose of transparency is to
interest such as, "the amount of domestic allow others who may be skeptical of the
violence" in various societies. Sally Engle findings the ability to exactly repeat the
Merry is correct when she says that few, "experiment" to see if the same or very
if any, quantitative indicators completely similar results can be obtained. Transpar-
capture the phenomena they seek to ency also allows others to develop alter-
measure, that there are many ways to native quantitative indicators of a concept
conceptualize a concept like "domestic like "domestic violence" emphasizing a
violence," and many different ways to different conceptualization of the idea of
quantify almost any concept. The author domestic violence. Both types of replica-
makes an additional, important contribu- tions are common in the social sciences.
tion by providing numerous concrete Qualitative research, on the other
examples of these points in chapters hand, is rarely "objective" in any sense
Three through Seven. of the word. The term, "qualitative," cov-
But most quantitative scholars already ers a wide range of approaches, but by
recognize and openly acknowledge these definition none of those approaches rely
problems. Quantitative studies are politi- on quantitative (numerical) indicators.
cal and biased in the sense Sally Engle Such work has tended to focus on one or
Merry says they are. To a large extent, a small number of cases, to use intensive
values determine what is studied, what interviews or depth analysis of historical
is measured, and how indicators are materials, to be discursive in method, and
constructed. This is the author's emphasis to be concerned with a rounded or com-
in her book. prehensive account of some event or unit.
The "objective" aspect of quantitative Even though they have a small number
studies stems from the fact that they use of cases, qualitative researchers generally
unearth enormous amounts of information
the scientific method. The purpose of
from their studies. Sometimes this kind of
quantitative research is to abstract from work in the social sciences is linked with
particular instances to develop a general area or case studies where the focus is on
description, to test causal hypotheses, a particular event, decision, institution,
and to build explanatory theories. The location, issue, or piece of legislation.5
methods of science are designed so

5. Gary King, Robert O. Keohane, & Sidney Verba, Designing Social Inquiry: Scientific Inference in
Qualitative Research 4 (1 994).

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
2017 Book Reviews 243
While qualitative research can be sci- capricious uses of discretion by those in
entific, usually it is not. When it is not power positions within organizations. On
scientific, qualitative research has its the minus side, the only side discussed
own serious limitations, which the author by the author of this book, evidence-
ignores. The weaknesses most commonly based governance may give too much
cited in the literature are subjectivity, lack power to experts in quantification who
of general izabi lity, and lack of replicabil- develop and implement measurement
ity. These drawbacks in most qualitative systems. The quantified systems are
research designs are important. They help usually transparent, but are perceived
us understand the strong trend towards by many as biased. The author uses the
use of quantitative indicators and scien- example of the use of quantitative indica-
tific methods. For this reason, the author tors to evaluate higher education faculty
should have discussed the limitations of and institutions in the British education
qualitative research. Instead, the author system. Ftowever, she never mentions the
simply provides examples showing how problems in the British faculty evaluation
qualitative studies sometimes revealed system before administrators turned to
insights missed when only quantitative the use of more quantitative indicators
indicators were used. of faculty performance.
One of the most interesting parts of Quantification is here to stay, mainly
Chapter One is the author's discussion because citizens want to know how their
of the growing emphasis on "evidence- governments are performing. They want
based" governance. By this, she refers to make numerical comparisons over
to a broad range of strategies that rely time and across places. They want to
on quantitative knowledge as the basis know, for example, how many unarmed
of making policy decisions. Benchmark- civilians are killed by police each year, in
ing is a well-known example of this what places, and whether the number of
approach. The author is correct in her police killings is increasing or decreasing.
conclusion that there has been a move to- Politicians responsible for making public
wards more evidence-based governance policy need the same kind of information.
around the world. For example, in the In order to change anything, it must
United States President Obama, unlike first be measured (or quantified). Intergov-
his predecessors, advocated that mem- ernmental organizations (IGOs) such as,
bers of his administration use evidence the United Nations, that are responsible
from quantitative social science research for monitoring the compliance of national
to help shape their proposals for the governments with international human
design and funding of social programs. rights norms against state killing, want
The author notes that, in its extreme form, similar information about all governments
evidence-based governance means shift- of the world. It will be used in a variety of
ing responsibility for decision making, ways including the UN Universal Periodic
"from individual, discretionary judgment Review of national human rights records.
to systems of measurement established Qualitative information of various
by experts."6 types is good, but it is not as useful for
On the plus side, this type of change theory building or policy making. Sticking
reduces possibilities for arbitrary and with the example of police killings of un-

6. Merry, supra note 1 , at 1 1 .

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
244 HUMAN RIGHTS QUARTERLY Vol. 39
armed civilians, while it is good that we other important human rights, such as the
have other sources of information, such as rights of children to freedom of speech.8
cell phone videos of deadly police-citizen Consistent with this line of argument, in
interactions, that type of non-quantitative Chapter One she observes that the most
information also has its limitations. If well-known and widely used quantitative
we relied on it alone for making public indicators of the concepts highlighted in
policy, we certainly would do so at our her book were developed and used by
peril. Even if we combined videos with leading, powerful institutions such as
other non-quantitative evidence collected Freedom House or the World Bank.
in a non-systematic way, such as citizen To this account, I would add that even
testimonials, it would not be adequate as the research questions that are apparently
the sole basis for making policy. freely chosen by many university-based
It is not obvious how the two types of scholars are, in fact, strongly biased
evidence should be combined by those towards those questions of greatest in-
who would like to make new policies terest to powerful national and global
that will reduce such killing in the fu- organizations that provide funding for
ture. The author probably recognizes the social science research. A human rights
drawbacks of too heavy reliance on either scholar interested in studying the rela-
quantitative or qualitative research for tionship between the level of human
decision-making, but she contends that rights repression in a nation state and
the current balance trusts the experts in the amount of terrorism it experienced,
quantification too much. I would have for example, would have an excellent
liked a more even-handed discussion of chance of attracting funding. Research
the dilemma. on most other topics would be much
Chapter Two, titled "Indicators as a less likely to be funded. Because of the
Technology of Knowledge," emphasizes biases of funding organizations, proposals
the relationship between power, technol- for quantitative research about terror-
ogy, and knowledge creation. It is the ism, insurgency, and counter-insurgency
most interesting, provocative, abstract, also would have a much better chance
and philosophical of all the chapters of success than research proposals that
in the book. It is valuable for several took a more humanistic, ethnographic,
reasons including an excellent review of philosophical, legal, or in other ways
previous, related literature. For example, non-quantitative approach.
there is a thought-provoking discussion of Chapter Two also explains how and
Michael Foucaulťs analysis of the power why the technology of quantification
and knowledge relationship.7 is important too. Scholars tend to study
The author explains that governments, those things that are easier to quantify,
IGOs, and foundations pour large sums while ignoring important subjects where
of money into measurement of some quantification is more difficult, and per-
concepts while ignoring measurement of haps, impossible. In general, she notes,
others. She notes that those who are influ- it is much easier to measure economic
ential are interested in measuring human phenomena, such as, "gross domestic
and drug trafficking, but not in measuring product" (GDP) than it is to measure

7. Id. at 28-29.
8. Id. at 29.

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
2017 Book Reviews 245
social ones. Among social phenomena, transparent. Perhaps there are some or-
it is easier to measure some rather than ganizations that hide their value biases,
others. This is an interesting discussion. but for the most part, I have found them
The author notes that many concepts to be apparent.
thought to be "unmeasu rabie" in the past, The middle chapters provide detailed
such as GDP are now routinely quantified examples of the serious, distinctive limita-
and the "new" measures are no longer tions of quantification of gender violence
controversial. Many of the most impor- (Chapters Three and Four), sex trafficking
tant contributions to the social science (Chapter Five and Six), and human rights
literature developed new concepts and (Chapter Seven). These chapters are filled
quantitative measures of those concepts. with important information about the
Despite the accuracy and importance history of the effort to quantify these
of the points the author makes in the first concepts. She describes the develop-
two chapters, I was put off by the vaguely ments that led to the identification of
conspiratorial tone of the writing. To me, gender violence and sex trafficking as
these chapters convey a sense that the problems, the motivations behind quan-
author has discovered a dark, exploitative tification, the various forms of counting,
scheme and is now revealing it to us, her the different organizations that developed
readers. For example, in the conclusion indicators, debates over what numbers
to Chapter One, she writes: "Those who are best, what ideas different indicators
are measured typically lack a voice in measure, what they leave out, why some
the construction of the categories and ideas are ignored, and how the indicators
measurements. Moreover, subordinated have been used by scholars and policy
groups have difficulty resisting or chang- makers. These chapters are thorough,
ing indicators."9 up-to-date, and informative. Even though
While true, these observations are not I had already encountered much of the
particularly constructive. What is it that information contained in the chapters
the author wants? Is there any process of about sex trafficking and human rights,
quantification of problems such as do- I benefited from the comprehensive, ac-
mestic violence or human rights violation curate, detailed, chronological treatment
that would meet the author's standard of the author provided in the chapters on
fairness? Is there any existing quantitative those topics.
measure that meets it? I do not think so. In Sally Engle Merry is right about many
a capitalist system, we should not expect things. Quantification is only one way
things to be much different from the way of knowing about the world, and quan-
they are right now. tification as a way of knowing has seri-
How could powerful organizations ous, distinctive limitations. I especially
that design quantitative measurement agree with the very last sentence of her
systems be more explicit about the book: "We rely on numbers alone at our
value systems which underlie their work? peril."10 The main contribution of this
Despite her contentions, I do not think book is that it raises our consciousness
there is any way to make the production about the limitations of quantification
of quantitative indicators more open and of concepts and quantitative research as
ways of knowing about the world.

9. Id. at 25.
10. Id. at 222.

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms
246 HUMAN RIGHTS QUARTERLY Vol. 39
However, as I read her book, I was Philippe Sands, East West Street:
reminded of a quotation often attributed On the Origins of "Genocide" and
to Winston Churchill, who allegedly said,
"Crimes Against Humanity" (New
"Democracy is the worst form of govern-
York: Alfred A. Knopf, 2016) ISBN
ment, except for all the others." His ob-
servation brings us to the main weakness 978-0385350716, 425 pages.
of this book: the author does not present
any practical solution to the problem she Philippe Sands may not be the first per-
describes. It is not enough to say that son to look at the lives and teachings of
there are problems with quantification. Hersch Lauterpacht and Raphael Lemkin,
What should we do to reduce the bad examine their seminal contributions to in-

effect of those problems? Quantification ternational human rights law, or note that
is so central to the method of science they both attended the same (to North
that an argument against quantification Americans, unfamiliar) law school in
East Galicia.1 But a confluence of several
comes perilously close to an argument
against science as a method of knowing. coincidences led Sands, a law professor
Abandoning the scientific approach to and practicing human rights attorney, to
making public policies (or even relying see the promise of an extraordinary story.
more heavily on non-scientific ways of Or rather, the promise of East West Street's

knowing) would be about as useful as several intersecting extraordinary stories.

forsaking the idea that democracy is the The chain of coincidences began with
best way to organize political systems. an invitation for Sands to lecture at Jan
Kazimierz University Law School in the
David Cingranelli* city now known as Lviv, in the Ukraine
Binghamton University, SUNY where Lauterpacht and Lemkin first stud-
ied law. Sands claims, not without reason,
* David Cingranelli is a Professor of Political Sci-
ence. He has written widely on human rights , that the university marks the origin of the
democracy, and governance. His 2007 book modern system of international justice.
with Rodwan Abouharb, Human Rights and Lviv, with its neoclassical architecture
Structural Adjustment, (Cambridge University
and art neuveau trim, was once dubbed
Press) demonstrated the negative human rights
the "Paris of the East."
impacts of World Bank and IMF program lend-
ing in developing countries. He is a former The city has quite a history. At the
President of the Human Rights Section of the beginning of Sands' story, Lviv was
American Political Science Association. Until
called Lemberg. The historic city is also
2012, he served as the co-director of the
Cingranelli and Richards (CIRI) Human Rights
identified on maps as Lvov, Lwow, or
Data Project, the largest and most widely used Lviv when the city was under alternating
human rights data set in the world. Presently, German, Polish, Russian, or Ukrainian
he and Mikhail Filippov are working in collabo- control. Located at what Sands calls the
ration with the United States Political Instability
very fault lines between North and South
Task Force on a successor to the CIRI project,
which will be called the "Rights" data project. and East and West Europe, this medium
sized city changes sovereign ownership

1. The first page of a Google search of "Lauterpacht" yielded a 2010 article Ana Filipa
Vrdoljak, Human Rights and Genocide : The Work of Lauterpacht and Lemkin in Modern
International Law, 20 Eur. J. Int'l L. 1163 (2010).

This content downloaded from 192.16.140.105 on Fri, 03 Oct 2025 07:29:29 UTC
All use subject to https://about.jstor.org/terms