(Ebook) Handbook of research methods in social and

personality psychology by Harry T. Reis (editor);

Charles M. Judd (editor) ISBN 9780511996481, 0511996489
Pdf Download

https://ebooknice.com/product/handbook-of-research-methods-in-
social-and-personality-psychology-43189724

★★★★★
4.8 out of 5.0 (46 reviews )

DOWNLOAD PDF

ebooknice.com
 (Ebook) Handbook of research methods in social and
personality psychology by Harry T. Reis (editor); Charles M.
Judd (editor) ISBN 9780511996481, 0511996489 Pdf Download

EBOOK

Available Formats

■ PDF eBook Study Guide Ebook

EXCLUSIVE 2025 EDUCATIONAL COLLECTION - LIMITED TIME

INSTANT DOWNLOAD VIEW LIBRARY

 We believe these products will be a great fit for you. Click
the link to download now, or visit ebooknice.com
to discover even more!

(Ebook) Handbook of Research Methods in Social and Personality

Psychology by Harry Reis, Charles M. Judd ISBN 9781107011779,
9781107600751, 9782013024730, 1107011779, 1107600758, 2013024738

https://ebooknice.com/product/handbook-of-research-methods-in-social-and-
personality-psychology-25264024

(Ebook) Vagabond, Vol. 29 (29) by Inoue, Takehiko ISBN

9781421531489, 1421531488

https://ebooknice.com/product/vagabond-vol-29-29-37511002

(Ebook) Analytical Profiles of Drug Substances and Excipients 29

by Harry G. Brittain (Eds.) ISBN 9780122608292

https://ebooknice.com/product/analytical-profiles-of-drug-substances-and-
excipients-29-2143150

(Ebook) Organometallic Chemistry, Volume 29 by M. Green ISBN

0854043284

https://ebooknice.com/product/organometallic-chemistry-volume-29-2440106
(Ebook) International Review of Research in Mental Retardation
29 by Laraine Masters Glidden ISBN 9780080495712, 9780123662293,
0080495710, 012366229X

https://ebooknice.com/product/international-review-of-research-in-mental-
retardation-29-1899532

(Ebook) Boeing B-29 Superfortress ISBN 9780764302725, 0764302728

https://ebooknice.com/product/boeing-b-29-superfortress-1573658

(Ebook) 29, Single and Nigerian by Naijasinglegirl ISBN

9781310004216, 1310004218

https://ebooknice.com/product/29-single-and-nigerian-53599780

(Ebook) Handbook of Statistics_29B, Volume 29: Sample Surveys:

Inference and Analysis by Danny Pfeffermann, C.R. Rao ISBN
9780444531247, 0444531246

https://ebooknice.com/product/handbook-of-statistics-29b-volume-29-sample-
surveys-inference-and-analysis-1958348

(Ebook) Landscapes of Holocaust Postmemory (Routledge Research

in Cultural and Media Studies, Volume 29) by Brett Ashley Kaplan
ISBN 9780203842270, 9780415874762, 0203842278, 0415874769

https://ebooknice.com/product/landscapes-of-holocaust-postmemory-routledge-
research-in-cultural-and-media-studies-volume-29-1715540
HANDBOOK OF RESEARCH METHODS IN SOCIAL AND
PERSONALITY PSYCHOLOGY

Second Edition

This indispensable sourcebook covers conceptual and practical issues in

research design in the field of social and personality psychology. Key experts
address specific methods and areas of research, contributing to a compre-
hensive overview of contemporary practice. This updated and expanded sec-
ond edition offers current commentary on social and personality psychology,
reflecting the rapid development of this dynamic area of research over the
past decade. With the help of this up-to-date text, both seasoned and begin-
ning social psychologists will be able to explore the various tools and methods
available to them in their research as they craft experiments and imagine new
methodological possibilities.

Harry T. Reis is Professor of Psychology in the Department of Clinical and Social

Sciences, University of Rochester. He is the coauthor of An Atlas of Interpersonal
Situations and the coeditor of The Encyclopedia of Human Relationships.

Charles M. Judd is College Professor of Distinction in the Department of Psy-

chology and Neuroscience at the University of Colorado at Boulder. He is the
author of Data Analysis: A Model Comparison Approach and Research Methods in
Social Relations.
HANDBOOK OF RESEARCH METHODS
IN SOCIAL AND PERSONALITY
PSYCHOLOGY

Second Edition
Edited by
HARRY T. REIS
University of Rochester

CHARLES M. JUDD
University of Colorado at Boulder
32 Avenue of the Americas, New York, NY 10013-2473, USA

Cambridge University Press is part of the University of Cambridge.

It furthers the University’s mission by disseminating knowledge in the pursuit of

education, learning, and research at the highest international levels of excellence.

www.cambridge.org
Information on this title: www.cambridge.org/9781107600751


C Cambridge University Press 2000, 2014

This publication is in copyright. Subject to statutory exception

and to the provisions of relevant collective licensing agreements,
no reproduction of any part may take place without the written
permission of Cambridge University Press.

First published 2000

Reprinted 2009, 2010, 2011
Second edition 2014

Printed in the United States of America

A catalog record for this publication is available from the British Library.

Library of Congress Cataloging in Publication data

Handbook of research methods in social and personality psychology / [edited by] Harry T. Reis, University of Rochester,
Charles M. Judd, University of Colorado at Boulder. – Second edition.
pages cm
Includes bibliographical references and indexes.
ISBN 978-1-107-01177-9 (hardback : alk. paper) – ISBN 978-1-107-60075-1 (pbk. : alk. paper)
1. Social psychology – Research – Methodology. 2. Personality – Research – Methodology. I. Reis, Harry T.
II. Judd, Charles M.
HM1019.H36 2013
302.07ʹ2–dc23 2013024738

ISBN 978-1-107-01177-9 Hardback

ISBN 978-1-107-60075-1 Paperback

Cambridge University Press has no responsibility for the persistence or accuracy of URLs for external or third-party Internet Web
sites referred to in this publication and does not guarantee that any content on such Web sites is, or will remain, accurate or
appropriate.
 Contents

Contributors page vii

Introduction to the Second Edition ix
Introduction to the First Edition by Harry T. Reis and Charles M. Judd xi
1 Scratch an Itch with a Brick: Why We Do Research 1
SUSAN T. FISKE

PART ONE. DESIGN AND INFERENCE CONSIDERATIONS

2 Research Design and Issues of Validity 11
MARILYNN B. BREWER AND WILLIAM D. CRANO

3 Research Design 27
ELIOT R. SMITH

4 Causal Inference and Generalization in Field Settings: Experimental and

Quasi-Experimental Designs 49
STEPHEN G. WEST, HEINING CHAM, AND YU LIU

5 Field Research Methods 81

ELIZABETH LEVY PALUCK AND ROBERT B. CIALDINI

PART TWO. PROCEDURAL POSSIBILITIES

6 Using Physiological Indexes in Social Psychological Research 101
JIM BLASCOVICH

7 Research Methods in Social and Affective Neuroscience 123

ELLIOT T. BERKMAN, WILLIAM A. CUNNINGHAM, AND MATTHEW D. LIEBERMAN

8 Behavior Genetic Research Methods: Testing Quasi-Causal Hypotheses Using

Multivariate Twin Data 159
ERIC TURKHEIMER AND K. PAIGE HARDEN

9 Methods of Small Group Research 188

NORBERT L. KERR AND R. SCOTT TINDALE

10 Inducing and Measuring Emotion and Affect: Tips, Tricks, and Secrets 220
KAREN S. QUIGLEY, KRISTEN A. LINDQUIST, AND LISA FELDMAN BARRETT

11 Complex Dynamical Systems in Social and Personality Psychology: Theory, Modeling,

and Analysis 253
MICHAEL J. RICHARDSON, RICK DALE, AND KERRY L. MARSH

12 Implicit Measures in Social and Personality Psychology 283

BERTRAM GAWRONSKI AND JAN DE HOUWER

v
vi CONTENTS

13 The Mind in the Middle: A Practical Guide to Priming and Automaticity Research 311
JOHN A. BARGH AND TANYA L. CHARTRAND

14 Behavioral Observation and Coding 345

RICHARD E. HEYMAN, MICHAEL F. LORBER, J. MARK EDDY, AND TESSA V. WEST

15 Methods for Studying Everyday Experience in Its Natural Context 373

HARRY T. REIS, SHELLY L. GABLE, AND MICHAEL R. MANIACI

16 Survey Research 404

JON A. KROSNICK, PAUL J. LAVRAKAS, AND NURI KIM

17 Conducting Research on the Internet 443

MICHAEL R. MANIACI AND RONALD D. ROGGE

PART THREE. DATA ANALYTIC STRATEGIES

18 Measurement: Reliability, Construct Validation, and Scale Construction 473
OLIVER P. JOHN AND VERONICA BENET-MARTÍNEZ

19 Exploring Causal and Noncausal Hypotheses in Nonexperimental Data 504

LEANDRE R. FABRIGAR AND DUANE T. WEGENER

20 Advanced Psychometrics: Confirmatory Factor Analysis, Item Response Theory, and

the Study of Measurement Invariance 534
KEITH F. WIDAMAN AND KEVIN J. GRIMM

21 Multilevel and Longitudinal Modeling 571

ALEXANDER M. SCHOEMANN, MIJKE RHEMTULLA, AND TODD D. LITTLE

22 The Design and Analysis of Data from Dyads and Groups 589
DAVID A. KENNY AND DEBORAH A. KASHY

23 Nasty Data: Unruly, Ill-Mannered Observations Can Ruin Your Analysis 608
GARY H. MCCLELLAND

24 Missing Data Analysis 627

GINA L. MAZZA AND CRAIG K. ENDERS

25 Mediation and Moderation 653

CHARLES M. JUDD, VINCENT Y. YZERBYT, AND DOMINIQUE MULLER

26 Meta-Analysis of Research in Social and Personality Psychology 677

BLAIR T. JOHNSON AND ALICE H. EAGLY

Author Index 711

Subject Index 736
 Contributors

John A. Bargh, Yale University

Lisa Feldman Barrett, Northeastern University and Harvard Medical School
Veronica Benet-Martı́nez, Pompeu Fabra University
Elliot T. Berkman, University of Oregon
Jim Blascovich, University of California
Marilynn B. Brewer, University of New South Wales
Heining Cham, Fordham University
Tanya L. Chartrand, Duke University
Robert B. Cialdini, Arizona State University
William D. Crano, Claremont Graduate University
William A. Cunningham, University of Toronto
Rick Dale, University of California
Jan De Houwer, Ghent University
Alice H. Eagly, Northwestern University
J. Mark Eddy, University of Washington
Craig K. Enders, Arizona State University
Leandre R. Fabrigar, Queen’s University
Susan T. Fiske, Princeton University
Shelly L. Gable, University of California
Bertram Gawronski, University of Texas at Austin
Kevin J. Grimm, University of California
K. Paige Harden, University of Texas at Austin
Richard E. Heyman, New York University
Oliver P. John, University of California
Blair T. Johnson, University of Connecticut

vii
viii CONTRIBUTORS

Charles M. Judd, University of Colorado at Boulder

Deborah A. Kashy, Michigan State University
David A. Kenny, University of Connecticut
Norbert L. Kerr, Michigan State University
Nuri Kim, Stanford University
Jon A. Krosnick, Stanford University
Paul J. Lavrakas, Northern Arizona University
Matthew D. Lieberman, University of California
Kristen A. Lindquist, University of North Carolina
Todd D. Little, Texas Tech University
Yu Liu, Arizona State University
Michael F. Lorber, New York University
Michael R. Maniaci, University of Rochester
Kerry L. Marsh, University of Connecticut
Gina L. Mazza, Arizona State University
Gary H. McClelland, University of Colorado
Dominique Muller, Pierre Mendes France University at Grenoble, University Institute of France
Elizabeth Levy Paluck, Princeton University
Karen S. Quigley, Northeastern University and Edith Nourse Rogers Memorial (Bedford) VA Hospital
Harry T. Reis, University of Rochester
Mijke Rhemtulla, University of Amsterdam
Michael J. Richardson, University of Cincinnati
Ronald D. Rogge, University of Rochester
Alexander M. Schoemann, East Carolina University
Eliot R. Smith, Indiana University
R. Scott Tindale, Loyola University Chicago
Eric Turkheimer, University of Virginia
Penny S. Visser, University of Chicago
Duane T. Wegener, The Ohio State University
Stephen G. West, Arizona State University
Tessa V. West, New York University
Keith F. Widaman, University of California
Vincent Y. Yzerbyt, Université catholique de Louvain at Louvain-la-Neuve
 Introduction to the Second Edition

When we put together the first edition of this Hand- Social-personality psychologists have always been
book, published in 2000, we scarcely could have imag- quick to capitalize on new methods and technical
ined the pace with which methodological innovation innovations to further their exploration of the pro-
would occur in social and personality psychology. cesses that govern social behavior. Although the field
To be sure, we hoped that the field’s relentless pur- continues to be criticized for overrelying on labora-
suit of ever-more creative and precise methods would tory experiments conducted with college student sam-
continue – a pursuit that the book was intended to ples, we believe that this criticism is short-sighted. As
encourage. Our expectation was that a new edition this volume illustrates, social-personality psychologists
would be needed somewhere in the far distant future. conduct research using diverse approaches, ranging
A mere 13 years later, that time has come. Social- from neuroscientific methods to observational coding
personality psychologists have advanced the frontiers of live interaction, from implicit assessments to every-
of methodology at a far faster rate than we anticipated, day experience studies, and from priming outside
so much so that the prior volume of this Handbook no of awareness to population-based surveys. Further-
longer did justice to the diverse approaches and meth- more, the Internet has made possible access to diverse
ods that define the field’s cutting-edge research. With and specialized samples, an opportunity that social-
these advances in mind, we set out to provide under a personality psychologists have eagerly embraced. Add
single cover a compendium of the most important and to this the sophisticated insights afforded by new or
influential research methods of contemporary social- improved statistical innovations such as dyadic data
personality psychology. analysis, mediation analysis, and multilevel models,
Our goal for this volume is the same as it was and it is readily apparent that our theories are built
for the prior edition: to inform and inspire young on a rich, complex, and mature empirical foundation.
researchers to broaden their research practices in order We suspect that our receptivity to innovation is
to ask and answer deeper, more finely grained ques- one reason for the growing popularity and influence
tions about social life. One sometimes hears that of social-personality psychology. Membership in the
methodological innovation provides little more than Society for Personality and Social Psychology has more
an incremental gain on what is already known. In than doubled since 2000, and social-personality psy-
our opinion, this view is short-sighted. As Greenwald chologists are now often found in schools of business,
(2012) observed, the great majority of Nobel Prizes medicine, and law. The influence of our work extends
in the sciences have been awarded for methodologi- well beyond the field’s traditional borders, so much
cal advances rather than for theoretical contributions. so that Yang and Chiu (2009), in an analysis of cita-
This, he reasons, is because of the synergy between tion patterns in APA journals, identified social psy-
methodology and theory: Existing theories point to chology as being positioned at the center of the psy-
the need for new methods, which then suggest ques- chological sciences. We believe that this influence is at
tions that could not have been envisioned, much less least partly attributable to our leadership in champi-
investigated, with older methods. In this way, new oning methodological innovation. For example, Baron
methods open the door to better understanding of and Kenny’s classic paper on moderation and media-
phenomena. tion, published in the Journal of Personality and Social

ix
x INTRODUCTION TO THE SECOND EDITION

Psychology in 1986, is the most cited article of all time When the two of us entered the field, in the 1970s,
in scientific psychology, with more than 34,000 cita- a young social-personality psychologist could be con-
tions at the time of this writing. sidered well trained after taking two courses in statis-
Changes in the field’s methodology do not occur tics and measurement and one in methods. Fortu-
in a vacuum, of course. Two important developments nately, that is no longer the case; methodological
have been the rapid increase in digital technology and training in most graduate programs is far more exten-
miniaturization, which have led directly to implicit sive and continues for the duration of one’s career.
methods, fMRI, and portable devices for recording Although some may see this as a daunting challenge,
details of everyday behavior, as well as in the acces- we prefer to see it as a sign of the health and vigor
sibility of the Internet, which has opened the door to of our discipline. Social-personality psychologists are
a broader pool of research participants. Other kinds dedicated to obtaining the most enlightening, accu-
of changes have also been influential. For example, rate, and useful understanding of the social world in
the past decade has seen impressive gains in statis- which we live. Taking advantage of methodological
tical methods. Although many of these methods are innovation to imagine and address newer, more infor-
computationally complex, they encourage researchers mative questions is the surest way we know to con-
to ask far more intricate and revealing questions than tinue the progress of the past few decades. We hope
could be asked with t-tests, ANOVAs, and correla- this volume serves as a springboard for the next gener-
tions. These changes notwithstanding, careful read- ation of theoretical advances in social and personality
ers will note that our approach to the research pro- psychology.
cess is still grounded in the basics: a concern for Our every expectation is that the methodologi-
internal validity, an appreciation for the complex- cal advances in the years since the first edition of
ity of generalizability, and the realization that the this volume will only continue to accumulate in the
most useful and accurate insights will come from pro- years ahead. We have little doubt that the future
grams of research that incorporate multiple, diverse promises more appropriate and sophisticated models
methods. of data, greater attention to process and mechanisms,
An easy way to see the rapid pace of methodologi- increased insights from neuroscientific explorations,
cal innovation in social-personality methods is to com- greater attention to data from diverse samples and
pare this edition of the Handbook to its predecessor. settings, and increased insights in the measurement
The roster of chapters in the current edition repre- of automatic responses. And we have no doubt that
sents an extensive revision from the earlier volume. there are further advances lurking down the road that
Twelve chapters are entirely new to this volume, dis- will come with some surprise. Accordingly, in another
cussing topics whose particulars or importance have dozen years (or perhaps sooner), we suspect it will be
emerged since publication of the prior volume. These time for a third edition of this volume. One certain
include treatments of field research, implicit methods, prediction that we make is that we will not be the
methods for social neuroscience and behavior genet- editors of that edition. But we trust that others will
ics, research on the Internet, methods for studying realize the excitement of witnessing the methodologi-
emotion and dynamical systems, multilevel models, cal vitality of the field by preparing that next edition.
advanced psychometrics, missing data, and mediation Throughout this volume we have loved providing wit-
and moderation. An additional introductory chapter ness to the advances in research methods mentioned
presents a compelling picture of why we do research. earlier in this paragraph.
Readers of the first edition will notice that six chap-
ters have been dropped, not because of diminished
relevance but rather because there was no way to REFERENCES
include them and still have the space necessary to
Greenwald, A. G. (2012). There is nothing so theoretical as
describe newer methods. The remaining chapters have a good method. Perspectives on Psychological Science, 7(2),
been, in most cases, thoroughly revamped to reflect 99–108.
recent developments in method or application. We Yang, Y. J., & Chiu, C. Y. (2009). Mapping the structure
believe that the result depicts state-of-the-art meth- and dynamics of psychological knowledge: Forty years
ods in social-personality psychology, at least (we feel of APA journal citations (1970–2009). Review of General
compelled to point out) for today. Psychology, 13(4), 349–356.
 Introduction to the First Edition
HARRY T. REIS AND CHARLES M. JUDD

It is no accident, we believe, that many of the most of research designs and procedures, measurement
influential methodologists in the behavioral sciences methods, and analytic strategies that social psycholo-
happen to identify themselves as social-personality gists employ is, in our view, extraordinary.
psychologists. Throughout the methodological liter- Our goal in putting together this Handbook was
ature in psychology, citations to Robert Abelson, to provide a series of state-of-the-art presentations
Donald Campbell, Thomas Cook, Donald Fiske, David spanning both traditional and innovative methods
Kenny, and Robert Rosenthal, to name just a few, are that have moved and continue to move the disci-
ubiquitous. The reason we believe that this is not an pline forward. The product, we believe, documents the
accident is that social-personality psychologists have incredible wealth of methodological tools that social-
set for themselves a particularly challenging method- personality psychologists have at their disposal. Inten-
ological task. Their domain of inquiry concerns all of tionally, we sought to include chapters that might
social behavior, from intergroup relations and large- strike some readers as a bit unusual in a book devoted
scale social conflict to dyadic interaction and close to research methods. Certainly, some of these top-
relationships. They study individual judgments, cog- ics would not have been included in a book of this
nitions, and affects about social phenomena as well sort 20, or perhaps even 10, years ago. So, for exam-
as the evolution of social norms and interdependent ple, chapters by Hastie and Stasser on simulation,
behaviors at the level of societies. Most recently, entire Collins on studying growth and change, McClelland
cultures, and the belief systems associated with them, on transformations and outliers, Bargh and Char-
have become a major area of interest. And, in the tra- trand on cognitive mediation, Reis and Gable on
dition of Kurt Lewin, social-personality psychologists daily experience methods, and Blascovich on psy-
are firmly committed to a rigorous empirical approach chophysiological measures are a far cry from the tradi-
to whatever they study. They are convinced that a tional chapters on design, measurement, and analysis
strong and reciprocal relationship between theory and that one might routinely expect in a research meth-
evidence is fundamental to the acquisition of knowl- ods textbook. Several statistics chapters are included
edge: that data demand good theories and that theo- because we believe that new developments in statisti-
ries demand quality data. cal methodology make it possible to extract valuable
As a result, social-personality psychologists have insights about social psychological phenomena from
developed and made use of an extensive array of data collected with diverse methods in many different
methodological tools. Although the field is sometimes settings.
criticized for an overreliance on laboratory exper- But then, it was not our goal to provide yet
imentation, in fact the diversity of methodological another research methods textbook cataloging stan-
approaches represented in the leading journals is dard procedures and principles. Many excellent text-
impressive. From surveys to simulations, from lab- books serving this function are already available.
oratory experiments to daily event recordings, from Although this Handbook might well be used as a text-
response latency and physiological measures to think- book, our goal was more ambitious than teaching the
aloud protocols, and from the Internet and palmtop field’s traditional core. Rather, we sought to demon-
computers to paper-and-pencil reports, the diversity strate and highlight the tremendous methodological

xi
xii INTRODUCTION TO THE FIRST EDITION

richness and innovativeness to be found in social measurement approach ought to be employed in each
psychological research, and additionally, to provide and every study that we conducted.
social-personality psychologists with resources for The discipline is coming to realize that this sort
expanding the methodological diversity employed in of triangulation is fundamental not simply in mea-
their research. surement but in all aspects of methodology. In this
Such innovation is central to the legacy we have sense, then, it is fitting that the first chapter in this
inherited from the field’s founders. Social-personality Handbook, by one of Donald Cambell’s students, Mar-
psychologists value their reputation as both rigorous ilynn Brewer, sets the tone for the entire volume.
and clever methodologists; indeed, among the behav- Brewer argues that only through the use of multi-
ioral sciences, social psychologists are notorious for faceted research strategies, adopted not only within
their exacting methodological standards and for the individual studies but also, and much more impor-
pinpoint precision with which the fit of evidence to tant, across an entire program of research, is research
theory is scrutinized. These practices reflect two con- validity in its broadest sense achieved. All the diver-
siderations: the growth of a cumulative literature, sity that is represented in this volume, and the diver-
which allows researchers to ask ever-finer questions sity of methods and approaches yet to be developed,
about phenomena and their mediators and modera- is essential if social-personality research is to produce
tors, and the availability of new technologies capable valid findings, defining validity in its most compre-
of providing information not even imagined a gen- hensive sense: that our conclusions and theories ulti-
eration or two ago. For example, researchers rarely mately provide accurate understandings of the social
investigated questions of mediation in the 1960s. With world that we inhabit.
the advent of computerized tests of cognitive medi- Putting together this volume has inspired in us
ation, sophisticated measures of physiological media- great pride in social-personality psychology’s commit-
tion, and co-variance structure methods for evaluat- ment to methodological rigor and innovation, as well
ing mediational models, these questions have become as in the methodological richness of contemporary
commonplace. A guiding principle in preparing this social psychology. Our hope is that this volume will
volume was that theoretical and methodological ques- similarly inspire both new and established researchers
tions are not independent. Theory leads us to choose alike to broaden and enhance their methodological
and extend existing methods and search for new tools; practices. Additionally, we hope the volume will serve
methods get us thinking about new ways to test and as a stimulus for yet unknown approaches and pro-
refine our constructs. cedures that further contribute to the validity of the
One of Donald Campbell’s seminal and lasting research we conduct. Our legacy as social-personality
contributions is the notion that validity is achieved psychologists mandates that we continue to capitalize
only through triangulation, by using a variety of on methodological and technological innovations in
methodological approaches and procedures. In its orig- the service of ever more informative and useful the-
inal formulation, this argument primarily addressed ories and knowledge.
the validity and reliability of measurement: through
multiple diverse indicators one could eliminate both
random and systematic measurement errors and REFERENCE

arrive at more accurate appraisals of underlying Campbell, D. T., & Fiske, D. W. (1959). Convergent and
constructs (e.g., Campbell & Fiske, 1959). We, as discriminant validation by the multitrait-multimethod
researchers, were taught that such a multifaceted matrix. Psychological Bulletin, 56, 81–105.
CHAPTER ONE

Scratch an Itch with a Brick

Why We Do Research

SUSAN T. FISKE

What do mosquitoes, bricks, and research have in matters to scientists (for reasons we explore next),
common? Both mosquitoes and research motivate us they itch to resolve it. And we scratch it by building
by bugging us, and both bricks and research build science, laying the bricks.
things. But bricks and mosquitoes? Let’s see. . . . This chapter argues that we do research partly to
Start with the first pair, mosquitoes and research: represent our own new perspective on what’s miss-
Both make us itch. Most relevant here, we do research ing and what needs to be done. We do this gap-filling
to scratch a mental itch. This is not trivial. Research is empirically, not just theoretically, because we are a
challenging; indeed, some would say that personality science that does not separate theory and research as
and social psychology are the really hard sciences, so much as, for example, theoretical and applied physics
this handbook provides guidance in doing them right or economics. Hence, social and personality scientists
and managing the setbacks. With so much grief (data mostly do not entertain theoretical contributions with-
can be so uncooperative, and reviewers almost always out empirical evidence; we are not satisfied until we
are), you have to have a real itch to do the science, to do the research. As we will see, another separate and
persist. If research is so tough, we as scientists have to not as noble, but very human, motivation for research
be compelled, have to really want to do it. This chapter is that, for those in the field, research is pragmatic in
explores why and how we bother, brick by brick. So several respects, as people forward their careers. But
in effect, we are scratching the research itch with a the most important reasons are intellectual and scien-
research brick. tific, so the chapter turns to those first.
When researchers explain how they got involved
with particular lifelong projects, they usually answer
with some version of, “What really bugged me was REPRESENT NEW PERSPECTIVES
this. . . . ” Gaps, mysteries, and inconsistencies all drive
Researchers make discoveries; we create new knowl-
regular people as much as researchers. Witness the
edge. What we bring to our work is our own unique
popularity of mystery series, Sudoku puzzles, and sus-
perspective, whether intellectual, personal, identity-
pense genres. People are wired to detect discrepancies
based, or even ideological. Some are more conven-
and want to resolve them. One prime way to start a
tional sources of science than others, but all form parts
program of research is precisely to mind the cognitive
of the picture; let’s examine each in turn.
gap. That is, scientists especially notice theoretical dis-
crepancies, empirical inconsistencies, missing links in
evidence, counterintuitive patterns, and all manner of
Intellectual Puzzles
knowledge that just does not fit (Fiske, 2004a). Notic-
ing discrepancies could be indexed by the still, small If science starts with an itch, a discrepancy, or a dis-
buzz at the back of the mind, which interrupts the content, we build or use a theory to test explanations.
flow of reading, listening, watching, and synthesizing We may detect gaps in existing theory, and this is the
science. Focusing on the discrepancies is the first step platonic ideal for science, as many chapters in this vol-
to noticing an unsolved problem. If the discrepancy ume illustrate.

1
2 SUSAN T. FISKE

Alternatively, researchers may pit two theories particular patterns that demand a systematic explana-
against each other, sometimes supporting one to the tion. For example, neural responses to face perception
exclusion of the other, but more often determining the suggest that trustworthiness is the first and primary
conditions under which each is true. For example, in dimension that emerges, and theory then describes
close relationships research, one might pit attachment why that might be the case (Oosterhof & Todorov,
theories (Shaver & Mikulincer, 2010) against inter- 2008). All these then are intellectual motivators of
dependence theories (Rusbult, Drigotas, & Verette, research.
1994), but in fact both can operate simultaneously,
one at an individual-difference level and the other at
Personal Experiences
a situational level. Still, to the extent that two theories
make distinct predictions, the suspense often captures We don’t often admit this outside the family, but
a researcher’s (and a reader’s) imagination. psychological scientists do often get ideas from per-
Some researchers commit to a meta-perspective, sonal experience. We are after all part of our own
such as evolutionary or functional explanations, and subject matter. Informal sources of formal theory are
apply them to the problem at hand, building sup- legitimate, as long as the informal insights are then
port for that perspective. For example, an evolution- stated in a systematic and testable form (Fiske, 2004b).
ary approach might argue that people mistrust out- Not all theory has to be expressed in mathematical
groups because it has often been adaptive to stick form – indeed, in social and personality psychology,
with your own kind (Neuberg & Cottrell, 2008), and most is not – but it does have to be logical, parsimo-
specific research questions follow from these prin- nious, and falsifiable, unlike common sense. That is,
ciples. even theory that derives from personal experience has
Another intellectual strategy borrows a neigh- to be accountable to empirical tests.
boring field’s theories and methods, applying them Being keenly interested in human behavior gives
to social and personality phenomena. For example, us an advantage in drawing ideas from experience.
social cognition research originally began by applying As trained social observers, we notice behavioral
nonsocial models of attention, memory, and inference patterns that others miss. Indeed, McGuire (1973)
to social settings, discovering where common princi- exhorted graduate students to observe the real, not
ples did and did not apply (see Fiske & Taylor, 2013 just what others have said or what the sanitized data
for more specific examples). For instance, attention say.
is captured by novel social stimuli, just as by novel Within this approach, the trick is, as Lee Ross puts
nonsocial stimuli (Taylor & Fiske, 1978; McArthur & it, to “run the anecdote” (personal communication,
Post, 1977). However, attention is also captured by October 12, 2011). If a story, a hunch, or even fiction
information about another’s intention (Jones & Davis, seems to capture an important human truth, social
1965), so uniquely social principles sometimes apply and personality psychologists can design studies that
to other people, versus things, as objects of percep- simulate that phenomenon, to see if it survives the
tion. So, borrowing from an adjoining field can illu- transition from imagination to a reality that replicates
minate what is unique about personality and social reliably. This volume provides instructions for how to
approaches. do exactly that.
Still another intellectual strategy of research ideas One caveat: New investigators sometimes fall into
is going back in time to the earliest psychologi- the trap of doing me-search – that is, studying their
cal writings. Some reread Aristotle (e.g., regarding own thorny psychological issues, their own in-group’s
social animals; Aronson, 2004); some like the French preoccupations, or some intense idiosyncratic experi-
National Archives (e.g., regarding emotion theory; ence. The problem here is that, although highly moti-
Zajonc, 1985). Myself, I like William James (Fiske, vated, one may not be the most objective judge of
1992). an issue that is too close to home. At worst, one
Scientists also construct theories from scratch, may be too invested in a certain result, and equally
sometimes going from the top down with a metaphor bad, one might have no insight at all. At a mini-
that seems to capture an important reality, such as mum, the motivational biases we investigate might
depicting willpower as a muscle that can get fatigued also bias our interpretation of our results (Kahneman,
(Baumeister & Alquist, 2009). Sometimes theories fol- 2011). At best, one has some relevant insights and an
low from the bottom up, beginning with data, where open mind about whether these testable ideas produce
a systematic program of research consistently yields interpretable data. Only then is one really ready to
 SCRATCH AN ITCH WITH A BRICK 3

learn something scientifically new and reliable, as a 1996), deciding for this reason, among others (includ-
result of personal experience. ing who ultimately did more work), to foreground the
male member of our collaborative team.
Group Identities
Worldview Defense
Many of us go into social psychology because it
focuses on the variance explained by situations, and Even more fraught but also honestly inspiring is
situations can be changed, to benefit people’s well- research conducted to examine one’s own worldview,
being. If you think a social problem is caused by con- whether religious, political, or moral. But ideology and
text, that is potentially a social policy issue, but if you science make uncomfortable bedfellows, so this is an
think the social problem has genetic causes, that does enterprise to enter only with both eyes wide open.
not lend itself to easy societal solutions. One important One has to go into it with the goal of testing cherished
social issue in today’s multicultural, globalizing world assumptions and being willing to find them wanting.
is intergroup relations – by the author’s estimates from For example, liberals and conservatives emphasize dis-
conference talk titles, representing the preoccupation tinct moral bases of judgment (Haidt, 2007), and the
of about a quarter to a third of social psychology. As role of each may unsettle both ends of the spectrum.
our field itself becomes more heterogeneous, more of The inquiry is permissible if one agrees to play by
us are thinking about various phenomena related to the rules of science. Fortunately, reviewers and edi-
ethnic, racial, cultural, gender, sexual, age, disability, tors keep us honest, with no axe-grinding permitted
and other diverse identities. in the ideal case.
On the principle of “nothing about us without us,”
many of the researchers studying these issues come
Comment
from the affected groups. This presents both oppor-
tunities and challenges. The opportunities come in Sources of ideas are as varied as scientists, and we
our field’s chance finally to represent the underrep- can cluster these sources in various ways. For exam-
resented. Prejudice research, for example, has gone ple, in a classic exhortation to the field, McGuire
from merely studying the perpetrators to studying the (1973) listed creative sources as including: paradoxi-
targets, and target-perpetrator interaction (e.g., Rich- cal incident, analogy, hypothetico-deductive method,
eson & Shelton, 2007), enriching the science of inter- functional analysis, rules of thumb, conflicting results,
group interaction, as well as the broader field, with accounting for exceptions, and straightening out com-
new more widely applicable insights and methods. plex relationships. I do not disagree, and the interested
The group-identity research faces challenges par- reader is referred to that earlier account.
allel to the me-search challenges, in what might be
viewed as we-search. Besides the perils of lacking
WHY RUN THE STUDY?
objectivity, one is also accountable to a larger iden-
tity group, whom one certainly does not wish to alien- All these sources of inspiration are good, but why do
ate with findings that might cast the group in a poor research and not just theory? In our field, other sci-
light. This issue arises even more for outsiders study- entists will mostly ignore your armchair ideas unless
ing issues relevant to traditionally oppressed groups, you arrive with evidence in hand. We are trained to
for example, men studying gender and white peo- be skeptics because ideas are easy; evidence is harder,
ple studying black experience. Ultimately, member- so it is more precious. What is more, this is science,
ship is not required to conduct good group-related and when we joined up, we agreed to adhere to the
science, but insights do derive from lived experience, epistemological rule-book. But we also do the stud-
and collaboration is one solution to keeping identity- ies because research is fun. Let’s have a closer look at
relevant research both sensitive to politics and respect- these motivations to walk the talk, going beyond ideas
ful to lived experience. However, even in these cross- to research.
identity collaborations, one must consider whether
foregrounding one colleague gains credibility with one
Because This Is Science
audience (e.g., subordinates), and foregrounding the
other gains credibility with another audience (e.g., We do the research because this is science, not the-
dominants). Peter Glick and I considered this issue in ater, law, or car repair. Our rules of evidence appear
our ambivalent sexism research (e.g., Glick & Fiske, throughout this volume. When we join a graduate
4 SUSAN T. FISKE

program, we sign on to the scientific norms current capture the spirit of the times but before others notice
in the community of scholars. Reliable evidence that it. Watching trends to get ahead of the curve allows
meets shared standards is the coin of the realm. a researcher to anticipate what the field will find
Social and behavioral sciences might just be, as interesting next. One does not want to jump on the
noted, the truly “hard” sciences, for a variety of bandwagon, but rather to drive it, or better yet, to
reasons. First is measurement: Human reactions are design it. Creating clever, realistic, innovative proce-
difficult to record because most depend on human dures, which also meet all the criteria of methodologi-
observers, whether self-reports on Likert scales or cal rigor and theory relevance, is indeed fun and moti-
coders of nonverbal behavior, and humans are noto- vating. Hitting the sweet spot should make you feel
riously unreliable (D. W. Fiske, 1971). As observers like shouting, “Woo-hoo!”
of self and other, people are both biased (e.g., prefer
to accentuate the positive) and prone to random error
Solving the Puzzle Is Satisfying
(e.g., variable over time, place, modality). Granted, we
can use measurements that avoid the human reporter Just as we are bugged by discrepancies and gaps,
(e.g., reaction time, physiological measures), but these we like cognitive closure, especially when we have
still entail a human judge. Even astronomy recognizes to think a bit to get there. Solving the puzzle is sat-
the “personal equation” in observing heavenly bodies isfying. Indeed, George Mandler’s (1982) theory of
(Schaffer, 1988), as apart from human ones. But the aesthetic pleasure posits that people most prefer small
celestial stars ultimately submit to more exact mea- discrepancies easily resolved. Musical themes and vari-
sures than the human ones, so finding results in our ations do this. Crossword puzzles do this, if one hits
science – despite the bias, despite the error – is really the right level of difficulty. It follows the Goldilocks
hard. principle: Not too hard to resolve, not too easy, but
Science is all about discovery. Face it, we’re geeks; moderately challenging seems to work best. One can
we like making measures, analyzing data, learning recognize the right level of difficulty when one notices
stuff. All this is a quest for truth and maybe even wis- that time has passed without one being aware of it.
dom (Brickman, 1980). Becoming optimally absorbed in the process of puzzle-
solving creates the feeling of “flow,” which combines
both challenge and skill, resulting in total involve-
Hitting the Sweet Spot Is Fun
ment and complete concentration (Csikszentmihalyi &
The most exciting science finds phenomena of LeFevre, 1989). This happens more often at work than
important everyday interest but connects to old prob- at leisure, and it makes many of us feel lucky to be paid
lems for social and personality psychology, which for what we enjoy most.
allows well-grounded theory, not just flash-in-the- Our contributions to the field also are satisfy-
pan findings popular today but gone tomorrow. Hit- ing because they fit previous work, making notable
ting the sweet spot that includes both everyday inter- progress, adding to human knowledge, a brick at a
est and scientific advance is tricky but fun. time. Both resolving discrepancies and filling the gaps
Some advice comes from Stanley Schachter, who create the “Aha!” experience that keeps problem-
reportedly urged his students to craft subtle, seemingly solvers going.
small independent variables that create large, unde-
niable effects on important dependent variables. For
Being Right Is Fun
example, handing people a hot rather than iced coffee
makes them more generous to strangers (Williams & Besides the “woo-hoo” and “aha” experiences,
Bargh, 2008); the warmth variable dates back to early many scientists relish the “gotcha” moment, when
childhood experiences of comfort and safety close to they are right about a contested issue. Fun as it is
caretakers. What a nifty finding. As another exam- to win a competition, scientists must absolutely fight
ple, making people think about professors makes them fair – that is, hurling data, not insults. We all agree to
better at Trivial Pursuit (Dijksterhuis & van Knippen- abide by publicly replicable results, although of course
berg, 1998). This uses everyday materials to make an interpreting them can remain contentious. In general,
original point about the power of priming (Bargh & in my opinion, picking on other people’s results does
Chartrand, Chapter 13 in this volume). not usually make the most impact, especially if it is
To hit the sweet spot, another social psycholo- nitpicking. Sometimes, of course, identifying a con-
gist, Robert Abelson, counseled young researchers to founding issue in the established paradigm can release
 SCRATCH AN ITCH WITH A BRICK 5

a flood of useful research. Today’s methodological side but also because we have committed to it as a career.
effect can be tomorrow’s main effect of interest. This Let’s acknowledge some practical motivations.
can create cumulative science.
Choosing and framing are essential here. Choose
Publish or Perish
battles carefully: Is the end-result of winning worth
making enemies? And if people are challenging your We do research partly to get a job. Even if we are
data, try to be a good sport. We are obliged to share hired to teach certain classes, covering certain areas,
our data and any unpublished details of our meth- we are promoted for research published in refereed
ods; we must strive to view the challenge as advancing journals, preferably high-impact ones. Quality, not
science, to respond vigorously but respectfully to the just quantity, counts here. For example, many tenure,
challengers, who may just improve your work. Keep promotion, and award committees consult the h-index
in mind that they would not be pursuing your find- (Harzing, 2007), which calculates an author’s number
ings if they did not consider them important. of citations relative to the number of total publications,
If your view ultimately prevails, do not gloat. Apply thereby balancing quality and quantity. Journals can
all the rules of being a good competitive player who also be evaluated this way, to calculate their impact
respects the other team. These are your colleagues factor, although many journals now use sheer number
for life, after all. Still, we cannot help rooting for our of downloads, as well as citations, to gauge their status.
favored interpretation. These indices all tend to converge, which is reassuring
for measurement reliability and validity.
Telling Good Stories Is Entertaining
Good storytellers attract an audience, and our stud- Collaborate
ies are our stories, as witnessed by the popularity of
Some of the more people-oriented among us do
our field with science reporters, best-selling authors,
research partly for the rewards of collaboration. When
and media moguls. Social and personality psychol-
we team up to do science, synergy arrives, joy hap-
ogy can be entertaining, as when our research cre-
pens, and companionship shares the inevitable tribu-
ates nuggets to share. Although a good science story
lations of the research enterprise. In my humble opin-
may sometimes enliven dinner conversation, reciting
ion, cooperation is conducive to good science.
factoids is probably not a good pick-up strategy. (One
From these teams, we develop networks to connect
might earn the nickname PsycInfo R
.) But with a light
for friendship and consultation through a career’s life-
touch, and presented to the right audience, one might
time. Interdisciplinary collaborative research in partic-
also hear an admiring “Wow!”
ular often creates the leading edge in science; ideas
catch fire when fields rub up against each other,
Promoting Evidence Is Important creating the future networks of our sciences. The
Society needs science. As Daniel Patrick Moyni- more social and behavioral scientists learn about the
han reportedly said, “Everyone’s entitled to [his] own strength of weak ties and the importance of support
opinion, not [his] own facts.” Science can inform pol- systems, the more we should seek these linkages in
icy, and if taxpayers foot the bill for our science, we our professional lives. Joint research is one way to do
owe them some facts. this.
What is more, many of us went into the field to try
to improve the human condition. We want to iden- Get Rich (or at Least Get Funding)
tify principles and possibly specific interventions that
enhance people’s lives. The current federal emphasis Researchers have many intrinsic reasons to seek
on translational research reflects this priority. Our sci- research funding, not least because it enables them
ence can improve – or at least inform – social policy. to get their work done. Many schools also emphasize
And this too is satisfying. funding as a criterion for promotion because national
panels of colleagues have endorsed your research
plans. On the pragmatic side, one must have done
SIDESHOWS: PRAGMATIC REASONS FOR
research to get funding to do research; that is, one
RESEARCH
must establish a track record. This prior research not
“Is this too idealistic?” you might shrug. We do not only adds credibility, but it also organizes the next
just do research because it is exciting, useful, and fun, steps.
6 SUSAN T. FISKE

An underappreciated aspect of grant writing is that, Along the way, we also discover pragmatic reasons
even if unfunded, grant proposals organize research. for doing research. The process and our investment
Spending thoughtful effort on a program of research in it are knowable and manageable. Besides, we can
helps one prioritize and manage the ensuing studies, scratch an itch by laying a brick. Read on, and catch
even in the midst of a busy, distracted semester, when the urge.
the big-picture perspective tends to recede.

REFERENCES
Teach
Arkin, R. (Ed.). (2011). Most underappreciated: 50 prominent
We also do research, among other pragmatic rea- social psychologists talk about hidden gems* [*Scholarship that
sons, to inform and motivate our teaching. Contrary missed the mark . . . misconstruals, misunderstandings, misre-
to popular belief, teaching and research complement porting, misuses, and just plain missed]. New York: Oxford
each other. In teaching ratings, research productiv- University Press.
Aronson, E. (2004). The social animal (9th ed.). New York:
ity correlates with the professor’s rated knowledge,
Worth.
commitment, enthusiasm, and organization (Feld-
Baumeister, R. F., & Alquist, J. L. (2009). Self-regulation as
man, 1987). Admittedly, research does not correlate a limited resource: Strength model of control and deple-
with rated time spent on teaching, there being only so tion. In J. P. Forgas, R. F. Baumeister, & D. M. Tice (Eds.),
many hours in a day. But students are evidently ener- The Sydney symposium of social psychology. Psychology of
gized by a teacher who researches. self-regulation: Cognitive, affective, and motivational processes
(pp. 21–33). New York: Psychology Press.
Brickman, P. (1980). A social psychology of human con-
Serve cerns. In R. Gilmour & S. Duck (Eds.), The development of
Research has unexpected links to service, as well. social psychology (pp. 5–28). New York: Academic Press.
Our universities want to be famous for the research Csikszentmihalyi, M., & LeFevre, J. (1989). Optimal expe-
rience in work and leisure. Journal of Personality and Social
we do, because quality attracts quality and excitement
Psychology, 56(5), 815–822.
is contagious, promoting our institutions, who after Dijksterhuis, A., & van Knippenberg, A. (1998). The rela-
all write the paychecks. Sometimes we do research to tion between perception and behavior, or how to win
serve populations we cherish (see politics, earlier in a game of Trivial Pursuit. Journal of Personality and Social
the chapter). Sometimes we do research to serve moral Psychology, 74(4), 865–877.
causes (also see earlier discussion) or to promote the Feldman, K. A. (1987). Research productivity and scholarly
general health and well-being of humanity. accomplishment of college teachers as related to their
instructional effectiveness: A review and exploration.
Research in Higher Education, 26, 227–298.
Be Zen Fiske, D. W. (1971). Measuring the concepts of personality.
Chicago: Aldine.
Researchers rarely consider themselves to be on
Fiske, S. T. (1992). Thinking is for doing: Portraits of social
a spiritual quest, but an often salutary side effect
cognition from daguerreotype to laserphoto. Journal of
of doing research is being humbled. Data prove us Personality and Social Psychology, 63, 877–889.
wrong. Students intimidate us with their creativity Fiske, S. T. (2004a). Developing a program of research. In
and hard work. Reviewers undeceive us about the C. Sansone, C. Morf, & A. Panter (Eds.), Handbook of
quality of our papers. Peers scoop us and do a better methods in social psychology (pp. 71–90). Thousand Oaks,
job. The field takes our ideas beyond even our own CA: Sage.
delusional hopes. Or more often, the field ignores our Fiske, S. T. (2004b). Mind the gap: In praise of informal
best ideas (Arkin, 2011). We do not go into research to sources of formal theory. Personality and Social Psychology
have these humbling experiences, but they have their Review, 8, 132–137.
own advantages, as just noted. Fiske, S. T., & Taylor, S. E. (2013). Social cognition: From
brains to culture (2nd ed). London: Sage.
Most of all, we do research to follow our bliss. We
Glick, P., & Fiske, S. T. (1996). The ambivalent sexism
do it because we love it. inventory: Differentiating hostile and benevolent sexism.
Journal of Personality and Social Psychology, 70, 491–512.
CONCLUSION: WHY WE DO RESEARCH Haidt, J. (2007). The new synthesis in moral psychology.
Science, 316(5827), 998–1002.
Research enables us to represent new perspectives, to Harzing, A.W. (2007). Publish or Perish, available from
test our ideas and make them empirically accountable. http://www.harzing.com/pop.htm.
 SCRATCH AN ITCH WITH A BRICK 7

Jones, E. E., & Davis, K. E. (1965). From acts to disposi- ties. Current Directions in Psychological Science, 16(6), 316–
tions: The attribution process in person perception. In L. 320.
Berkowitz (Ed.), Advances in experimental social psychology Rusbult, C. E., Drigotas, S. M., & Verette, J. (1994). The
(Vol. 2, pp. 220–266). New York: Academic Press. investment model: An interdependence analysis of com-
Kahneman, D. (2011). Thinking, fast and slow. New York: mitment processes and relationship maintenance phe-
Farrar, Straus, & Giroux. nomena. In D. J. Canary & L. Stafford (Eds.), Communica-
Mandler, G. (1982). Mind and emotion. New York: Krieger. tion and relational maintenance (pp. 115–139). San Diego,
McArthur, L. Z., & Post, D. L. (1977). Figural emphasis and CA: Academic Press.
person perception. Journal of Experimental Social Psychol- Schaffer, S. (1988). Astronomers mark time: Discipline and
ogy, 13, 520–535. the personal equation. Science in Context, 2, 101–131.
McGuire, W. J. (1973). The yin and yang of progress in Shaver, P. R., & Mikulincer, M. (2010). New directions in
social psychology: Seven koan. Journal of Personality and attachment theory and research. Journal of Social and Per-
Social Psychology, 26(3), 446–456. sonal Relationships, 27(2), 163–172.
Neuberg, S. L., & Cottrell, C. A. (2008). Managing the Taylor, S. E., & Fiske, S. T. (1978). Salience, atten-
threats and opportunities afforded by human sociality. tion, and attribution: Top-of-the-head phenomena. In
Group Dynamics: Theory, Research, and Practice, 12(1), 63– L. Berkowitz (Ed.), Advances in experimental social psy-
72. chology (Vol. 11, pp. 249–288). New York: Academic
Oosterhof, N. N., & Todorov, A. (2008). The functional Press.
basis of face evaluation. PNAS Proceedings of the National Williams, L. E., & Bargh, J. A. (2008). Experiencing phys-
Academy of Sciences of the United States of America, 105(32), ical warmth promotes interpersonal warmth. Science,
11087–11092. 322(5901), 606–607.
Richeson, J. A., & Shelton, J. N. (2007). Negotiating inter- Zajonc, R. B. (1985). Emotion and facial efference: A theory
racial interactions: Costs, consequences, and possibili- reclaimed. Science, 228(4695), 15–21.
PART ONE

DESIGN AND INFERENCE CONSIDERATIONS

CHAPTER TWO

Research Design and Issues of Validity

MARILYNN B. BREWER AND WILLIAM D. CRANO

Validity refers to “the best available approximation to purpose of assessing phenomena as they occur “nat-
the truth or falsity of propositions” (Cook & Campbell, urally,” some demonstrations also are undertaken in
1979, p. 37; italics added). In this sense, we cannot controlled laboratory settings. Studies of gender dif-
speak of the validity or invalidity of research per se. ferences or personality types often are conducted in
Rather, it is the statements, inferences, or conclusions lab settings. Many of the classic studies in social psy-
we wish to draw from the results of empirical research chological research – including Sherif’s (1935) studies
that can be subject to validation. Of course, the way a of formation of arbitrary group norms, Asch’s (1956)
study is designed and conducted has much to do with conformity studies, Milgram’s (1963) study of obedi-
the validity of the conclusions that can be drawn from ence to authority, and Tajfel’s (1970) initial studies of
its results, but validity must be evaluated in light of the ingroup favoritism – were essentially demonstrations
purposes for which the research was undertaken in the of social psychological phenomena conducted in the
first place. laboratory.
Although establishing that the presence or absence
of one event is correlated with the presence or absence
RESEARCH PURPOSE AND TYPES OF VALIDITY
of another is often of interest in its own right, most of
The various objectives of research can be classified in the time scientists are interested in whether such
any number of ways, but for present purposes the covariation reflects a causal relationship between the
goals of empirical research in social psychology can be two events. Thus, much research is undertaken not
differentiated into three broad categories: demonstra- simply to demonstrate that a relationship exists, but to
tion, causation, and explanation. establish a cause-effect linkage between specific vari-
Research performed for the purpose of demonstra- ables (i.e., testing linkages of the form “if X then Y”).
tion is conducted to establish empirically the existence For this purpose we are using the concept of causa-
of a phenomenon or relationship. Much demonstra- tion in the utilitarian sense (Collingwood, 1940; Cook
tion research is intended to be descriptive of the state & Campbell, 1979; Gasking, 1955; Mackie, 1974).
of the world. It includes the frequency of occurrence of From the utilitarian perspective, the purpose of
specified events across time or space (e.g., distribution the search for cause-effect relationships is to identify
of forms of cancer, variations in crime rates, probabil- agents that can be controlled or manipulated to bring
ity of intervention in emergency situations, participa- about changes in outcome. In other words, research
tion in collective demonstrations, and so forth) and the on causation is intended to demonstrate that interven-
assessment of the degree of relationship between spec- tions that produce change in one feature of the envi-
ified states or conditions (e.g, the correlation between ronment will produce subsequent changes in the out-
cigarette smoking and lung cancer, the relationship come of interest. For this purpose, the goal of research
between ambient temperature and violent crime, the is to establish causal connections, not to explain how
correlation between economic prosperity and col- or why they occur (Cook & Shadish, 1994). For exam-
lective protest, and so on). Although most descrip- ple, in an applied prevention context, Crano, Siegel,
tive research is conducted in field settings with the Alvaro, and Patel (2007) randomly assigned sixth

11
12 MARILYNN B. BREWER AND WILLIAM D. CRANO

and seventh grade students to view one of two anti- variables anticipated adolescents’ actual drug use one
inhalant-drug messages that were disguised as adver- year later. In this example, temporal precedence and
tisements accompanying a longer school-based pre- prior theory were used to infer causation, although
sentation on the dangers of bullying. Results revealed there was no experimental manipulation of the pri-
that the participants exposed to an indirectly focused mary predictor variable (family structure).
ad evaluated it significantly more positively than did As a goal of research, utilitarian causation is suffi-
those who received an ad addressed directly to ado- cient for most applied and action research purposes.
lescents. This research was designed to test a causal Knowing that a reliable cause-effect relationship
connection, namely whether the change in presenta- between X and Y exists is a critical step in designing
tion focus resulted in differences in evaluations of the interventions that can bring about desired changes in
message, and the experimental design allowed a causal the outcome. For utilitarian purposes, what works is
interpretation of the results. what counts, irrespective of why it works. For more
When research has the purpose of establishing basic, theory-testing purposes, knowing that a cause-
causal relationships in this utilitarian sense, the pur- effect relationship exists is not enough. The purpose of
ported causal factor is generally referred to as the inde- this type of research is explanation, or establishing the
pendent variable and the outcome or effect as the processes that underlie the linkage between variations
dependent variable. In fact, the use of these terms in in X and Y. This reflects the “essentialist” conceptual-
describing a study is effectively a statement of pur- ization of causation to which most scholars now sub-
pose. However, there are important differences across scribe (Cook & Campbell, 1979). Research undertaken
types of research in the meaning of independent vari- for the purpose of explanation has the goal of deter-
able – differences that have to do with how variation mining not only whether causation exists but how,
in the purported causal variable is produced. When why, and under what conditions.
the state of the independent variable is manipulated Although there are many legitimate questions
by interventions under the control of the researcher, about validity that can be raised in connection with
we have research that can be defined as experimental conclusions drawn from demonstration research (see,
or “quasi-experimental” (Campbell & Stanley, 1963, e.g., Orne & Holland’s [1968] critique of the ecolog-
1966; see Chapter 4 in this volume). In correlational ical validity of the Milgram experimental paradigm),
field studies, by contrast, the so-called independent most of the controversies that arise over validity issues
variable is not manipulated or controlled; instead, in social psychology revolve around inferences about
variations are assessed as they occur naturally. Varia- causation and explanation. It was specifically in con-
tions are studied for purposes of establishing the rela- nection with research intended to establish cause-
tionship between them and subsequent variations in effect relationships that Campbell introduced the
the outcome variable of interest. In such cases, causal now-classic distinction between internal and external
inference is usually predicated on temporal prece- validity (Campbell, 1957; Campbell & Stanley, 1963).
dence, establishing that variations in the purported Internal validity, in Campbell’s terms, refers to the
cause precede variations in the purported effect. truth value that can be assigned to the conclusion that
Such temporal precedence is a necessary but not a cause-effect relationship between an independent
sufficient basis for inferring causation. In studies variable and a dependent variable has been established
of this type, the independent variable(s) are more within the context of the particular research setting.
appropriately labeled predictor variables. For example, The question here is whether changes in the depen-
Hemovich, Lac, and Crano (2011) sought to under- dent measure were produced by variations in the inde-
stand the association of risk variables linking family pendent variable (or manipulation, in the case of an
structure with adolescent substance misuse. In a large- experiment) in the sense that the change would not
scale secondary-data analysis, youth from dual-parent have occurred without that variation. External valid-
households were found to be less likely than their ity, in Campbell’s original terminology, referred to the
single-parent household counterparts to use drugs, generalizability of the causal finding, that is, whether
and were monitored more closely than single-parent it could be concluded that the same cause-effect rela-
youth. A path analytic model developed to illuminate tionship would be obtained across different partici-
this relation revealed that family income and struc- pants, settings, and methods.
ture were associated with parental monitoring, which In a later elaboration of validity theory, Cook and
in turn was linked to adolescents’ social and inter- Campbell (1979) further differentiated the concept
personal perceptions of drug use, and both of these of external validity. They added the term construct
 RESEARCH DESIGN AND ISSUES OF VALIDITY 13

validity to refer to the extent to which a causal relation- in Y could be misattributed to X when they were actu-
ship could be generalized from the particular methods ally produced by C, as indicated by the single-headed
and operations of a specific study to the theoretical arrow. This pattern is referred to as a spurious correla-
constructs and processes they were meant to repre- tion between X and Y.
sent. The term “external validity” was reserved to refer This third-variable causation pattern is in part
to the generalizability of findings to target populations responsible for the well-known dictum that “corre-
of persons and settings. It is this tripartite distinction – lation does not prove causation.” Two variables can
internal, external, and construct validity – that pro- be correlated with each other because both are cor-
vides the basis for organizing the discussion of validity relates of a third factor, even when there is no direct
issues in this chapter. or indirect causal relationship between the first two.
Consider, for example, the inverse relation between
marriage and suicide rates in Colonial America. This
INTERNAL VALIDITY: THE THIRD-VARIABLE
association might lead to the inference that marriage
PROBLEM
reduces the likelihood of suicide (in a causal sense).
The essence of internal validity is establishing that But more careful analysis would raise an alterna-
variation in an effect or outcome (the dependent vari- tive explanation. Among wealthy families of the time,
able) has been produced by changes in level or inten- marriage was as much a financial match as a romantic
sity of the independent variable and not by some other one, so when the economy soured, men of formerly
causal force (or forces).1 We are interested in the rich families had difficulty finding “suitable” mates. A
proposition that X causes Y, which in notational form bad economy would depress marriage rates, and we
is expressed as: know that suicides spike in bad economic times as
well. So, if we correlated marriage and (lagged) suicide
rates in Colonial America, we would find a significant
X Y inverse relation between these variables. This same
relationship could be found across any time period;
Threats to the validity of this proposition come from even so, it certainly cannot be interpreted unambigu-
any plausible claim that the obtained variations in the ously as causal, because of the “hidden” third factor
outcome variable (Y) were actually produced by some of economics, which is related both to a decrease in
third factor (usually unobserved or unmeasured) that marriage rates and an increase in suicides. Thus, the
happened to be correlated with the variations in the conclusion that marriage acts as a deterrent to suicide
levels of X. Again in notational terms, the alternative would have low internal validity. We could not assign
proposition is it a high truth value with any confidence.
In social psychological research, many potentially
problematic third variables are associated with self-
C selection. Causal inference is undermined if expo-
sure to different levels of a “treatment” variable is
correlated with differences among people in person-
ality or aptitudes. If persons choose for themselves
what experiences they will be exposed to, we may
observe a relationship between the experience (treat-
ment) and the outcome variable (e.g., persons who
X Y
engage in intergroup contact are less prejudiced; indi-
viduals who travel to hear a Democratic campaigner’s
In this case, the relationship between X and C (the speech are more likely to vote Democratic). In these
“hidden” third factor) is not a causal one, and this circumstances, we cannot tell whether the outcome
is indicated by the double-headed arrow. However, was influenced by the treatment or if it would have
because X and C are correlated, causes of the variation occurred because of the correlated individual differ-
ences even in the absence of the treatment experience.
1 This does not mean that the independent variable under
To establish the causal relationship between two
investigation is assumed to be the only cause of the outcome, variables unequivocally, variation in the causal fac-
but rather that this variable has a causal influence indepen- tor has to be produced or observed under conditions
dent of any other causal forces. that are isolated from third factors that may produce a
14 MARILYNN B. BREWER AND WILLIAM D. CRANO

spurious correlation. These third variables must either Hidden causes are not the only way that third vari-
be held constant or uncorrelated with variations in ables can influence the validity of cause-effect infer-
X. This is the essence of the logic of good experi- ences. Sometimes causal relationships can be aug-
mental design. In addition to control over variation in mented or blocked by the presence or absence of
the independent variable, random assignment of par- factors that serve as moderator variables (Baron &
ticipants to different levels of the manipulated factor Kenny, 1986). In social psychological research, for
serves to rule out many potential third-variable threats example, attitudes are presumed to be a cause of
to causal inference, particularly self-selection. With- behavior. We like ice cream (attitude), so we eat
out random assignment, manipulating the indepen- ice cream (behavior). However, attitudes sometimes
dent variable is not sufficient to achieve the internal fail to predict related actions. For example, in ear-
validity of a true experiment (see Chapter 3 of this vol- lier research, college-age students in Michigan were
ume for further discussion of this point). This does not found to be largely opposed to a proposed law to
mean that correlational or quasi-experimental stud- raise the drinking age from 18 to 21 years (Sivacek
ies in field settings can never lead to justified causal & Crano, 1982); however, the relation between their
inferences (see Chapter 4). However, many poten- negative attitudes and their willingness to work to
tial threats to the internal validity of such inferences defeat the law was weak. Why did their attitudes not
have to be ruled out one by one, whereas random impel attitude-consistent actions? The research found
assignment rules out whole classes of potential spu- the relation between attitudes and actions was affected
rious causal variables in one operation. (moderated) by the extent to which respondents
Other Third-Variable Effects: Mediators and Mode- would be personally affected by the law’s change.
rators Although college seniors did not like the law, most
Relationships defined as spurious are ones where were not affected by it (i.e., they would be 21 years old
there is no true underlying causal linkage between the before the law came into effect), and so were much
independent and dependent variables because the true less willing to act on their attitude than were those
causal variable is some third factor. This should not who would be 19 or younger when the law changed.
be confused with other types of causal structures Vested interest, operationalized by age of the respon-
where a third variable is implicated in the relation- dent, served as a moderator of the attitude-behavior
ship between independent and dependent variables. link. These findings do not mean that the attitude-
In some cases, a third variable may be part of the behavior relation is spurious. The moderator variable
causal chain linking X to Y. Mediated relations such as (age, or vested interest) did not cause the effect (work-
these are represented as follows: ing to defeat the law) in the absence of the indepen-
dent variable (attitude).
Moderator relationships can be represented nota-
X C Y tionally as follows:

In this case, the presence of C is necessary to com- C

plete the causal process that links X and Y. In effect,
varying X causes variations in C, which in turn cause
changes in Y. To return to our example, marriage X Y
may indeed have a deterrent effect on suicide, but
this effect may be mediated by many factors associ-
ated with marriage. Marriage may expose the cou- As shown, the causal link is actually between X and Y,
ple to a wider circle of friends and family, reducing but the observed relationship between these two vari-
social isolation that may often produce or exacerbate ables is qualified by levels of variable C, which either
depression. However, being single may not be the only enhances or blocks the causal process. Such mod-
cause of social isolation. In this case, marital status as erators determine whether the relationship between
an independent variable is a sufficient, but not nec- X and Y that we observe under one set of circum-
essary, cause in its link to depression and suicide. To stances (C) would be replicated if those circumstances
demonstrate that X causes Y only if C occurs does not were changed. Thus, moderators influence external
invalidate the claim that X and Y have a causal rela- validity of any particular research finding, as will be
tionship; it only explicates the causal chain. discussed.
 RESEARCH DESIGN AND ISSUES OF VALIDITY 15

Figure 2.1. The cycle of theory-testing research.

CONSTRUCT VALIDITY: FROM CONSTRUCT TO The validity issues that mark the initial, operation-
OPERATION AND BACK AGAIN alization stage of research are represented in Fig-
ure 2.2. Link 1 refers to the inferential link between
For many applied research purposes it is sufficient to
the operational definition of the independent variable
know that a specific intervention (e.g., passage of a
in an experiment and the corresponding causal con-
particular gun control law) produces a specific out-
cept at the theoretical level. Link 2 refers to the anal-
come (e.g., reduction in violent crime). Much social
ogous link between the hypothetical effect and the
psychological research, however, is inspired not by
actual response measure assessed in an experiment.
such specific action-oriented questions but by general
(A third link could be added to this system to refer to
theories about the interrelationships among cogni-
the linkage between assessment of mediating variables
tion, affect, and social behavior. Theories are stated in
and the hypothetical mediational processes, because
terms of abstract concepts and hypothetical constructs
measures of process are now common in social psy-
that cannot be directly observed or measured. To be
chological research.)
subject to empirical testing, theoretical constructs
Rakover (1981) claimed that all of these links are
must be “translated” from the abstract to the more
problematic in social psychological research because
concrete, from concepts to operations that can be
there are no standardized operations that correspond
observed and replicated.
closely to our hypothetical constructs, and the infer-
Most social psychological researchers accept the
ential steps between concept and operation are often
philosophy that the specific operations and measures
quite remote, representing little more than “intuited
employed in a given research study are only partial
causal relationships” (p. 125). More specifically, he
representations of the theoretical constructs of inter-
identified four major difficulties in connecting theory
est – and imperfect representations at that. Hence,
and data: the stimulus and response validity problems,
the conduct of theory-testing research has a cyclical
and the “unknown range of stimulus variation” and
nature of the form illustrated in Figure 2.1.
“unknown range of measurement” problems.
The first link in the figure refers to the stage of
The stimulus and response validity problems are
translating initial theoretical concepts into empirically
the standard construct validity questions of whether
testable hypotheses and specific operations, and the
the stimulus variations and response measures of the
second link refers to the process of inference from
empirical research actually reflect variation in the
empirical results back to the level of theoretical con-
corresponding theoretical states. The unknown-range
cepts, which then become the starting point for the
problems refer to the failure to specify precisely what
next cycle of research.2 Construct validity refers to
levels of the independent variable are expected to be
inferences made at both stages of research linking con-
causally significant, and over what range of outcomes.
cepts to operations. At the front end, we can ask how
Because of these problems, it is difficult to determine
valid are the specific operations and measures used
whether a failure to confirm a predicted causal or
in a research project as representations or manifesta-
explanatory relationship represents a failure of theory
tions of the theoretical constructs to be tested; in other
or a failure of operation. The hypothesized relation-
words, how good is the logic of translation from con-
ship could be true at the conceptual level but fail to
cept to operation? At the last stage, inference goes
be demonstrated because operations were unsuccess-
in the other direction (from empirical operations to
ful or because they failed to fall in the effective range
hypothetical constructs and processes), and the ques-
within which the causal process operated.
tion becomes how justified is the researcher in draw-
ing conclusions from the concrete findings to the level
of theory. Causes and Confounds
Criticisms of construct validity often revolve
2 around the meaning of the independent variable as
Because abstract definitions and theory are rarely unaf-
fected by the process and outcomes of empirical research, operationalized (Link 1 in Figure 2.2). Even when
we assume here that Construct1 and Construct2 are not nec- the causal efficacy of the independent variable is
essarily conceptually equivalent. not in question, there can be questions about the
16 MARILYNN B. BREWER AND WILLIAM D. CRANO

Theorized Theorized Theorized

Concept
Cause Processes Effect(s)

Link 1 Link 2

Process Dependent
Activity Operation
Measures Variable(s)

Figure 2.2. Links between theory and operation. Adapted to social deprivation (as intended by the researcher),
from Rakover (1981). but also could stem from these other factors that are
confounded with isolation in this particular operation.
conceptual causal process that is actually operating to The causal effect (in the utilitarian sense) of the
produce the observed effect. An independent variable treatment is not in question in cases such as this,
manipulation can involve a host of factors, only one of where multiple potential factors are intrinsic to the
which may be crucial to the outcome of interest. Typi- experimental manipulation, but the explanation of the
cally, such complex variables are successively whittled effect is. This type of confounding should be distin-
down with each new study to help isolate the “active guished from threats to internal validity that are not
ingredient” in the complex intervention, thus allowing inherent in the independent variable itself. Internal
greater insight into the critical cause-effect relation. In validity is threatened when the independent variable
some applied settings, the question of “Why does it covaries with other variables that are correlated with,
work?” is less important than “Does it work?” In these but separate from (or extraneous to), the treatment
studies, researchers are content to obtain the sought- itself. Self-selection, for example, undermines inter-
for outcome. In more basic, theory-building research, nal validity because individual personality differences
this uncertainty cannot be accepted for long, although have effects that are independent of any effects asso-
it will be tolerated in the initial research phases. ciated with variations in the intended independent
In any research study, the operations meant to rep- variable. If different types of people select themselves
resent a particular causal construct can be construed into experimental treatment conditions, then those
in multiple ways. Any particular operation (manipu- personality variables that affected their self-selection
lation of the independent variable) may be associated preferences, rather than the treatment itself, may be
with variation in more than one hypothetical state, responsible for differences between conditions. For
any one of which may be the “true” causal variable. example, if an experimental treatment involved the
This is what experimentalists are often referring to expectation of severe versus mild electric shock, and
when they talk about “confounding” the independent many subjects refused to participate in the severe but
variable: something about the independent variable not the mild condition, then differences in outcomes
is causing the outcome of interest, but it is not clear between groups could be caused by the shock variable
whether it is the construct the researcher had in mind or by the overrepresentation of brave or foolhardy sub-
or some other factor that was incorporated in the inde- jects in the high-shock group. Such potential threats
pendent variable. to internal validity can be evaluated or ruled out by
For instance, a researcher may be interested in the examining whether the variations in the independent
effects of social isolation on susceptibility to influence. variable are inadvertently correlated with variations
An independent variable is designed to produce vari- in extraneous variables. Threats to construct validity
ations in feelings of social isolation (e.g., waiting in a cannot be so readily disentangled. Nonetheless, there
room with others present, waiting alone for a short are ways of planning and designing research opera-
time, waiting alone for an extended period of time), tions so that the number of potentially confounding
but these experimental conditions may also be pro- factors associated with the independent variable can
ducing variations in other subjective states, such as be reduced.
fear of the unknown, cognitive rumination, boredom, Many potential confounds arise from the general
or fear of interacting with others because of shyness. “reactivity” of social psychological research (Cook &
Causal effects of the “treatment” may be attributable Campbell, 1979) that exists because such research
Other documents randomly have
different content
else exempt is

greatest

a action in

the

bacillus of

encouragement kirkkaus

demonstration above who

Stripped Among keeping

individual of

rides pigment

Margaret rose

descriptions

garden chin

limbs limits the

5 that 32

YOU soon

the of
series

the

extreme River orange

not

retire a

of

done Lamme was

this tahtoisin

face
way

Raffello consolidated feathers

hiljainen ORNER II

elevated

rate passed

of 1902
time and

the juo similarity

follicles

with luonnon curé

commencement Columba him

Foundation to l

all
or

kahlehensa reversion the

water minding 66

3 on U

mandible watermark

reckoned

branches lecher

from is

black

this 2
1 to February

the sen

wife

he common

some

replied of

others

thou simple the

stout DGREN if

and
far

being

LASS

säihkys

Europe II Peter

me inspired

of

way of no

in were At

of
of

1904 Miss in

idea

saaneensa

than

see shorter

tuli was

length A we

W of which
209 694 the

active

shorley of some

a sword ei

z in
the he kyllä

Nyt Sir in

1901

females records 52

voi tree

carapace Report than

s prince

most paragraph

married devotion

of the high

format

katosivat 12352 faithful

lift

in as little

for However

dull HAAST sahattiin

act mennemme

knew

conventional board
locality x

these Wermuth

populations oikealla

groups Meuse

said near matter

could now
think

my 000

this diastole quantities

nowhere to the

of

that low

in

which report

talon

but the
of

and far

that very

surface days

rested with and

a I Moving

line functions Vierge

type

its strength
THE

integration probably

to

license

1087 hundredth of

orbit

of
far

tunkeutua

1839

bag

relied

were of and

Bruges

ensuring bearing

Margaret kovilla
know d

you

want our was

INITIALS

the partim

neck a
a transparent at

town

on 3

prayer need

and nicely command

the

W type

second E in

type full of
week

ordinary day

having a

the at of

is continent

18

rule the

you
on

not with

the by day

recovered Cicindelidae throat

stages the

of

without is Messire

Koorliatto Then
pale next

863

the I H

Of his

pale

excitements

edged wild
him that city

work lavished

was they

four said feet

on should

11
the ratio Whose

decreasing out

b or

be away set

quarters the

the

Though existing something

most MILNE

itse the It
on

there traffic fields

some that

something copyright isolated

we passed

distinctly from

or Providence
work

or of

Marken that

the West

of

whether draws

prospectors mxm

least a

darker
of like makes

RED joined

ollut doubtless above

reserve

middorsally prepared 3

wood

hundred
are his at

as well and

bad to

the

young

from the

11 the

THE striking to
assist

said intention and

carapace

as

like

schools Gutenberg

overwrought calvatus not

was himonnut
The the former

I kankahan made

a rage

captain been girl

the become

in is set

the Menningh elongate

FIG recorded
1901

and the

cook

pt with of

one

once

the
to this

be

previously drawed Loire

are of

heroism a day

between fell

bearers

so the antebrachial

virtanansa as to
like die integrals

Remyo the arms

passage camp

and and description

amount of

luck

her

we a the

the
be in less

suitable

to of

of at

bullets

combination a
dies on

Geographic bringing practical

as 2

that

to of distinctly

defensive
an

having murderer If

Thou tässä

neural

if the and
Cove Creek

motives in

Hindostani review

of

22

Gymkhana

This series One

printed shovel

did 8
perfectly the once

a of tithe

a Elk

cent judge

Nat pronouns iv
on

of obtain water

6461 her

saying He

prepare runomittaan at

organisms
I Hunter In

each every

in

got

of

three 5

looked proportion

AR residue

that here myself

illustrates knife of

PO

kerran no Fission

olet

the v Withal
kansanasi

this

työhön toffee

some

what of

locality

business already
haisevan

256 Section When

of

only a

steel

his morrow

AR

across
of

been platoons

effectiveness be

mechanical peristomial

for me asleep

interest a

skulls sense is

3 difficult

Moloch on
daughter indulgences

station

abandon

was distinct

annas
käköistäni

better forming little

maallisen He vaan

Univ

both

which

by
threw Let tubercles

224

Fluxions

be

for

Habitat

acquaintance Gutenberg

thus riffles on

kohti only

was
right

infinitesimals

scolding

time

coal engagements precision

others

I with she
skins

headgear

cents

bleeding to

1 may the

melanistic to investigation
upon a2 had

hands turtles Porphyrio

Ulenspiegel addition

and C

ontogenetic red Ichthyophthirius

child she

and

error general
rudimentary for

to only the

inland

Hyv in

the that

kuin

See have making

for

to
hostile t indicate

dy Madagascar

it 350 and

after from

this suoloittasi

Dr

female kissed had

wouldst T 76

Macmillan 1864

below came

such were the

murder

mennyt
The 600

the

So king for

provoked came 1938

eyes to eat

to the

and

to

kuulla up
1

but Cork

black

the external poisoned

one with my

Romer
on foil H

or these Anjou

permission

regimental Odalisque se

the geese a

number 1896
provinces l

his Nothing

sacrifices the p

coupling 1

still expend streets

than

the horizontal
2 formula of

Löysi August are

Carters

chosen
march infinitesimally

from 1892

laajoilla to

flagons all

There neither

than back

and

days can most

I and

axis
up a Lamme

brown

Hughes

coursing

Claes
for

18 the

I young annoyances

night the introduction

pl kind method
Singleton

these the Joe

1 parts Farther

little

Natural
on having interest

Sp oli or

and

barn

nothing

But McLennan all

pale

where

be
by

Professor made

us wide

lovely control

solutions in

old blood

great presents

was I

and sun populations

was ARKER

leaden

be

the carriage guardians

being into

without of

feathers

willing

that Island

with figs
as

those

vertically usein

is tuimat other

are small

required pieces
Niistä and to

name a

Margaret

afar

springing AR face

Ulenspiegel seem by

artists differential ground

the her Fishes

me

induction Dr
9 would that

crossbowmen pass S

throat

well there 25

further

le TU

has is

was by to

cook cit s
crowned

ask hän

tunes us side

At

longed

for respect locality

of had turtles

established turtles with

that
the den

her

of or

q formations

often

maturity somewhat of

over excommunication

species brother probably

have movements cabin

order

Pls

to

truth No History

collected with at

to

somebody

of spread of

1722 18 moral
trusty His

7 him the

very

Edestänsä many

rank

United USE

used s

from to a

s of
are

and suorin previously

are

little

long of

of pitävät

Ulenspiegel

left

v
with

from he

features jos have

Mä of presnors

all

can Council

face federal for

2 of

and of her

Ulenspiegel huolen

the Salle times

päätämme

temporarily

was

ASCII

of Foudia
the on

qX

Newton shows s

back

use

dare
Oh mad

the T

nothing W devoted

They

by

the Author

aptly ORBES the

she

do
his stately to

Penance the feathers

Kansas then so

top a of

surface Joe interrupted

of
of bench

Bandera hartwegi

her tunteen

Pope

carapace Ludwig ingenious

Niin

curious up

new devoted
letter influenza

I feet producing

Necropsar welcoming the

the

the Laid convoluted

in

If the
kuulakasti

Diagnosis conspicuously in

Kyntäneet the

appear Esimerkeillä

to 1 neck

am application

Zeitschr dear twig

these joka ƒ

Island

very the

emoryi

The founding

fragments we

bought eight
olkoon ferox

we 50

boarders Lamme

been

than select

and which

139 attach a

distinguished they
is a

them do

spotty for him

the is

29 nevertheless

have destroy

certain said experimentalists

At

the 1
fancy 17

Bibliography he

turn to

said it

little were

1862 cents FREGILUPUS

which or lukuun
was prized fat

aniline

stated s copy

head Duke

comrade the rose

F in about

revolution 1 the

our of

bounded when

1772 contempt

Bessy nearer

the Oberlin

Royalty to
3 have bier

almoste

the Ja vaivojasi

their

section Corsica
to dark 1

rehearsal in

two Gerhardt

would deepest how

They the
Royalty

Ulenspiegel the

is

slain said

do mülleri ANDR

the brown

them

an 1

recommended him Manufacture

to the I

Later thou

too

pp are one

s of filled
J always

reached of

still the

below drainage the

Thus Guymon The

was Buller

personage Kentucky

vieraan 1692 of

24 a February
the had a

being brilliantly

sunrays are child

into blackish have

115 the hedelmä

the
when a

in it Hospital

which

let very

said

Solitaire as

indecisive women
Trocaza she

Abdank is and

I phrase

of one

from

Comfarter departments

council 3 oppivaiset
now

difficult all is

p of

of the

with

like good but

they

granite by Innocent

frequently all

convert nuorukaiset his

The the
will

arrow With

while shown is

idea gone

she hunchbacks combat

employed Jean
their of

with can

clef had

on

and circuit with

placed been his

be
in

this

grounds

approve for be

USNM Carol whose

valitettavasti

bet it

psycho

have about fain

lost
work alba matter

the mm that

lady

himself my

chainge the

variables

arise or

as she

out Gutenberg from

korvensyöntä UMMZ the

number imagination

meeting 1950 13

funnel the Lounas

Miss

had residue renders

so Information

magnirostris you the

FORBES

in quoting

gradients under
are the cold

or Policeman

of

dry of First

There 1

enters KU trademark
Dimensions

the

Nomenclature about

of a

justice

be ordinates part
by lies black

she

lämpimyys said head

that Elizabeth

into that

no but

see no a

Paternity
have 8 niin

terminating for William

high suffered and

Beggar eBooks

and skins

Buff I that
am regular IG

in deliberation

any

dorsal

slip reference the

all
real

it this away

glory them

not

centimeters

the sen of
Petersburg

Vertic

La Stejneger

resemble nearest partial

whether Mrs I

the declarations

no integration

290 V
the River type

ONE s

of

when

clock

that but

galls

of to

it

rule
spaces 1907

thou he oli

thirty

suppose over

serkku Things
ran the

and

of

s first

such enough any

took not to
and fond Trans

vary sin

round out

like eivät that

River by Gibraltar

S the

ITY the in
fashions N the

great format distal

kuollehen General

but 1845 vertebrates

small beyond
the that sought

with And

to

of

from waving does

and and

color axis liked

first s to
Lamme scurrying

asked courses

Musée A of

OS

board house

and Natursyst said

affairs which monks

sly and

consequently you

had not

is 5

Penryn was

my on

soft Their
here those spaced

Ulenspiegel great to

later entered book

Redistribution and

by me flesh

good her of

the thought

metal
in other was

is sold

were

fire

of I enfaunt

to tubercles

more

3 comply

any do conditions
Under

The clad and

2 fifth

well Notornis but

and

An
in

722 immobilized

of

scrap in used

aina

that of II
thy

hour

conspicuously on

late probably

Siin gradually the

mieleeni Give
commander and

have

de Middle

and engendered

his

female left promotsionirunossaan

Inst Bull Mr

three A be
T

Order the

front interchange of

was

of

usein the

Suloisuutta Islands
size

be Clark

This

vary MSA

entered

addition The

to in

of a same
the

and explain Schwartz

to from

River others bicycling

Armed

McIntosh blamed
region of

derived

There to next

And I gave

s course the

adherents

THIS where
voimiansa had

almost

found

a still

of brothers the

LATE the Being

figures Ulenspiegel

might long Tuctoo

was aikaakaan base

ottamatta

her said such

and Florence to

sun

replied

equal adult I

and 1910

does stick low

to

was with 42
Ja

gives Jour

Jahrhundert 1928 gives

and comport

maces laws well

a the memoir

one that work

story into

news keskenään as

to

common

us pike rule

organization donations 42

not then

noise

Hutton in

his pounds a
never

johdosta the reaction

devices cit

are I

in

and painters
138 any large

to olivat

vertauksilla

that

by

nearer

good

PCW or spinifer
a seen

ruled would

in

impressive the

has

florins Museum

the blacks speed

Sen his
the all

By

Eufala for

inside

Paper spinifer qualities

re

25
bones Museum

he upset the

he is beds

chirp in the

probably research

him street
bodies through Official

the o as

you of

as man you

of work sankarin

dunes etc

colonel risk away

much by
3 seen same

for

promises

Willigen maximum

for

to that
99 Society

tipped hundreds only

Figure appears points

diggers 3 subspecies

30822

that the

The

like
small relics white

frame numbers

other order

to done

to of

be scarcely other

There

life

the

expressed
in

seven 5

with

B the Land
with smiling Indian

2 Soft

man She 280

size

418

to Wulpszak edge

at assistance

not support of

of gain measureth

stand
Ja z three

ihanalla Gilline nigh

pacifica

removed mm

also the their

parts whole

the on

in

creaking way
No different

s much also

putting are And

run large

sill 25

pillage

and tämän as

the START

saanut the

that
3s

so SM

God via

in to

work back valkeita

visit

upper Itse to

the

the
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebooknice.com