Nonreactive Research

&
Secondary Analysis

I.

II.

III.

Nonreactive Research
a. Experiments and survey research are both reactive; that is, the people being studied are
aware of that fact. The following quantitative research techniques are nonreactive; that is,
those being studied are not aware that they are part of a research project.
Nonreactive Measurement
a. The Logic of Nonreactive Research
i. The critical thing about nonreactive or unobtrusive measures (i.e., measures that are
not obtrusive or intrusive) is that the people being studied are not aware of it but
leave evidence of their social behavior or actions naturally. The observant
researcher infers from the evidence to behavior or attitudes without disrupting
those being studied.
b. Varieties of Nonreactive or Unobtrusive Observation
i. Erosion Measures
1. Where selective wear is used as a measure.
ii. Accretion Measures
1. Where the measures are deposits of something left behind.
c. Recording and Documentation
i. Steps for Success When Using Nonreactive Approaches
1. Conceptualize a concept.
2. Link the construct to nonreactive empirical evidence.
3. Clearly operationalize the construct.
4. Collect data following clear rules and guidelines.
Content Analysis
a. Is a technique for gathering and analyzing the content of recordable elements in
communication. The content refers to words, meanings, pictures, symbols, ideas, themes, or
any message that can be communicated. The recordable elements in communication include
anything written, visual, or spoken that serves as a medium of communication.
b. Topics Appropriate for Content Analysis
i. Problems involving large volumes of text.
ii. Helpful when a topic can be studied at a distance.
iii. When the researcher is attempting to reveal messages in a text that are difficult to
see with casual observation.
c. Measurement and Coding
i. General Issues
1. Social researchers must carefully design and document procedures for
coding and recording so that replication is possible. Constructs in content
analysis are operationalized with a coding system: a set of instructions or
rules on how to systematically observe and record content from text.
ii. Unit of Analysis
1. The unit of analysis can vary greatly in content analysis. It can be a word,
phrase, theme, plot, newspaper article, character, and so forth.
iii. What is Measured?
1. Measurement in content analysis uses structured observation; systematic,
careful observation based on written rules. The rules explain how to
categorize and classify observations.
2. Characteristics of the Coding System
a. Frequency
i. Simply means counting whether or not something
occurs and, if it occurs, how often.
b. Direction

47

IV.

i. Involves noting the direction of the messages in the

content along some continuum (e.g., positive, negative,
supportive, opposed).
c. Intensity
i. Involves the strength or power of a message in a
direction.
d. Space
i. A researcher can record the size of a text message or
the amount of space or volume allocated to it. Space in
written text is measured by counting words, sentences,
paragraphs, or space on a page (e.g., square inches). For
video or audio text, space can be measured by the
amount of time allocated.
d. Coding, Validity, and Reliability
i. Manifest Coding
1. Coding the visible, surface content in a text is called manifest coding
a. Reliability
i. Manifest coding is very reliable because the phrase or
word either is or is not present
b. Validity
i. The same word can take on different meanings
depending on the context. The possibility that there are
multiple meanings of a word limits the measurement
validity.
ii. Latent coding
1. A researcher uses latent coding (also known as semantic analysis) and
looks for the underlying, implicit meaning in the content of a text. Before
determining the implicit meanings, the researcher must provide general
guidelines for interpretation of the text and for determining whether
particular themes or moods are present.
a. Reliability
i. Tends to be less reliable than manifest coding because
it depends on a researchers knowledge of language and
social meaning.
b. Validity
i. The validity of latent coding can exceed that of
manifest coding because people communicate meaning
in many implicit ways that depend on context, not just
in specific words.
c. Intercoder Reliability
i. Reliability is usually determined using intercoder
reliability, a type of equivalence reliability, with a
statistical coefficient that tells the degree of consistency
among coders. The coefficient is always reported with
the results of content analysis research.
Existing Statistics/Document and Secondary Analysis
i. Topics Appropriate for Existing Statistics Research
1. Any topic on which information has been collected and is publicly
available can be studied. Existing statistics research is best for topics that
involve information collected by large bureaucratic organizations both
public and private.
ii. Social Indicators
1. Is any measure of social well being used in policy. An example of a social
indicator is the FBIs uniform crime index. It indicates the amount of
crime in U.S. society. Social indicators can measure negative aspects of
social life, such as infant mortality rate and so forth.
iii. Locating Data

48

1.

The main sources of existing statistics are government or international

agencies and private sources. An enormous volume and variety of
information exists. If you plan to conduct existing statistics research, it is
wise to discuss your interests with an information professional- in some
cases, a reference librarian, who can point you in the direction of possible
resources. The single most valuable source of statistical information about
the United States is the Statistical Abstract of the United States, which has
been published annually (with a few exceptions) since 1878.
iv. Secondary Survey Data
1. Secondary analysis is a special case of existing statistics; it is the re-analysis
of previously collected survey or other data that were originally collected
by others. It is relatively inexpensive; it permits comparisons across
groups, nations or time; it facilitates replication; and it permits asking
about issues not thought of by the original researchers.
a. Questions a Researcher Interested in Secondary Research Should
Ask!
i. Are the secondary data appropriate for the research
question?
ii. What theory and hypothesis can a researcher use with
the data?
iii. Is the researcher already familiar with the substantive
area?
iv. Does the researcher understand how the data were
originally gathered and coded?
2. Common Mistakes Made by Researchers Who Use Existing Statistics
a. Existing statistics and secondary data are not trouble free simply
because a government agency or other source gathered the
original data. A common error is the fallacy of misplaced
concreteness. It occurs when someone gives a false impression
of accuracy by quoting statistics in greater detail than warranted
by how the statistics are collected and by overloading detail. For
example, in order to impress an audience of ones command of
particulars, a politician might say that the population of South
Africa is 36,075,861, when they should say it is roughly 36
million.
3. Reliability
a. Problems with reliability can plague existing statistics research.
Listed below are some problems researchers may encounter
regarding reliability:
i. Stability Reliability
1. Stability reliability problems develop when
official definitions or the method of collecting
information changes over time.
ii. Equivalence Reliability
1. Equivalence reliability can also be a problem.
For example, a measure of crime across a
nation depends on each police departments
providing accurate information. If
departments in one region of a country have
sloppy bookkeeping, the measure losses
equivalence reliability.
iii. Representative Reliability
1. Representative reliability can be a serious
problem in official government statistics. This
could result from discrimination by police,
institutional racism or poor instrument

49

construction, which could lead to the

misidentification of a person.

4.

V.

Validity
a. Listed below are some problems researchers may encounter
regarding validity:
i. Validity problems can occur when the researchers
theoretical definition does not match that of the
government agency or organization that collected the
information. Official policies and procedures specify
definitions for official statistics. For example, a
researcher defines a work injury as including minor
cuts, bruises, and sprains that occur on the job, but the
official definition in government reports only includes
injuries that require a visit to a physician or hospital.
ii. Another validity problem arises when official statistics
are a surrogate or proxy for a construct in which a
researcher is really interested. This is necessary because
the researcher cannot collect original data. For
example, the researcher wants to know how many
people have been robbed, so he or she uses police
statistics on robbery arrests as a proxy. But the measure
is not entirely valid because many robberies are not
reported to the police, and reported robberies do not
always result in an arrest.
iii. A third validity problem arises because the researcher
lacks control over how information is collected.
5. Units of Analysis and Variable Attributes
a. A common problem in existing statistics is finding the
appropriate unit of analysis. Many statistics are published for
aggregates, not the individual. A related problem involves the
categories of variable attributes used in existing documents or
survey questions. This is not a problem if the initial data were
gathered in many highly refined categories. The problem arises
when the original data were collected in broad categories or ones
that do not match the needs of the researcher.
6. Missing Data
a. One problem that plagues researchers who use existing statistics
and documents is that of missing data. Sometimes, the data were
collected but have been lost. More frequently, the data were
never collected.
Issues of Inference and Theory Testing
a. A researchers ability to infer causality or test a theory on the basis of nonreactive data is
limited. It is difficult to use unobtrusive measures to establish temporal order and eliminate
alternative explanations. In content analysis, a researcher cannot generalize from the content
to its effect on those who read the text, but can only use the correlation logic of survey
research to show an association among variables. Unlike the ease of survey research, a
researcher does not ask respondents direct questions to measure variables, but relies on the
information available in the text.

50