AMA EDUCATION SYSTEM

RSCH 6258: ACCOUNTING RESEARCH METHODS

Introduction and Overview

Research in accounting is concerned with solving problems, investigating relationships and

building a body of knowledge. Because we rely to such a great extent on prior research in the natural
and social sciences to do so, this volume will take a similar approach in leaning on work in other
disciplines where it helps to inform accounting research. Bennett (1991) identifies four basic levels of
research:

• Description – concerned with the collection and reporting of data related to what is, or was, the
case. This would include means and standard deviations of individual variables, and correlations
between pairs of variables.
• Classification – still descriptive, but easing the reporting process, and highlighting similarities
and clustering through grouping and classifying (e.g., through the familiar cross-tabulation
facility in most basic statistical packages).
• Explanation – an attempt to make sense of observations by explaining the relationships
observed and attributing causality based on some appropriate theory.
• Prediction – going beyond the understanding and explaining of the prior stage, to model
observations in a way that allows testable predictions to be made of unknown events.

Good research generates the sound evidence needed to overturn or revise existing theories.
These assertions will, in turn, yield to revised theories based on better evidence, so that healthy
competition between rival ideas will lead to better explanations and more reliable predictions. Two
major processes of reasoning, ‘deductive’ (theory to observation) and ‘inductive’ (observation to
theory), are important for theory construction and observation testing. Inductive reasoning starts with
specific observations (data) from which theories can be generated; a generalizable pattern may emerge
from further observations and repeated testing for compliance.

Theory as testable explanation

Faced with a set of diverse observations, we can establish a set of tentative explanations which
help to make sense of the diversity. Such explanations constitute theory. In any set of circumstances
there will usually be multiple theories available to explain the observations. The systematic collection of
further data allows for the testing of the alternative theories so that we can establish which of the
exiting theories best explains the facts. A layman’s perspective of ‘theory’ is cynically expressed in
Michael Crichton’s The Lost World as: ‘A theory is nothing more than a substitute for experience put
forward by someone who does not know what they are talking about’ (1995, p. 67).

The data collection itself allows only a descriptive approach (e.g., means, standard deviations,
ranges, correlations); we cannot attempt to attribute causation in any meaningful way without recourse
to an explanatory theory. We are always looking for another theory which may fit better, so that, as
Popper (1959, p. 104) suggests, a ‘genuine test of a theory is an attempt to falsify it or refute it’. We look
for disconfirmations rather than confirmations.

Week 1: Introduction and Overview

Page 1 of 6
 AMA EDUCATION SYSTEM
RSCH 6258: ACCOUNTING RESEARCH METHODS

In the short term this may not be successful. In accounting, we witness the frequent and
numerous ‘anomalies’ to which the Efficient Markets Hypothesis (EMH) is subject, but we have no other
widely accepted theory of the manner in which stock prices react to the availability of relevant
information.

Popper’s suggestions are very attractive in providing a powerful empirical methodology for
subjecting theories to attempts to refute them. However, this position is not always ideal because the
process of ‘observation’ in itself may be fallible. Thus Hawking (1998) reports Heisenberg’s Uncertainty
Principle:

If we are to predict the future position and speed of any particle,

then we require accurate measurement of both its present position
and current speed. Heisenberg did this in 1926 by shining light on
a particle, and observing the resultant scattering of light in order to
reveal its position. However, to determine the position of the
particle accurately an amount of light needed to be used which
changed the speed of the particle in an unpredictable way: the
more accurately he tried to measure position, the less accurate
was the measurement of speed!

The Uncertainty Principle has wide implications for research conducted in any environment,
where it is impossible to measure the size and speed of a particle without altering all other
characteristics in the process of measurement. We have a parallel situation in accounting research
where the actions of the participants in ethnographic, experimental, survey or fieldwork impacts on the
outcomes of the measurement process.

Three fundamental criteria exist to judge whether theory fits observation:

1) Co-variation – even where no causality exists we would expect the two variables to
move together so that a high degree of correlation exists between the two variables.
Where there is no co-variation it will be difficult to establish a causal link.
2) Cause prior to effect – if a causal link is to be established then the ‘causal event’ should
occur before the ‘effect event’. The sequence of events can therefore help to establish
an explanatory direction.
3) Absence of plausible rival hypotheses – the third rule seeks to eliminate alternative
explanations of the events as being implausible. This may only be possible in the
present, because future researchers may develop competing explanations of the events
from a re-analysis of the data.

Consider, for example, the voluntary disclosure of information in corporate reports and analyst
following (i.e., the number of analysts examining the performance and reporting on the disclosures of
large companies). There is a relationship between these two variables – they co-vary: the volume of
voluntary disclosures and the number of analysts reporting move together.

Week 1: Introduction and Overview

Page 2 of 6
 AMA EDUCATION SYSTEM
RSCH 6258: ACCOUNTING RESEARCH METHODS

But which is causing which? Rival hypotheses suggest:

a) companies are supplying more information voluntarily to the market to signal their
intentions and reputation, attracting the attention of more investment analysis;
b) investment analysts are focusing their attention on particular companies and
demanding more information and more detailed disclosures.

The existing empirical evidence is less than convincing: Lang and Lundholm (1996) find (a); but
Walker and Tsalta (2001) provide only weak evidence for (a) but stronger evidence to support (b).
Clearly more empirical work is required to clarify the direction of causation.

A critical approach to accounting research

Researchers must demonstrate a healthy skepticism towards both their own findings and those
of other researchers. They must adopt a critical posture, questioning everything that they read until
sufficient evidence has been provided for them to be satisfied with the quality of the outcomes. The
development of critical appraisal skills is a fundamental requirement in researchers, so that they can
distinguish between good and bad research, and clearly identify flaws of argument, methodology and
analysis.

Honest and transparent reporting of research practice is an ethical duty of those participating.
Researchers should report everything that they did, why they did it, and how they did it. If they have
doubts about any stage of the procedure, then these should be stated, along with their likely
implications and what, if anything, has been done to overcome these doubts. Where researchers have
been ‘economical with the truth’, this is usually apparent in their papers and is often an indicator of bad
research.

With appropriate guidelines as to the appropriate questions to ask, students can quickly develop
some confidence in their ability to spot flaws and omissions. We would usually want to address the
following:

1) Why is this article interesting/important? The paper must offer some new insights which
constitute a contribution to knowledge. These insights should be non-trivial, so that they
can be embraced in either further theory development or recommendations for
improvement.
2) Are the outcomes important? Effectively, does the paper pass the ‘so what’ test? Will
anyone be interested in the outcomes of this research, or will it have any implications for
future practice? Would the scope of the research be well regarded by competitive grant
authorities? This has important implications for those papers which produce ‘negative’
findings, that is, they test reasonable hypotheses based on the research literature, but their
datasets fail to support any of the expectations. These findings still make a contribution in

Week 1: Introduction and Overview

Page 3 of 6
 AMA EDUCATION SYSTEM
RSCH 6258: ACCOUNTING RESEARCH METHODS

that they demonstrate that findings from elsewhere (often other disciplines) do not hold in
accounting, but their negativity may restrict their publication opportunities.
3) What motivates the authors to write this article now? The paper may be clearly addressing
issues of contemporary concern; on the other hand, it may be addressing more historical
issues and/or be using ‘old’ data. If we have the latter, we may be dealing with an old paper
recently recycled, or a paper which has been through many iterations at several different
journals before being deemed ‘publishable’.
4) What is the research problem/question? We are looking for a clear statement of the
problem very early on in the paper, so that its objectives are readily apparent. If we reach
page 11, say, of the paper without a clear idea of its direction, or any sort of research model,
then perhaps the authors need to readdress the fundamental purpose of the research.
5) What theory or theoretical framework underpins the research? Without some theoretical
foundations we have a problem-solving exercise or a consultancy project, neither of which
should be gracing the pages of a refereed journal. There must be some theoretical
justification for the question being addressed and the research approach adopted. Theory
will often not come first in the research process – it will frequently be preceded by an
interesting idea or a perplexing observation. But we require some theoretical explanation
for the relationships under investigation before we have the basics of a refereed journal
article. Observed deficiencies in this area usually fall into one of four categories:

• the underlying theory is either non-existent or extremely thin;

• the theoretical context is there but appears to have been tacked on as an
afterthought – usually at the beginning of the paper and often written by a co-
author. Examination of writing styles suggests that we frequently do not have a
seamless divide between ‘theory’ and ‘conduct of research’;
• the theoretical arguments are unconvincing, so that there are competing
theories that may reasonably have been adopted in the paper but have been
overlooked;
• a sound theoretical framework but findings which are totally at odds with
theory. Apparently, a competing theory may be more appropriate, although this
is unknown to the authors at the time.
6) What are the key motivating literatures on which the study depends? There will normally
be a small number of seminal pieces of literature which are driving the research. If any of
these are themselves unreliable, it may cast doubt on the state of the foundations on which
the paper is based. If one of the papers is an unpublished conference or working paper from
several years before, then alarm bells ring to question why that piece has not itself been
published in the refereed literature. If key seminal pieces of literature have been
overlooked, then again the integrity of the findings is reduced.
7) Which research method has been chosen? There should be a justification for the chosen
method, and a clear preference over alternatives. The method should be consistent with

Week 1: Introduction and Overview

Page 4 of 6
 AMA EDUCATION SYSTEM
RSCH 6258: ACCOUNTING RESEARCH METHODS

both theory and literature and, ideally, prior empirical studies in the field will have adopted
similar methods. Most importantly, we want to see a research method that has evolved
rather than one that has been selected first, even before the research question has been
fully developed. The use of survey methods should always be questioned in this way since
frequently they seem to have been selected without explanation of the elimination of
alternatives. Ideally, we should be able to trace through the emergence of abstract
concepts, from theory, through their operationalization and measurement, so that any
hypotheses are entirely consistent with both theory and literature.
8) How has the sample been selected? Details on sample selection are often sketchy in many
articles, perhaps because the authors feel vulnerable about the procedures adopted.
Sometimes (see, for example, Young, 1996) the actual sample size employed is omitted, as is
the response rate. Both omissions should be regarded as bad news. It is usually clear that
scientific methods have not been adopted (unfortunately far too commonly in accounting
research) where there is an over-reliance on convenience samples. What may be apparent is
an attempt by the authors to obfuscate in this regard, to overlook detail and try to create an
impression that the sample selection is more systematic than it has actually been.
9) How have questions of validity been addressed? Choice of research method should address
issues of validity. Where experimental methods have been employed we would anticipate
questions of internal validity to be paramount; where field studies are involved we would
expect issues of external validity to be addressed. For survey methods we would anticipate
the focus to be more on the reliability of the test instrument and the rigor of the subsequent
statistical analysis, rather than on validity issues.
10) How have the results been analyzed? We want to see the simplest analysis of the results
consistent with the relationships being explored. We do not wish to see unnecessary
complexity; this will make the paper less readable and tend to mask the findings and their
significance. On the other hand, most academic accountants are only ‘amateur’ statisticians;
if the level of their analysis is inadequate, then they may need to bring in a statistician as co-
author (evidenced by the number of ‘quant jocks’ appearing as third or fourth authors on
accounting papers to satisfy the reviewers). Importantly, we do not wish to see the method
of analysis driving the study. In just the same way as the research method should not
precede the research question, then neither should the method of analysis. For example, I
recall a paper of my own (M. Smith, 1992) presented at a conference but never published. It
attempted to show the advantages of using multidimensional scaling (MDS, then a little-
used technique in the accounting literature) for problem-solving, but the journal referees
rightly observed that the method was inappropriately sophisticated for a relatively simple
research question. MDS was abandoned, simpler methods instituted and the revised paper
eventually published as Smith (1996).
11) Are the conclusions and recommendations consistent with the findings? Effectively, does
the paper hold together? Is the title appropriate? Do the abstract and introduction lead us
to expect what we find at the end of the paper? In many papers the final sections are the

Week 1: Introduction and Overview

Page 5 of 6
 AMA EDUCATION SYSTEM
RSCH 6258: ACCOUNTING RESEARCH METHODS

weakest and may not do justice to the breadth of the research conducted. We look for
explanations, limitations and a future research agenda.

Week 1: Introduction and Overview

Page 6 of 6