The ``art'' of questionnaire construction: some

important considerations for manufacturing studies

Nicolaos E. Synodinos
Department of Marketing, University of Hawaii at Manoa, Honolulu, Hawaii, USA

Keywords order to generalize to their parent

Surveys, Questionnaires, Introduction population. The answers are obtained via self
Development, Methodology
Manufacturing decisions should be guided or interviewer-administered questionnaires
Abstract not only by technical feasibility, but also by and typically are structured and undisguised.
This article reviews research many other factors. These include a detailed That is, various aspects of the questionnaire
findings related to the ``art'' of understanding of the ever-changing needs of are clearly specified and each question's
constructing survey questionnaires.
potential customers, the variables of the purpose is apparent to the respondent.
It discusses some of the important
issues that should be considered in marketing mix, and the factors of the Surveys may sample citizens, consumers,
gathering quality data via external environment. Occasionally, employees, managers, organizations, or other
questionnaires, provides general knowledge and intuition are sufficient for entities. The studies in Table I employed a
suggestions for their construction,
developing appropriate managerial wide variety of samples. For example, their
includes a comprehensive list of
important reference sources, and strategies. Almost invariably, these are not samples consisted of groups such as
examines some of the survey-based enough and additional information is practitioner members of the Institute of
studies published in Integrated required. Frequently, survey research is the Operations Management (Burcher and Lee,
Manufacturing Systems. 2000), managers of automotive assemblers
chosen (and most appropriate) approach to
Constructing a good questionnaire
requires a thorough grasp of the gather the additional data needed for sound and suppliers (Burgess et al., 1997), and
intricacies of the topical area and manufacturing strategy decisions. furniture manufacturers with more than 30
detailed knowledge of the The origins of modern surveys are in early employees (Huang and Mak, 1998).
strengths and weaknesses of the Although surveys are susceptible to
twentieth century public opinion polls and
different survey administration
modes. In addition, questionnaire marketing research. Since that time, they various errors, one of the most critical and
construction entails close attention have proliferated in many fields and are preventable threats to their validity comes
to details about the wording of commonly used to obtain diverse types of from the design of their questions (Fowler,
questions, their instructions, their information in organizational settings (Kraut 1995). Question clarity and consistent
response choices, and their
and Saari, 1999). Surveys can measure meaning to all respondents can be
sequence. Most importantly, the
research instrument should be managers' attitudes about certain issues, instrumental in reducing bias. Also, well-
refined based on guidance from gather subjective appraisals of constructed questionnaires may contribute
repeated pretests. Well- manufacturing processes, or obtain to reductions in item and unit nonresponse.
constructed questionnaires can
expectations of various outcomes. Also, Nevertheless, high nonresponse does not
ensure the consistent meaning of
the questions across respondents surveys are used ``as strategic tools to drive necessarily equate to high nonresponse
and can contribute to data quality and measure organizational change'' (Kraut, error; it indicates the potential for such error
by decreasing both item and unit 1996, p. 11) and in studies of ergonomic because respondents are probably different
nonresponse.
improvements in manufacturing and from nonrespondents.
production (Lockhart and Russo, 1994). Many articles in manufacturing
Over the years, Integrated Manufacturing technology have been plagued with high unit
Received July 2001 Systems (IMS) published numerous articles nonresponse (see Table I). For instance, the
Revised March 2002 that collected their data via surveys. Table I authors of two of these articles (Newman and
Accepted July 2002 summarizes the characteristics of some of Sridharan, 1995; Riedel and Pawar, 1997) were
these studies. In addition, IMS published disappointed with their return rates (12.3
case studies that relied mostly on percent and 13 percent respectively), but
unstructured or semi-structured suggested that these rates were comparable
questionnaires. Although not included in the to those frequently obtained in other
table, such cases can also benefit from industrial surveys. The limitations imposed
guidelines of questionnaire construction. by low response rates were clearly noted by
Integrated Manufacturing Surveys rely on self-reported answers Gascoigne et al. (1997). They commented that
Systems obtained from a sample of respondents in their study may provide some useful
14/3 [2003] 221-237
# MCB UP Limited
[ISSN 0957-6061] The Emerald Research Register for this journal is available at The current issue and full text archive of this journal is available at
[DOI 10.1108/09576060310463172] http://www.emeraldinsight.com/researchregister http://www.emeraldinsight.com/0957-6061.htm

[ 221 ]
[ 222 ]
Systems
considerations for
Nicolaos E. Synodinos

manufacturing studies

14/3 [2003] 221-237

Integrated Manufacturing
The ``art'' of questionnaire
construction: some important

Table I
Selected studies published in IMS that used surveys
Author(s) and year of Survey Reported rate(s)b
publication Topic of article Composition of contacted samplea Additional notes mode (%)
Batley (1993) Measurement of product 350 operations managers of manufacturing companies in New Mail 39.7 (usable)
quality Zealand
Bennett et al. (1997) Transfer of manufacturing UK survey: Approximately 1,200 UK and UK-based foreign Received 207 questionnaires from the UK survey and 193 from Mail
technology to the PRC companies in the database of ``China-Britain Trade Group'' with the PRC survey 17 (overall average
involvement or interest in the PRC of both surveys)
PRC survey: Approximately 1,200 PRC companies that were Mail
members of the ``Ministry of Machinery Industry Technology
Import Information Centre'' and some non-machinery industry
companies listed in various Chinese enterprises directories
Burcher (1992) Production scheduling and User survey: 2,700 manufacturing practitioner members of the Mail 14.8
capacity planning in ``British Production and Inventory Control Society'' and personal 12.9 (usable)
manufacturing organizations contacts of the author
Supplier survey: 90 suppliers (comprising the total population) For the supplier survey, 27 questionnaires were reported as Mail 30 (usable)
of production management software packages listed in the usable; it is not specified whether the number returned was larger
1990-91 ``Industrial Computing Sourcebook'' of the British
Production and Inventory Control Society
Burcher and Lee Investment decisions in Manufacturing practitioners in the UK belonging to the The number of questionnaires mailed was not reported; 161 Mail Not given
(2000) AMT ``Institute of Operations Management'' persons replied
Burgess et al. (1997) AMT implementation among 155 companies (15 assemblers and 140 suppliers) from It is unclear how the given rate of 59 percent was derived as it Personal 59
automotive supply-chain automotive industry databases in Turkey was also noted that 83 of the 155 participated interview
members Up to 3 managers from each participating company were
interviewed
Gascoigne et al. Marketplace for cell control 100 industrial companies comprising vendors and end-users of Self-Administered presumed to be a mail survey. Mail
(1997) systems in Britain cell control systems in the UK 25 questionnaires were returned; 4 of these were excluded from
Two similar versions of the questionnaire were used (one for the analyses because they were not from vendors or end-users
vendors and one for end-users)
Gieskes and Continuous improvement Project managers of organizations in The Netherlands involved Two additional contacts by phone (a pre-notification and a Mail 40
ten Broeke (2000) implementation and in infrastructure management reminder) were used
organizational learning Unclear whether the number mailed was 92 or 86 ± if the latter,
the return rate was 43 percent (using the authors' method to
compute it)
(continued)
 Systems
considerations for
Nicolaos E. Synodinos

manufacturing studies

14/3 [2003] 221-237

Integrated Manufacturing
The ``art'' of questionnaire
construction: some important

Table I
Author(s) and year of Survey Reported rate(s)b
publication Topic of article Composition of contacted samplea Additional notes mode (%)
Gilgeous (1998) Managers' working life 500 manufacturing managers of UK companies in the ``Financial The reported rate by industry sector varied from a low of 6 Mail 10.8
quality Analysis Made Easy'' database from five industry sectors percent for textiles to 33 percent for food and drink. No
(automotive; printing; textiles; food and drink; electronics) deductions should be drawn from this variation because only 54
questionnaires were returned from all industry sectors
Gupta et al. (1998) Employees' organizational 500 managers (with responsibility of strategic decision-making One additional contact (a second mailing of the questionnaire) Mail 20.2 (usable)
commitment and AMT and knowledge of manufacturing operations) of USA was used
intensity manufacturing firms listed in the searched directories with
Standard Industrial Classification codes 34 to 38
Huang and Mak Product design by furniture Approximately 200 office and residential furniture 30 questionnaires were returned and 25 were deemed usable Mail  15c
(1998) manufacturers manufacturers in UK with more than 30 employees
Newman and Alternative manufacturing 1,500 manufacturing firms in Mid-Western USA with more than Mail 12.3
Sridharan (1995) planning and control 150 employees 11 (usable)
systems
Orr (1996) Use of industrial robotics First project (in 1990): 74 companies in Australia using Personal 43
industrial robotics interview
Second project (in 1993): 112 companies in Australia using 2 additional contacts (a reminder letter and a telephone call) Mail 52
industrial robotics. (Second project was a replication of the first were used in the second project
by a mail survey)
Orr (1999) Role of technology in 826 registered wine producers in Australia Self-administered questionnaire presumed to be a mail survey. Mail Not given
manufacturing strategy of Those replying represented 37 percent of the population of wine
wine producers producers in terms of employment numbers; 99 percent in terms
of production volume; and 99 percent in terms of wholesale sales
Riedel and Pawar Product design and Senior managers of mechanical engineering companies in the 109 usable questionnaires were returned Mail 13 (usable)
(1997) production aspects of UK included in the ``Financial Analysis Made Easy'' database
mechanical engineering
firms
Sohal et al. (1996) Decision-making processes 290 senior managers of New Zealand manufacturing One additional contact (a telephone reminder) was used. Mail 20 (usable)
in AMT organizations As incentive, the respondents could request copies of the
research report and of a previous related study
Tummala et al. Manufacturers' opinions on 400 companies of various manufacturing industries in Hong 18 of the 400 questionnaires were returned as undeliverable Mail 15.5 (usable)
(2000) global competitiveness Kong

Notes: AMT = Advanced manufacturing technology. aIn some articles, the ultimate recipients of the questionnaire within an organization were not clearly specified; bThis is the rate reported in the article. It is referred to
here as the ``reported rate'' because there are different ways of computing various outcome rates and the articles have not always included all the necessary details. Rates that are followed by the word ``usable'' are
those that the article specifically reported as usable questionnaires; cApproximate value deduced based on information given in the article

[ 223 ]
Nicolaos E. Synodinos insights, but it ``cannot be claimed to Questionnaire construction can be
The ``art'' of questionnaire represent the UK cell control marketplace deceptively simple (Birn et al., 1990; Sudman
construction: some important fully due to the limited response'' (Gascoigne and Bradburn, 1982). This apparent
considerations for
manufacturing studies et al., 1997, p. 181). With return rates as low as simplicity creates many problems because
Integrated Manufacturing those obtained in some of the studies in poorly constructed instruments can lead to
Systems Table I, the findings can be outright erroneous conclusions. In fact, Schwarz
14/3 [2003] 221-237 misleading if respondents differ from (1996, p. 72) noted that:
nonrespondents on some critical dimensions. Survey methodology has long been
Some of the articles of Table I reported characterized by rigorous theories of
their return and their usable questionnaire sampling on the one hand, and the so-called
``art of asking questions'' on the other.
rates. In these studies, the usable
questionnaires were approximately two Thus, the most critical element of surveys
percentage points below the number may end up being its weakest link (Bradburn
returned. For instance, Burcher (1992) and Sudman, 1988).
obtained a return rate of 14.8 percent and a The present article draws from the various
usable rate of 12.9 percent; Huang and Mak academic disciplines that contributed to the
(1998) had return and usable rates of development of survey research and
approximately 15 percent and 12.5 percent discusses the important issues that should be
respectively; and Newman and Sridharan considered in the development of high
(1995) reported a return rate of 12.3 percent quality questionnaires. Typically,
and a usable rate of 11 percent. It is unclear manufacturing-management researchers are
how liberal or conservative were the various not specialists in survey measurement.
authors' criteria of determining usability. Therefore, it is useful to summarize for them
Also, details of the mailing (e.g. number of some of the important guidelines and provide
questionnaires returned as undeliverable) references that they ought to consult during
were not usually given in the articles of the process of constructing their
Table I. All researchers of survey-based questionnaires.
studies must peruse the guidelines of the Guidelines should not be considered in a
American Association for Public Opinion vacuum, but in the context of the unique
Research (AAPOR, 2000) about computing circumstances surrounding a particular
and reporting detailed outcome rates. survey. Indeed, experts (Labaw, 1980;
Citations of survey methodology sources Oppenheim, 1966; Payne, 1951; Peterson, 2000;
were largely absent in the articles of Table I. Sheatsley, 1983; Sudman and Blair, 1998) have
An exception is Orr's (1999) article that cites warned explicitly against blind adherence to
textbooks in marketing research (i.e. rules of questionnaire construction. Payne
Zikmund) and sociological methods (i.e. (1951, pp. 98-9), in the classic book The Art of
Denzin). Empirical studies in manufacturing Asking Questions, emphasized that:
strategy are relatively recent and have been An open mind is especially needed in
traced to the mid 1970s (Swink and Way, research, and flat rules or arbitrary
1995). It is not surprising that many judgments might do more harm than good.
researchers in this and related fields ``do not The guidelines presented here should be
have a strong foundation in gathering and viewed with this important admonition in
using empirical data'' (Flynn et al., 1990, mind.
p. 250). However, such studies have gained The construction of a questionnaire
momentum and scholars in the area ``are consists of various interrelated steps that
learning to use the empirical methods that start with the research objectives and end
have been developed in other related with the final version of the instrument. This
academic disciplines'' (Minor et al., 1994, progression is summarized in Figure 1 and
p. 22). discussed in the remainder of this article.
An authoritative review of empirical
studies in operations management pointed
out that many of the ``questionnaires appear
to have been thrown together hastily, with
Administration method
little thought of reliability, validity or Almost invariably, there is no method that is
generalizability'' (Flynn et al., 1990, p. 259). superior to the others in all circumstances
Other authors (Fowler, 1995; Lockhart and (Bradburn, 1983). Several factors should be
Russo, 1994) noted that it is not uncommon considered in the selection of the most
for researchers ± usually in disciplines appropriate survey administration mode.
outside of survey/marketing research ± to Among them are the objectives of the study,
hold the mistaken belief that questionnaires the target group and its geographic
can be easily written by anyone distribution, the types of questions, and the
knowledgeable in the topical area. available resources. In multi-national
[ 224 ]
Nicolaos E. Synodinos studies, the selection is complicated further Personal interviews are appropriate for
The ``art'' of questionnaire by the fact that an appropriate method in one surveys where the sequence of the questions
construction: some important is consequential and where there is a need to
considerations for country may be inappropriate in another.
manufacturing studies Cost is an important determinant in choosing use visual materials. Usually, personal
Integrated Manufacturing the survey administration mode, but it interviews achieve higher response rates
Systems should never overshadow data quality than telephone or mail surveys and can be
14/3 [2003] 221-237 used for lengthier and more complex
considerations.
Survey administration methods can be questionnaires. Unfortunately, personal
classified into interviews and self- interviews are costly, especially when their
administered questionnaires (SAQs). samples are geographically widely dispersed.
Interviews can be conducted in person or via Generally, personal interviews are perceived
telephone. SAQs include postal and all other as being less anonymous than mail
forms of questionnaires designed to be questionnaires and telephone surveys. In
respondent administered (e.g. fax, e-mail, some cases, personal interviews can access
Web surveys). In this article, the discussion respondents that are unreachable by other
of SAQs is almost exclusively about mail methods. In other cases, some respondents
questionnaires because they have been used may be difficult to reach because of various
by most of the studies published in IMS (see practical impediments (e.g. organizational
Table I). gatekeepers may prevent direct contact with
Personal interviews rely on verbal reports the manager in charge of strategic decisions).
and thus are less burdensome to respondents Personal interviews have been used in two
than SAQs that require written answers. The of the IMS surveys shown in Table I. These
presence of the interviewer provides were Orr's (1996) ``first project'' and the
flexibility and the opportunity to observe Burgess et al. (1997) study. Personal
respondents. However, the results can be interviews have been used to collect data in
influenced by biases resulting from the other types of studies published in IMS (e.g.
Driva et al., 2001; Kidd, 1995; Woodcock and
interviewer-respondent interaction
Chen, 2000).
(Bradburn et al., 1979; Kwong See and Ryan,
Telephone surveys are appropriate for
1999). Fortunately, response effects are
surveys where question order is
relatively small when the interviewers are
consequential because the interviewer (as in
well trained and well supervised (Bradburn,
personal interviews) controls the flow of the
1983; DeLamater, 1982). Such effects are more
questionnaire. Most importantly, telephone
likely ``when the respondent has not arrived
interviews ± conducted from centralized
at a firm position on the issue and when the
calling facilities ± allow for better
subject of the study is highly related to the
supervision of the interviewers and thus can
respondent or interviewer characteristics''
achieve higher data quality (Lavrakas, 1993).
(Sudman and Bradburn, 1974, p. 137).
The data can be collected in a relatively short
time and telephone interviews are ideally
Figure 1 suited for obtaining information for ongoing
Questionnaire contruction process or recently completed events. Generally,
telephone interviews are perceived as more
anonymous than personal surveys but less so
than postal questionnaires. Some
practitioners espouse the view that telephone
interviews should not exceed 10-15 minutes.
Although certain general population
telephone surveys can be substantially
longer, the number of partially completed
interviews increases after 45 minutes (Lake
and Harper, 1987). Rea and Parker (1997, p. 7)
have suggested that:
The cost of implementing a telephone survey
is considerably less than that of an in-person
survey and, under certain circumstances it
can be less than that of a mail-out survey.

With the current telephone technology that is

in widespread use, questions that require
visual stimuli (e.g. package design,
advertisements) or facilitating devices (e.g.
showcards) are better suited for other
methods.
[ 225 ]
Nicolaos E. Synodinos Telephone interviews were not used in any format may lead to re-evaluation and change
The ``art'' of questionnaire of the studies in Table I. However, two of of the selected mode of survey administration
construction: some important them used telephone calls to encourage
considerations for (see Figure 1). A questionnaire developed for
manufacturing studies survey participation. That is, calls were made a particular mode will require some degree of
Integrated Manufacturing to notify (Gieskes and ten Broeke, 2000) and to change to make it suitable for another.
Systems remind (Gieskes and ten Broeke, 2000; Orr, Although not discussed in Orr's (1996) article,
14/3 [2003] 221-237 1996) respondents. various adaptations to the questionnaire
For mail surveys ± as for other forms of were probably necessary as his ``first project''
SAQs ± ease of administration and was a personal interview, whereas the
professional appearance of the questionnaire ``second project'' was a replication using mail
are important considerations. There is an (see Table I).
increased need to have exceedingly clear and The potential of some new technologies for
unambiguous overall and question-specific gathering questionnaire data (e.g. Web
instructions because of their self- surveys) will probably change the relative
administered nature. Mail questionnaires use of the different modes for administering
are usually the least costly and most surveys. In organizational surveys, the
standardized alternative. Indeed, their sampling concerns ± that plague Web surveys
ability to reach inexpensively geographically of the general population ± can be resolved in
dispersed groups was probably a major some cases. However, the issue of
determinant of the fact that almost all the confidentiality of sensitive company
studies of Table I were postal surveys. Mail information has different dimensions than
questionnaires can be completed at the those encountered in general population
respondent's convenience and are generally surveys. Notwithstanding some important
perceived as more anonymous than the other differences between traditional and Web
methods. Complex questions can be questionnaires (e.g. novel ways of presenting
facilitated with graphical presentations but visual and auditory stimuli), the basic
SAQs may not be appropriate for some principles of good item construction are the
groups because they require a certain level of same. In fact, new technologies increase the
literacy. Thus, this is not a common concern importance of understanding the
for most IMS survey-based studies because implications of the differences between
their samples usually consist of literate questionnaire administration methods
individuals. Mail questionnaires are (Tourangeau et al., 2000).
inappropriate for studies of rapidly changing
opinions. In typical manufacturing
technology applications, the magnitude of Questionnaire construction
this concern is small compared to surveys of
political and social attitudes. Generally, mail Questionnaires should be designed to gather
questionnaires have lower response rates responses in an unbiased manner. The
than personal and telephone surveys. obtained answers should not reflect
However, with meticulous procedures, postal differences due to the instrument but should
surveys can achieve response rates indicate differences between respondents
comparable to those of the other (Fowler, 2002). This article discusses the
administration methods (Dillman, 1978, 2000; concerns related to questionnaire
Mangione, 1995). construction under four sub-categories:
Mail and other SAQs are more susceptible question wording, response choices, question
to question context effects (Schwarz et al., sequence, and other considerations (see
1991). Also, in such instruments it is Figure 1).
impossible to control the order with which
the questions will be answered and there are Question wording
no assurances that the intended person The wording of questions (including their
completed the questionnaire. The latter stem, response choices, and instructions) can
concern can be especially problematic for have pronounced effects on the results. Even
manufacturing studies that do not address a small difference in wording may produce
the questionnaire to a specified individual or substantial response effects. Thus, it is
send the questionnaire to one person in an appropriate to characterize good
organization with instructions to pass it to questionnaire construction as ``a highly
another. developed art form within the practice of
The questionnaire should be constructed to scientific inquiry'' (Rea and Parker, 1997,
fit the method of survey administration. A p. 27).
question format that is appropriate for one Questions should ask information that
method may not be for another. Indeed, the respondents can access readily (Tourangeau,
necessity to include questions of a particular 2000). Asking for detailed and/or not easily
[ 226 ]
Nicolaos E. Synodinos accessible information may antagonize some vis-aÁ-vis the burden imposed on respondents
The ``art'' of questionnaire persons. Also, questions that ask ``people to and their ability/willingness to make the
construction: some important predict their response to a future or differentiations requested by the questions.
considerations for
manufacturing studies hypothetical situation should be done with Unfortunately, it is not uncommon for
Integrated Manufacturing considerable caution ± particularly when researchers to be deeply engrossed in the
Systems respondents are likely to have limited minutia of their topic, leading them to ask
14/3 [2003] 221-237 ``two or more questions which sound alike to
experience on which to base their answers''
(Fowler, 1995, p. 80). Questions that request the respondents'' (Payne, 1951, p. 125).
confidential information (e.g. certain Items should be asked in frames of
company records) can lead to high item reference that are meaningful to the
nonresponse. Such items may be respondents (e.g. appropriate measurement
instrumental in the decision of some persons units and typical time frames for a particular
to forego participation in the survey, causing activity). For example, asking respondents to
higher unit nonresponse as well. Therefore, report by calendar year may lead to
researchers should limit their questions to difficulties and confusion if it differs from
those that the selected persons are able and their fiscal year. All assumptions should be
willing to answer. stated explicitly and the phrasing should be
The intrusiveness of certain questions is in specific rather than in indefinite terms.
probably one of the reasons for the very low Most importantly, the wording must be
return rates obtained by most of the surveys neutral. That is, researchers should avoid
in Table I. Although not usually reported in leading questions (i.e. suggesting a response
the articles of Table I, item nonresponse was to the respondent) or loaded questions (i.e.
probably high as well. A hint about this including emotionally charged descriptions).
concern is given by Newman and Sridharan In addition, researchers should be constantly
(1995, p. 38) who commented that: vigilant of the potential effects of social
Not all questions were answered by each firm, desirability on the respondents' answers
the chief reasons being that some of the (DeMaio, 1984).
questions were not applicable to every firm Each question should cover a single issue
that responded and in some cases the only. Items that inquire about two or more
required data were not available or
issues must be divided into separate
proprietary.
questions. In the second figure of their IMS
Also, they suggested that this tendency is article, Gardiner and Gregory (1996)
prevalent in manufacturing surveys. presented two questions of their audit
The respondent's understanding of a questionnaire. Both of these should have
question should correspond to the meaning been broken into more than one question as
intended by the researcher (Schwarz, 1999) respondents may be agreeing with one of the
bearing in mind that ``question issues of the stem but disagreeing with the
comprehension involves extensive other(s). Some of the questions of Gieskes
inferences about the speaker's intentions to and ten Broeke (2000) suffer from the same
determine the pragmatic meaning of the problem (e.g. statements 5 and 15 in Table AI
question'' (Schwarz et al., 1998a, p. 152). of their article).
Therefore, a question should be as clear and For studies repeated at different times,
precise as possible so that all respondents researchers should reevaluate each and
interpret it as intended and all understand every question in light of topical and
the same thing. Belson (1981) provided linguistic changes. Rapid changes in some
detailed examples of possible interpretations topics can lead to question obsolescence.
and misinterpretations of questions. His Researchers of manufacturing technology
book and examples can be very instructive to must be especially vigilant given the fast
all researchers employing surveys. pace of change taking place in that field.
Questions should use simple structure Wording problems increase exponentially
with familiar words and avoid any slang or in multi-culture and/or multi-country
jargon. Also, items should never resort to questionnaires (Behling and Law, 2000;
double negatives. Questions should be as Johnson et al., 1997). These problems can be
concise as possible to convey the intended especially noticeable when the languages and
meaning and respondents should be able to the underlying cultures differ substantially.
answer them with relatively minimal effort. Questions developed within a particular
As a general rule, questions should be easy to cultural context may be meaningless or
understand by persons with little formal offensive in another. Indeed, creating
education. The emphasis should be ``on directly comparable questionnaires in
communication rather than grammar and different languages is an extremely difficult
style'' (Wolfe, 1990, p. 95). Furthermore, task. Researchers should strive to construct
research objectives should be examined research instruments that are lexically
[ 227 ]
Nicolaos E. Synodinos equivalent, conceptually equivalent, generally easier for respondents to answer
The ``art'' of questionnaire equivalent in measurement, and equivalent and have fewer missing data than open-ended
construction: some important questions. Generally, closed-ended questions
considerations for in response (Bulmer and Warwick, 1983;
manufacturing studies Warwick and Lininger, 1975). Certain basic are difficult to construct but are relatively
Integrated Manufacturing universals may exist in some manufacturing- simpler to code and analyze. In most
Systems related questions. However, this does not instances, closed-ended items are the most
14/3 [2003] 221-237 appropriate response format and researchers
necessarily eliminate some biases when such
questions are answered by native speakers of should expend the necessary effort to create
the language vis-aÁ-vis those who are not. The them.
presumption that meaning will be equivalent Special attention should be placed on the
across all respondents because they ``know'' ± choices of closed-ended questions bearing in
at various levels of proficiency ± the language mind that ``identically worded questions may
is unwarranted. Indeed, the meaning of some acquire different meanings, depending on the
questions may differ among speakers of response alternatives by which they are
variations of the same language. accompanied'' (Schwarz, 1996, p. 75). The
choices may clarify the underlying meaning
Response choices because they provide ``guidelines'' regarding
Based on their response format, questions the expected answers (Schwarz, 1999) and are
can be classified as being either open-ended more likely ``to communicate the same frame
or closed-ended. In open-ended items, the of reference to all respondents'' (Converse
respondents phrase their own replies rather and Presser, 1986, p. 33). However, they
than trying to fit their answers into the reduce the likelihood of obtaining not-
provided choices. In closed-ended items, the specified answers (Schwarz, 1996).
respondent selects one (or more, if Closed-ended questions should provide
applicable) answer from the given response alternatives that are exhaustive and
alternatives. Hence, the former are referred mutually exclusive. That is, they should
to as ``free response'' and the latter as ``fixed cover all possible response options and these
response'' or ``fixed alternative'' items. A should not overlap. The choices should be
detailed discussion of open-ended vis-aÁ-vis comprehensive but researchers should not
closed-ended questions can be found in Foddy overwhelm the respondents with too many
(1993), who devoted an entire chapter of his alternatives. Whenever pertinent, it is
book on this issue. advisable to construct the questions and
In open-ended questions, the answer is their categories so that they can be compared
given from the respondent's frame of readily with secondary data.
reference rather than that of the writer of the In certain instances, the substantive
questionnaire. Free response items tend to be choices of a closed-ended question consist of
burdensome to respondents, especially when value ranges. Response choices that specify
the questionnaire is self-administered. Also, ranges rather than specific values can be
free response questions are more likely to very useful for sensitive items. In such
result in vague and useless responses questions, it is more likely that respondents
(Fowler, 2002; Hague, 1987). Consequently, will be willing to give a range rather than an
open-ended items should be used sparingly exact amount (Dillman, 1978).
because they require substantial respondent The number of response alternatives is
determined by various factors including the
effort. However, they can be useful in
questionnaire administration method and
exploratory research and in the early stages
type of question. For questions with
of questionnaire development. According to
categorical choices, Rea and Parker (1997)
Peterson (2000), open-ended questions are
suggested using less than ten and up to a
necessary in five situations. He noted that
maximum of 15 answers in SAQs; a
these occur in instances when the:
maximum of 20 choices in personal
. answers are not readily foreseeable;
interviews provided they are accompanied
. responses can be influenced by the
with a showcard; and up to six options in
presented choices;
telephone surveys. The order of the
. variables measured impose the use of
alternatives may influence the results
open-ended format as in the case of
(Schuman and Presser, 1981; Schwarz and
unaided recall;
Hippler, 1991; Sudman et al., 1996) and such
. potential for unanticipated events
effects interact with respondent
demands flexibility; and
characteristics such as age (KnaÈuper, 1999).
. initial responses necessitate asking
There are several possible ways of ordering
follow-up questions.
the choices of categorical variables. For
Fixed response items oversimplify the example, they can be presented randomly,
complexity of some opinions but are alphabetically, or in a sequence appropriate
[ 228 ]
Nicolaos E. Synodinos to fulfill particular research objectives. With order should be carefully planned in the
The ``art'' of questionnaire computer-administered questionnaires, it is construction of every survey''.
construction: some important Various devices can be used to scrutinize
considerations for feasible to vary the order of the alternatives
manufacturing studies for different respondents or build the relationships between the questions and
Integrated Manufacturing experiments within the survey to examine their sequence. For instance, the use of flow
Systems response sequence effects. charts is one such invaluable tool (Jabine,
14/3 [2003] 221-237 1985). Also, it is helpful to conceptualize the
For choices on a continuum, there are
various other factors that should be questionnaire as consisting of three parts:
considered. These include issues such as the The ``introduction'', the ``main body'', and the
degree of possible differentiation ``characteristics of the respondent and/or
(respondents can make) of the alternatives, organization.''
the use of verbal descriptions for all or for The introduction provides a brief
some of the categories, the length of such description of the study. In postal
descriptions, and the decision to use questionnaires, a portion of the information
numbers instead of verbal labels to describe is included (or duplicated) in the cover letter
the choices. Undoubtedly, the answers can be that should always accompany such surveys.
influenced by the psychological meaning of The description of the study should state
factors such as the numeric values chosen, clearly who the researcher represents, how/
the selected verbal labels, and the graphic why the respondent was selected, and the
layout of the scale (Schwarz et al., 1998b). importance of the respondent's answers to
In many instances, the researcher must the research. For opinion questions, it should
decide whether to include explicit be stressed to respondents that there are no
nonsubstantive (i.e. ``don't know,'' ``no right or wrong answers. In cases where the
opinion'') choices. Obviously, if respondents questionnaire is marked with identifying
have an opinion it is important to record it. information (e.g. identification number), the
However, questions that provide respondents must be informed of its presence
nonsubstantive options may discourage and purpose. Frequently, manufacturing
respondents from reporting their meaningful studies seek sensitive information and this
opinions (Krosnick, 1999; Weisberg et al., may lead to high item and unit nonresponse.
1996). As a guide, it has been suggested that: The latter can occur if respondents decide to
Questions about which nearly everyone has forego answering the questionnaire because
enough information to form some opinion . . . they find some portion(s) objectionable.
should be stated without a ``no opinion'' Thus, it is critical for the researcher to
option. Questions of a specific, narrow, or explain to respondents how the provided
detailed nature . . . should be prefaced by information and their privacy will be
screening questions to see whether the protected.
respondent has any information on the In postal surveys, an explicit deadline for
subject. (Scheaffer et al., 1990, p. 45). the return of the questionnaire must be
given. Also, respondents should be provided
Question sequence with a stamped addressed envelope along
The context within which a question is with appropriate instructions for returning
presented can influence the respondents' the questionnaire and for making inquiries.
answers (Bradburn, 1983; Schuman and Some of these instructions may be repeated
Presser, 1981; WaÈnke and Schwarz, 1997; at the end of the questionnaire.
Schwarz, 1996; Strack, 1992; Sudman and Various screening questions may be
Bradburn, 1974; Sudman et al., 1996; included at the end of the introduction.
Tourangeau and Rasinski, 1988). In fact, it is Screening questions can be used to select
possible to find context effects in SAQs that respondents that meet certain criteria and to
are caused by subsequent items (Schwarz ensure that those selected meet the necessary
and Hippler, 1995). Also, context effects are requirements assumed by the researcher.
more likely to occur when a single question That is, respondents that are not part of the
is used to measure a complex issue (Schuman sampling frame should be excluded. For
et al., 1981). Context effects can be especially instance, Gascoigne et al. (1997) were able to
problematic in studies investigating issues identify (though it is not clear from the
across time because such studies may narrative whether this was done via screener
incorrectly attribute their findings to questions) and exclude from their analyses
changes while they may simply reflect the respondents that were not part of their
different contextual factors within which the intended sample (see Table I).
questions have been presented (Schuman The main body of the questionnaire
et al., 1981). Although question sequence contains the topical questions. Proper
effects are not ubiquitous, McFarland's (1981, sequencing of the items facilitates
p. 213) findings suggested ``that the question questionnaire administration and minimizes
[ 229 ]
Nicolaos E. Synodinos confusion. The questions should be ordered In each of these studies, at least some of the
The ``art'' of questionnaire logically and in a manner non-threatening to questions were probably not pertinent to
construction: some important segments of respondents. Indeed, Bennett
considerations for respondents. Usually, similar questions
manufacturing studies should be grouped together and the within- et al. (1997) attributed their nonresponse
Integrated Manufacturing topic order should be from the general to the partly to the inapplicability of the
Systems specific. Other sequencing options can be questionnaire (in totality or in parts) to some
14/3 [2003] 221-237 of their potential respondents. Researchers
used to satisfy particular research objectives
(Labaw, 1980). should be in a position to determine those not
Questions pertaining to respondent and/or participating in the survey because of their
organizational characteristics usually ineligibility. Indeed, the issue of eligibility is
comprise the last section of the questionnaire a major consideration in the computation of
because they tend to ask the most sensitive various survey outcome measures (AAPOR,
information. Within this section, items 2000).
should be organized topically and from the The questionnaire should have a
least to the most sensitive. In a postal survey professional appearance and its formatting
at an organizational setting, it has been should make it easy for the respondent to
demonstrated that placing sensitive complete or for the interviewer to
administer. Instructions should be clear in
questions at the end of the questionnaire
terms of content and should be
resulted in higher return rates (Robertson
distinguishable stylistically from questions.
and Sundstrom, 1990).
Branching instructions must be user-friendly
The sequence of questions is not always
and as unambiguous as possible. The
indicated in the write-up of the studies in
questionnaire should be constructed to
Table I. Some of the studies (Burcher, 1992;
facilitate coding and data entry, bearing in
Gascoigne et al., 1997; Gilgeous, 1998)
mind the available resources and the
mentioned that the organizational/
intended analyses. Detailed suggestions for
respondent characteristics were placed at the
questionnaire design and layout can be found
beginning of the questionnaire. Depending
in various sources (Dillman, 2000; Salant and
on the sensitivity of these questions vis-aÁ-vis
Dillman, 1994; Sudman and Blair, 1998;
that of the others, this placement may have Sudman and Bradburn, 1982).
contributed to the high unit nonresponse. For postal questionnaires, the two
preeminent factors that influence response
Other considerations rates are incentives and number of contacts
The questionnaire must be tailored to its (Harvey, 1987). In addition, various other
audience. To be able to do this, researchers factors (e.g. type of postage, questionnaire
must be well versed with the survey topic appearance) should be considered. Increases
and realize the limits of their respondents' in response rates can be achieved by taking
knowledge. The questions should be into account potentially influencing factors
applicable to all respondents and pertinent in a comprehensively designed system
branching must be included for the cases (Dillman, 1978, 1991, 2000).
where particular questions apply only to Over the years, various survey incentives
some members of the sample (Warwick and (monetary and non-monetary) have been
Lininger, 1975). Questionnaires should have examined. The preponderance of the
appropriate branching and means by which research findings suggests that token
it can be distinguished if a respondent did not monetary incentives included (rather than
answer an item because of inapplicability or promised) with the questionnaire can
failure to reply. That is, the type of item increase response rates (Brennan et al., 1991;
nonresponse is important information and Dommeyer, 1988; Furse and Stewart, 1982;
must be distinguished in the questionnaire Gajraj et al., 1990; Gendall et al., 1998;
and reported in the findings. Hopkins and Gullickson, 1992; Hopkins and
The samples of some of the IMS studies Podolak, 1983; James and Bolstein, 1992;
shown in Table I consisted of heterogeneous Paolillo and Lorenzi, 1984).
respondents. For example, the selected Discussions of incentives and their
companies had between 50 and 3,000 feasibility were largely absent in the IMS
employees in the Gilgeous (1998) study; the studies of Table I. With the exception of the
number ranged from 10 to 10,000 employees Sohal et al. (1996) study, none reported the
in Orr's (1996) ``second project''; and use of incentives, despite the fact that in
Tummala et al. (2000, p. 372) noted that in many cases their questionnaires appear to
their study: have been very demanding. Some form of
The positions held by the people who incentives should have been used in
completed the questionnaire varied from top appreciation of the respondents' time and
management to supervisors and engineers. effort. Token monetary amounts or small
[ 230 ]
Nicolaos E. Synodinos gifts are the typical incentives used to attention. According to Paxson et al. (1995),
The ``art'' of questionnaire increase participation in surveys. Incentives these center around five factors:
construction: some important 1 the definition of what constitutes a
considerations for can take other forms such as a donation to a
manufacturing studies charity of the respondent's choice or a particular organization;
Integrated Manufacturing promise of a summary of the study's results. 2 the importance of selecting the
Systems Indeed, the Sohal et al. (1996) study used the appropriate person for the survey;
14/3 [2003] 221-237 3 the presence of various gatekeepers that
latter.
Response rates can be improved by may control the access to the respondent;
increasing the number of contacts with 4 the possible existence of organizational
respondents (Martin et al., 1989; Peterson policies regarding participation in
et al., 1989; Ruggles et al., 1984; Schlegelmilch surveys; and
and Diamantopoulos, 1991; Sutton and Zeits, 5 the generally higher level of effort needed
1992; Taylor and Lynn, 1998). This increase as organizational surveys frequently
can be accomplished with a combination of a require information that is not readily
preliminary notification and a series of accessible to respondents.
follow-up contacts. Schlegelmilch and The presence/absence of such characteristics
Diamantopoulos (1991) expressed skepticism must be addressed explicitly in all survey-
about the efficacy of the preliminary based studies published in IMS.
notification in industrial surveys. However,
Yammarino et al. (1991, p. 628) ± in a meta-
analysis of 115 articles that used mail surveys Questionnaire pretesting
± found that:
Follow-ups/repeated contacts seemed to have After the questionnaire is constructed it
a greater effect on institutional [which must be pretested (piloted). At times, the
comprised educational, industrial, terms ``pretesting'' and ``piloting'' are
healthcare, governmental, and ``other differentiated (Babbie, 1990), but here will be
institutional'' samples] than consumer used interchangeably. Pretests assist the
groups' response rates. researcher to refine the instrument and
fielding procedures. Pretesing should be
Respondent burden is usually high in many
viewed as an iterative process (see Figure 1)
manufacturing-management studies.
aimed to ``perfect'' the questionnaire for its
However, not enough attempts (in the form of
intended purpose. Based on the findings of
a preliminary notification and follow-ups)
this process, the questionnaire may have to
were made to increase response rates in most
be restructured and various items may have
of the studies of Table I. Indeed, some authors
to be re-written bearing in mind their inter-
appear to espouse the mistaken belief that
relationships within the new sequence.
multiple contacts should be avoided. This is Wolfe (1990, p. 102) stressed the importance
exemplified by the comment that: of pretesting by pointing out:
it is considered satisfactory rather than that more disasters in market research
disappointing that no efforts (such as happen through bad questionnaires than
telephoning or preliminary questionnaires) anything else, and most of these failures can
were made to boost the response to this be traced to inadequate piloting.
questionnaire survey (Huang and Mak, 1998,
p. 384). Other authors criticized the undesirable, but
all too frequent, state of affairs of conducting
Multiple contact attempts have been used in
pretests on small convenience samples
some of the IMS surveys shown in Table I. (Bolton, 1991) and in a nonsystematic and
For example, Gieskes and ten Broeke (2000) hurried fashion (Hunt et al., 1982). The
used a preliminary phone notification and a practice of pretesting questionnaires on
telephone reminder in their postal survey; in university students only (while they are
Orr's (1996) ``second project'' the respondents intended for a different population) has also
were mailed a follow-up letter and called in been strongly admonished (DeLamater, 1982;
order to increase the response rate; and Oppenheim, 1966).
Gupta et al. (1998) used two mailings of the Questionnaire pretesting must be an
questionnaire. indispensable phase of all studies, and it is
Pressley and Dunn (1985) stressed the puzzling why it is frequently ``handled so
importance of investigating empirically on casually, given its importance'' (Sykes and
organizational populations the impact of Morton-Williams, 1987, p. 192). Not all
various design features on response rates pretests are created equal and some can be
rather than relying solely on findings from more effective in detecting particular types of
general population studies. Indeed, there are problems than others (Diamantopoulos et al.,
some important characteristics of 1994; Hunt et al., 1982; Presser and Blair,
organizational surveys that merit special 1994). Although pretests may uncover some
[ 231 ]
Nicolaos E. Synodinos difficulties, their error detection rate tends to et al., 1999; Sudman et al., 1996; Tanur, 1992;
The ``art'' of questionnaire be relatively low (Hunt et al., 1982). Thus, it is Tourangeau et al., 2000) to the cognitive
construction: some important useful to conduct pretests with specialists in
considerations for aspects of survey methodology (CASM) is a
manufacturing studies questionnaire construction in addition to welcomed merging of academic disciplines.
Integrated Manufacturing those that must be done with potential Application of CASM can be traced to an
Systems respondents. important seminar on the topic under the
14/3 [2003] 221-237 Early pretests should be done in person auspices of the Committee on National
(Boyd et al., 1989; Churchill and Iacobucci, Statistics (Jabine et al., 1984). Cognitive
2002; Kinnear and Taylor, 1996; Weiers, 1988). approaches are grounded in psychology and
However, the final pretest must be done with aim for in-depth understanding of the
potential respondents and with the intended processes underlying responses in surveys.
questionnaire administration method. Application of approaches based on the
Whether pretests should use typical CASM can ``be effective for identifying
respondents or extreme cases is an issue that cognitive sources of response error that are
needs to be tested empirically (Reynolds et al., not uncovered by the usual question-and-
1993). Nevertheless, it has been proposed answer exchange between the interviewer
(Babbie, 1990) that pretests should aim to and the respondent that occurs in a typical
cover the whole range of intended field test of a questionnaire'' (Willis et al.,
respondents, including those that may be 1991, p. 263). Thus, cognitive methods have
considered atypical. For multi-language been touted as invaluable additions to
studies it is imperative to pretest the existing pretesting tools (Bolton, 1991;
instrument in the intended languages and in DeMaio and Rothgeb, 1996; Dippo et al., 1995;
the particular fieldwork environments. Such Schwarz, 1997; Schwarz et al., 1998a; Turner
pretests may uncover unique difficulties not et al., 1992; Willis et al., 1991).
ascertainable in the original language of the Uses of the CASM can assist researchers in
questionnaire (McKay et al., 1996). their quest for ``solutions to practical
Some authors (Aaker et al., 2001; Boyd et al., questions in questionnaire design, and `rules'
1989; McDaniel and Gates, 2001) suggested of question formulation'' (O'Muircheartaigh,
that only expert interviewers should conduct 1997, p. 18). Undoubtedly, such pretesting can
the pretests. Others (Hunt et al., 1982; be costly because of the time and labor
Malhotra, 1999; Nelson, 1985) stressed the involved (Bolton, 1991). However, the
need to run the gamut of interviewer increased effort and costs should not
experience. Be that as it may, it is advisable dissuade researchers because pretests can be
that researchers themselves conduct some extremely beneficial in the research process
pretest interviews (Malhotra, 1999; Sheatsley, (Fowler, 1995).
1983; Tull and Hawkins, 1993).
The use of pretests is noted explicitly in
several of the articles shown in Table I Conclusion
(Batley, 1993; Bennett et al., 1997; Gieskes and
ten Broeke, 2000; Gupta et al., 1998; Newman Gathering data via surveys is ubiquitous in
and Sridharan, 1995; Orr, 1999; Sohal et al., many fields and used frequently in empirical
1996; Tummala et al., 2000). The most studies of manufacturing strategy. A
extensive appears to be the pretest conducted fundamental part of this research process is
by Newman and Sridharan (1995). The the development of a quality questionnaire.
pretests of other studies in Table I varied in Contrary to what some may believe, just
their extensiveness and purpose. writing a set of questions does not result in
Presumably, all studies found their pretests an appropriate research instrument because
useful. For example, Batley's (1993) pretest ± ``a questionnaire is not simply a series of
which consisted of personal interviews with questions, nor is a question merely a series of
potential respondents ± assisted in the words'' (Labaw, 1980, p. 1).
construction of the postal questionnaire and This article addressed important issues
refinement of the fielding procedures. The surrounding the collection of data via
pretest led him to the decision to exclude questionnaires. It provided general
very small firms for which the instrument suggestions for the construction of high
would have been inappropriate because the quality questionnaires and a comprehensive
management requirements pertaining to bibliography that should be consulted by
quality in such companies ``were mostly those wishing to explore the matter in
unwritten and often difficult to identify'' greater detail. In addition, the article
(Batley, 1993, p. 5). examined and critiqued some of the survey-
The recent attention (Forsyth and Lessler, based studies published in IMS.
1991; Hippler et al., 1987; Jobe and Mingay, Researchers of manufacturing strategy will
1991; Schwarz and Sudman, 1996; Schwarz create increasingly refined questionnaires as
[ 232 ]
Nicolaos E. Synodinos they gain sophistication in survey research and Outcome Rates for Surveys, AAPOR, Ann
The ``art'' of questionnaire methodology. Well-constructed Arbor, MI.
construction: some important Babbie, E. (1990), Survey Research Methods, 2nd
considerations for questionnaires will contribute to a deeper
manufacturing studies understanding of the intricacies of ed., Wadsworth, Belmont, CA.
manufacturing strategy. Also, Batley, T.W. (1993), ``Managing product quality in
Integrated Manufacturing
Systems New Zealand firms'', Integrated
questionnaires that are not burdensome to
14/3 [2003] 221-237 Manufacturing Systems, Vol. 4 No. 4, pp. 4-9.
respondents will have probably lower item
Behling, O. and Law, K.S. (2000), Translating
and unit nonresponse.
Questionnaires and Other Research
Manufacturing-management researchers Instruments: Problems and Solutions, Sage,
must address explicitly the implications of Thousand Oaks, CA.
the high nonresponse rate of their studies Belson, W.A. (1981), The Design and
and take appropriate actions to ensure those Understanding of Survey Questions, Gower,
responding represent the pertinent Aldershot.
population accurately. At the very least, Bennett, D., Hongyu, Z., Vaidya, K. and
some of the nonrespondents must be followed Xing Ming, W. (1997), ``Transferring
and their responses compared with those of manufacturing technology to China: supplier
the respondents. Also, researchers should perceptions and acquirer expectations'',
include, in their articles, details of their Integrated Manufacturing Systems, Vol. 8
questionnaires (such as the questions' exact No. 5, pp. 283-91.
Birn, R., Hague, P. and Vangelder, P. (1990),
phrasing) and make them readily available.
``Introduction'', in Birn, R., Hague, P. and
Furthermore, published expositions should
Vangelder, P. (Eds), A Handbook of Market
report their findings in sufficient detail,
Research Techniques, Kogan Page, London,
include lengthy discussions of the limitations pp. 17-23.
of their surveys, and offer recommendations Bolton, R.N. (1991), ``An exploratory investigation
for improvements to the questionnaire (e.g. of questionnaire pretesting with verbal
items that produced high item nonresponse, protocol analysis'', Advances in Consumer
conflicting answers) and to the survey Research, Vol. 18, pp. 558-65.
process. Boyd, H.W. Jr, Westfall, R. and Stasch, S.F. (1989),
As Payne (1951) envisioned, constructing a Marketing Research: Text and Cases, 7th ed.,
good questionnaire is and will remain an Irwin, Homewood, IL.
``art'' that considers simultaneously a Bradburn, N.M. (1983), ``Response effects'', in
multitude of factors following various Rossi, P.H., Wright, J.D. and Anderson, A.B.
flexible guidelines. This process requires (Eds), Handbook of Survey Research,
Academic Press, Orlando, FL, pp. 289-328.
thorough knowledge of the topical area, great
Bradburn, N.M. and Sudman, S. (1988), Polls and
attention to details, as seemingly slight
Surveys: Understanding What they Tell us,
changes in wording or structure can
Jossey-Bass, San Francisco, CA.
influence the results, and continuous Bradburn, N.M. et al. (1979), Improving Interview
revisions in light of pretests. Cognitive Method and Questionnaire Design: Response
psychology can offer valuable insights into Effects to Threatening Questions in Survey
the development of better survey Research, Jossey-Bass, San Francisco, CA.
instruments. However, it is only ``recently Brennan, M., Hoek, J. and Astridge, C. (1991),
that the cognitive and communicative ``The effects of monetary incentives on the
processes underlying question answering in response rate and cost-effectiveness of a mail
surveys have received sustained theoretical survey'', Journal of the Market Research
attention from psychologists'' (Schwarz et al., Society, Vol. 33 No. 3, pp. 229-41.
1998a, p. 150). Indeed, some researchers seem Bulmer, M. and Warwick, D.P. (1983), ``Data
to downplay the data gathering stage and collection'', in Bulmer, M. and Warwick, D.P.
(Eds), Social Research in Developing
simply concentrate on elaborate statistical
Countries: Surveys and Censuses in the Third
procedures. It can be stated unequivocally
World, Wiley, Chichester, pp. 145-60.
that no amount of sophistication in statistical Burcher, P. (1992), ``Master production scheduling
analyses can correct fundamental and capacity planning: the link?'', Integrated
shortcomings stemming from a poorly Manufacturing Systems, Vol. 3 No. 4, pp. 16-22.
constructed questionnaire. Burcher, P.G. and Lee, G.L. (2000), ``Competitive
strategies and AMT investment decisions'',
References Integrated Manufacturing Systems, Vol. 11
Aaker, D.A., Kumar, V. and Day, G.S. (2001), No. 5, pp. 340-7.
Marketing Research, 7th ed., Wiley, New Burgess, T.F., Gules, H.K. and Tekin, M. (1997),
York, NY. ``Supply-chain collaboration and success in
American Association for Public Opinion technology implementation'', Integrated
Research (AAPOR) (2000), Standard Manufacturing Systems, Vol. 8 No. 5,
Definitions: Final Dispositions of Case Codes pp. 323-32.

[ 233 ]
Nicolaos E. Synodinos Churchill, G.A. Jr and Iacobucci, D. (2002), Biemer, P.B., Groves, R.M., Lyberg, L.E.,
The ``art'' of questionnaire Marketing Research: Methodological Mathiowetz, N.A. and Sudman, S. (Eds),
construction: some important Foundations, 8th ed., Harcourt, Fort Worth, Measurement Errors in Surveys, Wiley, New
considerations for
manufacturing studies TX. York, NY, pp. 393-418.
Converse, J.M. and Presser, S. (1986), Survey Fowler, F.J. Jr (1995), Improving Survey
Integrated Manufacturing
Systems Questions: Handcrafting the Standardized Questions: Design and Evaluation, Sage,
14/3 [2003] 221-237 Questionnaire, Sage, Beverly Hills, CA. Thousand Oaks, CA.
DeLamater, J. (1982), ``Response-effects of Fowler, F.J. Jr (2002), Survey Research Methods,
question content'', in Dijkstra, W. and van der 3rd ed., Sage, Thousand Oaks, CA.
Zouwen, J. (Eds), Response Behaviour in the Furse, D.H. and Stewart, D.W. (1992), ``Monetary
Survey-Interview, Academic Press, London, incentives versus promised contribution to
pp. 13-48. charity: new evidence on mail survey
DeMaio, T.J. (1984), ``Social desirability and response'', Journal of Marketing Research,
survey measurement: a review'', in Vol. 19 No. 3, pp. 375-80.
Turner, C.F. and Martin, E. (Eds), Surveying Gajraj, A.M., Faria, A.J. and Dickinson, J.R.
Subjective Phenomena, Vol. 2, Russell Sage, (1990), ``A comparison of the effect of
New York, NY, pp. 257-82. promised and provided lotteries, monetary
DeMaio, T.J. and Rothgeb, J.M. (1996), ``Cognitive and gift incentives on mail survey response
interviewing techniques: in the lab and in the rate, speed and cost'', Journal of the Market
field'', in Schwarz, N. and Sudman S. (Eds), Research Society, Vol. 32 No. 1, pp. 141-62.
Answering Questions: Methodology for Gardiner, G.S. and Gregory, M.J. (1996), ``An
Determining Cognitive and Communicative audit-based approach to the analysis,
Processes in Survey Research, Jossey-Bass, redesign and continuing assessment of a new
San Francisco, CA, pp. 177-95. product introduction system'', Integrated
Diamantopoulos, A., Reynolds, N. and
Manufacturing Systems, Vol. 7 No. 2, pp. 52-9.
Schlegelmilch, B. (1994), ``Pretesting in Gascoigne, J.D., Zhang, B.L. and Weston, R.H.
questionnaire design: the impact of
(1997), ``A report on the UK cell control
respondent characteristics on error
marketplace'', Integrated Manufacturing
detection'', Journal of the Market Research
Systems, Vol. 8 No. 3, pp. 181-4.
Society, Vol. 36 No. 4, pp. 295-313.
Gendall, P., Hoek, J. and Brennan, M. (1998), ``The
Dillman, D.A. (1978), Mail and Telephone Surveys:
tea bag experiment: more evidence on
The Total Design Method, Wiley, New York,
incentives in mail surveys'', Journal of the
NY.
Market Research Society, Vol. 40 No. 4,
Dillman, D.A. (1991), ``The design and
pp. 347-51.
administration of mail surveys'', Annual
Gieskes, J.F.B. and ten Broeke, A.M. (2000),
Review of Sociology, Vol. 17, pp. 225-49.
``Infrastructure under construction:
Dillman, D.A. (2000), Mail and Internet Surveys:
continuous improvement and learning in
The Tailored Design Method, 2nd ed., Wiley,
projects'', Integrated Manufacturing Systems,
New York, NY.
Dippo, C.S., Chun, Y.I. and Sander, J. (1995), Vol. 11 No. 3, pp. 188-98.
Gilgeous, V. (1998), ``Manufacturing managers:
``Designing the data collection process'', in
Cox, B.G., Binder, D.A., Chinnappa, B.N., their quality of working life'', Integrated
Christianson, A., Colledge, M.J. and Kott, P.S. Manufacturing Systems, Vol. 9 No 3, pp. 173-81.
(Eds), Business Survey Methods, Wiley, New Gupta, A., Prinzinger, J. and Messerschmidt, D.C.
York, NY, pp. 283-301. (1998), ``Role of organizational commitment in
Dommeyer, C.J. (1988), ``How form of the advanced manufacturing technology and
monetary incentive affects mail survey performance relationship'', Integrated
response'', Journal of the Market Research Manufacturing Systems, Vol. 9 No. 5, pp. 272-8.
Society, Vol. 30 No. 3, pp. 379-85. Hague, P.N. (1987), The Industrial Market
Driva, H., Pawar, K.S. and Menon, U. (2001), Research Handbook, 2nd ed., Kogan Page,
``Performance evaluation of new product London.
development from a company perspective'', Harvey, L. (1987), ``Factors affecting response
Integrated Manufacturing Systems, Vol. 12 rates to mailed questionnaires: a
No. 5, pp. 368-78. comprehensive literature review'', Journal of
Flynn, B.B., Sakakibara, S., Schroeder, R.G., the Market Research Society, Vol. 29 No. 3,
Bates, K.A. and Flynn E.J. (1990), ``Empirical pp. 341-53.
research methods in operations Hippler, H.J., Schwarz, N. and Sudman, S. (Eds)
management'', Journal of Operations (1987), Social Information Processing and
Management, Vol. 9 No. 2, pp. 250-84. Survey Methodology, Springer-Verlag, New
Foddy, W. (1993), Constructing Questions for York, NY.
Interviews and Questionnaires: Theory and Hopkins, K.D. and Gullickson, A.R. (1992),
Practice in Social Research, Cambridge ``Response rates in survey research: a meta-
University Press, Cambridge. analysis of the effects of monetary gratuities'',
Forsyth, B.H. and Lessler, J.T. (1991), ``Cognitive Journal of Experimental Education, Vol. 61
laboratory methods: a taxonomy'', in No. 1, pp. 52-62.

[ 234 ]
Nicolaos E. Synodinos Hopkins, K.D. and Podolak, J. (1983), ``Class-of- Krosnick, J.A. (1999), ``Maximizing questionnaire
The ``art'' of questionnaire mail and the effects of monetary gratuity on quality'', in Robinson, J.P., Shaver, P.R. and
construction: some important the response rates of mailed questionnaires'', Wrightsman, L.S. (Eds), Measures of Political
considerations for
manufacturing studies Journal of Experimental Education, Vol. 51 Attitudes, Vol. 2, Academic Press, San Diego,
No. 4, pp. 169-70. CA, pp. 37-57.
Integrated Manufacturing
Systems Huang, G.Q. and Mak, K.L. (1998), ``A survey Kwong See, S.T. and Ryan, E.B. (1999),
14/3 [2003] 221-237 report on design for manufacture in the UK ``Intergenerational communication: The
furniture manufacturing industry'', survey interview as a social exchange'', in
Integrated Manufacturing Systems, Vol. 9 Schwarz, N., Park, D., KnaÈuper, B. and
No. 6, pp. 383-7. Sudman, S. (Eds), Cognition, Aging, and Self-
Hunt, S.D., Sparkman, R.D. Jr and Wilcox, J.B. Reports, Psychology Press, Philadelphia, PA,
(1982), ``The pretest in survey research: issues pp. 245-62.
and preliminary findings'', Journal of Labaw, P.J. (1980), Advanced Questionnaire
Marketing Research, Vol. 19 No. 2, pp. 269-73. Design, Abt Books, Cambridge, MA.
Jabine, T.B. (1985), ``Flow charts: a tool for Lake, C.C. and Harper, P.C. (1987), Public Opinion
developing and understanding survey Polling: A Handbook for Public Interest and
questionnaires'', Journal of Official Statistics, Citizen Advocacy Groups, Island Press,
Vol. 1 No. 2, pp. 189-207. Washington, DC.
Jabine, T.B., Straf, M.L., Tanur, J.M. and Lavrakas, P.J. (1993), Telephone Survey Methods:
Tourangeau, R. (Eds) (1984), Cognitive Aspects Sampling, Selection, and Supervision, 2nd ed.,
of Survey Methodology: Building a Bridge Sage, Newbury Park, CA.
Between Disciplines, National Academy Press, Lockhart, D.C. and Russo J.R. (1994), ``Mail and
Washington, DC. telephone surveys in marketing research: a
James, J.M. and Bolstein, R. (1992), ``Large perspective from the field'', in Bagozzi, R.P.
monetary incentives and their effects on mail (Ed.), Principles of Marketing Research,
Blackwell, Cambridge, MA, pp. 116-61.
survey response rates'', Public Opinion
McDaniel, C. Jr and Gates, R. (2001), Marketing
Quarterly, Vol. 56 No. 4, pp. 442-53.
Research Essentials, 3rd ed., South-Western,
Jobe, J.B. and Mingay, D.J. (1991), ``Cognition and
Cincinnati, OH.
survey measurement: history and overview'',
McFarland, S.G. (1981), ``Effects of question order
Applied Cognitive Psychology, Vol. 5 No. 3,
on survey responses'', Public Opinion
pp. 175-92.
Quarterly, Vol. 45 No. 2, pp. 208-15.
Johnson, T., O'Rourke, D., Chavez, N., Sudman, S.,
McKay, R.B., Breslow, M.J., Sangster, R.L.,
Warnecke, R., Lacey, L. and Horm, J. (1997),
Gabbard, S.M., Reynolds, R.W.,
``Social cognition and responses to survey
Nakamoto, J.M. and Tarnai, J. (1996),
questions among culturally diverse
``Translating survey questionnaires: lessons
populations'', in Lyberg, L., Biemer, P.,
learned'', in Braverman, M.T. and Slater, J.K.
Collins, M., De Leeuw, E., Dippo, C.,
(Eds), Advances in Survey Research (New
Schwarz, N. and Trewin, D. (Eds), Survey
Directions for Evaluation, No. 70), Jossey-
Measurement and Process Quality, Wiley, New
Bass, San Francisco, CA, pp. 93-104.
York, NY, pp. 87-113. Malhotra, N.K. (1999), Marketing Research: An
Kidd, J. B. (1995), ``Subcontractors, JIT and
Applied Orientation, 3rd ed., Prentice-Hall,
kanbans: a brief review of spring Upper Saddle River, NJ.
manufacturing in Japan and South Korea'', Mangione, T.W. (1995), Mail Surveys: Improving
Integrated Manufacturing Systems, Vol. 6 the Quality, Sage, Thousand Oaks, CA.
No. 6, pp. 15-22. Martin, W.S., Duncan, W.J., Powers, T.L. and
Kinnear, T.C. and Taylor, J.R. (1996), Marketing Sawyer, J.C. (1989), ``Costs and benefits of
Research: An Applied Approach, 5th ed., selected response inducement techniques in
McGraw-Hill, New York, NY. mail survey research'', Journal of Business
KnaÈuper, B. (1999), ``Age differences in question Research, Vol. 19 No. 1, pp. 67-79.
and response order effects'', in Schwarz, N., Minor, E.D. III, Hensley, R.L. and Wood, D.R. Jr
Park, D., KnaÈuper, B. and Sudman, S. (Eds), (1994), ``A review of empirical manufacturing
Cognition, Aging, and Self-Reports, strategy studies'', International Journal of
Psychology Press, Philadelphia, PA, pp. 341-63. Operations & Production Management, Vol. 14
Kraut, A.I. (1996), ``Introduction: an overview of No. 1, pp. 5-25.
organizational surveys'', in Kraut, A.I. (Ed.), Nelson, D.D. (1985), ``Informal testing as a means
Organizational Surveys: Tools for Assessment of questionnaire development'', Journal of
and Change, Jossey-Bass, San Francisco, CA, Official Statistics, Vol. 1 No. 2, pp. 179-88.
pp. 1-14. Newman, W.R. and Sridharan, V. (1995), ``Linking
Kraut, A.I. and Saari, L.M. (1999), ``Organization manufacturing planning and control to the
surveys: coming of age for a new era'', in manufacturing environment'', Integrated
Kraut, A.I. and Korman, A.K. (Eds), Evolving Manufacturing Systems, Vol. 6 No. 4, pp. 36-42.
Practices in Human Resource Management: O'Muircheartaigh, C. (1997), ``Measurement error
Responses to a Changing World of Work, in surveys: a historical perspective'', in
Jossey-Bass, San Francisco, CA, pp. 302-27. Lyberg, L., Biemer, P., Collins, M.,

[ 235 ]
Nicolaos E. Synodinos De Leeuw, E., Dippo, C., Schwarz, N. and attitude questionnaire'', Journal of Applied
The ``art'' of questionnaire Trewin, D. (Eds), Survey Measurement and Psychology, Vol. 75 No. 3, pp. 354-7.
construction: some important Process Quality, Wiley, New York, NY, Ruggles, D.R., Dea, J.Y., Kwok, F.K. and
considerations for
manufacturing studies pp. 1-25. Carman, C.A. (1984), ``Evaluation of the
Oppenheim, A.N. (1966), Questionnaire Design effectiveness of data collection procedures for
Integrated Manufacturing
Systems and Attitude Measurement, Basic Books, New the 1982 census of agriculture'', in American
14/3 [2003] 221-237 York, NY. Statistical Association 1984 Proceedings of the
Orr, S.C. (1996), ``A longitudinal survey of robot Section on Survey Research Methods,
usage in Australia'', Integrated American Statistical Association,
Manufacturing Systems, Vol. 7 No. 5, pp. 33-46. Washington, DC, pp. 588-93.
Orr, S. (1999), ``The role of technology in Salant, P. and Dillman, D.A. (1994), How to
manufacturing strategy: experiences from the Conduct Your Own Survey, Wiley, New York,
Australian wine industry'', Integrated NY.
Manufacturing Systems, Vol. 10 No. 1, pp. 45-55. Scheaffer, R.L., Mendenhall, W. and Ott, L. (1990),
Paolillo, J.G.P. and Lorenzi, P. (1984), ``Monetary Elementary Survey Sampling, 4th ed., PWS-
incentives and mail questionnaire response Kent, Boston, MA.
rates'', Journal of Advertising, Vol. 13 No. 1, Schlegelmilch, B.B. and Diamantopoulos, A.
pp. 46-8. (1991), ``Prenotification and mail survey
Paxson, M.C., Dillman, D.A. and Tarnai, J. (1995), response rates: a quantitative integration of
``Improving response to business mail the literature'', Journal of the Market
surveys'', in Cox, B.G., Binder, D.A., Research Society, Vol. 33 No. 3, pp. 243-55.
Chinnappa, B.N., Christianson, A., Schuman, H. and Presser, S. (1981), Questions and
Colledge, M.J. and Kott, P.S. (Eds), Business Answers in Attitude Surveys Experiments on
Survey Methods, Wiley, New York, NY, Question Form, Wording, and Context,
Academic Press, New York, NY.
pp. 303-16.
Schuman, H., Presser, S. and Ludwig, J. (1981),
Payne, S.L. (1951), The Art of Asking Questions,
``Context effects on survey responses to
Princeton University Press, Princeton, NJ.
questions about abortion'', Public Opinion
Peterson, R.A. (2000), Constructing Effective
Quarterly, Vol. 45 No. 2, pp. 216-23.
Questionnaires, Sage, Thousand Oaks, CA.
Schwarz, N. (1996), ``Survey research: collecting
Peterson, R.A., Albaum, G. and Kerin, R.A. (1989),
data by asking questions'', in Semin, G.R. and
``A note on alternative contact strategies in
Fiedler, K. (Eds), Applied Social Psychology,
mail surveys'', Journal of the Market Research
Sage, London, pp. 65-90.
Society, Vol. 31 No. 3, pp. 409-18.
Schwarz, N. (1997), ``Questionnaire design: the
Presser, S. and Blair, J. (1994), ``Survey pretesting:
rocky road from concepts to answers'', in
do different methods produce different
Lyberg, L., Biemer, P., Collins, M., De
results?'', Sociological Methodology, Vol. 24,
Leeuw, E., Dippo, C., Schwarz, N. and
pp. 73-104.
Trewin, D. (Eds), Survey Measurement and
Pressley, M.M. and Dunn, M.G. (1985), ``A factor-
Process Quality, Wiley, New York, NY,
interactive experimental investigation of
pp. 29-45.
inducing response to questionnaires mailed
Schwarz, N. (1999), ``Self-reports: How the
to commercial populations'', in Lusch, R.F.,
questions shape the answers'', American
Ford, G.T., Frazier, G.L., Howell, R.D., Psychologist, Vol. 54 No. 2, pp. 93-105.
Ingene, C.A., Reilly, M. and Stampfl, R.W. Schwarz, N. and Hippler, H.J. (1991), ``Response
(Eds), 1985 AMA Educators' Proceedings alternatives: the impact of their choice and
(Series No. 51), American Marketing presentation order'', in Biemer, P.B.,
Association, Chicago, IL, pp. 356-61. Groves, R.M., Lyberg, L.E., Mathiowetz, N.A.
Rea, L.M. and Parker, R.A. (1997), Designing and and Sudman, S. (Eds), Measurement Errors in
Conducting Survey Research: A Surveys, Wiley, New York, NY, pp. 41-56.
Comprehensive Guide, 2nd ed., Jossey-Bass, Schwarz, N. and Hippler, H.J. (1995), ``Subsequent
San Francisco, CA. questions may influence answers to
Reynolds, N., Diamantopoulos, A. and preceding questions in mail surveys'', Public
Schlegelmilch, B. (1993), ``Pretesting in Opinion Quarterly, Vol. 59 No. 1, pp. 93-7.
questionnaire design: a review of the Schwarz, N. and Sudman S. (Eds) (1996),
literature and suggestions for further Answering Questions: Methodology for
research'', Journal of the Market Research Determining Cognitive and Communicative
Society, Vol. 35, pp. 171-82. Processes in Survey Research, Jossey-Bass,
Riedel, J.C.K.H. and Pawar, K.S. (1997), ``The San Francisco, CA.
consideration of production aspects during Schwarz, N., Grayson, C.E. and KnaÈuper, B.
product design stages'', Integrated (1998b), ``Formal features of rating scales and
Manufacturing Systems, Vol. 8 No. 4, the interpretation of question meaning'',
pp. 208-14. International Journal of Public Opinion
Robertson, M.T. and Sundstrom, E. (1990), Research, Vol. 10 No. 2, pp. 177-83.
``Questionnaire design, return rates, and Schwarz, N., Groves, R.M. and Schuman, H.
response favorableness in an employee (1998a), ``Survey methods'', in Gilbert D.T.,

[ 236 ]
Nicolaos E. Synodinos Fiske, S.T. and Lindzey, G. (Eds), The Bachrach, C.A., Jobe, J.B., Kurtzman, H.S.
The ``art'' of questionnaire Handbook of Social Psychology, 4th ed., (Vol. and Cain, V.S. (Eds), The Science of Self-
construction: some important AÃ 1), McGraw-Hill, Boston, MA, pp. 143-79. Report: Implications for Research and
considerations for
manufacturing studies Schwarz, N., Park, D.C., KnaÈuper, B. and Practice, LEA, Mahwah, NJ, pp. 29-47.
Sudman, S. (Eds) (1999), Cognition, Aging, and Tourangeau, R. and Rasinski, K.A. (1988),
Integrated Manufacturing
Systems Self-Reports, Psychology Press, Philadelphia. ``Cognitive processes underlying context
14/3 [2003] 221-237 Schwarz, N., Strack, F., Hippler, H.J. and effects in attitude measurement'',
Bishop, G. (1991), ``The impact of Psychological Bulletin, Vol. 103 No. 3,
administration mode on response effects in pp. 299-314.
survey measurement'', Applied Cognitive Tourangeau, R., Rips, L.J. and Rasinski, K. (2000),
Psychology, Vol. 5 No. 3, pp. 193-212. The Psychology of Survey Response,
Sheatsley, P.B. (1983), ``Questionnaire Cambridge University Press, Cambridge.
construction and item writing'', in Rossi, Tull, D.S. and Hawkins, D I. (1993), Marketing
P.H., Wright, J.D. and Anderson, A.B. (Eds), Research: Measurement & Method, 6th ed.,
Handbook of Survey Research, Academic Macmillan, New York, NY.
Press, Orlando, FL, pp. 195-230. Tummala, V.M.R., Lee, H.Y.H. and Yam, R.C.M.
Sohal, A.S., Maguire, W.A.A. and Putterill, M.S. (2000), ``Strategic alliances of China and Hong
(1996), ``AMT investments in New Zealand: Kong in manufacturing and their impact on
purpose, pattern and outcomes'', Integrated global competitiveness of Hong Kong
Manufacturing Systems, Vol. 7 No. 2, pp. 27-36. manufacturing industries'', Integrated
Strack, F. (1992), ```Order effects' in survey Manufacturing Systems, Vol. 11 No. 6,
research: activation and information pp. 370-84.
functions of preceding questions'', in Turner, C.F., Lessler, J.T. and Gfroerer, J.C.
Schwarz, N. and Sudman, S. (Eds), Context (1992), ``Future directions for research and
Effects in Social and Psychological Research, practice'', in Turner, C.F., Lessler, J.T. and
Springer-Verlag, New York, NY, pp. 23-34. Gfroerer, J.C. (Eds), Survey Measurement of
Sudman, S. and Blair, E. (1998), Marketing Drug Use: Methodological Studies, National
Research: A Problem-Solving Approach, Institute on Drug Abuse, Rockville, MD,
McGraw-Hill, Boston, MA. pp. 299-306.
Sudman, S. and Bradburn, N.M. (1974), Response WaÈnke, M. and Schwarz, N. (1997), ``Reducing
Effects in Surveys: A Review and Synthesis, question order effects: the operation of buffer
Aldine, Chicago, IL. items'', in Lyberg, L., Biemer, P., Collins, M.,
Sudman, S. and Bradburn, N.M. (1982), Asking De Leeuw, E., Dippo, C., Schwarz, N. and
Questions: A Practical Guide to Questionnaire Trewin, D. (Eds), Survey Measurement and
Design, Jossey-Bass, San Francisco, CA. Process Quality, Wiley, New York, NY,
Sudman, S., Bradburn, N.M. and Schwarz, N. pp. 115-40.
(1996), Thinking about Answers: The Warwick, D.P. and Lininger, C.A. (1975), The
Application of Cognitive Processes to Survey Sample Survey: Theory and Practice, McGraw-
Methodology, Jossey-Bass, San Francisco, CA. Hill, New York, NY.
Sutton, R.J. and Zeits, L.L. (1992), ``Multiple prior Weiers, R.M. (1988), Marketing Research, 2nd ed.,
notifications, personalization, and reminder Prentice-Hall, Englewood Cliffs, NJ.
surveys'', Marketing Research, Vol. 4 No. 4, Weisberg, H.F., Krosnick, J.A. and Bowen, B.D.
pp. 14-21. (1996), An Introduction to Survey Research,
Swink, M. and Way, M.H. (1995), ``Manufacturing Polling, and Data Analysis, 3rd ed., Sage,
strategy: propositions, current research, Thousand Oaks, CA.
renewed directions'', International Journal of Willis, G.B., Royston, P. and Bercini, D. (1991),
Operations and Production Management, ``The use of verbal report methods in the
Vol. 15 No. 7, pp. 4-26. development and testing of survey
Sykes, W. and Morton-Williams, J. (1987), questions'', Applied Cognitive Psychology,
``Evaluating survey questions'', Journal of Vol. 5 No. 3, pp. 251-67.
This article is an expanded Official Statistics, Vol. 3 No. 2, pp. 191-207. Wolfe, A. ( 1990), ``Questionnaire design'', in
and updated version of a Tanur, J.M. (Ed.) (1992), Questions about Birn, R., Hague, P. and Vangelder, P. (Eds), A
paper presented to the Questions: Inquiries into the Cognitive Bases of Handbook of Market Research Techniques,
International Symposium on
Surveys, Russell Sage, New York, NY. Kogan Page, London, pp. 89-103.
Manufacturing Strategy
(ISMS '98) at Waseda Taylor, S. and Lynn P. (1998), ``The effect of a Woodcock, D. and Chen, C.Y. (2000), ``Skills and
University, Tokyo, Japan preliminary notification letter on response to knowledge of senior Taiwanese manufacturing
November 18-20, 1998. The a postal survey of young people'', Journal of managers'', Integrated Manufacturing Systems,
author is grateful to Sophia the Market Research Society, Vol. 40 No. 2, Vol. 11 No. 6, pp. 393-404.
University (Tokyo, Japan) pp. 165-73. Yammarino, F.J., Skinner, S.J. and Childers, T.L.
where portions of this
Tourangeau, R. (2000), ``Remembering what (1991), ``Understanding mail survey response
manuscript were completed
while being there as a happened: memory errors and survey behavior: a meta-analysis'', Public Opinion
``visiting researcher.'' reports'', in Stone, A.A., Turkkan, J.S., Quarterly, Vol. 55 No. 4, pp. 613-39.

[ 237 ]