1

Survey and Questionnaire based Research

Survey research is a quantitative and qualitative method with two important

characteristics. First, the variables of interest are measured using self-reports. In essence, survey
researchers ask their participants (who are often called respondents in survey research) to report
directly on their own thoughts, feelings, and behaviors. Second, considerable attention is paid to
the issue of sampling. In particular, survey researchers have a strong preference for large random
samples because they provide the most accurate estimates of what is true in the population. In
fact, survey research may be the only approach in psychology in which random sampling is
routinely used. Beyond these two characteristics, almost anything goes in survey research
(Montero & León, 2007).

Types of Surveys

There is much variety when it comes to surveys. This variety comes both in terms
of time—when or with what frequency a survey is administered—and in terms
of administration—how a survey is delivered to respondents (Zechmeister, Jeanne, &
Shaughnessy, 2009).

Time. In terms of time, there are two main types of surveys: cross-sectional and
longitudinal.
I. Cross-sectional surveys are those that are administered at just one point in time.
These surveys offer researchers a sort of snapshot in time and give us an idea
about how things are for our respondents at the particular point in time that the
survey is administered.
II. Longitudinal surveys are those that enable a researcher to make observations over
some extended period of time. There are several types of longitudinal surveys,
including trend, panel, and cohort surveys.

Administration. Surveys vary not just in terms of when they are administered but also
in terms of how they are administered. One common way to administer surveys is in the form
of self-administered questionnaires.
 2

Steps in survey research

Step one:
Identify the Research question. Good research has the characteristic that its purpose is
to address a single clear and explicit research question; conversely, the end product of a study
that aims to answer a number of diverse questions is often weak. Weakest of all, however, are
those studies that have no research question at all and whose design simply is to collect a wide
range of data and then to ‘trawl’ the data looking for ‘interesting’ or ‘significant’ associations.
This is a trap novice researchers in particular fall into (McBurney & White, 2009).
Step two :

Define the population and sample. Before you start conducting survey research, you
should already have a clear research question that defines what you want to find out. Based on
this question, you need to determine exactly who you will target to participate in the survey.
Population. A population is the set of all cases of interest. For example, if you are
interested in the attitudes of students on your campus toward computer services, your population
is all students on your campus. Contacting everyone in a large population is often practically
impossible. Therefore, researchers usually select a subset of the population to represent the
population as a whole.
Sampling frame. We need to develop a specific list of the members of the population in
order to select a subset of that population. This specific list is called a sampling frame and is, in a
sense, an operational definition of the population of interest.
Sample. The subset of the population actually drawn from the sampling frame is called
the sample. We might select 100 students from the registrar’s list to serve as the sample for our
computer survey. How closely the attitudes of this sample of students will represent all students’
attitudes depends critically on how the sample is selected. Each member of the population is
called an element. The identification and selection of elements that will make up the sample are
at the heart of all sampling techniques.

Types of Samples.

I. Haphazard samples. population subgroup for whose selection the researcher uses
hit-or-miss methods.
 3

II. Purposive samples. a nonrandom sample that is chosen for some characteristic
that it possesses.A purposive sample is not selected randomly, but for some
particular reason.
III. Convenience samples. a nonrandom sample that is chosen for practical [Link]
is similar to the purposive sample in that it selects a desirable group of people but
differs in that it may not come close to sampling all of a population.
IV. Probability Samples. The most satisfactory surveys generally obtain their
respondents in some manner such that the researcher knows the probability that
any given individual will appear in the sample. Whereas the other three types of
samples permit only subjective evaluation of the validity of the results, probability
samples permit one to apply various statistics. Our focus, therefore, is on
probability samples.

Sampling procedure.

I. Systematic Samples. A systematic sample is a probability sample but not a

random sample. It involves selecting elements from an ordered sampling frame.
II. Simple Random Samples. The basic simple random sample is used when we
believe that the population is relatively homogeneous with respect to the
questions of interest.
III. Stratified Random Samples. If you are surveying a population that has
identifiable subgroups that are likely to differ markedly in their responses, you
can improve the validity of your study by obtaining a stratified random sample.
IV. Cluster Samples. Many populations would be impossible or impractical to
number. For instance, making a list of every person in a country would be
impossible.
A sophisticated form of cluster sampling is known as multistage sampling.
Commercial polls, such as the Gallup Poll, use multistage sampling.
Step three:
Decide on the type of survey,there are two main types of survey:

[Link]. A questionnaire, where a list of questions is distributed by mail, online

or in person, and respondents fill it out [Link] out a paper survey by mail is a
 4

common method of gathering demographic information (for example, in a government

census of the population).
[Link]. An interview, where the researcher asks a set of questions by phone or in
person and records the responses Oral interviews are a useful method for smaller sample
sizes. They allow you to gather more in-depth information on people’s opinions and
preferences. You can conduct interviews by phone or in person.
Like questionnaires, interviews can be used to collect quantitative data: the researcher
records each response as a category or rating and statistically analyzes the results. But they are
more commonly used to collect qualitative data: the interviewees’ full responses
are transcribed and analyzed individually to gain a richer understanding of their opinions and
[Link] type you choose depends on the sample size and location, as well as the focus of
the research.

Step four:

Design the survey questions. Next, you need to decide which questions you will ask and
how you will ask them. It’s important to consider: type of questions, content of the questions
phrasing of the questions, ordering and layout of the survey (McBurney & White, 2009).

There are two main forms of survey questions: open-ended and closed-ended. Many
surveys use a combination of both.

I. Closed-ended questions are best for quantitative research. They provide you with
numerical data that can be statistically analyzed to find patterns, trends,
and correlations. Closed-ended questions give the respondent a predetermined set
of answers to choose from. A closed-ended question can include: a binary answer
(e.g. yes/no or agree/disagree), a scale (e.g. five points ranging from strongly
agree to strongly disagree), a list of options with a single answer possible (e.g.
age categories), a list of options with multiple answers possible (e.g. leisure
interests).

II. Open-ended questions are best for qualitative research. This type of question has
no predetermined answers to choose from. Instead, the respondent answers in
their own words. Open questions are most common in interviews, but you can
 5

also use them in questionnaires. They are often useful as follow-up questions to
ask for more detailed explanations of responses to the closed questions.

Step five:
Content of the survey [Link] ensure the validity and reliability of your results, you
need to carefully consider each question in the survey. All questions should be narrowly focused
with enough context for the respondent to answer accurately. Avoid questions that are not
directly relevant to the survey’s purpose. If you include a list of options that isn’t exhaustive, you
can add an “other” field.
Phrasing the survey questions. In terms of language, the survey questions should be as
clear and precise as possible. Tailor the questions to your target population, keeping in mind
their level of knowledge of the topic. Make sure your questions are phrased neutrally, with no
bias towards one answer or another.
Ordering the survey questions. The questions should be arranged in a logical order. Start
with easy, non-sensitive, closed-ended questions that will encourage the respondent to
[Link] the survey covers several different topics or themes, group together related questions.
You can divide a questionnaire into sections to help respondents understand what is being asked
in each part.
Step six:
Distribute the survey and collect [Link] are essentially four different modes
of administering surveys: faceto-face, written, computerized, and by telephone. Each one has its
advantages and disadvantages, as we shall see. The one that is best depends on the circumstances
(McBurney & White, 2009).
Face-to-Face. Personal interviews have the advantage that the interviewers can
establish rapport with the people being interviewed. Interviewers can direct the attention
of the respondents to the material and motivate them to answer the questions carefully.
Interviewers can guarantee the order in which questions are administered, thus making
sure that people answer the survey in the order intended.
Written Responses. Written administration of questionnaires can take several
forms: The questionnaires may be administered to a group, they may be dropped off at a
particular location, or they may be mailed to the respondents. Group administration is
 6

familiar to you if you have completed course evaluations in college. Group

administration is a very efficient use of time and money and can have a very high
response rate if attendance by group members is high, as in a class.
Computerized [Link] the Internet and social media continues to rise
in popularity, it may be convenient to administer a questionnaire by cell phone or
computer. The questionnaire may be distributed via e-mail, posted as a Web-based survey
that is open to the general public, or administered in a laboratory setting. The computer
has the advantage of being impersonal, so social desirability may be reduced. The
computer is also absolutely consistent.
Telephone [Link] main advantage of the telephone as a method of
administering surveys is low cost. Nowadays the percentage of people who can be
reached by telephone is about as high as the percentage that can be reached by other
means. Even unlisted numbers are not a problem when random-digit dialing is used.

Step seven:

Analyze the survey [Link] are many methods of analyzing the results of your
survey. First you have to process the data, usually with the help of a computer program to sort all
the responses. You should also clean the data by removing incomplete or incorrectly completed
responses.
If you asked open-ended questions, you will have to code the responses by assigning
labels to each response and organizing them into categories or themes. You can also use more
qualitative methods, such as thematic analysis, which is especially suitable for analyzing
interviews.
Statistical analysis is usually conducted using programs like SPSS or Stata. The same set
of survey data can be subject to many analyses.

Step eight:
Write up the survey results. Finally, when you have collected and analyzed all the
necessary data, you will write it up as part of your thesis, dissertation or research paper.
 7

In the methodology section, you describe exactly how you conducted the survey. You
should explain the types of questions you used, the sampling method, when and where the survey
took place, and the response rate. You can include the full questionnaire as an appendix if
relevant.
Then introduce the analysis by describing how you prepared the data and the statistical
methods you used to analyze it. In the results section, you summarize the key results from your
analysis.
In the discussion and conclusion, you give your explanations and interpretations of these
results, answer your research

Correlation and Causality

Surveys are often used in correlational research, and correlational research is an excellent
method for meeting the scientific goals of description and prediction. For example, studies
demonstrating correlations between physical health and psychological well-being allow
researchers to make predictions regarding health-related problems (McBurney & White, 2009).
Correlational evidence allows researchers to make predictions for the correlated
variables. However, the ability to make causal inferences based solely on a correlation between
two variables is very limited. Not being able to determine the direction of the relationship in a
correlation is only one challenge we face. It’s possible there is another causal interpretation for
the correlation between the two variables. For example, a third variable, number of friends, could
cause people to be more outgoing and more satisfi ed with their lives. A correlation that can be
explained by a third variable is called a spurious relationship.

Advantages and disadvantages of survey

Advantages:
• The breadth of coverage of many people or events means that it is more likely than
some other approaches to obtain data based on a representative sample, and can therefore
be generalizable to a population.
• Surveys can produce a large amount of data in a short time for a fairly low cost.
Researchers can therefore set a finite time-span for a project, which can assist in planning
and delivering end results.
 8

Disadvantages:
• The significance of the data can become neglected if the researcher focuses too much on
the range of coverage to the exclusion of an adequate account of the implications of those
data for relevant issues, problems, or theories.
• The data that are produced are likely to lack details or depth on the topic being
investigated.
• Securing a high response rate to a survey can be hard to control, particularly when it is
carried out by post, but is also difficult when the survey is carried out face-to-face or
over the telephone.

Validity and Reliability Issues in Survey Questions

Validity is concerned with the accuracy of our measurement. Reliability, on the other
hand, is concerned with the consistency of our measurement. This is the degree to which the
questions elicit the same type of information each time we use them, under the same conditions.
Validity and reliability are not always aligned. Reliability is needed, but not sufficient to
establish validity. We can get high reliability and low validity. This would happen when we ask
the wrong questions over and over again, consistently yielding bad information. Also, if the
results show large variability, they may be valid, but not reliable. In short, don’t assume
reliability and validity, unless you design surveys that really measure what you want and do it
consistently (Zechmeister et al., 2009).

Ethical issues
Anyone involved in collecting data from patients has an ethical duty to respect each
individual participant’s autonomy. Any survey should be conducted in an ethical manner and one
that accords with best research practice. Two important ethical issues to adhere to when
conducting a survey are confidentiality and informed consent.
The respondent’s right to confidentiality should always be respected and any legal
requirements on data protection should be adhered to. In the majority of surveys, the respondents
should be fully informed about the aims of the survey, and the consent to participate in the
survey must be obtained and recorded.
 9

It is much more important to know whether a biased sample has been used, or whether
the wording of questions has been slanted, or whether the data have been selectively analyzed or
reported. Any of these aspects of survey research can bias the results, and unethical researchers
can use these techniques to make the results “turn out right.” The best protection against
unethical researchers and poor quality research is to examine carefully the procedures and
analyses used in the survey research (Zechmeister et al., 2009).

Researches in Pakistan Employing Survey method

Following are some Pakistani researches employing survey and questionnaire method:

A study by Bashir, Azam, Butt, Javed, and Tanvir (2013) investigates the influence of
demographics (residential area, age, gender, marital status, education background) and
personality traits (extraversion, openness, conscientiousness, neuroticism, and agreeableness) on
the financial behavioral biases (overconfidence, herding/mass behavior and disposition effect)
and risk taking behavior in [Link] dimensions are categorized under Big Five
Personality model. Questionnaire survey method is used to collect the data from a Sample size of
225 respondents that includes bankers, finance students as well as investors. Structure equation
modeling (SEM) analysis is used to analyze the impact of personality traits and demographics on
the investment biases through Amos 20. The results show that big five personality traits have a
significant relationship with overconfidence, herding/mass behavior and risk taking except
disposition effect.
Another study by Nadeem, Ali, Maqbool, and Zaidi (2012) is based on a study that was
designed to study the "Impact of Anxiety on the Academic Achievement of Students at
University level in Bahawalpur, Pakistan." Being a descriptive study, survey method was
adopted for data collection to find out the results. For sample size out of 200 students 97 students
were selected by stratified sampling. The researcher made three groups of all the students and
three groups of male and female students. In this research questionnaire (Otis selfadministering
test of mental ability) and anxiety measurement scale was selected as an instrument for the
purpose of data [Link] in depth investigation of the findings obtained through this
analyzed data reveals that anxiety had its impact on academic achievement of students. The
results show that when anxiety increases, academic achievement decreases both in male and
 10

female students. It is also noteworthy in the results that there is more impact of anxiety on female
students as compared to male students.
A recent study by Nand, Pitafi, Kanwal, Pitafi, and Rasheed (2019) investigated whether
adopting smartphones by students improve learning or generates interference. The purpose of
this study was to examine the usage of smartphone among university students in Pakistan. Using
a survey approach, data were collected from different universities/colleges of Karachi, Sindh,
Pakistan. total, 585 entries was analyzed using SPSS 21.0 and AMOS 21.0 software. Results of
analysis, validated most of the hypothesis showing the intention of Pakistani students towards
smartphone usage for academic purpose. In addition, findings confirm that students' self‐efficacy
related to a smartphone works as a moderating role of intention to use a smartphone for learning
and actual usage of the smartphone. This study has several implications in terms of smartphone
usage and student academic performance.

References

Bashir, T., Azam, N., Butt, A. A., Javed, A., & Tanvir, A. (2013). Are behavioral biases
influenced by demographic characteristics & personality traits? Evidence from Pakistan.
European Scientific Journal, 9(29).
McBurney, D. H., & White, T. L. (2009). Research methods (9th ed.). 20 Davis Drive,Belmont,
CA 94002-3098,USA: Cengage Learning.
Montero, I., & León, O. G. (2007). A guide for naming research studies in Psychology.
International Journal of clinical and Health psychology, 7(3), 847-862.
Nadeem, M., Ali, A., Maqbool, S., & Zaidi, S. U. (2012). Impact of anxiety on the academic
achievement of students having different mental abilities at university level in
Bahawalpur (Southern Punjab) Pakistan. International Online Journal of Educational
Sciences, 4(3), 519-528.
Nand, S., Pitafi, A. H., Kanwal, S., Pitafi, A., & Rasheed, M. I. (2019). Understanding the
academic learning of university students using smartphone: Evidence from Pakistan.
Journal of Public Affairs, e1976.
Zechmeister, E., Jeanne, & Shaughnessy, J. (2009). Research methods in psychology (9th ed.).
1221 Avenue of the Americas, New York: McGraw-Hill Higher Education.