Week 8 sampling and measurements 2015
- 2. More reminders .. Get your variables sorted by now Chapters 1, 2, 3,4 are not yours Strong remark Use of phrases from another article Citing secondary sources Block quotation and Page reference In text citation Colloquial – “In my honest opinion….” Connecting ideas – on the other hand, moreover, in support of, This is supported by.., in contrast to, in addition, nonetheless….
- 3. Instructor assigned task (25%) TASK : Design a research Survey Correlational Experimental Causal Comparative Case study – Mixed method Ethnography TOPIC : 21st century learners Group work Group of 3 (max) Study task description on i-Learn Assignment/Projects Instructor Assigned Task Use Blendspace Due 30th May 2015 Marks given for product and comments/discussions
- 4. Where we are now… Observation Background survey Broad area of research Literature review Problem definition Problem statement Research questions Theoretical framework Variables clearly identified Hypothesis generation Research design Methods Data collection Analysis Interpretation Deduction Hypothesis substantiated? RQ answered?
- 5. Lesson Objectives Describe procedures Obtaining permissions for data collection Selecting participants for data collection Identifying data options Recording and administering data collection
- 6. Let’s move on to data collection What is data? Collection of facts from which conclusions may be drawn
- 7. Key ideas Who will you study (unit, sampling, sample size)? What permissions will you need? (levels,MOE) What information will you collect? (types of data, links to questions/variables) What instrument(s) will you use? (selecting an instrument, scales of measurement, validity, reliability) How will you administer the data collection? (standardization, ethical issues)
- 8. Who will you study? Unit of analysis is the level (e.g. individual, family, school, school district) the data will be gathered. There may be different units of analysis one for the dependent variable one for the independent variable
- 9. Procedures for Collecting Quantitative Data (1) Obtain permissions secure permissions obtain informed consent from participants
- 10. Obtaining Permissions Institutional or organizational (e.g. school district) Site-specific (e.g. secondary school) Individual participants or parents Campus approval (e.g. university or college)
- 11. Obtaining Informed Consent from EPRD,MOE Obtain Approval via State Education Department Have participants sign an informed consent form
- 12. Procedures for Collecting Quantitative Data (2) Select participants specify a population and sample use probability and non- probability sampling choose a sample size
- 13. Procedures for Collecting Quantitative Data (3) Identify data options specify variables operationalize variables select scales of measurement choose types of data measures
- 14. Procedures for Collecting Quantitative Data (4) Record and administer data collection locate or develop an instrument obtain reliable and valid data develop administrative procedures for data collection
- 16. POPULATION and SAMPLING A population is a group of individuals that comprise the same characteristics A sample is a sub-group of the target population that the researcher plans to study
- 17. Select Participants: Specify a Population and Sample Samples for the purpose about making generalizations about the target population (quantitative research). samples are only estimates the difference between the sample estimate and the true population is the “sampling error.”
- 18. Manageable research population Populations and Samples Sample Target Population Population • All science teachers in secondary schools in Kuantan • College students in all community colleges • Adult educators in all faculties of education Better? All sec. school biology teachers in Kuantan Students in one community college Adult educators in faculties of education in the East Malaysia
- 19. Probability and Non-Probability Sampling Probability sampling is the selection of individuals from the population so that they are representative of the population Non-probability sampling is the selection of participants because they are available, convenient, or represent some characteristic the investigator wants to study.
- 20. Types of Sampling Sampling Strategies Probability/Random Sampling Non-Probability/Purposeful Sampling Simple Stratified Cluster Random Sampling Sampling Sampling Convenience Snowball Sampling Sampling
- 21. Differences Between Random and Purposeful Sampling Random “Quantitative” Sampling Select Representative individuals To generalize from sample to population To make claims about the population To build/test “theories” that explain the population Purposeful “Qualitative” Sampling Select people/sites who can best help us understand our phenomenon To develop detailed understanding That might be “useful: information That might help people “learn” about the phenomenon That might give voice to “silenced” people
- 22. Types of Probability Sampling Simple Random: selecting a sample from the population so all in the population have an equal chance of being selected Systematic: choosing every “nth” individual or site in the population until the desired sample size is achieved
- 23. Types of Probability Sampling Stratified sampling: stratifying the population on a characteristic (e.g. gender), then sampling from each stratum. Boys N=6000 Girls N=3000 Population (N=9000) .66 of pop. 200 .33 of pop 100 Sample = 300
- 24. Types of Probability Sampling Cluster Sampling: Selects groups, not individuals All members in the groups have similar characteristics Useful when the population is large or spread over a wide geographical area
- 25. Example: Cluster Sampling Population : All primary school teachers in Klang valley (5000) The desired sample : 400 Cluster: School No. of primary schools in Klang valley: 150 Average number of teachers per school : 40 Number of cluster : 400/ 40 10 out of 150 schools are randomly selected All teachers in the selected schools make up the sample
- 26. Try one You want to study the resilience of UiTM students. Population? Sampling?
- 27. Select Participants: Choose a Sample Size Select a sample size as large as possible from the individuals available Select a sufficient number of participants for the statistical tests you will use Calculate the sample size using a sample size formula
- 28. Select Participants: Choose a Sample Size A rough estimate: 15 participants in each grp in an expt 30 participants for a correlational study 350 individuals for a survey study but depend of several factors
- 29. Calculating sample size Krejcie and Morgan (1970)
- 30. EXAMPLE Gender difference in vocational interest of post matriculation at UiTM Independent variable: Dependent variable: Research Design: Population: Sample size: Instrument:
- 31. EXAMPLE Gender difference in vocational interest of post matriculation at UiTM Independent variable: Gender Dependent variable: Vocational interest Research Design: Survey Population: Post matriculation students Sample size: 350 Instrument: Self developed
- 32. Calculating sample size Use web calculator http://www.raosoft.com/samplesize.html http://survey.pearsonncs.com/sample-calc.htm
- 35. Types of Purposeful Sampling When Does Sampling Occur? After Data Collection has startedWhat is the intent? To develop many perspectives Extreme Case Sampling To describe particularly troublesome or enlightening cases Typical Sampling To describe what is “typical” to those unfamiliar with the case What is the intent? To take advantage of whatever case unfolds Opportunistic Sampling To locate people or sites to study Snowball Sampling To explore confirming or disconfirming cases Confirming/ Disconfirming Sampling Maximal Variation Sampling To generate a theory or concept Theory or Concept Sampling To describe some sub-group in depth Homogenous Sampling Before Data Collection
- 36. One may sample.. Maximal variation Most hardworking/ Highest achiever Lowest achiever Extreme case ??
- 37. Examples of Non-Probability Samples Convenience Sampling: participants are selected because they are willing and available to be studied Snowball Sampling: the researcher asks participants to identify other participants to become members of the sample.
- 38. Non Probability sampling example Study delinquent behaviour during recess. I selected 1 school out of 4 - Interviewed all 4 principals and toured all 4 schools. I chose school to which I was given most access with fewest restrictions. Also school that reported the widest variations in delinquent behaviour during recess (very high to virtually no display). Then sampled different locations with camera to find most varied activity and least self-conscious/guarded behavior. Where? Turned out to be behind the surau. Later used snowball approach in choosing children to interview.
- 39. Your sample?
- 40. Proceed to collecting quantitative data
- 42. Flow of Activities in Collecting Data 1.Identify the variable 2.Operationally define the variable 3.Locate data (measures, observations, documents with questions and scales) 4.Collect data on instruments yielding numeric scores
- 43. Identify Data Options: Specify Variables Independent Variables Dependent Variables Intervening Moderating
- 44. Identify Data Options: Operationalize Variables Operational Definition: The specification of how the variable will be defined and measured typically based on the literature often found in reports under “definition of terms” Sometimes the researcher must construct it
- 45. Flow of Activities in Collecting Data Identify the variable Operationally define the variable Locate data (measures, observations, documents with questions and scales) Collect data on instruments yielding numeric scores Self-efficacy for learning from others Level of confidence that an individual can learn something by being taught by others 13 items on a self-efficacy attitudinal scale from Bergin (1989) Scores of each item ranged from 0-10 with 10 being “completely confident.” Flow of Activities Example
- 46. Flow of Activities in Collecting Data Identify the variable Operationally define the variable Locate data (measures, observations, documents with questions and scales) Collect data on instruments yielding numeric scores Learning motivation of adult learners Interest and level of engagement of an individual ----- items on a questionnaire developed Scores of each item ranged from 1-5 with 5 being “most interested” Flow of Activities Example
- 48. Identify Data Options: Select Scales of Measurement Nominal (Categorical): categories that describe traits or characteristics participants can check Female Male Ordinal: participants rank the order of a characteristic, trait or attribute
- 49. Identify Data Options: Select Scales of Measurement Interval: provides “continuous” response possibilities to questions with assumed equal distance ; scale with no true zero Discrete (SD ---------------------SA) Metric (oC) Ratio: a scale with a true zero and equal distances among units
- 50. Practice Identify the level of measurement
- 51. Measurement level? Age Religion Gender Income bracket Test scores CGPA Frequency of asking questions Time spent on task Level of acceptance (0 – never, 5 – all the time) Activity ( 0 –not active, 5 – very active) Ratio ? Nominal? Ordinal? Interval?
- 52. Measurement level? Age Religion Gender Income bracket Test scores CGPA Frequency of asking questions Time spent on task Level of acceptance (0 – never, 5 – all the time) Activity ( 0 –not active, 5 – very active) Ratio Nominal Nominal Ordinal Ratio Ordinal Ratio Ratio Interval/ Ordinal Interval/Ordinal
- 53. Identify Data Options: Choose Types of Data Measures An instrument is a tool for measuring, observing, or documenting quantitative data Types of Instruments Performance Measures (e.g. test performance) Attitudinal Measures (measures feelings toward educational topics) Behavioral Measures (observations of behavior) Factual Measures (documents, records)
- 54. Record and Administer Data Collection: Locate or Develop an Instrument Develop your own instrument Locate an existing instrument Modify an existing instrument
- 55. Record and Administer Data Collection: Locate or Develop an Instrument Strategies to use Look in published journal articles Run an ERIC search and use the term “instruments” and the topic of the study Go to ERIC web site for Evaluation and Assessment Examine guides to commercially available tests
- 57. Obtain Reliable and Valid Data Reliability: individual scores from an instrument should be nearly the same or stable on repeated administrations of the instrument Bathroom scale
- 58. Reliability Types of reliability Test-retest (scores are stable over time) Internal consistency (consistent scores across the instrument) Cronbach coefficient alpha if items are scored as continuous variables (SA—SD) Inter-rater reliability (similarity in observation of a behavior by two or more individuals)
- 59. Validity Validity: the ability to draw meaningful and justifiable inferences from the scores about a sample or a population
- 60. Types of validity Content (representative of all possible questions that could be asked) Criterion-referenced (scores are a predictor of an outcome or criterion they are expected to predict Construct (determination of the significance, meaning, purpose and use of the scores)
- 61. Let’ s look at a Rosenberg Self Esteem Scale
- 63. Key Ideas Gaining site permission Purposive sampling Types of qualitative data Protocols and Issues regarding administering and recording qualitative
- 64. Gaining Permission Gain permission from Institutional Review Board Gain permission from “gatekeepers” at the research site Gatekeepers: individuals at the site who provide site access, helps researcher locate people and identifies places to study The gatekeeper may require written permission about the project
- 65. Information for the gatekeeper Why their site was chosen What time and resources are required What will be accomplished at the site What potential there is for your presence to be disruptive What individuals at the site will gain from the study How will you use and report the results
- 66. Types of data to Collect Observations Interviews Documents Audio-Visual Materials
- 67. Sources of Qualitative Data From People: Interviews Surveys Focus Groups Participant Observation (field notes) From Things: Agency case records Miscellaneous documents Historical Artifacts Media Published materials
- 69. Types of Data to Collect: Observations An observation is the process if gathering first-hand information by observing people and places at a research site. Observational roles Participant observer Non-Participant observer Observational roles can be changed
- 70. Types of Data to Collect: Observations Conduct multiple observations Record both descriptive and reflective field notes during the observation Descriptive field notes describe the events, activities and people Reflective field notes record personal reflections that relate to their insights, hunches or broad themes that emerge
- 71. Administering and Recording Data: Observational Protocols The header: essential information about the observation Left column to record descriptive notes Right column to record reflective notes A picture of the site may be sketched
- 72. Types of Data to Collect: Interviews Types of Interviews Individual Focus group Telephone e-mail
- 73. Structured, Unstructured, and Semi-Structured Interviews Approach to Data Collection Type of Response Options to Questions Types of Interviews Leading to Data Quantitative Close- Ended Structured/ semi-structured Interviews Scores to answers Qualitative Open- Ended Unstructured Interviews Transcription of words
- 74. Types of Data to Collect: Interviews General open-ended questions are asked allows the participant to create options for responding participants can voice their experiences and perspectives Information is recorded then transcribed for analysis
- 75. Administering and Recording Data: Interview Protocols The header: essential information about the interview Open-ended questions include “ice-breaker” ones that address major research questions probes that clarify and elaborate Closing comments thanking the participant
- 76. EXAMPLE: Semi structured Interview protocols Name: Date: Time: Venue: Experience – please describe Questions 1. Is the method a valid approach to addressing the learning needs of adult learners? Please describe your thoughts of the method 2. What do you think are the barriers to the utilization of the method in UiTM? 3. How would you rate the quality of learning using the method? 4. What overall suggestions do you have for improving the method?
- 77. Types of Data to Collect: Documents Public and private records Good source for text data You must obtain permission before using documents Scan documents when possible
- 78. Types of Data to Collect: Audio- Visual Materials Determine the material that can provide evidence to address your research questions Determine if the material is available and obtain permission to use it Check the accuracy and authenticity of the material if you do not record it yourself Collect the data and organize it
- 79. Administering and Recording Data: Field Issues Time needed for data collection Limit initial collection or one or two observations or interviews Time is needed to establish a substantial data base
- 80. Administering and Recording Data: Field Issues Obtaining permission to use materials Ethical issues Anonymity of participants Convey true purpose of study without deception
- 81. Let’s gather data Fill in the Qnaire Tabulate results Answer research Qs Qualitative data ??