Design-based research: an introduction
- 1. 1
- 2. Motives More relevance for pedagogical decision making in schools. (Anne Brown) Capture the innovations that teachers (not only researchers) produce (Alan Collins) DBR
- 3. A definition “A systematic but flexible methodology aimed to improve educational practices through iterative analysis, design, development, and implementation, based on collaboration among researchers and practitioners in real-world settings, and leading to contextually-sensitive design principles and theories.” Wang & Hannafin, 2005, p. 6
- 4. Aspiration “DBR involves the creation of a theoretically-inspired innovation, usually a learning environment, to directly address a local problem”. But credible evidence for local gains is not sufficient. In addition, the researcher needs to: “…generate evidence-based claims about learning that address contemporary theoretical issues and further the theoretical knowledge of the field” (Barab, 2008, p. 155)
- 5. Positioning DBR Low Control High R e l e v a n c e L H Action Research Correlation Studies Experiments DBR Quasi- Experiments
- 6. Variables and context Levels: individual, interpersonal, group, resource, institution Types of dependent variables: climate, learning, and systemic Types of contextual variables: setting, nature of learners, resources for implementation, professional development, financial requirements, implementation path. Time horizon and measurement points.
- 7. Factorial designs DVs IV IV IV CVs DV = f(IV1, IV2, …Ivk | CV1…m) with f usually linear
- 8. Repeated Measurements Design Time Design&Refine Implement&ObserveAnalyze Design&Refine Implement&ObserveAnalyze Design&Refine Implement&ObserveAnalyze Design Change Design Change
- 9. Repeated Measurements Design Time Interventions Relevant performance measure Baseline
- 10. Not only a research process The design artifact is essential: it outlasts the study and can be adopted, adapted, and re-used by others.To the extent that the design artifact reifies a model of teaching/learning, that model/theory gets tested (elsewhere) as well. Process (models) as artifacts Teaching experiments Software as artifact “learning environments”
- 13. What kind of knowledge is produced by DBR? 1. Designs for process and/or software. 2. Software (where applicable). 3. “Design narratives”: How to allow others to re-contextualize (and perhaps re-implement) a design with respect to local particulars? Rich descriptions how designs are enacted. 4. Theoretical accounts: Propositional systems reporting “rich models of interaction” .
- 14. How is a design study (proto- )typically done? Phase 1: Preparing the experiment Clarifying the instructional goals Documenting the instructional starting points Delineating an envisioned learning trajectory Placing the experiment in a theoretical context Phase 2: Experimenting to support learning Collecting data in cycles of design and analysis Applying interpretive frameworks Formulating and testing domain-specific instructional theories Phase 3: Conducting retrospective analyses Explicating the argumentative grammar Establishing trust in the findings Ensuring repeatability Ensuring generalizability
- 15. Delineating a learning trajectory Testable conjectures about significant shifts in students’ reasoning, and the means for supporting and organising these shifts, including: “Tangibles” such as instructional tasks, materials, ICTs,... “Intangibles” such as classrooms norms and nature of classroom discourse, … Teacher as co-designer of the ‘means’ and part of them. Basis for the conjectures: Research literature Experience Creativity
- 16. Placing the experiment in a theoretical context The goal of DBR is to produce knowledge that will be useful in providing guidance to others in their attempts to support learning processes. Hence, the specific experiment needs to be framed as a paradigmatic instance of broader phenomena. This framing is best done in terms of ‘mid-range’ theories E.g., meta-representational competence (diSessa, 1992) rather than ‘constructivism’.
- 17. Data analysis Huge range of methods, ranging from mathematical and computational modeling over statistical analysis to content analysis and ethnomethdological or phenomenological methods. Video analysis very prominent Log file analysis very prominent in some quarters (CSCL) Typically, mixed methods approach applied. (Statistical) “Generalisations” often required by policy makers.
- 18. DBR’s Argumentative Grammar How are claims to knowledge justified? What form does the warrant take? Demarcation: Can the set of methods establish boundaries between sound and unsound claims about learning and teaching? Meaningfulness: Can the set of methods help to generate meaningful/powerful (construct-advancing) hypotheses/models?
- 19. The full research process 1 Grounded Models 2 Development of Artifact 3 Feasability Study 4 Prototyping & Trials (DBR) 5 Field Study (Quasi- Experiment, DBR) 6 Definitive Test (Full Experiment) 7 Dissemination & Impact
- 20. Teacher-led design research Required for long-term design research. Required for continuous design improvements. Develops capacity in teachers and in the organisation “school” to not only ‘implement’, but t0 generate (sustainable) innovations.
- 21. Challenges Methodological Researcher involvement, Hawthorne effect Reliability Incremental contributions, comparison across studies Design methodology Data gathering and analysis Capturing context and process Managing and integrating various, often ‘rich’ data formats Data deluge Dissemination and scalability Publishing design studies Going beyond the case Researcher “training” (including teacher “training”)
- 22. Key Readings Kelly, A. E., Lesh, R.A., & Baek, J.Y. (Eds.). (2008). Handbook of design research methods in education. NewYork: Routledge. Reimann, P. (2009).Time is precious:Variable- and event- centered approaches to process analysis in CSCL research. International Journal of Computer-supported Collaborative Learning, 4, 239-257. Suthers, D. D., Dwyer, N., Medina, R., &Vatrapu, R. (2007). A framework for eclectic analysis of collaborative interaction. In C. Chinn, G. Erkens & S. Puntambekar (Eds.), Minds, mind, and society. Proceedings of the 6th International Conference on Computer-supported Collaborative Learning (CSCL 2007) (pp. 694-703). New Brunswick, NJ: International Society of the Learning Sciences.
Editor's Notes
- #6: The question is: can this mixture work? And what are the epistemological tensions?