design based research paper

Change Password

Your password must have 8 characters or more and contain 3 of the following:.

• a lower case character, 
• an upper case character, 
• a special character 

Password Changed Successfully

Your password has been changed

• Sign in / Register

Request Username

Can't sign in? Forgot your username?

Enter your email address below and we will send you your username

If the address matches an existing account you will receive an email with instructions to retrieve your username

Design-Based Research: A Methodology to Extend and Enrich Biology Education Research

• Emily E. Scott
• Mary Pat Wenderoth
• Jennifer H. Doherty

*Address correspondence to: Emily E. Scott ( E-mail Address: [email protected] ).

Department of Biology, University of Washington, Seattle, WA 98195

Search for more papers by this author

Recent calls in biology education research (BER) have recommended that researchers leverage learning theories and methodologies from other disciplines to investigate the mechanisms by which students to develop sophisticated ideas. We suggest design-based research from the learning sciences is a compelling methodology for achieving this aim. Design-based research investigates the “learning ecologies” that move student thinking toward mastery. These “learning ecologies” are grounded in theories of learning, produce measurable changes in student learning, generate design principles that guide the development of instructional tools, and are enacted using extended, iterative teaching experiments. In this essay, we introduce readers to the key elements of design-based research, using our own research into student learning in undergraduate physiology as an example of design-based research in BER. Then, we discuss how design-based research can extend work already done in BER and foster interdisciplinary collaborations among cognitive and learning scientists, biology education researchers, and instructors. We also explore some of the challenges associated with this methodological approach.

INTRODUCTION

There have been recent calls for biology education researchers to look toward other fields of educational inquiry for theories and methodologies to advance, and expand, our understanding of what helps students learn to think like biologists ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Lo et al. , 2019 ). These calls include the recommendations that biology education researchers ground their work in learning theories from the cognitive and learning sciences ( Coley and Tanner, 2012 ) and begin investigating the underlying mechanisms by which students to develop sophisticated biology ideas ( Dolan, 2015 ; Lo et al. , 2019 ). Design-based research from the learning sciences is one methodology that seeks to do both by using theories of learning to investigate how “learning ecologies”—that is, complex systems of interactions among instructors, students, and environmental components—support the process of student learning ( Brown, 1992 ; Cobb et al. , 2003 ; Collins et al. , 2004 ; Peffer and Renken, 2016 ).

The purpose of this essay is twofold. First, we want to introduce readers to the key elements of design-based research, using our research into student learning in undergraduate physiology as an example of design-based research in biology education research (BER). Second, we will discuss how design-based research can extend work already done in BER and explore some of the challenges of its implementation. For a more in-depth review of design-based research, we direct readers to the following references: Brown (1992) , Barab and Squire (2004) , and Collins et al. (2004) , as well as commentaries by Anderson and Shattuck (2012) and McKenney and Reeves (2013) .

WHAT IS DESIGN-BASED RESEARCH?

Design-based research is a methodological approach that aligns with research methods from the fields of engineering or applied physics, where products are designed for specific purposes ( Brown, 1992 ; Joseph, 2004 ; Middleton et al. , 2008 ; Kelly, 2014 ). Consequently, investigators using design-based research approach educational inquiry much as an engineer develops a new product: First, the researchers identify a problem that needs to be addressed (e.g., a particular learning challenge that students face). Next, they design a potential “solution” to the problem in the form of instructional tools (e.g., reasoning strategies, worksheets; e.g., Reiser et al. , 2001 ) that theory and previous research suggest will address the problem. Then, the researchers test the instructional tools in a real-world setting (i.e., the classroom) to see if the tools positively impact student learning. As testing proceeds, researchers evaluate the instructional tools with emerging evidence of their effectiveness (or lack thereof) and progressively revise the tools— in real time —as necessary ( Collins et al. , 2004 ). Finally, the researchers reflect on the outcomes of the experiment, identifying the features of the instructional tools that were successful at addressing the initial learning problem, revising those aspects that were not helpful to learning, and determining how the research informed the theory underlying the experiment. This leads to another research cycle of designing, testing, evaluating, and reflecting to refine the instructional tools in support of student learning. We have characterized this iterative process in Figure 1 after Sandoval (2014) . Though we have portrayed four discrete phases to design-based research, there is often overlap of the phases as the research progresses (e.g., testing and evaluating can occur simultaneously).

FIGURE 1. The four phases of design-based research experienced in an iterative cycle (A). We also highlight the main features of each phase of our design-based research project investigating students’ use of flux in physiology (B).

Design-based research has no specific requirements for the form that instructional tools must take or the manner in which the tools are evaluated ( Bell, 2004 ; Anderson and Shattuck, 2012 ). Instead, design-based research has what Sandoval (2014) calls “epistemic commitments” 1 that inform the major goals of a design-based research project as well as how it is implemented. These epistemic commitments are: 1) Design based research should be grounded in theories of learning (e.g., constructivism, knowledge-in-pieces, conceptual change) that both inform the design of the instructional tools and are improved upon by the research ( Cobb et al. , 2003 ; Barab and Squire, 2004 ). This makes design-based research more than a method for testing whether or not an instructional tool works; it also investigates why the design worked and how it can be generalized to other learning environments ( Cobb et al. , 2003 ). 2) Design-based research should aim to produce measurable changes in student learning in classrooms around a particular learning problem ( Anderson and Shattuck, 2012 ; McKenney and Reeves, 2013 ). This requirement ensures that theoretical research into student learning is directly applicable, and impactful, to students and instructors in classroom settings ( Hoadley, 2004 ). 3) Design-based research should generate design principles that guide the development and implementation of future instructional tools ( Edelson, 2002 ). This commitment makes the research findings broadly applicable for use in a variety of classroom environments. 4) Design-based research should be enacted using extended, iterative teaching experiments in classrooms. By observing student learning over an extended period of time (e.g., throughout an entire term or across terms), researchers are more likely to observe the full effects of how the instructional tools impact student learning compared with short-term experiments ( Brown, 1992 ; Barab and Squire, 2004 ; Sandoval and Bell, 2004 ).

HOW IS DESIGN-BASED RESEARCH DIFFERENT FROM AN EXPERIMENTAL APPROACH?

Many BER studies employ experimental approaches that align with traditional scientific methods of experimentation, such as using treatment versus control groups, randomly assigning treatments to different groups, replicating interventions across multiple spatial or temporal periods, and using statistical methods to guide the kinds of inferences that arise from an experiment. While design-based research can similarly employ these strategies for educational inquiry, there are also some notable differences in its approach to experimentation ( Collins et al. , 2004 ; Hoadley, 2004 ). In this section, we contrast the differences between design-based research and what we call “experimental approaches,” although both paradigms represent a form of experimentation.

The first difference between an experimental approach and design-based research regards the role participants play in the experiment. In an experimental approach, the researcher is responsible for making all the decisions about how the experiment will be implemented and analyzed, while the instructor facilitates the experimental treatments. In design-based research, both researchers and instructors are engaged in all stages of the research from conception to reflection ( Collins et al. , 2004 ). In BER, a third condition frequently arises wherein the researcher is also the instructor. In this case, if the research questions being investigated produce generalizable results that have the potential to impact teaching broadly, then this is consistent with a design-based research approach ( Cobb et al. , 2003 ). However, when the research questions are self-reflective about how a researcher/instructor can improve his or her own classroom practices, this aligns more closely with “action research,” which is another methodology used in education research (see Stringer, 2013 ).

A second difference between experimental research and design-based research is the form that hypotheses take and the manner in which they are investigated ( Collins et al. , 2004 ; Sandoval, 2014 ). In experimental approaches, researchers develop a hypothesis about how a specific instructional intervention will impact student learning. The intervention is then tested in the classroom(s) while controlling for other variables that are not part of the study in order to isolate the effects of the intervention. Sometimes, researchers designate a “control” situation that serves as a comparison group that does not experience the intervention. For example, Jackson et al. (2018) were interested in comparing peer- and self-grading of weekly practice exams to if they were equally effective forms of deliberate practice for students in a large-enrollment class. To test this, the authors (including authors of this essay J.H.D., M.P.W.) designed an experiment in which lab sections of students in a large lecture course were randomly assigned to either a peer-grading or self-grading treatment so they could isolate the effects of each intervention. In design-based research, a hypothesis is conceptualized as the “design solution” rather than a specific intervention; that is, design-based researchers hypothesize that the designed instructional tools, when implemented in the classroom, will create a learning ecology that improves student learning around the identified learning problem ( Edelson, 2002 ; Bell, 2004 ). For example, Zagallo et al. (2016) developed a laboratory curriculum (i.e., the hypothesized “design solution”) for molecular and cellular biology majors to address the learning problem that students often struggle to connect scientific models and empirical data. This curriculum entailed: focusing instruction around a set of target biological models; developing small-group activities in which students interacted with the models by analyzing data from scientific papers; using formative assessment tools for student feedback; and providing students with a set of learning objectives they could use as study tools. They tested their curriculum in a novel, large-enrollment course of upper-division students over several years, making iterative changes to the curriculum as the study progressed.

By framing the research approach as an iterative endeavor of progressive refinement rather than a test of a particular intervention when all other variables are controlled, design-based researchers recognize that: 1) classrooms, and classroom experiences, are unique at any given time, making it difficult to truly “control” the environment in which an intervention occurs or establish a “control group” that differs only in the features of an intervention; and 2) many aspects of a classroom experience may influence the effectiveness of an intervention, often in unanticipated ways, which should be included in the research team’s analysis of an intervention’s success. Consequently, the research team is less concerned with controlling the research conditions—as in an experimental approach—and instead focuses on characterizing the learning environment ( Barab and Squire, 2004 ). This involves collecting data from multiple sources as the research progresses, including how the instructional tools were implemented, aspects of the implementation process that failed to go as planned, and how the instructional tools or implementation process was modified. These characterizations can provide important insights into what specific features of the instructional tools, or the learning environment, were most impactful to learning ( DBR Collective, 2003 ).

A third difference between experimental approaches and design-based research is when the instructional interventions can be modified. In experimental research, the intervention is fixed throughout the experimental period, with any revisions occurring only after the experiment has concluded. This is critical for ensuring that the results of the study provide evidence of the efficacy of a specific intervention. By contrast, design-based research takes a more flexible approach that allows instructional tools to be modified in situ as they are being implemented ( Hoadley, 2004 ; Barab, 2014 ). This flexibility allows the research team to modify instructional tools or strategies that prove inadequate for collecting the evidence necessary to evaluate the underlying theory and ensures a tight connection between interventions and a specific learning problem ( Collins et al. , 2004 ; Hoadley, 2004 ).

Finally, and importantly, experimental approaches and design-based research differ in the kinds of conclusions they draw from their data. Experimental research can “identify that something meaningful happened; but [it is] not able to articulate what about the intervention caused that story to unfold” ( Barab, 2014 , p. 162). In other words, experimental methods are robust for identifying where differences in learning occur, such as between groups of students experiencing peer- or self-grading of practice exams ( Jackson et al. , 2018 ) or receiving different curricula (e.g., Chi et al. , 2012 ). However, these methods are not able to characterize the underlying learning process or mechanism involved in the different learning outcomes. By contrast, design-based research has the potential to uncover mechanisms of learning, because it investigates how the nature of student thinking changes as students experience instructional interventions ( Shavelson et al. , 2003 ; Barab, 2014 ). According to Sandoval (2014) , “Design research, as a means of uncovering causal processes, is oriented not to finding effects but to finding functions , to understanding how desired (and undesired) effects arise through interactions in a designed environment” (p. 30). In Zagallo et al. (2016) , the authors found that their curriculum supported students’ data-interpretation skills, because it stimulated students’ spontaneous use of argumentation during which group members coconstructed evidence-based claims from the data provided. Students also worked collaboratively to decode figures and identify data patterns. These strategies were identified from the researchers’ qualitative data analysis of in-class recordings of small-group discussions, which allowed them to observe what students were doing to support their learning. Because design-based research is focused on characterizing how learning occurs in classrooms, it can begin to answer the kinds of mechanistic questions others have identified as central to advancing BER ( National Research Council [NRC], 2012 ; Dolan, 2015 ; Lo et al. , 2019 ).

DESIGN-BASED RESEARCH IN ACTION: AN EXAMPLE FROM UNDERGRADUATE PHYSIOLOGY

To illustrate how design-based research could be employed in BER, we draw on our own research that investigates how students learn physiology. We will characterize one iteration of our design-based research cycle ( Figure 1 ), emphasizing how our project uses Sandoval’s four epistemic commitments (i.e., theory driven, practically applied, generating design principles, implemented in an iterative manner) to guide our implementation.

Identifying the Learning Problem

Understanding physiological phenomena is challenging for students, given the wide variety of contexts (e.g., cardiovascular, neuromuscular, respiratory; animal vs. plant) and scales involved (e.g., using molecular-level interactions to explain organism functioning; Wang, 2004 ; Michael, 2007 ; Badenhorst et al. , 2016 ). To address these learning challenges, Modell (2000) identified seven “general models” that undergird most physiology phenomena (i.e., control systems, conservation of mass, mass and heat flow, elastic properties of tissues, transport across membranes, cell-to-cell communication, molecular interactions). Instructors can use these models as a “conceptual framework” to help students build intellectual coherence across phenomena and develop a deeper understanding of physiology ( Modell, 2000 ; Michael et al. , 2009 ). This approach aligns with theoretical work in the learning sciences that indicates that providing students with conceptual frameworks improves their ability to integrate and retrieve knowledge ( National Academies of Sciences, Engineering, and Medicine, 2018 ).

Before the start of our design-based project, we had been using Modell’s (2000) general models to guide our instruction. In this essay, we will focus on how we used the general models of mass and heat flow and transport across membranes in our instruction. These two models together describe how materials flow down gradients (e.g., pressure gradients, electrochemical gradients) against sources of resistance (e.g., tube diameter, channel frequency). We call this flux reasoning. We emphasized the fundamental nature and broad utility of flux reasoning in lecture and lab and frequently highlighted when it could be applied to explain a phenomenon. We also developed a conceptual scaffold (the Flux Reasoning Tool) that students could use to reason about physiological processes involving flux.

Although these instructional approaches had improved students’ understanding of flux phenomena, we found that students often demonstrated little commitment to using flux broadly across physiological contexts. Instead, they considered flux to be just another fact to memorize and applied it to narrow circumstances (e.g., they would use flux to reason about ions flowing across membranes—the context where flux was first introduced—but not the bulk flow of blood in a vessel). Students also struggled to integrate the various components of flux (e.g., balancing chemical and electrical gradients, accounting for variable resistance). We saw these issues reflected in students’ lower than hoped for exam scores on the cumulative final of the course. From these experiences, and from conversations with other physiology instructors, we identified a learning problem to address through design-based research: How do students learn to use flux reasoning to explain material flows in multiple physiology contexts?

The process of identifying a learning problem usually emerges from a researcher’s own experiences (in or outside a classroom) or from previous research that has been described in the literature ( Cobb et al. , 2003 ). To remain true to Sandoval’s first epistemic commitment, a learning problem must advance a theory of learning ( Edelson, 2002 ; McKenney and Reeves, 2013 ). In our work, we investigated how conceptual frameworks based on fundamental scientific concepts (i.e., Modell’s general models) could help students reason productively about physiology phenomena (National Academies of Sciences, Engineering, and Medicine, 2018; Modell, 2000 ). Our specific theoretical question was: Can we characterize how students’ conceptual frameworks around flux change as they work toward robust ideas? Sandoval’s second epistemic commitment stated that a learning problem must aim to improve student learning outcomes. The practical significance of our learning problem was: Does using the concept of flux as a foundational idea for instructional tools increase students’ learning of physiological phenomena?

We investigated our learning problem in an introductory biology course at a large R1 institution. The introductory course is the third in a biology sequence that focuses on plant and animal physiology. The course typically serves between 250 and 600 students in their sophomore or junior years each term. Classes have the following average demographics: 68% male, 21% from lower-income situations, 12% from an underrepresented minority, and 26% first-generation college students.

Design-Based Research Cycle 1, Phase 1: Designing Instructional Tools

The first phase of design-based research involves developing instructional tools that address both the theoretical and practical concerns of the learning problem ( Edelson, 2002 ; Wang and Hannafin, 2005 ). These instructional tools can take many forms, such as specific instructional strategies, classroom worksheets and practices, or technological software, as long as they embody the underlying learning theory being investigated. They must also produce classroom experiences or materials that can be evaluated to determine whether learning outcomes were met ( Sandoval, 2014 ). Indeed, this alignment between theory, the nature of the instructional tools, and the ways students are assessed is central to ensuring rigorous design-based research ( Hoadley, 2004 ; Sandoval, 2014 ). Taken together, the instructional tools instantiate a hypothesized learning environment that will advance both the theoretical and practical questions driving the research ( Barab, 2014 ).

In our work, the theoretical claim that instruction based on fundamental scientific concepts would support students’ flux reasoning was embodied in our instructional approach by being the central focus of all instructional materials, which included: a revised version of the Flux Reasoning Tool ( Figure 2 ); case study–based units in lecture that explicitly emphasized flux phenomena in real-world contexts ( Windschitl et al. , 2012 ; Scott et al. , 2018 ; Figure 3 ); classroom activities in which students practiced using flux to address physiological scenarios; links to online videos describing key flux-related concepts; constructed-response assessment items that cued students to use flux reasoning in their thinking; and pretest/posttest formative assessment questions that tracked student learning ( Figure 4 ).

FIGURE 2. The Flux Reasoning Tool given to students at the beginning of the quarter.

FIGURE 3. An example flux case study that is presented to students at the beginning of the neurophysiology unit. Throughout the unit, students learn how ion flows into and out of cells, as mediated by chemical and electrical gradients and various ion/molecular channels, sends signals throughout the body. They use this information to better understand why Jaime experiences persistent neuropathy. Images from: uz.wikipedia.org/wiki/Fayl:Blausen_0822_SpinalCord.png and commons.wikimedia.org/wiki/File:Figure_38_01_07.jpg.

FIGURE 4. An example flux assessment question about ion flows given in a pre-unit/post-unit formative assessment in the neurophysiology unit.

Phase 2: Testing the Instructional Tools

In the second phase of design-based research, the instructional tools are tested by implementing them in classrooms. During this phase, the instructional tools are placed “in harm’s way … in order to expose the details of the process to scrutiny” ( Cobb et al. , 2003 , p. 10). In this way, researchers and instructors test how the tools perform in real-world settings, which may differ considerably from the design team’s initial expectations ( Hoadley, 2004 ). During this phase, if necessary, the design team may make adjustments to the tools as they are being used to account for these unanticipated conditions ( Collins et al. , 2004 ).

We implemented the instructional tools during the Autumn and Spring quarters of the 2016–2017 academic year. Students were taught to use the Flux Reasoning Tool at the beginning of the term in the context of the first case study unit focused on neurophysiology. Each physiology unit throughout the term was associated with a new concept-based case study (usually about flux) that framed the context of the teaching. Embedded within the daily lectures were classroom activities in which students could practice using flux. Students were also assigned readings from the textbook and videos related to flux to watch during each unit. Throughout the term, students took five exams that each contained some flux questions as well as some pre- and post-unit formative assessment questions. During Winter quarter, we conducted clinical interviews with students who would take our course in the Spring term (i.e., “pre” data) as well as students who had just completed our course in Autumn (i.e., “post” data).

Phase 3: Evaluating the Instructional Tools

The third phase of a design-based research cycle involves evaluating the effectiveness of instructional tools using evidence of student learning ( Barab and Squire, 2004 ; Anderson and Shattuck, 2012 ). This can be done using products produced by students (e.g., homework, lab reports), attitudinal gains measured with surveys, participation rates in activities, interview testimonials, classroom discourse practices, and formative assessment or exam data (e.g., Reiser et al. , 2001 ; Cobb et al. , 2003 ; Barab and Squire, 2004 ; Mohan et al. , 2009 ). Regardless of the source, evidence must be in a form that supports a systematic analysis that could be scrutinized by other researchers ( Cobb et al. , 2003 ; Barab, 2014 ). Also, because design-based research often involves multiple data streams, researchers may need to use both quantitative and qualitative analytical methods to produce a rich picture of how the instructional tools affected student learning ( Collins et al. , 2004 ; Anderson and Shattuck, 2012 ).

In our work, we used the quality of students’ written responses on exams and formative assessment questions to determine whether students improved their understanding of physiological phenomena involving flux. For each assessment question, we analyzed a subset of student’s pretest answers to identify overarching patterns in students’ reasoning about flux, characterized these overarching patterns, then ordinated the patterns into different levels of sophistication. These became our scoring rubrics, which identified five different levels of student reasoning about flux. We used the rubrics to code the remainder of students’ responses, with a code designating the level of student reasoning associated with a particular reasoning pattern. We used this ordinal rubric format because it would later inform our theoretical understanding of how students build flux conceptual frameworks (see phase 4). This also allowed us to both characterize the ideas students held about flux phenomena and identify the frequency distribution of those ideas in a class.

By analyzing changes in the frequency distributions of students’ ideas across the rubric levels at different time points in the term (e.g., pre-unit vs. post-unit), we could track both the number of students who gained more sophisticated ideas about flux as the term progressed and the quality of those ideas. If the frequency of students reasoning at higher levels increased from pre-unit to post-unit assessments, we could conclude that our instructional tools as a whole were supporting students’ development of sophisticated flux ideas. For example, on one neuromuscular ion flux assessment question in the Spring of 2017, we found that relatively more students were reasoning at the highest levels of our rubric (i.e., levels 4 and 5) on the post-unit test compared with the pre-unit test. This meant that more students were beginning to integrate sophisticated ideas about flux (i.e., they were balancing concentration and electrical gradients) in their reasoning about ion movement.

To help validate this finding, we drew on three additional data streams: 1) from in-class group recordings of students working with flux items, we noted that students increasingly incorporated ideas about gradients and resistance when constructing their explanations as the term progressed; 2) from plant assessment items in the latter part of the term, we began to see students using flux ideas unprompted; and 3) from interviews, we observed that students who had already taken the course used flux ideas in their reasoning.

Through these analyses, we also noticed an interesting pattern in the pre-unit test data for Spring 2017 when compared with the frequency distribution of students’ responses with a previous term (Autumn 2016). In Spring 2017, 42% of students reasoned at level 4 or 5 on the pre-unit test, indicating these students already had sophisticated ideas about ion flux before they took the pre-unit assessment. This was surprising, considering only 2% of students reasoned at these levels for this item on the Autumn 2016 pre-unit test.

Phase 4: Reflecting on the Instructional Tools and Their Implementation

The final phase of a design-based research cycle involves a retrospective analysis that addresses the epistemic commitments of this methodology: How was the theory underpinning the research advanced by the research endeavor (theoretical outcome)? Did the instructional tools support student learning about the learning problem (practical outcome)? What were the critical features of the design solution that supported student learning (design principles)? ( Cobb et al. , 2003 ; Barab and Squire, 2004 ).

Theoretical Outcome (Epistemic Commitment 1).

Reflecting on how a design-based research experiment advances theory is critical to our understanding of how students learn in educational settings ( Barab and Squire, 2004 ; Mohan et al. , 2009 ). In our work, we aimed to characterize how students’ conceptual frameworks around flux change as they work toward robust ideas. To do this, we drew on learning progression research as our theoretical framing ( NRC, 2007 ; Corcoran et al. , 2009 ; Duschl et al. , 2011 ; Scott et al. , 2019 ). Learning progression frameworks describe empirically derived patterns in student thinking that are ordered into levels representing cognitive shifts in the ways students conceive a topic as they work toward mastery ( Gunckel et al. , 2012 ). We used our ion flux scoring rubrics to create a preliminary five-level learning progression framework ( Table 1 ). The framework describes how students’ ideas about flux often start with teleological-driven accounts at the lowest level (i.e., level 1), shift to focusing on driving forces (e.g., concentration gradients, electrical gradients) in the middle levels, and arrive at complex ideas that integrate multiple interacting forces at the higher levels. We further validated these reasoning patterns with our student interviews. However, our flux conceptual framework was largely based on student responses to our ion flux assessment items. Therefore, to further validate our learning progression framework, we needed a greater diversity of flux assessment items that investigated student thinking more broadly (i.e., about bulk flow, water movement) across physiological systems.

Practical Outcome (Epistemic Commitment 2).

In design-based research, learning theories must “do real work” by improving student learning in real-world settings ( DBR Collective, 2003 ). Therefore, design-based researchers must reflect on whether or not the data they collected show evidence that the instructional tools improved student learning ( Cobb et al. , 2003 ; Sharma and McShane, 2008 ). We determined whether our flux-based instructional approach aided student learning by analyzing the kinds of answers students provided to our assessment questions. Specifically, we considered students who reasoned at level 4 or above as demonstrating productive flux reasoning. Because almost half of students were reasoning at level 4 or 5 on the post-unit assessment after experiencing the instructional tools in the neurophysiology unit (in Spring 2017), we concluded that our tools supported student learning in physiology. Additionally, we noticed that students used language in their explanations that directly tied to the Flux Reasoning Tool ( Figure 2 ), which instructed them to use arrows to indicate the magnitude and direction of gradient-driving forces. For example, in a posttest response to our ion flux item ( Figure 4 ), one student wrote:

Ion movement is a function of concentration and electrical gradients . Which arrow is stronger determines the movement of K+. We can make the electrical arrow bigger and pointing in by making the membrane potential more negative than Ek [i.e., potassium’s equilibrium potential]. We can make the concentration arrow bigger and pointing in by making a very strong concentration gradient pointing in.

Given that almost half of students reasoned at level 4 or above, and that students used language from the Flux Reasoning Tool, we concluded that using fundamental concepts was a productive instructional approach for improving student learning in physiology and that our instructional tools aided student learning. However, some students in the 2016–2017 academic year continued to apply flux ideas more narrowly than intended (i.e., for ion and simple diffusion cases, but not water flux or bulk flow). This suggested that students had developed nascent flux conceptual frameworks after experiencing the instructional tools but could use more support to realize the broad applicability of this principle. Also, although our cross-sectional interview approach demonstrated how students’ ideas, overall, could change after experiencing the instructional tools, it did not provide information about how a student developed flux reasoning.

Reflecting on practical outcomes also means interpreting any learning gains in the context of the learning ecology. This reflection allowed us to identify whether there were particular aspects of the instructional tools that were better at supporting learning than others ( DBR Collective, 2003 ). Indeed, this was critical for our understanding why 42% of students scored at level 3 and above on the pre-unit ion assessment in the Spring of 2017, while only 2% of students scored level 3 and above in Autumn of 2016. When we reviewed notes of the Spring 2017 implementation scheme, we saw that the pretest was due at the end of the first day of class after students had been exposed to ion flux ideas in class and in a reading/video assignment about ion flow, which may be one reason for the students’ high performance on the pretest. Consequently, we could not tell whether students’ initial high performance was due to their learning from the activities in the first day of class or for other reasons we did not measure. It also indicated we needed to close pretests before the first day of class for a more accurate measure of students’ incoming ideas and the effectiveness of the instructional tools employed at the beginning of the unit.

Design Principles (Epistemic Commitment 3).

Although design-based research is enacted in local contexts (i.e., a particular classroom), its purpose is to inform learning ecologies that have broad applications to improve learning and teaching ( Edelson, 2002 ; Cobb et al. , 2003 ). Therefore, design-based research should produce design principles that describe characteristics of learning environments that researchers and instructors can use to develop instructional tools specific to their local contexts (e.g., Edelson, 2002 ; Subramaniam et al. , 2015 ). Consequently, the design principles must balance specificity with adaptability so they can be used broadly to inform instruction ( Collins et al. , 2004 ; Barab, 2014 ).

From our first cycle of design-based research, we developed the following design principles: 1) Key scientific concepts should provide an overarching framework for course organization. This way, the individual components that make up a course, like instructional units, activities, practice problems, and assessments, all reinforce the centrality of the key concept. 2) Instructional tools should explicitly articulate the principle of interest, with specific guidance on how that principle is applied in context. This stresses the applied nature of the principle and that it is more than a fact to be memorized. 3) Instructional tools need to show specific instances of how the principle is applied in multiple contexts to combat students’ narrow application of the principle to a limited number of contexts.

Design-Based Research Cycle 2, Phase 1: Redesign and Refine the Experiment

The last “epistemic commitment” Sandoval (2014) articulated was that design-based research be an iterative process with an eye toward continually refining the instructional tools, based on evidence of student learning, to produce more robust learning environments. By viewing educational inquiry as formative research, design-based researchers recognize the difficulty in accounting for all variables that could impact student learning, or the implementation of the instructional tools, a priori ( Collins et al. , 2004 ). Robust instructional designs are the products of trial and error, which are strengthened by a systematic analysis of how they perform in real-world settings.

To continue to advance our work investigating student thinking using the principle of flux, we began a second cycle of design-based research that continued to address the learning problem of helping students reason with fundamental scientific concepts. In this cycle, we largely focused on broadening the number of physiological systems that had accompanying formative assessment questions (i.e., beyond ion flux), collecting student reasoning from a more diverse population of students (e.g., upper division, allied heath, community college), and refining and validating the flux learning progression with both written and interview data in a student through time. We developed a suite of constructed-response flux assessment questions that spanned neuromuscular, cardiovascular, respiratory, renal, and plant physiological contexts and asked students about several kinds of flux: ion movement, diffusion, water movement, and bulk flow (29 total questions; available at beyondmultiplechoice.org). This would provide us with rich qualitative data that we could use to refine the learning progression. We decided to administer written assessments and conduct interviews in a pretest/posttest manner at the beginning and end of each unit both as a way to increase our data about student reasoning and to provide students with additional practice using flux reasoning across contexts.

From this second round of designing instructional tools (i.e., broader range of assessment items), testing them in the classroom (i.e., administering the assessment items to diverse student populations), evaluating the tools (i.e., developing learning progression–aligned rubrics across phenomena from student data, tracking changes in the frequency distribution of students across levels through time), and reflecting on the tools’ success, we would develop a more thorough and robust characterization of how students use flux across systems that could better inform our creation of new instructional tools to support student learning.

HOW CAN DESIGN-BASED RESEARCH EXTEND AND ENRICH BER?

While design-based research has primarily been used in educational inquiry at the K–12 level (see Reiser et al. , 2001 ; Mohan et al. , 2009 ; Jin and Anderson, 2012 ), other science disciplines at undergraduate institutions have begun to employ this methodology to create robust instructional approaches (e.g., Szteinberg et al. , 2014 in chemistry; Hake, 2007 , and Sharma and McShane, 2008 , in physics; Kelly, 2014 , in engineering). Our own work, as well as that by Zagallo et al. (2016) , provides two examples of how design-based research could be implemented in BER. Below, we articulate some of the ways incorporating design-based research into BER could extend and enrich this field of educational inquiry.

Design-Based Research Connects Theory with Practice

One critique of BER is that it does not draw heavily enough on learning theories from other disciplines like cognitive psychology or the learning sciences to inform its research ( Coley and Tanner, 2012 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Davidesco and Milne, 2019 ). For example, there has been considerable work in BER developing concept inventories as formative assessment tools that identify concepts students often struggle to learn (e.g., Marbach-Ad et al. , 2009 ; McFarland et al. , 2017 ; Summers et al. , 2018 ). However, much of this work is detached from a theoretical understanding of why students hold misconceptions in the first place, what the nature of their thinking is, and the learning mechanisms that would move students to a more productive understanding of domain ideas ( Alonzo, 2011 ). Using design-based research to understand the basis of students’ misconceptions would ground these practical learning problems in a theoretical understanding of the nature of student thinking (e.g., see Coley and Tanner, 2012 , 2015 ; Gouvea and Simon, 2018 ) and the kinds of instructional tools that would best support the learning process.

Design-Based Research Fosters Collaborations across Disciplines

Recently, there have been multiple calls across science, technology, engineering, and mathematics education fields to increase collaborations between BER and other disciplines so as to increase the robustness of science education research at the collegiate level ( Coley and Tanner, 2012 ; NRC, 2012 ; Talanquer, 2014 ; Dolan, 2015 ; Peffer and Renken, 2016 ; Mestre et al. , 2018 ; Davidesco and Milne, 2019 ). Engaging in design-based research provides both a mechanism and a motivation for fostering interdisciplinary collaborations, as it requires the design team to have theoretical knowledge of how students learn, domain knowledge of practical learning problems, and instructional knowledge for how to implement instructional tools in the classroom ( Edelson, 2002 ; Hoadley, 2004 ; Wang and Hannafin, 2005 ; Anderson and Shattuck, 2012 ). For example, in our current work, our research team consists of two discipline-based education learning scientists from an R1 institution, two physiology education researchers/instructors (one from an R1 institution the other from a community college), several physiology disciplinary experts/instructors, and a K–12 science education expert.

Design-based research collaborations have several distinct benefits for BER: first, learning or cognitive scientists could provide theoretical and methodological expertise that may be unfamiliar to biology education researchers with traditional science backgrounds ( Lo et al. , 2019 ). This would both improve the rigor of the research project and provide biology education researchers with the opportunity to explore ideas and methods from other disciplines. Second, collaborations between researchers and instructors could help increase the implementation of evidence-based teaching practices by instructors/faculty who are not education researchers and would benefit from support while shifting their instructional approaches ( Eddy et al. , 2015 ). This may be especially true for community college and primarily undergraduate institution faculty who often do not have access to the same kinds of resources that researchers and instructors at research-intensive institutions do ( Schinske et al. , 2017 ). Third, making instructors an integral part of a design-based research project ensures they are well versed in the theory and learning objectives underlying the instructional tools they are implementing in the classroom. This can improve the fidelity of implementation of the instructional tools, because the instructors understand the tools’ theoretical and practical purposes, which has been cited as one reason there have been mixed results on the impact of active learning across biology classes ( Andrews et al. , 2011 ; Borrego et al. , 2013 ; Lee et al. , 2018 ; Offerdahl et al. , 2018 ). It also gives instructors agency to make informed adjustments to the instructional tools during implementation that improve their practical applications while remaining true to the goals of the research ( Hoadley, 2004 ).

Design-Based Research Invites Using Mixed Methods to Analyze Data

The diverse nature of the data that are often collected in design-based research can require both qualitative and quantitative methodologies to produce a rich picture of how the instructional tools and their implementation influenced student learning ( Anderson and Shattuck, 2012 ). Using mixed methods may be less familiar to biology education researchers who were primarily trained in quantitative methods as biologists ( Lo et al. , 2019 ). However, according to Warfa (2016 , p. 2), “Integration of research findings from quantitative and qualitative inquiries in the same study or across studies maximizes the affordances of each approach and can provide better understanding of biology teaching and learning than either approach alone.” Although the number of BER studies using mixed methods has increased over the past decade ( Lo et al. , 2019 ), engaging in design-based research could further this trend through its collaborative nature of bringing social scientists together with biology education researchers to share research methodologies from different fields. By leveraging qualitative and quantitative methods, design-based researchers unpack “mechanism and process” by characterizing the nature of student thinking rather than “simply reporting that differences did or did not occur” ( Barab, 2014 , p. 158), which is important for continuing to advance our understanding of student learning in BER ( Dolan, 2015 ; Lo et al. , 2019 ).

CHALLENGES TO IMPLEMENTING DESIGN-BASED RESEARCH IN BER

As with any methodological approach, there can be challenges to implementing design-based research. Here, we highlight three that may be relevant to BER.

Collaborations Can Be Difficult to Maintain

While collaborations between researchers and instructors offer many affordances (as discussed earlier), the reality of connecting researchers across departments and institutions can be challenging. For example, Peffer and Renken (2016) noted that different traditions of scholarship can present barriers to collaboration where there is not mutual respect for the methods and ideas that are part and parcel to each discipline. Additionally, Schinske et al. (2017) identified several constraints that community college faculty face for engaging in BER, such as limited time or support (e.g., infrastructural, administrative, and peer support), which could also impact their ability to form the kinds of collaborations inherent in design-based research. Moreover, the iterative nature of design-based research requires these collaborations to persist for an extended period of time. Attending to these challenges is an important part of forming the design team and identifying the different roles researchers and instructors will play in the research.

Design-Based Research Experiments Are Resource Intensive

The focus of design-based research on studying learning ecologies to uncover mechanisms of learning requires that researchers collect multiple data streams through time, which often necessitates significant temporal and financial resources ( Collins et al., 2004 ; O’Donnell, 2004 ). Consequently, researchers must weigh both practical as well as methodological considerations when formulating their experimental design. For example, investigating learning mechanisms requires that researchers collect data at a frequency that will capture changes in student thinking ( Siegler, 2006 ). However, researchers may be constrained in the number of data-collection events they can anticipate depending on: the instructor’s ability to facilitate in-class collection events or solicit student participation in extracurricular activities (e.g., interviews); the cost of technological devices to record student conversations; the time and logistical considerations needed to schedule and conduct student interviews; the financial resources available to compensate student participants; the financial and temporal costs associated with analyzing large amounts of data.

Identifying learning mechanisms also requires in-depth analyses of qualitative data as students experience various instructional tools (e.g., microgenetic methods; Flynn et al. , 2006 ; Siegler, 2006 ). The high intensity of these in-depth analyses often limits the number of students who can be evaluated in this way, which must be balanced with the kinds of generalizations researchers wish to make about the effectiveness of the instructional tools ( O’Donnell, 2004 ). Because of the large variety of data streams that could be collected in a design-based research experiment—and the resources required to collect and analyze them—it is critical that the research team identify a priori how specific data streams, and the methods of their analysis, will provide the evidence necessary to address the theoretical and practical objectives of the research (see the following section on experimental rigor; Sandoval, 2014 ). These are critical management decisions because of the need for a transparent, systematic analysis of the data that others can scrutinize to evaluate the validity of the claims being made ( Cobb et al. , 2003 ).

Concerns with Experimental Rigor

The nature of design-based research, with its use of narrative to characterize versus control experimental environments, has drawn concerns about the rigor of this methodological approach. Some have challenged its ability to produce evidence-based warrants to support its claims of learning that can be replicated and critiqued by others ( Shavelson et al. , 2003 ; Hoadley, 2004 ). This is a valid concern that design-based researchers, and indeed all education researchers, must address to ensure their research meets established standards for education research ( NRC, 2002 ).

One way design-based researchers address this concern is by “specifying theoretically salient features of a learning environment design and mapping out how they are predicted to work together to produce desired outcomes” ( Sandoval, 2014 , p. 19). Through this process, researchers explicitly show before they begin the work how their theory of learning is embodied in the instructional tools to be tested, the specific data the tools will produce for analysis, and what outcomes will be taken as evidence for success. Moreover, by allowing instructional tools to be modified during the testing phase as needed, design-based researchers acknowledge that it is impossible to anticipate all aspects of the classroom environment that might impact the implementation of instructional tools, “as dozens (if not millions) of factors interact to produce the measureable outcomes related to learning” ( Hoadley, 2004 , p. 204; DBR Collective, 2003 ). Consequently, modifying instructional tools midstream to account for these unanticipated factors can ensure they retain their methodological alignment with the underlying theory and predicted learning outcomes so that inferences drawn from the design experiment accurately reflect what was being tested ( Edelson, 2002 ; Hoadley, 2004 ). Indeed, Barab (2014) states, “the messiness of real-world practice must be recognized, understood, and integrated as part of the theoretical claims if the claims are to have real-world explanatory value” (p. 153).

CONCLUSIONS

providing a methodology that integrates theories of learning with practical experiences in classrooms,

using a range of analytical approaches that allow for researchers to uncover the underlying mechanisms of student thinking and learning,

fostering interdisciplinary collaborations among researchers and instructors, and

characterizing learning ecologies that account for the complexity involved in student learning

By employing this methodology from the learning sciences, biology education researchers can enrich our current understanding of what is required to help biology students achieve their personal and professional aims during their college experience. It can also stimulate new ideas for biology education that can be discussed and debated in our research community as we continue to explore and refine how best to serve the students who pass through our classroom doors.

1 “Epistemic commitment” is defined as engaging in certain practices that generate knowledge in an agreed-upon way.

ACKNOWLEDGMENTS

We thank the UW Biology Education Research Group’s (BERG) feedback on drafts of this essay as well as Dr. L. Jescovich for last-minute analyses. This work was supported by a National Science Foundation award (NSF DUE 1661263/1660643). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the NSF. All procedures were conducted in accordance with approval from the Institutional Review Board at the University of Washington (52146) and the New England Independent Review Board (120160152).

• Alonzo, A. C. ( 2011 ). Learning progressions that support formative assessment practices . Measurement , 9 (2/3), 124–129. Google Scholar
• Anderson, T., & Shattuck, J. ( 2012 ). Design-based research: A decade of progress in education research? Educational Researcher , 41 (1), 16–25. Google Scholar
• Andrews, T. M., Leonard, M. J., Colgrove, C. A., & Kalinowski, S. T. ( 2011 ). Active learning not associated with student learning in a random sample of college biology courses . CBE—Life Sciences Education , 10 (4), 394–405. Link ,  Google Scholar
• Badenhorst, E., Hartman, N., & Mamede, S. ( 2016 ). How biomedical misconceptions may arise and affect medical students’ learning: A review of theoretical perspectives and empirical evidence . Health Professions Education , 2 (1), 10–17. Google Scholar
• Barab, S. ( 2014 ). Design-based research: A methodological toolkit for engineering change . In The Cambridge handbook of the learning sciences (2nd ed., pp. 151–170). Cambridge University Press. https://doi.org/10.1017/CBO9781139519526.011 Google Scholar
• Barab, S., & Squire, K. ( 2004 ). Design-based research: Putting a stake in the ground . Journal of the Learning Sciences , 13 (1), 1–14. Google Scholar
• Bell, P. ( 2004 ). On the theoretical breadth of design-based research in education . Educational Psychologist , 39 (4), 243–253. Google Scholar
• Borrego, M., Cutler, S., Prince, M., Henderson, C., & Froyd, J. E. ( 2013 ). Fidelity of implementation of research-based instructional strategies (RBIS) in engineering science courses . Journal of Engineering Education , 102 (3), 394–425. Google Scholar
• Brown, A. L. ( 1992 ). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings . Journal of the Learning Sciences , 2 (2), 141–178. Google Scholar
• Chi, M. T. H., Roscoe, R. D., Slotta, J. D., Roy, M., & Chase, C. C. ( 2012 ). Misconceived causal explanations for emergent processes . Cognitive Science , 36 (1), 1–61. Medline ,  Google Scholar
• Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. ( 2003 ). Design experiments in educational research . Educational Researcher , 32 (1), 9–13. Google Scholar
• Coley, J. D., & Tanner, K. D. ( 2012 ). Common origins of diverse misconceptions: Cognitive principles and the development of biology thinking . CBE—Life Sciences Education , 11 (3), 209–215. Link ,  Google Scholar
• Coley, J. D., & Tanner, K. ( 2015 ). Relations between intuitive biological thinking and biological misconceptions in biology majors and nonmajors . CBE—Life Sciences Education , 14 (1). https://doi.org/10.1187/cbe.14-06-0094 Medline ,  Google Scholar
• Collins, A., Joseph, D., & Bielaczyc, K. ( 2004 ). Design research: Theoretical and methodological issues . Journal of the Learning Sciences , 13 (1), 15–42. Google Scholar
• Corcoran, T., Mosher, F. A., & Rogat, A. D. ( 2009 ). Learning progressions in science: An evidence-based approach to reform (CPRE Research Report No. RR-63) . Philadelphia, PA: Consortium for Policy Research in Education. Google Scholar
• Davidesco, I., & Milne, C. ( 2019 ). Implementing cognitive science and discipline-based education research in the undergraduate science classroom . CBE—Life Sciences Education , 18 (3), es4. Link ,  Google Scholar
• Design-Based Research Collective . ( 2003 ). Design-based research: An emerging paradigm for educational inquiry . Educational Researcher , 32 (1), 5–8. Google Scholar
• Dolan, E. L. ( 2015 ). Biology education research 2.0 . CBE—Life Sciences Education , 14 (4), ed1. Link ,  Google Scholar
• Duschl, R., Maeng, S., & Sezen, A. ( 2011 ). Learning progressions and teaching sequences: A review and analysis . Studies in Science Education , 47 (2), 123–182. Google Scholar
• Eddy, S. L., Converse, M., & Wenderoth, M. P. ( 2015 ). PORTAAL: A classroom observation tool assessing evidence-based teaching practices for active learning in large science, technology, engineering, and mathematics classes . CBE—Life Sciences Education , 14 (2), ar23. Link ,  Google Scholar
• Edelson, D. C. ( 2002 ). Design research: What we learn when we engage in design . Journal of the Learning Sciences , 11 (1), 105–121. Google Scholar
• Flynn, E., Pine, K., & Lewis, C. ( 2006 ). The microgenetic method—Time for change? The Psychologist , 19 (3), 152–155. Google Scholar
• Gouvea, J. S., & Simon, M. R. ( 2018 ). Challenging cognitive construals: A dynamic alternative to stable misconceptions . CBE—Life Sciences Education , 17 (2), ar34. Link ,  Google Scholar
• Gunckel, K. L., Mohan, L., Covitt, B. A., & Anderson, C. W. ( 2012 ). Addressing challenges in developing learning progressions for environmental science literacy . In Alonzo, A. C.Gotwals, A. W. (Eds.), Learning progressions in science: Current challenges and future directions (pp. 39–75). Rotterdam: SensePublishers. https://doi.org/10.1007/978-94-6091-824-7_4 Google Scholar
• Hake, R. R. ( 2007 ). Design-based research in physics education research: A review . In Kelly, A. E.Lesh, R. A.Baek, J. Y. (Eds.), Handbook of design research methods in mathematics, science, and technology education (p. 24). New York: Routledge. Google Scholar
• Hoadley, C. M. ( 2004 ). Methodological alignment in design-based research . Educational Psychologist , 39 (4), 203–212. Google Scholar
• Jackson, M., Tran, A., Wenderoth, M. P., & Doherty, J. H. ( 2018 ). Peer vs. self-grading of practice exams: Which is better? CBE—Life Sciences Education , 17 (3), es44. https://doi.org/10.1187/cbe.18-04-0052 Link ,  Google Scholar
• Jin, H., & Anderson, C. W. ( 2012 ). A learning progression for energy in socio-ecological systems . Journal of Research in Science Teaching , 49 (9), 1149–1180. Google Scholar
• Joseph, D. ( 2004 ). The practice of design-based research: Uncovering the interplay between design, research, and the real-world context . Educational Psychologist , 39 (4), 235–242. Google Scholar
• Kelly, A. E. ( 2014 ). Design-based research in engineering education . In Cambridge handbook of engineering education research (pp. 497–518). New York, NY: Cambridge University Press. https://doi.org/10.1017/CBO9781139013451.032 Google Scholar
• Lee, C. J., Toven-Lindsey, B., Shapiro, C., Soh, M., Mazrouee, S., Levis-Fitzgerald, M., & Sanders, E. R. ( 2018 ). Error-discovery learning boosts student engagement and performance, while reducing student attrition in a bioinformatics course . CBE—Life Sciences Education , 17 (3), ar40. https://doi.org/10.1187/cbe.17-04-0061 Link ,  Google Scholar
• Lo, S. M., Gardner, G. E., Reid, J., Napoleon-Fanis, V., Carroll, P., Smith, E., & Sato, B. K. ( 2019 ). Prevailing questions and methodologies in biology education research: A longitudinal analysis of research in CBE — life sciences education and at the society for the advancement of biology education research . CBE—Life Sciences Education , 18 (1), ar9. Link ,  Google Scholar
• Marbach-Ad, G., Briken, V., El-Sayed, N. M., Frauwirth, K., Fredericksen, B., Hutcheson, S., … & Smith, A. C. ( 2009 ). Assessing student understanding of host pathogen interactions using a concept inventory . Journal of Microbiology & Biology Education , 10 (1), 43–50. Medline ,  Google Scholar
• McFarland, J. L., Price, R. M., Wenderoth, M. P., Martinková, P., Cliff, W., Michael, J. , … & Wright, A. ( 2017 ). Development and validation of the homeostasis concept inventory . CBE—Life Sciences Education , 16 (2), ar35. Link ,  Google Scholar
• McKenney, S., & Reeves, T. C. ( 2013 ). Systematic review of design-based research progress: Is a little knowledge a dangerous thing? Educational Researcher , 42 (2), 97–100. Google Scholar
• Mestre, J. P., Cheville, A., & Herman, G. L. ( 2018 ). Promoting DBER-cognitive psychology collaborations in STEM education . Journal of Engineering Education , 107 (1), 5–10. Google Scholar
• Michael, J. A. ( 2007 ). What makes physiology hard for students to learn? Results of a faculty survey . AJP: Advances in Physiology Education , 31 (1), 34–40. Medline ,  Google Scholar
• Michael, J. A., Modell, H., McFarland, J., & Cliff, W. ( 2009 ). The “core principles” of physiology: What should students understand? Advances in Physiology Education , 33 (1), 10–16. Medline ,  Google Scholar
• Middleton, J., Gorard, S., Taylor, C., & Bannan-Ritland, B. ( 2008 ). The “compleat” design experiment: From soup to nuts . In Kelly, A. E.Lesh, R. A.Baek, J. Y. (Eds.), Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching (pp. 21–46). New York, NY: Routledge. Google Scholar
• Modell, H. I. ( 2000 ). How to help students understand physiology? Emphasize general models . Advances in Physiology Education , 23 (1), S101–S107. Medline ,  Google Scholar
• Mohan, L., Chen, J., & Anderson, C. W. ( 2009 ). Developing a multi-year learning progression for carbon cycling in socio-ecological systems . Journal of Research in Science Teaching , 46 (6), 675–698. Google Scholar
• National Academies of Sciences, Engineering, and Medicine . ( 2018 ). How People Learn II: Learners, Contexts, and Cultures . Washington, DC: National Academies Press. Retrieved June 24, 2019, from https://doi.org/10.17226/24783 Google Scholar
• National Research Council (NRC) . ( 2002 ). Scientific research in education . Washington, DC: National Academies Press. Retrieved January 31, 2019, from https://doi.org/10.17226/10236 Google Scholar
• NRC . ( 2007 ). Taking science to school: Learning and teaching science in grades K–8 . Washington, DC: National Academies Press. Retrieved March 22, 2019, from www.nap.edu/catalog/11625/taking-science-to-school-learning-and-teaching-science-in-grades . https://doi.org/10.17226/11625 Google Scholar
• NRC . ( 2012 ). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering . Washington, DC: National Academies Press. Retrieved from www.nap.edu/catalog/13362/discipline-based-education-research-understanding-and-improving-learning-in-undergraduate . https://doi.org/10.17226/13362 Google Scholar
• NRC . ( 2018 ). How people learn II: Learners, contexts, and cultures . Washington, DC: National Academies Press. Retrieved from www.nap.edu/read/24783/chapter/7 . https://doi.org/10.17226/24783 Google Scholar
• O’Donnell, A. M. ( 2004 ). A commentary on design research . Educational Psychologist , 39 (4), 255–260. Google Scholar
• Offerdahl, E. G., McConnell, M., & Boyer, J. ( 2018 ). Can I have your recipe? Using a fidelity of implementation (FOI) framework to identify the key ingredients of formative assessment for learning . CBE—Life Sciences Education , 17 (4), es16. Link ,  Google Scholar
• Peffer, M., & Renken, M. ( 2016 ). Practical strategies for collaboration across discipline-based education research and the learning sciences . CBE—Life Sciences Education , 15 (4), es11. Link ,  Google Scholar
• Reiser, B. J., Smith, B. K., Tabak, I., Steinmuller, F., Sandoval, W. A., & Leone, A. J. ( 2001 ). BGuILE: Strategic and conceptual scaffolds for scientific inquiry in biology classrooms . In Carver, S. M.Klahr, D. (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 263–305). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Google Scholar
• Sandoval, W. ( 2014 ). Conjecture mapping: An approach to systematic educational design research . Journal of the Learning Sciences , 23 (1), 18–36. Google Scholar
• Sandoval, W. A., & Bell, P. ( 2004 ). Design-based research methods for studying learning in context: Introduction . Educational Psychologist , 39 (4), 199–201. Google Scholar
• Schinske, J. N., Balke, V. L., Bangera, M. G., Bonney, K. M., Brownell, S. E., Carter, R. S. , … & Corwin, L. A. ( 2017 ). Broadening participation in biology education research: Engaging community college students and faculty . CBE—Life Sciences Education , 16 (2), mr1. Link ,  Google Scholar
• Scott, E., Anderson, C. W., Mashood, K. K., Matz, R. L., Underwood, S. M., & Sawtelle, V. ( 2018 ). Developing an analytical framework to characterize student reasoning about complex processes . CBE—Life Sciences Education , 17 (3), ar49. https://doi.org/10.1187/cbe.17-10-0225 Link ,  Google Scholar
• Scott, E., Wenderoth, M. P., & Doherty, J. H. ( 2019 ). Learning progressions: An empirically grounded, learner-centered framework to guide biology instruction . CBE—Life Sciences Education , 18 (4), es5. https://doi.org/10.1187/cbe.19-03-0059 Link ,  Google Scholar
• Sharma, M. D., & McShane, K. ( 2008 ). A methodological framework for understanding and describing discipline-based scholarship of teaching in higher education through design-based research . Higher Education Research & Development , 27 (3), 257–270. Google Scholar
• Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. ( 2003 ). On the science of education design studies . Educational Researcher , 32 (1), 25–28. Google Scholar
• Siegler, R. S. ( 2006 ). Microgenetic analyses of learning . In Damon, W.Lerner, R. M. (Eds.), Handbook of child psychology (pp. 464–510). Hoboken, NJ: John Wiley & Sons, Inc. https://doi.org/10.1002/9780470147658.chpsy0211 Google Scholar
• Stringer, E. T. ( 2013 ). Action research . Thousand Oaks, CA: Sage Publications, Inc. Google Scholar
• Subramaniam, M., Jean, B. S., Taylor, N. G., Kodama, C., Follman, R., & Casciotti, D. ( 2015 ). Bit by bit: Using design-based research to improve the health literacy of adolescents . JMIR Research Protocols , 4 (2), e62. Medline ,  Google Scholar
• Summers, M. M., Couch, B. A., Knight, J. K., Brownell, S. E., Crowe, A. J., Semsar, K. , … & Batzli, J. ( 2018 ). EcoEvo-MAPS: An ecology and evolution assessment for introductory through advanced undergraduates . CBE—Life Sciences Education , 17 (2), ar18. Link ,  Google Scholar
• Szteinberg, G., Balicki, S., Banks, G., Clinchot, M., Cullipher, S., Huie, R. , … & Sevian, H. ( 2014 ). Collaborative professional development in chemistry education research: Bridging the gap between research and practice . Journal of Chemical Education , 91 (9), 1401–1408. Google Scholar
• Talanquer, V. ( 2014 ). DBER and STEM education reform: Are we up to the challenge? Journal of Research in Science Teaching , 51 (6), 809–819. Google Scholar
• Wang, F., & Hannafin, M. J. ( 2005 ). Design-based research and technology-enhanced learning environments . Educational Technology Research and Development , 53 (4), 5–23. Google Scholar
• Wang, J.-R. ( 2004 ). Development and validation of a Two-tier instrument to examine understanding of internal transport in plants and the human circulatory system . International Journal of Science and Mathematics Education , 2 (2), 131–157. Google Scholar
• Warfa, A.-R. M. ( 2016 ). Mixed-methods design in biology education research: Approach and uses . CBE—Life Sciences Education , 15 (4), rm5. Link ,  Google Scholar
• Windschitl, M., Thompson, J., Braaten, M., & Stroupe, D. ( 2012 ). Proposing a core set of instructional practices and tools for teachers of science . Science Education , 96 (5), 878–903. Google Scholar
• Zagallo, P., Meddleton, S., & Bolger, M. S. ( 2016 ). Teaching real data interpretation with models (TRIM): Analysis of student dialogue in a large-enrollment cell and developmental biology course . CBE—Life Sciences Education , 15 (2), ar17. Link ,  Google Scholar
• Codéveloppement d’un programme d’autogestion de la douleur chronique en ligne: un projet de recherche basé sur la conception et axé sur l’engagement des patients 12 March 2024 | Canadian Journal of Pain, Vol. 8, No. 1
• Behavioral assessment of soft skill development in a highly structured pre-health biology course for undergraduates 11 June 2024 | Journal of Microbiology & Biology Education, Vol. 32
• Enhancing undergraduates’ engagement in a learning community by including their voices in the technological and instructional design 1 Jun 2024 | Computers & Education, Vol. 214
• Practice-Based Teacher Education Benefits Graduate Trainees and Their Students Through Inclusive and Active Teaching Methods 16 October 2023 | Journal for STEM Education Research, Vol. 7, No. 1
• Developing and Validating the Preschool Nutrition Education Practices Survey 1 Apr 2024 | Journal of Nutrition Education and Behavior, Vol. 22
• A Research-Led Contribution of Engineering Education for a Sustainable Future 1 June 2024
• Designing and Evaluating Generative AI-Based Voice-Interaction Agents for Improving L2 Learners’ Oral Communication Competence 2 July 2024
• Leveraging learning experience design: digital media approaches to influence motivational traits that support student learning behaviors in undergraduate online courses 11 October 2022 | Journal of Computing in Higher Education, Vol. 35, No. 3
• Investigating an Assessment Design that Prevents Students from Using ChatGPT as the Sole Basis to Pass Assessment at the Tertiary Level 30 November 2023 | E-Journal of Humanities, Arts and Social Sciences
• Spatial Variations in Aquatic Insect Community Structure in the Winam Gulf of Lake Victoria, Kenya 8 Sep 2023 | International Journal of Ecology, Vol. 2023
• The Perceived Effectiveness of Various Forms of Feedback on the Acquisition of Technical Skills by Advanced Learners in Simulation-Based Health Professions Education 28 Aug 2023 | Cureus, Vol. 44
• Occupational therapists' acceptance of 3D printing 22 August 2023 | South African Journal of Occupational Therapy, Vol. 53, No. 2
• An app by students for students – the DPaCK-model for a digital collaborative teamwork project to identify butterflies 4 August 2023 | Frontiers in Education, Vol. 8
• Applying DBR to design protocols for synchronous online Chinese learning: An activity theoretic perspective 1 Aug 2023 | System, Vol. 116
• Defining the Nature of Augmented Feedback for Learning Intraosseous Access Skills in Simulation-Based Health Professions Education 14 Jul 2023 | Cureus, Vol. 86
• Practice-based 21st-century teacher education: Design principles for adaptive expertise 1 Jul 2023 | Teaching and Teacher Education, Vol. 128
• Undergraduate students’ neurophysiological reasoning: what we learn from the attractive distractors students select 1 Jun 2023 | Advances in Physiology Education, Vol. 47, No. 2
• Oaks to arteries: the Physiology Core Concept of flow down gradients supports transfer of student reasoning 1 Jun 2023 | Advances in Physiology Education, Vol. 47, No. 2
• Audrey Chen ,
• Kimberley A. Phillips ,
• Jennifer E. Schaefer , and
• Patrick M. Sonner
• Kyle Frantz,, Monitoring Editor
• Optimizing the Learner’s Role in Feedback: Development of a Feedback-Preparedness Online Application for Medical Students in the Clinical Setting 8 May 2023 | Cureus, Vol. 42
• History, Status, and Development of AI-Based Learning Science 8 April 2023 | SN Computer Science, Vol. 4, No. 3
• An Analytical Dashboard of Collaborative Activities for the Knowledge Building 4 March 2023 | Technology, Knowledge and Learning, Vol. 29
• The Application of a Design-Based Research Framework for Simulation-Based Education 22 Nov 2022 | Cureus, Vol. 22
• Erin Stanfield ,
• Corin D. Slown ,
• Quentin Sedlacek , and
• Suzanne E. Worcester
• James Hewlett, Monitoring Editor
• 2022 | , Vol. 511
• The effect of the e-mentoring-based education program on professional development of preschool teachers 3 July 2021 | Education and Information Technologies, Vol. 27, No. 1
• 2022 | Education Sciences, Vol. 12, No. 8
• Training Digital Competences of Educators in Continuing Education: A Three-Level Approach 27 October 2022
• Design-based research as a framework for developing and deploying augmented reality applications and scenarios for intercultural exchange 13 December 2021
• Repetition Is Important to Students and Their Understanding during Laboratory Courses That Include Research 10 Sep 2021 | Journal of Microbiology & Biology Education, Vol. 22, No. 2
• Another look at the core concepts of physiology: revisions and resources 1 Dec 2020 | Advances in Physiology Education, Vol. 44, No. 4

Submitted: 18 November 2019 Revised: 3 March 2020 Accepted: 25 March 2020

© 2020 E. E. Scott et al. CBE—Life Sciences Education © 2020 The American Society for Cell Biology. This article is distributed by The American Society for Cell Biology under license from the author(s). It is available to the public under an Attribution–Noncommercial–Share Alike 3.0 Unported Creative Commons License (http://creativecommons.org/licenses/by-nc-sa/3.0).

Last updated 27/06/24: Online ordering is currently unavailable due to technical issues. We apologise for any delays responding to customers while we resolve this. For further updates please visit our website: https://www.cambridge.org/news-and-insights/technical-incident

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings .

Login Alert

• > The Cambridge Handbook of the Learning Sciences
• > Design-Based Research

Book contents

• The Cambridge Handbook of the Learning Sciences
• Copyright page
• Contributors
• 1 An Introduction to the Learning Sciences
• Part I Foundations
• Part II Methodologies
• 9 Design-Based Research
• 10 Analyzing Collaboration
• 11 Microgenetic Methods
• 12 A Learning Sciences Perspective on the Design and Use of Assessment in Education
• 13 Learning Analytics and Educational Data Mining
• Part III Grounding Technology in the Learning Sciences
• Part IV Learning Together
• Part V Learning Disciplinary Knowledge
• Part VI Moving Learning Sciences Research into the Classroom

9 - Design-Based Research

A Methodological Toolkit for Engineering Change

from Part II - Methodologies

Published online by Cambridge University Press:  14 March 2022

Design-based research (DBR) is a methodology used to study learning in environments that are designed and systematically changed by the researcher. The goal of DBR is to engage the close study of learning as it unfolds within a particular context that contains one or more theoretically inspired innovations and then to develop new theories, artifacts, and practices that can be used to inform research and learning in other related contexts beyond the one classroom being studied. In DBR, research improves practice at the same time as it results in fundamental research findings that can be generalized. The widespread use of DBR by learning scientists demonstrates the field’s commitment to impacting classroom practice, and is consistent with a focus on complex learning environments that involve many people in situated social practices.

Access options

Save book to kindle.

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle .

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service .

• Design-Based Research
• By Sasha Barab
• Edited by R. Keith Sawyer , University of North Carolina, Chapel Hill
• Book: The Cambridge Handbook of the Learning Sciences
• Online publication: 14 March 2022
• Chapter DOI: https://doi.org/10.1017/9781108888295.012

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox .

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive .

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Design-based research: Connecting theory and practice in pharmacy educational intervention research

Affiliations.

• 1 University of North Carolina, Eshelman School of Pharmacy, 329 Beard Hall, Chapel Hill, NC 27599, United States; University of North Carolina, School of Education, 3500 Peabody Hall, Chapel Hill, NC 27599, United States.
• 2 University of North Carolina, Eshelman School of Pharmacy, 329 Beard Hall, Chapel Hill, NC 27599, United States. Electronic address: [email protected].
• PMID: 30904155
• DOI: 10.1016/j.cptl.2018.12.002

Our situation: Interventional research in pharmacy education includes the study of complex challenges that can be difficult to navigate. Design-based research (DBR), is a systematic and iterative approach to interventional research that is attentive to the practical and theoretical contributions to education. Practical contributions include the creation of novel solutions to complex problems that improve learning while theoretical contributions include refining our understanding of context-specific learning. In this paper, we describe how we addressed challenges associated with student collaboration in pharmacy education by applying DBR to bridge theory and practice.

Methodological literature review: DBR is characterized as authentic, contextually aware, collaborative, theoretically focused, methodologically diverse, practical, iterative, and operation-oriented. DBR includes three iterative phases: (1) analysis and exploration, (2) design and construction, and (3) evaluation and reflection.

Our recommendations and their applications: To integrate DBR into interventional research, scholars should work collaboratively with diverse teams of experts. DBR also requires extensive planning, a toolkit of expansive research methodologies, and attention to practical and theoretical considerations. Finally, scholars should share their work as often as possible and engage in creative exercises to promote innovative solutions to challenges in education.

Potential impact: DBR offers an approach to generate practical, theoretical, and scholarly contributions to pharmacy education research. In summary, DBR can aid pharmacy educational scholars by using a flexible, iterative, and systematic process to generate novel and creative solutions to complex problems.

Keywords: Collaboration; Design-based research; Mixed methods; Research design; Research methods.

Copyright © 2018 Elsevier Inc. All rights reserved.

PubMed Disclaimer

Similar articles

• Using multiple linear regression in pharmacy education scholarship. Olsen AA, McLaughlin JE, Harpe SE. Olsen AA, et al. Curr Pharm Teach Learn. 2020 Oct;12(10):1258-1268. doi: 10.1016/j.cptl.2020.05.017. Epub 2020 Jun 12. Curr Pharm Teach Learn. 2020. PMID: 32739064 Review.
• Conducting and presenting qualitative research in pharmacy education. Bush AA, Amechi MH. Bush AA, et al. Curr Pharm Teach Learn. 2019 Jun;11(6):638-650. doi: 10.1016/j.cptl.2019.02.030. Epub 2019 Mar 27. Curr Pharm Teach Learn. 2019. PMID: 31213322 Review.
• Management education within pharmacy curricula: A need for innovation. Mospan CM. Mospan CM. Curr Pharm Teach Learn. 2017 Mar-Apr;9(2):171-174. doi: 10.1016/j.cptl.2016.11.019. Epub 2017 Jan 30. Curr Pharm Teach Learn. 2017. PMID: 29233399
• The effectiveness of internet-based e-learning on clinician behavior and patient outcomes: a systematic review protocol. Sinclair P, Kable A, Levett-Jones T. Sinclair P, et al. JBI Database System Rev Implement Rep. 2015 Jan;13(1):52-64. doi: 10.11124/jbisrir-2015-1919. JBI Database System Rev Implement Rep. 2015. PMID: 26447007
• Building bridges between theory and practice in medical education using a design-based research approach: AMEE Guide No. 60. Dolmans DH, Tigelaar D. Dolmans DH, et al. Med Teach. 2012;34(1):1-10. doi: 10.3109/0142159X.2011.595437. Med Teach. 2012. PMID: 22250671
• Missing from the Narrative: A Seven-Decade Scoping Review of the Inclusion of Black Autistic Women and Girls in Autism Research. Lovelace TS, Comis MP, Tabb JM, Oshokoya OE. Lovelace TS, et al. Behav Anal Pract. 2021 Sep 30;15(4):1093-1105. doi: 10.1007/s40617-021-00654-9. eCollection 2022 Dec. Behav Anal Pract. 2021. PMID: 36605161 Free PMC article. Review.
• The Learning Loop: Conceptualizing Just-in-Time Faculty Development. Yilmaz Y, Papanagnou D, Fornari A, Chan TM. Yilmaz Y, et al. AEM Educ Train. 2022 Feb 1;6(1):e10722. doi: 10.1002/aet2.10722. eCollection 2022 Feb. AEM Educ Train. 2022. PMID: 35224408 Free PMC article.
• Developing a dashboard to meet Competence Committee needs: a design-based research project. Thoma B, Bandi V, Carey R, Mondal D, Woods R, Martin L, Chan T. Thoma B, et al. Can Med Educ J. 2020 Mar 16;11(1):e16-e34. doi: 10.36834/cmej.68903. eCollection 2020 Mar. Can Med Educ J. 2020. PMID: 32215140 Free PMC article.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• Elsevier Science
• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

• DOI: 10.4135/9781506326139.n195
• Corpus ID: 197417670

Design-Based Research

• M. Armstrong , Cade Dopp , +1 author Welsh
• Published 2019

Figures from this paper

5 Citations

Emerging educational design in online training in higher education, exploring artifact-generated learning with digital technologies: advancing active learning with co-design in higher education across disciplines, change management for learning analytics, design and evaluation of a game for mobile platforms about periodic properties of the chemical elements, the experience of the kingdom of saudi arabia in the field of e-learning during the coronavirus pandemic, 9 references, design-based research: an emerging paradigm for educational inquiry, design research: theoretical and methodological issues, design-based research: putting a stake in the ground, design experiments: theoretical and methodological challenges in creating complex interventions in c.

• Highly Influential

Toward a Design Science of Education

Building a better mousetrap: how design-based research was used to improve homemade powerpoint games, conducting educational design research, related papers.

Showing 1 through 3 of 0 Related Papers

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Kimberly Christensen and Richard E. West

Design-Based Research (DBR) is one of the most exciting evolutions in research methodology of our time, as it allows for the potential knowledge gained through the intimate connections designers have with their work to be combined with the knowledge derived from research. These two sources of knowledge can inform each other, leading to improved design interventions as well as improved local and generalizable theory. However, these positive outcomes are not easily attained, as DBR is also a difficult method to implement well. The good news is that we can learn much from other disciplines who are also seeking to find effective strategies for intertwining design and research. In this chapter, we will review the history of DBR as well as Interdisciplinary Design Research (IDR) and then discuss potential implications for our field.

Shared Origins with IDR

These two types of design research, both DBR and IDR, share a common genesis among the design revolution of the 1960s, where designers, researchers, and scholars sought to elevate design from mere practice to an independent scholarly discipline, with its own research and distinct theoretical and methodological underpinnings. A scholarly focus on design methods, they argued, would foster the development of design theories, which would in turn improve the quality of design and design practice (Margolin, 2010). Research on design methods, termed design research, would be the foundation of this new discipline.

Design research had existed in primitive form—as market research and process analysis—since before the turn of the 20th century, and, although it had served to improve processes and marketing, it had not been applied as scientific research. John Chris Jones, Bruce Archer, and Herbert Simon were among the first to shift the focus from research for design (e.g., research with the intent of gathering data to support product development) to research on design (e.g., research exploring the design process). Their efforts framed the initial development of design research and science.

John Chris Jones

An engineer, Jones (1970) felt that the design process was ambiguous and often too abstruse to discuss effectively. One solution, he offered, was to define and discuss design in terms of methods. By identifying and discussing design methods, researchers would be able to create transparency in the design process, combating perceptions of design being more or less mysteriously inspired. This discussion of design methods, Jones proposed, would in turn raise the level of discourse and practice in design.

Bruce Archer

Archer, also an engineer, worked with Jones and likewise supported the adoption of research methods from other disciplines. Archer (1965) proposed that applying systematic methods would improve the assessment of design problems and foster the development of effective solutions. Archer recognized, however, that improved practice alone would not enable design to achieve disciplinary status. In order to become a discipline, design required a theoretical foundation to support its practice. Archer (1981) advocated that design research was the primary means by which theoretical knowledge could be developed. He suggested that the application of systematic inquiry, such as existed in engineering, would yield knowledge about not only product and practice, but also the theory that guided each.

Herbert Simon

It was multidisciplinary social scientist Simon, however, that issued the clarion call for transforming design into design science (Buchanan, 2007; Collins, 1992; Collins, Joseph, & Bielaczyc, 2004; Cross, 1999; Cross, 2007; Friedman, 2003; Jonas, 2007; Willemien, 2009). In The Sciences of the Artificial, Simon (1969) reasoned that the rigorous inquiry and discussion surrounding naturally occurring processes and phenomena was just as necessary for man-made products and processes. He particularly called for “[bodies] of intellectually tough, analytic, partly formalizable, partly empirical, teachable doctrine about the design process” (p. 132). This call for more scholarly discussion and practice resonated with designers across disciplines in design and engineering (Buchanan, 2007; Cross, 1999; Cross, 2007; Friedman, 2003; Jonas, 2007; Willemien, 2009). IDR sprang directly from this early movement and has continued to gain momentum, producing an interdisciplinary body of research encompassing research efforts in engineering, design, and technology.

Years later, in the 1980s, Simon’s work inspired the first DBR efforts in education (Collins et al., 2004). Much of the DBR literature attributes its beginnings to the work of Ann Brown and Allan Collins (Cobb, Confrey, diSessa, Lehrer, & Schauble, 2003; Collins et al., 2004; Kelly, 2003; McCandliss, Kalchman, & Bryant, 2003; Oh & Reeves, 2010; Reeves, 2006; Shavelson, Phillips, Towne, & Feuer, 2003; Tabak, 2004; van den Akker, 1999). Their work, focusing on research and development in authentic contexts, drew heavily on research approaches and development practices in the design sciences, including the work of early design researchers such as Simon (Brown, 1992; Collins, 1992; Collins et al., 2004). However, over generations of research, this connection has been all but forgotten, and DBR, although similarly inspired by the early efforts of Simon, Archer, and Jones, has developed into an isolated and discipline-specific body of design research, independent from its interdisciplinary cousin.

Current Issues in DBR

The initial obstacle to understanding and engaging in DBR is understanding what DBR is. What do we call it? What does it entail? How do we do it? Many of the current challenges facing DBR concern these questions. Specifically, there are three issues that influence how DBR is identified, implemented, and discussed. First, proliferation of terminology among scholars and inconsistent use of these terms have created a sprawling body of literature, with various splinter DBR groups hosting scholarly conversations regarding their particular brand of DBR. Second, DBR, as a field, is characterized by a lack of definition, in terms of its purpose, its characteristics, and the steps or processes of which it is comprised. Third, the one consistent element of DBR across the field is an unwieldy set of considerations incumbent upon the researcher.

Because it is so difficult to define and conceptualize DBR, it is similarly difficult to replicate authentically. Lack of scholarly agreement on the characteristics and outcomes that define DBR withholds a structure by which DBR studies can be identified and evaluated and, ultimately, limits the degree to which the field can progress. The following sections will identify and explore the three greatest challenges facing DBR today: proliferation of terms, lack of definition, and competing demands.

Proliferation of terminology

One of the most challenging characteristics of DBR is the quantity and use of terms that identify DBR in the research literature. There are seven common terms typically associated with DBR: design experiments, design research, design-based research, formative research, development research, developmental research, and design-based implementation research.

Synonymous terms

Collins and Brown first termed their efforts design experiments (Brown, 1992; Collins, 1992). Subsequent literature stemming from or relating to Collins’ and Brown’s work used design research and design experiments synonymously (Anderson & Shattuck, 2012; Collins et al., 2004). Design-based research was introduced to distinguish DBR from other research approaches. Sandoval and Bell (2004) best summarized this as follows:

We have settled on the term design-based research over the other commonly used phrases “design experimentation,” which connotes a specific form of controlled experimentation that does not capture the breadth of the approach, or “design research,” which is too easily confused with research design and other efforts in design fields that lack in situ research components. (p. 199)

Variations by discipline

Terminology across disciplines refers to DBR approaches as formative research, development research, design experiments, and developmental research. According to van den Akker (1999), the use of DBR terminology also varies by educational sub-discipline, with areas such as (a) curriculum, (b) learning and instruction, (c) media and technology, and (d) teacher education and didactics favoring specific terms that reflect the focus of their research (Figure 1).

Figure 1. Variations in DBR terminology across educational sub-disciplines.

Lack of definition

This variation across disciplines, with design researchers tailoring design research to address discipline-specific interests and needs, has created a lack of definition in the field overall. In addition, in the literature, DBR has been conceptualized at various levels of granularity. Here, we will discuss three existing approaches to defining DBR: (a) statements of the overarching purpose, (b) lists of defining characteristics, and (c) models of the steps or processes involved.

General purpose

In literature, scholars and researchers have made multiple attempts to isolate the general purpose of design research in education, with each offering a different insight and definition. According to van den Akker (1999), design research is distinguished from other research efforts by its simultaneous commitment to (a) developing a body of design principles and methods that are based in theory and validated by research and (b) offering direct contributions to practice. This position was supported by Sandoval and Bell (2004), who suggested that the general purpose of DBR was to address the “tension between the desire for locally usable knowledge, on the one hand, and scientifically sound, generalizable knowledge on the other” (p. 199). Cobb et al. (2003) particularly promoted the theory-building focus, asserting “design experiments are conducted to develop theories, not merely to empirically tune ‘what works’” (p. 10). Shavelson et al. (2003) recognized the importance of developing theory but emphasized that the testing and building of instructional products was an equal focus of design research rather than the means to a theoretical end.

The aggregate of these definitions suggests that the purpose of DBR involves theoretical and practical design principles and active engagement in the design process. However, DBR continues to vary in its prioritization of these components, with some focusing largely on theory, others emphasizing practice or product, and many examining neither but all using the same terms.

Specific characteristics

Another way to define DBR is by identifying the key characteristics that both unite and define the approach. Unlike other research approaches, DBR can take the form of multiple research methodologies, both qualitative and quantitative, and thus cannot be recognized strictly by its methods. Identifying characteristics, therefore, concern the research process, context, and focus. This section will discuss the original characteristics of DBR, as introduced by Brown and Collins, and then identify the seven most common characteristics suggested by DBR literature overall.

Brown’s concept of DBR. Brown (1992) defined design research as having five primary characteristics that distinguished it from typical design or research processes. First, a design is engineered in an authentic, working environment. Second, the development of research and the design are influenced by a specific set of inputs: classroom environment, teachers and students as researchers, curriculum, and technology. Third, the design and development process includes multiple cycles of testing, revision, and further testing. Fourth, the design research process produces an assessment of the design’s quality as well as the effectiveness of both the design and its theoretical underpinnings. Finally, the overall process should make contributions to existing learning theory.

Collins’s concept of DBR. Collins (1990, 1992) posed a similar list of design research characteristics. Collins echoed Brown’s specifications of authentic context, cycles of testing and revision, and design and process evaluation. Additionally, Collins provided greater detail regarding the characteristics of the design research processes—specifically, that design research should include the comparison of multiple sample groups, be systematic in both its variation within the experiment and in the order of revisions (i.e., by testing the innovations most likely to succeed first), and involve an interdisciplinary team of experts including not just the teacher and designer, but technologists, psychologists, and developers as well. Unlike Brown, however, Collins did not refer to theory building as an essential characteristic.

Current DBR characteristics. The DBR literature that followed expanded, clarified, and revised the design research characteristics identified by Brown and Collins. The range of DBR characteristics discussed in the field currently is broad but can be distilled to seven most frequently referenced identifying characteristics of DBR: design driven, situated, iterative, collaborative, theory building, practical, and productive.

Design driven.  All literature identifies DBR as focusing on the evolution of a design (Anderson & Shattuck, 2012; Brown, 1992; Cobb et al., 2003; Collins, 1992; Design-Based Research Collective, 2003). While the design can range from an instructional artifact to an intervention, engagement in the design process is what yields the experience, data, and insight necessary for inquiry.

Situated.  Recalling Brown’s (1992) call for more authentic research contexts, nearly all definitions of DBR situate the aforementioned design process in a real-world context, such as a classroom (Anderson & Shattuck, 2012; Barab & Squire, 2004; Cobb et al., 2003).

Iterative. Literature also appears to agree that a DBR process does not consist of a linear design process, but rather multiple cycles of design, testing, and revision (Anderson & Shattuck, 2012; Barab & Squire, 2004; Brown, 1992; Design-Based Research Collective, 2003; Shavelson et al., 2003). These iterations must also represent systematic adjustment of the design, with each adjustment and subsequent testing serving as a miniature experiment (Barab & Squire, 2004; Collins, 1992).

Collaborative.  While the literature may not always agree on the roles and responsibilities of those engaged in DBR, collaboration between researchers, designers, and educators appears to be key (Anderson & Shattuck, 2012; Barab & Squire, 2004; McCandliss et al., 2003). Each collaborator enters the project with a unique perspective and, as each engages in research, forms a role-specific view of phenomena. These perspectives can then be combined to create a more holistic view of the design process, its context, and the developing product.

Theory building.  Design research focuses on more than creating an effective design; DBR should produce an intimate understanding of both design and theory (Anderson & Shattuck, 2012; Barab & Squire, 2004; Brown, 1992; Cobb et al., 2003; Design-Based Research Collective, 2003; Joseph, 2004; Shavelson et al., 2003). According to Barab & Squire (2004), “Design-based research requires more than simply showing a particular design works but demands that the researcher . . . generate evidence-based claims about learning that address contemporary theoretical issues and further the theoretical knowledge of the field” (p. 6). DBR needs to build and test theory, yielding findings that can be generalized to both local and broad theory (Hoadley, 2004).

Practical.  While theoretical contributions are essential to DBR, the results of DBR studies “must do real work” (Cobb et al., 2003, p. 10) and inform instructional, research, and design practice (Anderson & Shattuck, 2012; Barab & Squire, 2004; Design-Based Research Collective, 2003; McCandliss et al., 2003).

Productive.  Not only should design research produce theoretical and practical insights, but also the design itself must produce results, measuring its success in terms of how well the design meets its intended outcomes (Barab & Squire, 2004; Design-Based Research Collective, 2003; Joseph, 2004; McCandliss et al., 2003).

Steps and processes

The third way DBR could possibly be defined is to identify the steps or processes involved in implementing it. The sections below illustrate the steps outlined by Collins (1990) and Brown (1992) as well as models by Bannan-Ritland (2003), Reeves (2006), and an aggregate model presented by Anderson & Shattuck (2012).

Collins’s design experimentation steps.  In his technical report, Collins (1990) presented an extensive list of 10 steps in design experimentation (Figure 2). While Collins’s model provides a guide for experimentally testing and developing new instructional programs, it does not include multiple iterative stages or any evaluation of the final product. Because Collins was interested primarily in development, research was not given much attention in his model.

Brown’s design research example.  The example of design research Brown (1992) included in her article was limited and less clearly delineated than Collins’s model (Figure 2). Brown focused on the development of educational interventions, including additional testing with minority populations. Similar to Collins, Brown also omitted any summative evaluation of intervention quality or effectiveness and did not specify the role of research through the design process.

Bannan-Ritland’s DBR model.  Bannan-Ritland (2003) reviewed design process models in fields such as product development, instructional design, and engineering to create a more sophisticated model of design-based research. In its simplest form, Bannan-Ritland’s model is comprised of multiple processes subsumed under four broad stages: (a) informed exploration, (b) enactment, (c) evaluation of local impact, and (d) evaluation of broad impact. Unlike Collins and Brown, Bannan-Ritland dedicated large portions of the model to evaluation in terms of the quality and efficacy of the final product as well as the implications for theory and practice.

Reeves’s development research model.  Reeves (2006) provided a simplified model consisting of just four steps (Figure 2). By condensing DBR into just a few steps, Reeves highlighted what he viewed as the most essential processes, ending with a general reflection on both the process and product generated in order to develop theoretical and practical insights.

Anderson and Shattuck’s aggregate model.  Anderson and Shattuck (2012) reviewed design-based research abstracts over the past decade and, from their review, presented an eight-step aggregate model of DBR (Figure 2). As an aggregate of DBR approaches, this model was their attempt to unify approaches across DBR literature, and includes similar steps to Reeves’s model. However, unlike Reeves, Anderson and Shattuck did not include summative reflection and insight development.

Comparison of models. Following in Figure 2, we provide a comparison of all these models side-by-side.

Competing demands and roles

The third challenge facing DBR is the variety of roles researchers are expected to fulfill, with researchers often acting simultaneously as project managers, designers, and evaluators. However, with most individuals able to focus on only one task at a time, these competing demands on resources and researcher attention and faculties can be challenging to balance, and excess focus on one role can easily jeopardize others. The literature has recognized four major roles that a DBR professional must perform simultaneously: researcher, project manager, theorist, and designer.

Researcher as researcher

Planning and carrying out research is already comprised of multiple considerations, such as controlling variables and limiting bias. The nature of DBR, with its collaboration and situated experimentation and development, innately intensifies some of these issues (Hoadley, 2004). While simultaneously designing the intervention, a design-based researcher must also ensure that high-quality research is accomplished, per typical standards of quality associated with quantitative or qualitative methods.

However, research is even more difficult in DBR because the nature of the method leads to several challenges. First, it can be difficult to control the many variables at play in authentic contexts (Collins et al., 2004). Many researchers may feel torn between being able to (a) isolate critical variables or (b) study the comprehensive, complex nature of the design experience (van den Akker, 1999). Second, because many DBR studies are qualitative, they produce large amounts of data, resulting in demanding data collection and analysis (Collins et al., 2004). Third, according to Anderson and Shattuck (2012), the combination of demanding data analysis and highly invested roles of the researchers leaves DBR susceptible to multiple biases during analysis. Perhaps best expressed by Barab and Squire (2004), “if a researcher is intimately involved in the conceptualization, design, development, implementation, and researching of a pedagogical approach, then ensuring that researchers can make credible and trustworthy assertions is a challenge” (p. 10). Additionally, the assumption of multiple roles invests much of the design and research in a single person, diminishing the likelihood of replicability (Hoadley, 2004). Finally, it is impossible to document or account for all discrete decisions made by the collaborators that influenced the development and success of the design (Design-Based Research Collective, 2003).

Quality research, though, was never meant to be easy! Despite these challenges, DBR has still been shown to be effective in simultaneously developing theory through research as well as interventions that can benefit practice—the two simultaneous goals of any instructional designer.

Researcher as project manager

The collaborative nature of DBR lends the approach one of its greatest strengths: multiple perspectives. While this can be a benefit, collaboration between researchers, developers, and practitioners needs to be highly coordinated (Collins et al., 2004), because it is difficult to manage interdisciplinary teams and maintain a productive, collaborative partnership (Design-Based Research Collective, 2003).

Researcher as theorist

For many researchers in DBR, the development or testing of theory is a foundational component and primary focus of their work. However, the iterative and multi-tasking nature of a DBR process may not be well-suited to empirically testing or building theory. According to Hoadley (2004), “the treatment’s fidelity to theory [is] initially, and sometimes continually, suspect” (p. 204). This suggests that researchers, despite intentions to test or build theory, may not design or implement their solution in alignment with theory or provide enough control to reliably test the theory in question.

Researcher as designer

Because DBR is simultaneously attempting to satisfy the needs of both design and research, there is a tension between the responsibilities of the researcher and the responsibilities of the designer (van den Akker, 1999). Any design decision inherently alters the research. Similarly, research decisions place constraints on the design. Skilled design-based researchers seek to balance these competing demands effectively.

What we can learn from IDR

IDR has been encumbered by similar issues that currently exist in DBR. While IDR is by no means a perfect field and is still working to hone and clarify its methods, it has been developing for two decades longer than DBR. The history of IDR and efforts in the field to address similar issues can yield possibilities and insights for the future of DBR. The following sections address efforts in IDR to define the field that hold potential for application in DBR, including how professionals in IDR have focused their efforts to increase unity and worked to define sub-approaches more clearly.

Defining Approaches

Similar to DBR, IDR has been subject to competing definitions as varied as the fields in which design research has been applied (i.e., product design, engineering, manufacturing, information technology, etc.) (Findeli, 1998; Jonas, 2007; Schneider, 2007). Typically, IDR scholars have focused on the relationship between design and research, as well as the underlying purpose, to define the approach. This section identifies three defining conceptualizations of IDR—the prepositional approach trinity, Cross’s -ologies, and Buchanan’s strategies of productive science—and discusses possible implications for DBR.

The approach trinity

One way of defining different purposes of design research is by identifying the preposition in the relationship between research and design: research into design, research for design, and research through design (Buchanan, 2007; Cross, 1999; Findeli, 1998; Jonas, 2007; Schneider, 2007).

Jonas (2007) identified research into design as the most prevalent—and straightforward—form of IDR. This approach separates research from design practice; the researcher observes and studies design practice from without, commonly addressing the history, aesthetics, theory, or nature of design (Schneider, 2007). Research into design generally yields little or no contribution to broader theory (Findeli, 1998).

Research for design applies to complex, sophisticated projects, where the purpose of research is to foster product research and development, such as in market and user research (Findeli, 1998; Jonas, 2007). Here, the role of research is to build and improve the design, not contribute to theory or practice.

According to Jonas’s (2007) description, research through design bears the strongest resemblance to DBR and is where researchers work to shape their design (i.e., the research object) and establish connections to broader theory and practice. This approach begins with the identification of a research question and carries through the design process experimentally, improving design methods and finding novel ways of controlling the design process (Schneider, 2007). According to Findeli (1998), because this approach adopts the design process as the research method, it helps to develop authentic theories of design.

Cross’s -ologies

Cross (1999) conceived of IDR approaches based on the early drive toward a science of design and identified three bodies of scientific inquiry: epistemology, praxiology, and phenomenology. Design epistemology primarily concerns what Cross termed “designerly ways of knowing” or how designers think and communicate about design (Cross, 1999; Cross, 2007). Design praxiology deals with practices and processes in design or how to develop and improve artifacts and the processes used to create them. Design phenomenology examines the form, function, configuration, and value of artifacts, such as exploring what makes a cell phone attractive to a user or how changes in a software interface affect user’s activities within the application.

Buchanan’s strategies of productive science

Like Cross, Buchanan (2007) viewed IDR through the lens of design science and identified four research strategies that frame design inquiry: design science, dialectic inquiry, rhetorical inquiry, and productive science (Figure 2). Design science focuses on designing and decision-making, addressing human and consumer behavior. According to Buchanan (2007), dialectic inquiry examines the “social and cultural context of design; typically [drawing] attention to the limitations of the individual designer in seeking sustainable solutions to problems” (p.57). Rhetorical inquiry focuses on the design experience as well as the designer’s process to create products that are usable, useful, and desirable. Productive science studies how the potential of a design is realized through the refinement of its parts, including materials, form, and function. Buchanan (2007) conceptualized a design research—what he termed design inquiry—that includes elements of all four strategies, looking at the designer, the design, the design context, and the refinement process as a holistic experience.

Implications for DBR

While the literature has yet to accept any single approach to defining types of IDR, it may still be helpful for DBR to consider similar ways of limiting and defining sub-approaches in the field. The challenges brought on by collaboration, multiple researcher roles, and lack of sufficient focus on the design product could be addressed and relieved by identifying distinct approaches to DBR. This idea is not new. Bell and Sandoval (2004) opposed the unification of DBR, specifically design-based research, across educational disciplines (such as developmental psychology, cognitive science, and instructional design). However, they did not suggest any potential alternatives. Adopting an IDR approach, such as the approach trinity, could serve to both unite studies across DBR and clearly distinguish the purpose of the approach and its primary functions. Research into design could focus on the design process and yield valuable insights on design thinking and practice. Research for design could focus on the development of an effective product, which development is missing from many DBR approaches. Research through design would use the design process as a vehicle to test and develop theory, reducing the set of expected considerations. Any approach to dividing or defining DBR efforts could help to limit the focus of the study, helping to prevent the diffusion of researcher efforts and findings.

In this chapter we have reviewed the historical development of both design-based research and interdisciplinary design research in an effort to identify strategies in IDR that could benefit DBR development. Following are a few conclusions, leading to recommendations for the DBR field.

Improve interdisciplinary collaboration

Overall, one key advantage that IDR has had—and that DBR presently lacks—is communication and collaboration with other fields. Because DBR has remained so isolated, only rarely referencing or exploring approaches from other design disciplines, it can only evolve within the constraints of educational inquiry. IDR’s ability to conceive solutions to issues in the field is derived, in part, from a wide variety of disciplines that contribute to the body of research. Engineers, developers, artists, and a range of designers interpose their own ideas and applications, which are in turn adopted and modified by others. Fostering collaboration between DBR and IDR, while perhaps not the remedy to cure all scholarly ills, could yield valuable insights for both fields, particularly in terms of refining methodologies and promoting the development of theory.

Simplify terminology and improve consistency in use

As we identified in this paper, a major issue facing DBR is the proliferation of terminology among scholars and the inconsistency in usage. From IDR comes the useful acknowledgement that there can be research into design, for design, and through design (Buchanan, 2007; Cross, 1999; Findeli, 1998; Jonas, 2007; Schneider, 2007). This framework was useful for scholars in our conversations at the conference. A resulting recommendation, then, is that, in published works, scholars begin articulating which of these approaches they are using in that particular study. This can simplify the requirements on DBR researchers, because instead of feeling the necessity of doing all three in every paper, they can emphasize one. This will also allow us to communicate our research better with IDR scholars.

Describe DBR process in publications

Oftentimes authors publish DBR studies using the same format as regular research studies, making it difficult to recognize DBR research and learn how other DBR scholars mitigate the challenges we have discussed in this chapter. Our recommendation is that DBR scholars publish the messy findings resulting from their work and pull back the curtain to show how they balanced competing concerns to arrive at their results. We believe it would help if DBR scholars adopted more common frameworks for publishing studies. In our review of the literature, we identified the following characteristics, which are the most frequently used to identify DBR:

• DBR is design driven and intervention focused
• DBR is situated within an actual teaching/learning context
• DBR is iterative
• DBR is collaborative between researchers, designers, and practitioners
• DBR builds theory but also needs to be practical and result in useful interventions

One recommendation is that DBR scholars adopt these as the characteristics of their work that they will make explicit in every published paper so that DBR articles can be recognized by readers and better aggregated together to show the value of DBR over time. One suggestion is that DBR scholars in their methodology sections could adopt these characteristics as subheadings. So in addition to discussing data collection and data analysis, they would also discuss Design Research Type (research into, through, or of design), Description of the Design Process and Product, Design and Learning Context, Design Collaborations, and a discussion explicitly of the Design Iterations, perhaps by listing each iteration and then the data collection and analysis for each. Also in the concluding sections, in addition to discussing research results, scholars would discuss Applications to Theory (perhaps dividing into Local Theory and Outcomes and Transferable Theory and Findings) and Applications for Practice. Papers that are too big could be broken up with different papers reporting on different iterations but using this same language and formatting to make it easier to connect the ideas throughout the papers. Not all papers would have both local and transferable theory (the latter being more evident in later iterations), so it would be sufficient to indicate in a paper that local theory and outcomes were developed and met with some ideas for transferable theory that would be developed in future iterations. The important thing would be to refer to each of these main characteristics in each paper so that scholars can recognize the work as DBR, situate it appropriately, and know what to look for in terms of quality during the review process.

Application Exercises

• According to the authors, what are the major issues facing DBR and what are some things that can be done to address this problem?
• Imagine you have designed a new learning app for use in public schools. How would you go about testing it using design-based research?

Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41 (1), 16–25.

Archer, L.B. (1965). Systematic method for designers. In N. Cross (ed.), Developments in design methodology. London, England: John Wiley, 1984, pp. 57–82.

Archer, L. B. (1981). A view of the nature of design research. In R. Jacques & J.A. Powell (Eds.), Design: Science: Method (pp. 36-39). Guilford, England: Westbury House.

Bannan-Ritland, B. (2003). The role of design in research: The integrative learning design framework. Educational Researcher, 32 (1), 21 –24. doi:10.3102/0013189X032001021

Barab, S., & Squire, K. (2004). Design-based research: Putting a stake in the ground. The Journal of the Learning Sciences, 13 (1), 1–14.

Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. The Journal of the Learning Sciences, 2 (2), 141–178.

Buchanan, R. (2007). Strategies of design research: Productive science and rhetorical inquiry. In R. Michel (Ed.), Design research now (pp. 55–66). Basel, Switzerland: Birkhäuser Verlag AG.

Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32 (1), 9–13. doi:10.3102/0013189X032001009

Collins, A. (1990). Toward a Design Science of Education. Technical Report No. 1.

Collins, A. (1992). Toward a design science of education. In E. Scanlon & T. O’Shea (Eds.), New directions in educational technology. Berlin, Germany: Springer-Verlag.

Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. The Journal of the Learning Sciences, 13 (1), 15–42.

Cross, N. (1999). Design research: A disciplined conversation. Design Issues, 15 (2), 5–10. doi:10.2307/1511837

Cross, N. (2007). Forty years of design research. Design Studies, 28 (1), 1–4. doi:10.1016/j.destud.2006.11.004

Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32 (1), 5–8. doi:10.3102/0013189X032001005

Findeli, A. (1998). A quest for credibility: Doctoral education and research in design at the University of Montreal. Doctoral Education in Design, Ohio, 8–11 October 1998.

Friedman, K. (2003). Theory construction in design research: Criteria: approaches, and methods. Design Studies, 24 (6), 507–522.

Hoadley, C. M. (2004). Methodological alignment in design-based research. Educational Psychologist, 39 (4), 203–212.

Jonas, W. (2007). Design research and its meaning to the methodological development of the discipline. In R. Michel (Ed.), Design research now (pp. 187–206). Basel, Switzerland: Birkhäuser Verlag AG.

Jones, J. C. (1970). Design methods: Seeds of human futures. New York, NY: John Wiley & Sons Ltd.

Joseph, D. (2004). The practice of design-based research: uncovering the interplay between design, research, and the real-world context. Educational Psychologist, 39 (4), 235–242.

Kelly, A. E. (2003). Theme issue: The role of design in educational research. Educational Researcher, 32 (1), 3–4. doi:10.3102/0013189X032001003

Margolin, V. (2010). Design research: Towards a history. Presented at the Design Research Society Annual Conference on Design & Complexity, Montreal, Canada. Retrieved from http://www.drs2010.umontreal.ca/data/PDF/080.pdf

McCandliss, B. D., Kalchman, M., & Bryant, P. (2003). Design experiments and laboratory approaches to learning: Steps toward collaborative exchange. Educational Researcher, 32 (1), 14–16. doi:10.3102/0013189X032001014

Michel, R. (Ed.). (2007). Design research now. Basel, Switzerland: Birkhäuser Verlag AG

Oh, E., & Reeves, T. C. (2010). The implications of the differences between design research and instructional systems design for educational technology researchers and practitioners. Educational Media International, 47 (4), 263–275.

Reeves, T. C. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney, & N. Nieveen (Eds.), Educational design research (Vol. 1, pp. 52–66). London, England: Routledge.

Reigeluth, C. M., & Frick, T. W. (1999). Formative research: A methodology for creating and improving design theories. In C. Reigeluth (Ed.), Instructional-design theories and models. A new paradigm of instructional theory (Vol. 2) (pp. 633–651), Mahwah, NJ: Lawrence Erlbaum Associates.

Richey, R. C., & Nelson, W. A. (1996). Developmental research. In D. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 1213–1245), London, England: Macmillan.

Sandoval, W. A., & Bell, P. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist, 39 (4), 199–201.

Schneider, B. (2007). Design as practice, science and research. In R. Michel (Ed.), Design research now (pp. 207–218). Basel, Switzerland: Birkhäuser Verlag AG.

Shavelson, R. J., Phillips, D. C., Towne, L., & Feuer, M. J. (2003). On the science of education design studies. Educational Researcher, 32 (1), 25–28. doi:10.3102/0013189X032001025

Simon, H. A. (1969). The sciences of the artificial. Cambridge, MA: The MIT Press.

Tabak, I. (2004). Reconstructing context: Negotiating the tension between exogenous and endogenous educational design. Educational Psychologist, 39 (4), 225–233.

van den Akker, J. (1999). Principles and methods of development research. In J. van den Akker, R. M. Branch, K. Gustafson, N. Nieveen, & T. Plomp (Eds.), Design approaches and tools in education and training (pp. 1–14). Norwell, MA: Kluwer Academic Publishers.

van den Akker, J., & Plomp, T. (1993). Development research in curriculum: Propositions and experiences. Paper presented at the annual meeting of the American Educational Research Association, April 12–14, Atlanta, GA.

Walker, D.F., (1992). Methodological issues in curriculum research, In Jackson, P. (Ed.), Handbook of research on curriculum (pp. 98–118). New York, NY: Macmillan.

Walker, D. & Bresler, L. (1993). Development research: Definitions, methods, and criteria.  Paper presented at the annual meeting of the American Educational Research Association, April 12–16, Atlanta, GA.

Willemien, V. (2009). Design: One, but in different forms. Design Studies, 30 (3), 187–223. doi:10.1016/j.destud.2008.11.004

Further Video Resource

Video available at  http://bit.ly/WestDBRX

Foundations of Learning and Instructional Design Technology Copyright © 2018 by Kimberly Christensen and Richard E. West is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Guide to Experimental Design | Overview, Steps, & Examples

Guide to Experimental Design | Overview, 5 steps & Examples

Published on December 3, 2019 by Rebecca Bevans . Revised on June 21, 2023.

Experiments are used to study causal relationships . You manipulate one or more independent variables and measure their effect on one or more dependent variables.

Experimental design create a set of procedures to systematically test a hypothesis . A good experimental design requires a strong understanding of the system you are studying.

There are five key steps in designing an experiment:

• Consider your variables and how they are related
• Write a specific, testable hypothesis
• Design experimental treatments to manipulate your independent variable
• Assign subjects to groups, either between-subjects or within-subjects
• Plan how you will measure your dependent variable

For valid conclusions, you also need to select a representative sample and control any  extraneous variables that might influence your results. If random assignment of participants to control and treatment groups is impossible, unethical, or highly difficult, consider an observational study instead. This minimizes several types of research bias, particularly sampling bias , survivorship bias , and attrition bias as time passes.

Table of contents

Step 1: define your variables, step 2: write your hypothesis, step 3: design your experimental treatments, step 4: assign your subjects to treatment groups, step 5: measure your dependent variable, other interesting articles, frequently asked questions about experiments.

You should begin with a specific research question . We will work with two research question examples, one from health sciences and one from ecology:

To translate your research question into an experimental hypothesis, you need to define the main variables and make predictions about how they are related.

Start by simply listing the independent and dependent variables .

Then you need to think about possible extraneous and confounding variables and consider how you might control  them in your experiment.

Finally, you can put these variables together into a diagram. Use arrows to show the possible relationships between variables and include signs to show the expected direction of the relationships.

Here we predict that increasing temperature will increase soil respiration and decrease soil moisture, while decreasing soil moisture will lead to decreased soil respiration.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Now that you have a strong conceptual understanding of the system you are studying, you should be able to write a specific, testable hypothesis that addresses your research question.

The next steps will describe how to design a controlled experiment . In a controlled experiment, you must be able to:

• Systematically and precisely manipulate the independent variable(s).
• Precisely measure the dependent variable(s).
• Control any potential confounding variables.

If your study system doesn’t match these criteria, there are other types of research you can use to answer your research question.

How you manipulate the independent variable can affect the experiment’s external validity – that is, the extent to which the results can be generalized and applied to the broader world.

First, you may need to decide how widely to vary your independent variable.

• just slightly above the natural range for your study region.
• over a wider range of temperatures to mimic future warming.
• over an extreme range that is beyond any possible natural variation.

Second, you may need to choose how finely to vary your independent variable. Sometimes this choice is made for you by your experimental system, but often you will need to decide, and this will affect how much you can infer from your results.

• a categorical variable : either as binary (yes/no) or as levels of a factor (no phone use, low phone use, high phone use).
• a continuous variable (minutes of phone use measured every night).

How you apply your experimental treatments to your test subjects is crucial for obtaining valid and reliable results.

First, you need to consider the study size : how many individuals will be included in the experiment? In general, the more subjects you include, the greater your experiment’s statistical power , which determines how much confidence you can have in your results.

Then you need to randomly assign your subjects to treatment groups . Each group receives a different level of the treatment (e.g. no phone use, low phone use, high phone use).

You should also include a control group , which receives no treatment. The control group tells us what would have happened to your test subjects without any experimental intervention.

When assigning your subjects to groups, there are two main choices you need to make:

• A completely randomized design vs a randomized block design .
• A between-subjects design vs a within-subjects design .

Randomization

An experiment can be completely randomized or randomized within blocks (aka strata):

• In a completely randomized design , every subject is assigned to a treatment group at random.
• In a randomized block design (aka stratified random design), subjects are first grouped according to a characteristic they share, and then randomly assigned to treatments within those groups.

Sometimes randomization isn’t practical or ethical , so researchers create partially-random or even non-random designs. An experimental design where treatments aren’t randomly assigned is called a quasi-experimental design .

Between-subjects vs. within-subjects

In a between-subjects design (also known as an independent measures design or classic ANOVA design), individuals receive only one of the possible levels of an experimental treatment.

In medical or social research, you might also use matched pairs within your between-subjects design to make sure that each treatment group contains the same variety of test subjects in the same proportions.

In a within-subjects design (also known as a repeated measures design), every individual receives each of the experimental treatments consecutively, and their responses to each treatment are measured.

Within-subjects or repeated measures can also refer to an experimental design where an effect emerges over time, and individual responses are measured over time in order to measure this effect as it emerges.

Counterbalancing (randomizing or reversing the order of treatments among subjects) is often used in within-subjects designs to ensure that the order of treatment application doesn’t influence the results of the experiment.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Finally, you need to decide how you’ll collect data on your dependent variable outcomes. You should aim for reliable and valid measurements that minimize research bias or error.

Some variables, like temperature, can be objectively measured with scientific instruments. Others may need to be operationalized to turn them into measurable observations.

• Ask participants to record what time they go to sleep and get up each day.
• Ask participants to wear a sleep tracker.

How precisely you measure your dependent variable also affects the kinds of statistical analysis you can use on your data.

Experiments are always context-dependent, and a good experimental design will take into account all of the unique considerations of your study system to produce information that is both valid and relevant to your research question.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Experimental design means planning a set of procedures to investigate a relationship between variables . To design a controlled experiment, you need:

• A testable hypothesis
• At least one independent variable that can be precisely manipulated
• At least one dependent variable that can be precisely measured

When designing the experiment, you decide:

• How you will manipulate the variable(s)
• How you will control for any potential confounding variables
• How many subjects or samples will be included in the study
• How subjects will be assigned to treatment levels

Experimental design is essential to the internal and external validity of your experiment.

The key difference between observational studies and experimental designs is that a well-done observational study does not influence the responses of participants, while experiments do have some sort of treatment condition applied to at least some participants by random assignment .

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

In a between-subjects design , every participant experiences only one condition, and researchers assess group differences between participants in various conditions.

In a within-subjects design , each participant experiences all conditions, and researchers test the same participants repeatedly for differences between conditions.

The word “between” means that you’re comparing different conditions between groups, while the word “within” means you’re comparing different conditions within the same group.

An experimental group, also known as a treatment group, receives the treatment whose effect researchers wish to study, whereas a control group does not. They should be identical in all other ways.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 21). Guide to Experimental Design | Overview, 5 steps & Examples. Scribbr. Retrieved June 30, 2024, from https://www.scribbr.com/methodology/experimental-design/

Is this article helpful?

Rebecca Bevans

Other students also liked, random assignment in experiments | introduction & examples, quasi-experimental design | definition, types & examples, how to write a lab report, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Trends in...

Trends in cardiovascular disease incidence among 22 million people in the UK over 20 years: population based study

• Related content
• Peer review
• Geert Molenberghs , professor 4 ,
• Geert Verbeke , professor 4 ,
• Francesco Zaccardi , associate professor 5 ,
• Claire Lawson , associate professor 5 ,
• Jocelyn M Friday , data scientist 1 ,
• Huimin Su , PhD student 2 ,
• Pardeep S Jhund , professor 1 ,
• Naveed Sattar , professor 6 ,
• Kazem Rahimi , professor 3 ,
• John G Cleland , professor 1 ,
• Kamlesh Khunti , professor 5 ,
• Werner Budts , professor 1 7 ,
• John J V McMurray , professor 1
• 1 School of Cardiovascular and Metabolic Health, British Heart Foundation Cardiovascular Research Centre, University of Glasgow, Glasgow, UK
• 2 Department of Cardiovascular Sciences, KU Leuven, Leuven, Belgium
• 3 Deep Medicine, Nuffield Department of Women’s and Reproductive Health, University of Oxford, Oxford, UK
• 4 Interuniversity Institute for Biostatistics and statistical Bioinformatics (I-BioStat), Hasselt University and KU Leuven, Belgium
• 5 Leicester Real World Evidence Unit, Diabetes Research Centre, University of Leicester, Leicester, UK
• 6 College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK
• 7 Congenital and Structural Cardiology, University Hospitals Leuven, Belgium
• Correspondence to: N Conrad nathalie.conrad{at}kuleuven.be (or @nathalie_conrad on X)
• Accepted 1 May 2024

Objective To investigate the incidence of cardiovascular disease (CVD) overall and by age, sex, and socioeconomic status, and its variation over time, in the UK during 2000-19.

Design Population based study.

Setting UK.

Participants 1 650 052 individuals registered with a general practice contributing to Clinical Practice Research Datalink and newly diagnosed with at least one CVD from 1 January 2000 to 30 June 2019.

Main outcome measures The primary outcome was incident diagnosis of CVD, comprising acute coronary syndrome, aortic aneurysm, aortic stenosis, atrial fibrillation or flutter, chronic ischaemic heart disease, heart failure, peripheral artery disease, second or third degree heart block, stroke (ischaemic, haemorrhagic, and unspecified), and venous thromboembolism (deep vein thrombosis or pulmonary embolism). Disease incidence rates were calculated individually and as a composite outcome of all 10 CVDs combined and were standardised for age and sex using the 2013 European standard population. Negative binomial regression models investigated temporal trends and variation by age, sex, and socioeconomic status.

Results The mean age of the population was 70.5 years and 47.6% (n=784 904) were women. The age and sex standardised incidence of all 10 prespecified CVDs declined by 19% during 2000-19 (incidence rate ratio 2017-19 v 2000-02: 0.80, 95% confidence interval 0.73 to 0.88). The incidence of coronary heart disease and stroke decreased by about 30% (incidence rate ratios for acute coronary syndrome, chronic ischaemic heart disease, and stroke were 0.70 (0.69 to 0.70), 0.67 (0.66 to 0.67), and 0.75 (0.67 to 0.83), respectively). In parallel, an increasing number of diagnoses of cardiac arrhythmias, valve disease, and thromboembolic diseases were observed. As a result, the overall incidence of CVDs across the 10 conditions remained relatively stable from the mid-2000s. Age stratified analyses further showed that the observed decline in coronary heart disease incidence was largely restricted to age groups older than 60 years, with little or no improvement in younger age groups. Trends were generally similar between men and women. A socioeconomic gradient was observed for almost every CVD investigated. The gradient did not decrease over time and was most noticeable for peripheral artery disease (incidence rate ratio most deprived v least deprived: 1.98 (1.87 to 2.09)), acute coronary syndrome (1.55 (1.54 to 1.57)), and heart failure (1.50 (1.41 to 1.59)).

Conclusions Despite substantial improvements in the prevention of atherosclerotic diseases in the UK, the overall burden of CVDs remained high during 2000-19. For CVDs to decrease further, future prevention strategies might need to consider a broader spectrum of conditions, including arrhythmias, valve diseases, and thromboembolism, and examine the specific needs of younger age groups and socioeconomically deprived populations.

Introduction

Since the 1970s, the prevention of coronary disease, both primary and secondary, has improved considerably, largely attributable to public health efforts to control risk factors, such as antismoking legislation, and the widespread use of drugs such as statins. 1 2

Improvements in mortality due to heart disease have, however, stalled in several high income countries, 3 and reports suggest that the incidence of heart disease might even be increasing among younger people. 4 5 6 Conversely, along with coronary heart disease, other cardiovascular conditions are becoming relatively more prominent in older people, altering the profile of cardiovascular disease (CVD) in ageing societies. The importance of non-traditional risk factors for atherosclerotic diseases, such as socioeconomic deprivation, has also been increasingly recognised. Whether socioeconomic deprivation is as strongly associated with other CVDs as with atherosclerosis is uncertain, but it is important to understand as many countries have reported an increase in socioeconomic inequalities. 7

Large scale epidemiological studies are therefore needed to investigate secular trends in CVDs to target future preventive efforts, highlight the focus for future clinical trials, and identify healthcare resources required to manage emerging problems. Existing comprehensive efforts, such as statistics on CVD from leading medical societies or the Global Burden of Diseases studies, have helped toward this goal, but reliable age standardised incidence rates for all CVDs, how these vary by population subgroups, and changes over time are currently not available. 8 9 10

We used a large longitudinal database of linked primary care, secondary care, and death registry records from a representative sample of the UK population 11 12 to assess trends in the incidence of 10 of the most common CVDs in the UK during 2000-19, and how these differed by sex, age, socioeconomic status, and region.

Data source and study population

We used anonymised electronic health records from the GOLD and AURUM datasets of Clinical Practice Research Datalink (CPRD). CPRD contains information on about 20% of the UK population and is broadly representative of age, sex, ethnicity, geographical spread, and socioeconomic deprivation. 11 12 It is also one of the largest databases of longitudinal medical records from primary care in the world and has been validated for epidemiological research for a wide range of conditions. 11 We used the subset of CPRD records that linked information from primary care, secondary care from Hospital Episodes Statistics (HES admitted patient care and HES outpatient) data, and death certificates from the Office for National Statistics (ONS). Linkage was possible for a subset of English practices, covering about 50% of the CPRD records. Data coverage dates were 1 January 1985 to 31 December 2019 for primary care data (including drug prescription data), 1 April 1997 to 30 June 2019 for secondary care data, and 2 January 1998 to 30 May 2019 for death certificates.

Included in the study were men and women registered with a general practice for at least one year during the study period (1 January 2000 to 30 June 2019) whose records were classified by CPRD as acceptable for use in research and approved for HES and ONS linkage.

Study endpoints

The primary endpoint was the first presentation of CVD as recorded in primary or secondary care. We investigated 10 CVDs: acute coronary syndrome, aortic aneurysm, aortic stenosis, atrial fibrillation or flutter, chronic ischaemic heart disease, heart failure, peripheral artery disease, second or third degree heart block, stroke (ischaemic, haemorrhagic, or unspecified), and venous thromboembolism (deep vein thrombosis or pulmonary embolism). We defined incident diagnoses as the first record of that condition in primary care or secondary care regardless of its order in the patient’s record.

Diseases were considered individually and as a composite outcome of all 10 CVDs combined. For the combined analyses, we calculated the primary incidence (considering only the first recorded CVD in each patient, reflecting the number of patients affected by CVDs) and the total incidence (considering all incident CVD diagnoses in each patient, reflecting the cumulative number of CVD diagnoses). We performed sensitivity analyses including diagnoses recorded on death certificates.

To identify diagnoses, we compiled a list of diagnostic codes based on the coding schemes in use in each data source following previously established methods. 13 14 15 We used ICD-10 (international classification of diseases, 10th revision) codes for diagnoses recorded in secondary care, ICD-9 (international classification of diseases, ninth revision) (in use until 31 December 2000) and ICD-10 codes for diagnoses recorded on death certificates (used in sensitivity analyses only), the UK Office of Population Censuses and Surveys classification (OPCS-4) for procedures performed in secondary care settings, and a combination of Read, SNOMED, and local EMIS codes for diagnoses recorded in primary care records (see supplementary table S1). 16 Supplementary texts S1, S2, and S3 describe our approach to the generation of the diagnostic code list as well as considerations and sensitivity analyses into the validity of diagnoses recorded in UK electronic health records.

We selected covariates to represent a range of known cardiovascular risk factors. For clinical data, including systolic and diastolic blood pressure, smoking status, cholesterol (total:high density lipoprotein ratio), and body mass index (BMI), we abstracted data from primary care records as the most recent measurement within two years before the incident CVD diagnosis. BMI was categorised as underweight (<18.5), normal (18.5-24.9), overweight (25-29.9), and obesity (≥30). Information on the prevalence of chronic kidney disease, dyslipidaemia, hypertension, and type 2 diabetes was obtained as the percentage of patients with a diagnosis recorded in their primary care or secondary care record at any time up to and including the date of a first CVD diagnosis. Patients’ socioeconomic status was described using the index of multiple deprivation 2015, 17 a composite measure of seven dimensions (income, employment, education, health, crime, housing, living environment) and provided by CPRD. Measures of deprivation are calculated at small area level, covering an average population of 1500 people, and are presented in fifths, with the first 20% and last 20% representing the least and most deprived areas, respectively. We extracted information on ethnicity from both primary and secondary care records, and we used secondary care data when records differed. Ethnicity was grouped into four categories: African/Caribbean, Asian, white, and mixed/other. Finally, we extracted information on cardiovascular treatments (ie, aspirin and other antiplatelets, alpha adrenoceptor antagonists, aldosterone antagonists/mineralocorticoid receptor antagonists, angiotensin converting enzyme inhibitors, angiotensin II receptor antagonists, beta blockers, calcium channel blockers, diuretics, nitrates, oral anticoagulants, and statins) as the number of patients with at least two prescriptions of each drug class within six months after incident CVD, among patients alive and registered with a general practitioner 30 days after the diagnosis. Supplementary table S2 provides a list of substances included in each drug class. Prescriptions were extracted from primary care records up to 31 December 2019.

Statistical analyses

Categorical data for patient characteristics are presented as frequencies (percentages), and continuous data are presented as means and standard deviations (SDs) for symmetrically distributed data or medians and interquartile ranges (IQRs) for non-symmetrically distributed data, over the whole CVD cohort and stratified by age, sex, socioeconomic status, region, and calendar year of diagnosis. For variables with missing entries, we present numbers and percentages of records with missing data. For categorical variables, frequencies refer to complete cases.

Incidence rates of CVD were calculated by dividing the number of incident diagnoses by the number of patient years in the cohort. Category specific rates were computed separately for subgroups of age, sex, socioeconomic status, region, and calendar year of diagnosis. Age calculations were updated for each calendar year. To ensure calculations referred to incident diagnoses, we excluded individuals, from both the numerator and the denominator populations, with a disease of interest diagnosed before the study start date (1 January 2000), or within the first 12 months of registration with their general practice. Time at risk started at the latest of the patient’s registration date plus 12 months, 30 June of their birth year, or study start date; and stopped at the earliest of death, transfer out of practice, last collection date of the practice, incidence of the disease of interest, or linkage end date (30 June 2019). Disease incidence was standardised for age and sex 18 using the 2013 European standard population 19 in five year age bands up to age 90 years.

Negative binomial regression models were used to calculate overall and category specific incidence rate ratios and corresponding 95% confidence intervals (CIs). 20 Models were adjusted for calendar year of diagnosis, age (categorised into five years age bands), sex, socioeconomic status, and region. We chose negative binomial models over Poisson models to account for potential overdispersion in the data. Sensitivity analyses comparing Poisson and negative binomial models showed similar results.

Study findings are reported according to the RECORD (reporting of studies conducted using observational routinely collected health data) recommendations. 21 We performed statistical analyses in R, version 4.3.3 (R Foundation for Statistical Computing, Vienna, Austria).

Patient and public involvement

No patients or members of the public were directly involved in this study owing to constraints on funding and time.

A total of 22 009 375 individuals contributed data between 1 January 2000 and 30 June 2019, with 146 929 629 patient years of follow-up. Among those we identified 2 906 770 new CVD diagnoses, affecting 1 650 052 patients. Mean age at first CVD diagnosis was 70.5 (SD 15.0) years, 47.6% (n=784 904) of patients were women, and 11.6% (n=191 421), 18.0% (n=296 554), 49.7% (n=820 892), and 14.2% (n=233 833) of patients had a history of chronic kidney disease, dyslipidaemia, hypertension, and type 2 diabetes, respectively, at the time of their first CVD diagnosis ( table 1 ).

Characteristics of patients with a first diagnosis of CVD, 2000-19. Values are number (percentage) unless stated otherwise

• View inline

During 2017-19, the most common CVDs were atrial fibrillation or flutter (age-sex standardised incidence 478 per 100 000 person years), heart failure (367 per 100 000 person years), and chronic ischaemic heart disease (351 per 100 000 person years), followed by acute coronary syndrome (190 per 100 000 person years), venous thromboembolism (183 per 100 000 person years), and stroke (181 per 100 000 patient years) ( fig 1 ).

Incidence of a first diagnosis of cardiovascular disease per 100 000 person years, 2000-19. Incidence rates are age-sex standardised to the 2013 European standard population. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease). See supplementary table S4 for crude incidence rates by age and sex groups. IRR=incidence rate ratio

• Download figure
• Open in new tab
• Download powerpoint

Temporal trends

The primary incidence of CVDs (ie, the number of patients with CVD) decreased by 20% during 2000-19 (age-sex standardised incidence rate ratio 2017-19 v 2000-02: 0.80 (95% CI 0.73 to 0.88)). However, the total incidence of CVD (ie, the total number of new CVD diagnoses) remained relatively stable owing to an increasing number of subsequent diagnoses among patients already affected by a first CVD (incidence rate ratio 2017-19 v 2000-02: 1.00 (0.91 to 1.10)).

The observed decline in CVD incidence was largely due to declining rates of atherosclerotic diseases, in particular acute coronary syndrome, chronic ischaemic heart disease, and stroke, which decreased by about 30% during 2000-19. The incidence of peripheral artery disease also declined, although more modestly (incidence rate ratio 2017-19 v 2000-02: 0.89 (0.80 to 0.98)) ( fig 1 ).

The incidence of non-atherosclerotic heart diseases increased at varying rates, with incidence of aortic stenosis and heart block more than doubling over the study period (2017-19 v 2000-02: 2.42 (2.13 to 2.74) and 2.22 (1.99 to 2.46), respectively) ( fig 1 ). These increasing rates of non-atherosclerotic heart diseases balanced the reductions in ischaemic diseases so that the overall incidence of CVD across the 10 conditions appeared to reach a plateau and to remain relatively stable from 2007-08 (incidence rate ratio 2017-19 v 2005-07: 1.00 (0.91 to 1.10)) ( fig 2 ).

Age standardised incidence of cardiovascular disease by sex, 2000-19. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease). IRR=incidence rate ratio

Age stratified analyses further showed that the observed decrease in incidence of chronic ischaemic heart disease, acute coronary syndrome, and stroke was largely due to a reduced incidence in those aged >60 years, whereas incidence rates in those aged <60 years remained relatively stable ( fig 3 and fig 4 ).

Sex standardised incidence of cardiovascular disease in all age groups. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease)

Sex standardised incidence of cardiovascular diseases by age subgroups <69 years. Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease)

Age at diagnosis

CVD incidence was largely concentrated towards the end of the life span, with a median age at diagnosis generally between 65 and 80 years. Only venous thromboembolism was commonly diagnosed before age 45 years ( fig 5 ). Over the study period, age at first CVD diagnosis declined for several conditions, including stroke (on average diagnosed 1.9 years earlier in 2019 than in 2000), heart block (1.3 years earlier in 2019 than in 2000), and peripheral artery disease (1 year earlier in 2019 than in 2000) (see supplementary figure S1). Adults with a diagnosis before age 60 years were more likely to be from lower socioeconomic groups and to have a higher prevalence of several risk factors, including obesity, smoking, and high cholesterol levels (see supplementary table S3).

Incidence rates of cardiovascular diseases calculated by one year age bands and divided into a colour gradient of 20 quantiles to reflect incidence density by age. IQR=interquartile range

Incidence by sex

Age adjusted incidence of all CVDs combined was higher in men (incidence rate ratio for women v men: 1.46 (1.41 to 1.51)), with the notable exception of venous thromboembolism, which was similar between men and women. The incidence of aortic aneurysms was higher in men (3.49 (3.33 to 3.65)) ( fig 2 ). The crude incidence of CVD, however, was similar between men and women (1069 per 100 000 patient years and 1176 per 100 000 patient years, respectively), owing to the higher number of women in older age groups. Temporal trends in disease incidence were generally similar between men and women ( fig 2 ).

Incidence by socioeconomic status

The most deprived socioeconomic groups had a higher incidence of any CVDs (incidence rate ratio most deprived v least deprived: 1.37 (1.30 to 1.44)) ( fig 6 ). A socioeconomic gradient was observed across almost every condition investigated. That gradient did not decrease over time, and it was most noticeable for peripheral artery disease (incidence rate ratio most deprived v least deprived: 1.98 (1.87 to 2.09)), acute coronary syndrome (1.55 (1.54 to 1.57)), and heart failure (1.50 (1.41 to 1.59)). For aortic aneurysms, atrial fibrillation, heart failure, and aortic stenosis, socioeconomic inequalities in disease incidence appeared to increase over time.

Age-sex standardised incidence rates of cardiovascular diseases by socioeconomic status (index of multiple deprivation 2015). Any cardiovascular disease refers to the primary incidence of cardiovascular disease across the 10 conditions investigated (ie, number of patients with a first diagnosis of cardiovascular disease). Yearly incidence estimates were smoothed using loess (locally estimated scatterplot smoothing) regression lines

Regional differences

Higher incidence rates were seen in northern regions (north west, north east, Yorkshire and the Humber) of England for all 10 conditions investigated, even after adjusting for socioeconomic status. Aortic aneurysms and aortic stenosis had the strongest regional gradients, with incidence rates about 30% higher in northern regions compared with London. Geographical variations remained modest, however, and did not appear to change considerably over time (see supplementary figure S2).

Sensitivity analyses

In sensitivity analyses that used broader disease definitions, that included diagnoses recorded on death certificates, that relied on longer lookback periods for exclusion of potentially prevalent diagnoses, or that were restricted to diagnoses recorded during hospital admissions, temporal trends in disease incidence appeared similar (see supplementary figures S3-S6).

Secondary prevention treatments

The proportion of patients using statins and antihypertensive drugs after a first CVD diagnosis increased over time, whereas the use of non-dihydropyridines calcium channel blockers, nitrates, and diuretics decreased over time. Non-vitamin K antagonist oral anticoagulants increasingly replaced vitamin K anticoagulants (see supplementary figure S7).

The findings of this study suggest that important changes occurred in the distribution of CVDs during 2000-19 and that several areas are of concern. The incidence of non-atherosclerotic heart diseases was shown to increase, the decline in atherosclerotic disease in younger people was stalling, and socioeconomic inequalities had a substantial association across almost every CVD investigated.

Implications for clinical practice and policy

Although no causal inference can be made from our data, the decline in rates of ischaemic diseases coincided with reductions in the prevalence of risk factors such as smoking, hypertension, and raised cholesterol levels in the general population over the same period, 22 and this finding suggests that efforts in the primary and secondary prevention of atherosclerotic diseases have been successful. The decline in stroke was not as noticeable as that for coronary heart disease, which may reflect the rising incidence of atrial fibrillation. The variation in trends for peripheral artery disease could be due to differences in risk factors (eg, a stronger association with diabetes), the multifaceted presentations and causes, and the introduction of systematic leg examinations for people with diabetes. 23 24

All the non-atherosclerotic diseases, however, appeared to increase during 2000-19. For some conditions, such as heart failure, the observed increase remained modest, whereas for others, such as aortic stenosis and heart block, incidence rates doubled. All analyses in this study were standardised for age and sex, to illustrate changes in disease incidence independently of changes in population demographics. Whether these trends solely reflect increased awareness, access to diagnostic tests, or even screening (eg, for abdominal aortic aneurysm 25 ) and coding practices, is uncertain. Reductions in premature death from coronary heart disease may have contributed to the emergence of these other non-atherosclerotic CVDs. Regardless, the identification of increasing numbers of people with these problems has important implications for health services, especially the provision of more surgical and transcatheter valve replacement, pacemaker implantation, and catheter ablation for atrial fibrillation. Importantly, these findings highlight the fact that for many cardiovascular conditions such as heart block, aortic aneurysms, and non-rheumatic valvular diseases, current medical practice remains essentially focused on the management of symptoms and secondary prevention and that more research into underlying causes and possible primary prevention strategies is needed. 26 27

These varying trends also mean that the contribution of individual CVDs towards the overall burden has changed. For example, atrial fibrillation or flutter are now the most common CVDs in the UK. Atrial fibrillation is also a cause (and consequence) of heart failure, and these two increasingly common problems may amplify the incidence of each other. Venous thromboembolism and heart block also appeared as important contributors to overall CVD burden, with incidence rates similar to those of stroke and acute coronary syndrome, yet both receive less attention in terms of prevention efforts.

The stalling decline in the rate of coronary heart disease in younger age groups is of concern, has also been observed in several other high income countries, and may reflect rising rates of physical inactivity, obesity, and type 2 diabetes in young adults. 4 6 28 The stalled decline suggests prevention approaches may need to be expanded beyond antismoking legislation, blood pressure control, and lipid lowering interventions to include the promotion of physical activity, weight control, and use of new treatments shown to reduce cardiovascular risk in people with type 2 diabetes. 29 Although CVD incidence is generally low in people aged <60 years, identifying those at high risk of developing CVD at a young age and intervening before problems occur could reduce premature morbidity and mortality and have important economic implications.

Our study further found that socioeconomic inequalities may contribute to CVD burden, and that this association is not restricted to selected conditions but is visible across most CVDs. The reasons behind the observed increase in risk in relation to socioeconomic inequalities are likely to be multifactorial and to include environmental, occupational, psychosocial, and behavioural risk factors, including established cardiovascular risk factors such as smoking, obesity, nutrition, air pollution, substance misuse, and access to care. 30 How these findings apply to different countries is likely to be influenced by socioeconomic structures and healthcare systems, although health inequalities have been reported in numerous countries. 30 One important factor in the present study is that access to care is free at the point of care in the UK, 31 and yet socioeconomic inequalities persist despite universal health coverage and they did not appear to improve over time. Independently of the specificities of individual countries, our findings highlight the importance of measuring and considering health inequalities and suggest that dealing with the social determinants of health—the conditions under which people are born, live, work, and age—could potentially bring substantial health improvements across a broad range of chronic conditions.

Finally, our results reflect disease incidence based on diagnostic criteria, screening practices, availability, and accuracy of diagnostic tests in place at a particular time and therefore must be interpreted within this context. 32 Several of the health conditions investigated are likely to being sought and detected with increased intensity over the study period. For example, during the study period the definition of myocardial infarction was revised several times, 33 34 35 and high sensitivity troponins were progressively introduced in the UK from 2010. These more sensitive markers of cardiac injury are thought to have increased the detection rates for less severe disease. 36 37 Similarly, increased availability of computed tomography may have increased detection rates for stroke. 38 These changes could have masked an even greater decline in these conditions than observed in the present study. Conversely, increased use of other biochemical tests (such as natriuretic peptides) and more sensitive imaging techniques might have increased the detection of other conditions. 39 40 41 The implementation of a screening programme for aortic aneurysm and incentive programmes aimed at improving coding practices, including the documentation of CVD, associated risk factors and comorbidities, and treatment of these, are also likely to have contributed to the observed trends. 25 42 43 As a result, the difference in incidence estimates and prevalence of comorbidities over time may not reflect solely changes in the true incidence but also differences in ascertainment of people with CVD. 44 Nonetheless, long term trends in large and unconstrained populations offer valuable insights for healthcare resource planning and for the design of more targeted prevention strategies that could otherwise not be answered by using smaller cohorts, cross sectional surveys, or clinical trials; and precisely because they are based on routinely reported diagnoses they are more likely to capture the burden of disease as experienced by doctors and health services.

Strengths and limitations of this study

A key strength of this study is its statistical power, with >140 million person years of data. The large size of the cohort allowed us to perform incidence calculations for a broad spectrum of conditions, and to examine the influence of age, sex, and socioeconomic status as well as trends over 20 years. One important limitation of our study was the modest ethnic diversity in our cohort and the lack of information on ethnicity for the denominator population, which precluded us from stratifying incidence estimates by ethnic group. Our analyses were also limited by the unavailability or considerable missingness of additional variables potentially relevant to the development of CVD, such as smoking, body mass index, imaging data, women specific cardiovascular risk factors (eg, pregnancy associated hypertension and gestational diabetes), and blood biomarkers. Further research may also need to consider an even wider spectrum of CVDs, including individual types of valve disease, pregnancy related conditions, and infection related heart diseases. Research using databases with electronic health records is also reliant on the accuracy of clinical coding input by doctors in primary care as part of a consultation, or in secondary care as part of a hospital admission. We therefore assessed the validity of diagnoses in UK electronic health records data and considered it to be appropriate in accordance with the >200 independent validation studies reporting an average positive predictive value of about 90% for recorded diagnoses. 45 Observed age distributions were also consistent with previous studies and added to the validity of our approach. Nevertheless, our results must be interpreted within the context and limitations of routinely collected data from health records, diagnostic criteria, screening practices, the availability and accuracy of diagnostic tests in place at that time, and the possibility that some level of miscoding is present or that some bias could have been introduced by restricting the cohort to those patients with at least 12 months of continuous data.

Conclusions

Efforts to challenge the notion of the inevitability of vascular events with ageing, and evidence based recommendations for coronary heart disease prevention, have been successful and can serve as a model for other non-communicable diseases. Our findings show that it is time to expand efforts to improve the prevention of CVDs. Broadening research and implementation efforts in both primary and secondary prevention to non-atherosclerotic diseases, tackling socioeconomic inequalities, and introducing better risk prediction and management among younger people appear to be important opportunities to tackle CVDs.

What is already known on this topic

Recent data show that despite decades of declining rates of cardiovascular mortality, the burden from cardiovascular disease (CVD) appears to have stalled in several high income countries

What this study adds

This observational study of a representative sample of 22 million people from the UK during 2000-19 found reductions in CVD incidence to have been largely restricted to ischaemic heart disease and stroke, and were paralleled by a rising number of diagnoses of cardiac arrhythmias, valve disease, and thromboembolic events

Venous thromboembolism and heart block were important contributors to the overall burden of CVDs, with incidence rates similar to stroke and acute coronary syndromes

Improvements in rates of coronary heart disease almost exclusively appeared to benefit those aged >60 years, and the CVD burden in younger age groups appeared not to improve

Ethics statements

Ethical approval.

This study was approved by the Clinical Practice Research Datalink Independent Scientific Advisory Committee.

Data availability statement

Access to Clinical Practice Research Datalink (CPRD) data is subject to a license agreement and protocol approval process that is overseen by CPRD’s research data governance process. A guide to access is provided on the CPRD website ( https://www.cprd.com/data-access ) To facilitate the subsequent use and replication of the findings from this study, aggregated data tables are provided with number of events and person years at risk by individual condition and by calendar year, age (by five year age band), sex, socioeconomic status, and region (masking field with fewer than five events, as per CPRD data security and privacy regulations) on our GitHub repository ( https://github.com/nathalieconrad/CVD_incidence ).

Acknowledgments

We thank Hilary Shepherd, Sonia Coton, and Eleanor L Axson from the Clinical Practice Research Datalink for their support and expertise in preparing the dataset underlying these analyses.

Contributors: NC and JJVM conceived and designed the study. NC, JJVM, GM, and GV designed the statistical analysis plan and NC performed the statistical analysis. All authors contributed to interpreting the results, drafting the manuscript, and the revisions. NC, GM, and GV had permission to access the raw data and NC and GM verified the raw data. All authors gave final approval of the version to be published and accept responsibility to submit the manuscript for publication. NC and JJVM accept full responsibility for the conduct of the study, had access to aggregated data, and controlled the decision to publish. They are the guarantors. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: This study was funded by a personal fellowship from the Research Foundation Flanders (grant No 12ZU922N), a research grant from the European Society of Cardiology (grant No App000037070), and the British Heart Foundation Centre of Research Excellence (grant No RE/18/6/34217). The funders had no role in considering the study design or in the collection, analysis, interpretation of data, writing of the report, or decision to submit the article for publication.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: NC is funded by a personal fellowship from the Research Foundation Flanders and a research grant from the European Society of Cardiology. JMF, PSJ, JGC, NS, and JJVM are supported by British Heart Foundation Centre of Research Excellence. PSJ and JJVM are further supported by the Vera Melrose Heart Failure Research Fund. JJVM has received funding to his institution from Amgen and Cytokinetics for his participation in the steering sommittee for the ATOMIC-HF, COSMIC-HF, and GALACTIC-HF trials and meetings and other activities related to these trials; has received payments through Glasgow University from work on clinical trials, consulting, and other activities from Alnylam, Amgen, AstraZeneca, Bayer, Boehringer Ingelheim, Bristol Myers Squibb, Cardurion, Dal-Cor, GlaxoSmithKline, Ionis, KBP Biosciences, Novartis, Pfizer, and Theracos; and has received personal lecture fees from the Corpus, Abbott, Hikma, Sun Pharmaceuticals, Medscape/Heart.Org, Radcliffe Cardiology, Alkem Metabolics, Eris Lifesciences, Lupin, ProAdWise Communications, Servier Director, and Global Clinical Trial Partners. NS declares consulting fees or speaker honorariums, or both, from Abbott Laboratories, Afimmune, Amgen, AstraZeneca, Boehringer Ingelheim, Lilly, Hanmi Pharmaceuticals, Janssen, Merck Sharp & Dohme, Novartis, Novo Nordisk, Pfizer, Roche Diagnostics, and Sanofi; and grant support paid to his university from AstraZeneca, Boehringer Ingelheim, Novartis, and Roche Diagnostics. KK has acted as a consultant or speaker or received grants for investigator initiated studies for Astra Zeneca, Bayer, Novartis, Novo Nordisk, Sanofi-Aventis, Lilly, Merck Sharp & Dohme, Boehringer Ingelheim, Oramed Pharmaceuticals, Roche, and Applied Therapeutics. KK is supported by the National Institute for Health and Care Research (NIHR) Applied Research Collaboration East Midlands (ARC EM) and the NIHR Leicester Biomedical Research Centre (BRC). CL is funded by an NIHR Advanced Research Fellowship (NIHR-300111) and supported by the Leicester BRC. PSJ has received speaker fees from AstraZeneca, Novartis, Alkem Metabolics, ProAdWise Communications, Sun Pharmaceuticals, and Intas Pharmaceuticals; has received advisory board fees from AstraZeneca, Boehringer Ingelheim, and Novartis; has received research funding from AstraZeneca, Boehringer Ingelheim, Analog Devices; his employer, the University of Glasgow, has been remunerated for clinical trial work from AstraZeneca, Bayer, Novartis, and Novo Nordisk; and is the Director of Global Clinical Trial Partners. HS is supported by the China Scholarship Council. Other authors report no support from any organisation for the submitted work, no financial relationships with any organisations that might have an interest in the submitted work in the previous three years, and no other relationships or activities that could appear to have influenced the submitted work.

Transparency: The lead author (NC) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as planned (and, if relevant, registered) have been explained.

Dissemination to participants and related patient and public communities: Results from this study will be shared with patient associations and foundations dedicated to preventing cardiovascular diseases, such as the European Heart Network and the American Heart Association. To reach the public, findings will also be press released alongside publication of this manuscript. Social media (eg, X) will be used to draw attention to the work and stimulate debate about its findings. Finally, the underlying developed algorithms will be freely available for academic use at https://github.com/nathalieconrad/CVD_incidence .

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/ .

• ↵ Institute for Health Metrics and Evaluation. Gobal Burden of Diseases Viz Hub. 2023. https://vizhub.healthdata.org/gbd-compare/
• Ananth CV ,
• Rutherford C ,
• Rosenfeld EB ,
• O’Flaherty M ,
• Allender S ,
• Scarborough P ,
• Andersson C ,
• Abdalla SM ,
• Mensah GA ,
• Johnson CO ,
• GBD-NHLBI-JACC Global Burden of Cardiovascular Diseases Writing Group
• Almarzooq ZI ,
• American Heart Association Council on Epidemiology and Prevention Statistics Committee and Stroke Statistics Subcommittee
• Townsend N ,
• Atlas Writing Group, European Society of Cardiology
• Herrett E ,
• Gallagher AM ,
• Bhaskaran K ,
• ↵ Wolf A, Dedman D, Campbell J, et al. Data resource profile: Clinical Practice Research Datalink (CPRD) Aurum. International Journal of Epidemiology. 2019 Mar 11 [cited 2019 Mar 22]; https://academic.oup.com/ije/advance-article/doi/10.1093/ije/dyz034/5374844
• Verbeke G ,
• Molenberghs G ,
• Ferguson LD ,
• ↵ Medicines and Healthcare products Regulatory Agency. What coding systems are used in CPRD data? 2023. https://www.cprd.com/defining-your-study-population
• ↵ Department for Communities and Local Government (DCLG). The English Index of Multiple Deprivation 2015: Guidance. 2015; pp1-7. https://www.gov.uk/government/statistics/english-indices-of-deprivation-2015
• ↵ Kirkwood B, Sterne J. Essential medical statistics. 2010.
• ↵ Eurostat’s task force. Revision of the European Standard Population Report. 2013.
• Benchimol EI ,
• Guttmann A ,
• RECORD Working Committee
• ↵ NHS Digital. Health Survey for England, 2021 part 1. https://digital.nhs.uk/data-and-information/publications/statistical/health-survey-for-england/2021
• Criqui MH ,
• ↵ Health and Social Care Information Centre. Quality and Outcomes Framework - Indicators 2011-12. 2011. https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data/quality-and-outcomes-framework-2011-12
• Jacomelli J ,
• Summers L ,
• Stevenson A ,
• Earnshaw JJ
• Vahanian A ,
• Beyersdorf F ,
• ESC/EACTS Scientific Document Group
• Glikson M ,
• Nielsen JC ,
• Kronborg MB ,
• ESC Scientific Document Group
• Stevenson C ,
• Peeters A ,
• Federici M ,
• Schultz WM ,
• Verbakel JY ,
• Thygesen K ,
• Alpert JS ,
• Joint ESC/ACCF/AHA/WHF Task Force for the Redefinition of Myocardial Infarction
• Joint ESC/ACCF/AHA/WHF Task Force for the Universal Definition of Myocardial Infarction
• Sandoval Y ,
• High-STEACS investigators
• Camargo ECS ,
• Singhal AB ,
• Wiener RS ,
• Schwartz LM ,
• Sappler N ,
• Roalfe AK ,
• Lay-Flurrie SL ,
• Ordóñez-Mena JM ,
• Herbert A ,
• Wijlaars L ,
• Zylbersztejn A ,
• Cromwell D ,
• ↵ NHS Digital. Quality and Outcomes Framework (QOF), 2019-20. Indicator definitions. 2020. https://digital.nhs.uk/data-and-information/publications/statistical/quality-and-outcomes-framework-achievement-prevalence-and-exceptions-data/2019-20
• Pronovost PJ
• Thomas SL ,
• Schoonen WM ,

Design Based Research of Multimodal Robotic Learning Companions

• Conference paper
• First Online: 02 July 2024
• Cite this conference paper

• Hae Seon Yun 9 ,
• Heiko Hübert 10 ,
• Niels Pinkwart 9 &
• Verena V. Hafner 9  

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 2151))

Included in the following conference series:

• International Conference on Artificial Intelligence in Education

This paper presents some initial findings from a design-based research study on multimodal robotic language learning companions. The study involved designing four robots with varying physical modalities, including two humanoid (Pepper and Nao) and two non-humanoid (Cozmo and MyKeepOn), to interact empathetically with participants during language learning scenarios. One German teacher participated in the development of these interactive scenarios. Pre- and post-tests were conducted to measure learning gains, and a survey was administered to gather insights from participants. The results showed that all participants exhibited improvements in targeted behaviors between pre- and post-tests, specifically by using longer and more grammatically correct sentences in their responses. However, unlike the strong preference for NAO, the survey revealed that Cozmo had the most characteristics of a learning companion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or Ebook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., Tanaka, F.: Social robots for education: a review. Sci. Robot. 3 (21), eaat5954 (2018)

Google Scholar  

Van den Berghe, R., et al.: A toy or a friend? Children’s anthropomorphic beliefs about robots and how these relate to second-language word learning. J. Comput. Assist. Learn. 37 (2), 396–410 (2021)

Article   Google Scholar  

Biswas, G., Segedy, J.R., Bunchongchit, K.: From design to implementation to practice a learning by teaching system: betty’s brain. Int. J. Artif. Intell. Educ. 26 , 350–364 (2016)

Cephei, A.: VOSK offline speech recognition API. https://alphacephei.com/vosk/ . Accessed 01 Mar 2022

Chan, T.W., Baskin, A.B.: Studying with the prince: the computer as a learning companion. In: Proceedings of the International Conference on Intelligent Tutoring Systems, vol. 194200 (1988)

Chan, T.W., Baskin, A.B.: Learning companion systems. Intell. Tutoring Syst. Crossroads Artif. Intell. Educ. 1 , 6–33 (1990)

Chu, J., Zhao, G., Li, Y., Fu, Z., Zhu, W., Song, L.: Design and implementation of education companion robot for primary education. In: 2019 IEEE 5th International Conference on Computer and Communications (ICCC), pp. 1327–1331. IEEE (2019)

Easterday, M.W., Rees Lewis, D.G., Gerber, E.M.: The logic of design research. Learn. Res. Pract. 4 (2), 131–160 (2018)

Hegel, F., Lohse, M., Wrede, B.: Effects of visual appearance on the attribution of applications in social robotics. In: RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp. 64–71. IEEE (2009)

Hübert, H., Yun, H.S.: Sobotify: a framework for turning robots into social robots. In: Proceedings of the 2024 ACM/IEEE International Conference on Human-Robot Interaction (HRI 2024), Boulder, CO, USA, 11–14 March 2024 (2024)

Kiesler, S., Powers, A., Fussell, S.R., Torrey, C.: Anthropomorphic interactions with a robot and robot-like agent. Soc. Cogn. 26 (2), 169–181 (2008)

Kim, Y.: Desirable characteristics of learning companions. Int. J. Artif. Intell. Educ. 17 (4), 371–388 (2007)

Kim, Y., Baylor, A.L., Shen, E.: Pedagogical agents as learning companions: the impact of agent emotion and gender. J. Comput. Assist. Learn. 23 (3), 220–234 (2007)

Kozima, H., Michalowski, M.P., Nakagawa, C.: Keepon: a playful robot for research, therapy, and entertainment. Int. J. Soc. Robot. 1 , 3–18 (2009)

Lee, H., Lee, J.H.: The effects of robot-assisted language learning: a meta-analysis. Educ. Res. Rev. 35 , 100425 (2022)

Liu, W.: Does teacher immediacy affect students? A systematic review of the association between teacher verbal and non-verbal immediacy and student motivation. Front. Psychol. 12 , 713978 (2021)

Miller, O.: Design and development of Cozmo as an empathetic languate trainer (2023). https://github.com/milleroski/cozmo_teaching

Stenzel, A., Chinellato, E., Bou, M.A.T., Del Pobil, Á.P., Lappe, M., Liepelt, R.: When humanoid robots become human-like interaction partners: corepresentation of robotic actions. J. Exp. Psychol. Hum. Percept. Perform. 38 (5), 1073 (2012)

de Wit, J., et al.: The effect of a robot’s gestures and adaptive tutoring on children’s acquisition of second language vocabularies. In: Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction, pp. 50–58 (2018)

Yun, H.S., Karl, M., Fortenbacher, A.: Designing an interactive second language learning scenario: a case study of Cozmo. In: HCIK 2020: Proceedings of HCI Korea, pp. 384–387 (2020)

Yun, H.S., et al.: Challenges in designing teacher robots with motivation based gestures. arXiv preprint arXiv:2302.03942 (2023)

Yun, H., Fortenbacher, A., Helbig, R., Pinkwart, N.: Design considerations for a mobile sensor-based learning companion. In: Herzog, M.A., Kubincová, Z., Han, P., Temperini, M. (eds.) ICWL 2019. LNCS, vol. 11841, pp. 344–347. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35758-0_35

Chapter   Google Scholar  

Yun, H., Fortenbacher, A., Pinkwart, N.: Improving a mobile learning companion for self-regulated learning using sensors. In: CSEDU (1), pp. 531–536 (2017)

Yun, H., Sardogan, A.: Design based research of a sensor based mobile learning companion. In: ICERI2022 Proceedings, pp. 8117–8124. IATED (2022)

Download references

Acknowledgements

This research was funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy - EXC 2002/1 “Science of Intelligence” - project number 390523135. We would like to express our special thanks to Ellen Donder and Detlef Steppuhn at Erich-Gutenberg-Berufskolleg (EGB) in Cologne for their invaluable inputs and active participation.

Author information

Authors and affiliations.

Humbolt University Berlin, Berlin, Germany

Hae Seon Yun, Niels Pinkwart & Verena V. Hafner

HTW Berlin, Berlin, Germany

Heiko Hübert

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Hae Seon Yun .

Editor information

Editors and affiliations.

University of Memphis, Memphis, TN, USA

Andrew M. Olney

University of Duisburg-Essen, Duisburg, Germany

Irene-Angelica Chounta

Jinan University, Guangzhou, China

UNED, Madrid, Spain

Olga C. Santos

Universidade Federal de Alagoas, Maceio, Brazil

Ig Ibert Bittencourt

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Cite this paper.

Yun, H.S., Hübert, H., Pinkwart, N., Hafner, V.V. (2024). Design Based Research of Multimodal Robotic Learning Companions. In: Olney, A.M., Chounta, IA., Liu, Z., Santos, O.C., Bittencourt, I.I. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky. AIED 2024. Communications in Computer and Information Science, vol 2151. Springer, Cham. https://doi.org/10.1007/978-3-031-64312-5_12

Download citation

DOI : https://doi.org/10.1007/978-3-031-64312-5_12

Published : 02 July 2024

Publisher Name : Springer, Cham

Print ISBN : 978-3-031-64311-8

Online ISBN : 978-3-031-64312-5

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Research on the design of zhuang brocade patterns based on automatic pattern generation.

1. Introduction

2. related literature and work, 2.1. automatic pattern generation technology, 2.2. works related to the pattern color extraction algorithm, 2.3. works related to pattern extraction algorithms, 2.4. works related to pattern organization form extraction, 3. construction of the feature dataset of zhuang brocade patterns and design of the extraction algorithm, 3.1. feature dataset construction, 3.2. color feature extraction algorithm design and results.

• Step 1: input the image and the number of clusters (the number of extracted colors).

3.3. Pattern Extraction Process and Results

3.3.1. image smoothing, 3.3.2. image segmentation processing, 3.3.3. image binarization, 3.3.4. pattern extraction results, 3.4. organizational form extraction process and results, 4. analysis of the zhuang brocade pattern generation algorithm based on pattern feature elements, 4.1. pattern sample encoding, 4.1.1. color encoding, 4.1.2. pattern encoding, 4.1.3. organizational form encoding, 4.2. zhuang brocade pattern design research, 4.2.1. algorithm design and framework.

• Use the cv2.imread(img) method to read the image. At this point, the image format is BGR.
• Convert the image to grayscale using the color space conversion function cv2.cvtColor(img, cv2.COLOR_BGR2GRAY). Here, img is the result returned by step A, and cv2.COLOR_BGR2GRAY indicates the conversion from BGR format to grayscale.
• Use the grayscale image obtained from step B as the src parameter in the cv2.threshold(src, thresh, maxval, type[, dst]) function. This function performs binary thresholding on the grayscale image, returning retVal and dst for subsequent image processing. In this context, src represents the image source (i.e., the grayscale image), thresh denotes the threshold (initial value), and maxval indicates the threshold (maximum value), set to 0 and 255, respectively. The type parameter selects the method, employing a combination of cv2.THRESH_BINARY_INV and cv2.THRESH_OTSU. cv2.THRESH_OTSU utilizes the least squares method to process pixel points, seeking the optimal threshold, while cv2.THRESH_BINARY_INV sets the binarization color—pixels greater than the threshold are set to 0 (black), and those less than the threshold are set to maxval, i.e., 255 (white). The returned value retVal is the threshold value, and dst represents the resulting black-and-white image.
• The color group colorGroup obtained in step A is in list format, containing all color codes and corresponding RGB values in the color group. RGB values are assigned corresponding to the colors to the parts outside the mask, i.e., img[mask] = RGB. The mask specifies the areas to be preserved from replacement, taking values of 0 or 255, and RGB represents the color to be substituted.

4.2.2. Case Examples

4.3. zhuang brocade pattern style similarity evaluation, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Huang, Y.; Pan, Y. Discovery and Extraction of Cultural Traits in Intangible Cultural Heritages Based on Kansei Engineering: Taking Zhuang Brocade Weaving Techniques as an Example. Appl. Sci. 2021 , 11 , 11403. [ Google Scholar ] [ CrossRef ]
• Nong, Z. Feature Extraction and Measurement Algorithm Based on Color in Image Database. J. Intell. Fuzzy Syst. 2020 , 38 , 3885–3891. [ Google Scholar ] [ CrossRef ]
• Tian, G.; Yuan, Q.; Hu, T.; Shi, Y. Auto-Generation System Based on Fractal Geometry for Batik Pattern Design. Appl. Sci. 2019 , 9 , 2383. [ Google Scholar ] [ CrossRef ]
• Jia, X.; Liu, Z. Element Extraction and Convolutional Neural Network-Based Classification for Blue Calico. Text. Res. J. 2021 , 91 , 261–277. [ Google Scholar ] [ CrossRef ]
• Jiang, S.; Lu, Z.; Li, M. Extraction and Application of Pearl S Buck’s Cultural Elements Based on Big Data Mining. Packag. Eng. 2021 , 42 , 337–346. [ Google Scholar ]
• Liang, W. Innovative Development of Egg Carving Cultural and Creative Products Using 3D Printing Technology Based on Internet of Things. Sci. Program. 2021 , 2021 , e3267155. [ Google Scholar ] [ CrossRef ]
• Moghadam, T.S.; Afjeh, M.G.; Amirshahi, S.H. Classification of Persian Carpet Patterns Based on Quantitativeaesthetic-Relatedfeatures. Color Res. Appl. 2021 , 46 , 195–206. [ Google Scholar ] [ CrossRef ]
• Xiang, J.; Zhang, J.; Pan, R.; Han, Y.; Zhang, J.; Gao, W. Graphic Contour Extraction for Printed Fabric Based on Texture Smoothing. J. Text. Res. 2017 , 38 , 162–167. [ Google Scholar ]
• Wang, W.; Deng, N.; Xin, B.; Wang, Y.; Lu, S. Novel Segmentation Algorithm for Jacquard Patterns Based on Multi-View Image Fusion. IET Image Process. 2020 , 14 , 4563–4570. [ Google Scholar ] [ CrossRef ]
• Liu, J.; Zhuang, M.; Shang, L.; Zhang, X. Pattern Segmentation of Shadow Puppetry Costumes Based on Color Clustering. Adv. Text. Technol. 2021 , 29 , 71–77. [ Google Scholar ]
• Abdi, A.; Safabakhsh, R. An Automatic Graphic Pattern Generation Algorithm and Its Application to the Multipurpose Camouflage Pattern Design. IEEE Trans. Cybern. 2023 , 53 , 4748–4762. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Xu, S.; Zhang, Y.; Yan, S. Automatic Mandala Pattern Design and Generation Based on COOM Framework. J. Comput. Lang. 2022 , 72 , 101138. [ Google Scholar ] [ CrossRef ]
• Wu, Y.; Kyungsun, K. Automatic Generation of Traditional Patterns and Aesthetic Quality Evaluation Technology. Inf. Technol. Manag. 2022 , 25 , 125–143. [ Google Scholar ] [ CrossRef ]
• Ju, F.; Wang, Q.; Tan, Z.; Li, Q. Intelligent Recognition of Colour and Contour from Ancient Chinese Embroidery Images. Fibres Text. East. Eur. 2022 , 30 , 79–92. [ Google Scholar ] [ CrossRef ]
• Molada-Tebar, A.; Marqués-Mateu, Á.; Lerma, J.L.; Westland, S. Dominant Color Extraction with K-Means for Camera Characterization in Cultural Heritage Documentation. Remote Sens. 2020 , 12 , 520. [ Google Scholar ] [ CrossRef ]
• Jing, J.; Pinghua, X.; Xiaowan, S.; Jingwen, C.; Ruibing, L. Automatic Coloration of Pattern Based on Color Parsing of Sung Porcelain. Text. Res. J. 2022 , 92 , 5066–5079. [ Google Scholar ] [ CrossRef ]
• He, K.; Wang, D.; Tong, M.; Zhu, Z. An Improved GrabCut on Multiscale Features. Pattern Recognit. 2020 , 103 , 107292. [ Google Scholar ] [ CrossRef ]
• Sudhakar, M.; Meena, M.J. An Efficient Interactive Segmentation Algorithm Using Color Correction for Underwater Images. Wirel. Netw. 2021 , 27 , 5435–5446. [ Google Scholar ] [ CrossRef ]
• Wang, W.; Zhuang, J.; Zhang, X.; Hsia, C.-H.; Li, C.-I.; Yang, C.-F. Household Goods Recognition Using Hierarchical Multi-Object Segmentation. Sens. Mater. 2021 , 33 , 1363. [ Google Scholar ] [ CrossRef ]
• Feng, Y.; Deng, S.; Yan, X.; Yang, X.; Wei, M.; Liu, L. Easy2Hard: Learning to Solve the Intractables From a Synthetic Dataset for Structure-Preserving Image Smoothing. IEEE Trans. Neural Netw. Learn. Syst. 2022 , 33 , 7223–7236. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Liu, J.; Zhuang, M.; Shi, L.; Gao, T. Intelligent Contour Extraction of Shadow Patterns Based on Texture Smoothing and GrabCut. J. Silk 2020 , 57 , 20–27. [ Google Scholar ]
• Castellanos, F.J.; Gallego, A.-J.; Calvo-Zaragoza, J. Unsupervised Neural Domain Adaptation for Document Image Binarization. Pattern Recognit. 2021 , 119 , 108099. [ Google Scholar ] [ CrossRef ]
• Orts, F.; Ortega, G.; Cucura, A.C.; Filatovas, E.; Garzón, E.M. Optimal Fault-Tolerant Quantum Comparators for Image Binarization. J. Supercomput. 2021 , 77 , 8433–8444. [ Google Scholar ] [ CrossRef ]
• Merzban, M.H.; Elbayoumi, M. Efficient Solution of Otsu Multilevel Image Thresholding: A Comparative Study. Expert Syst. Appl. 2019 , 116 , 299–309. [ Google Scholar ] [ CrossRef ]
• Walter, N.; Ligler, H.; Gürsoy, B. From Graphical Treatment of Combinatorics to Tiling Grammars. Nexus Netw. J. 2023 , 25 , 321–332. [ Google Scholar ] [ CrossRef ]
• Hou, Y.; Lv, J.; Liu, X.; Hu, T.; Zhao, Z. Innovative method of ethnic pattern based on neural style transfer network. J. Graph. 2020 , 41 , 606–613. [ Google Scholar ]
• Wang, W.; Lv, J.; Pan, W.; Zhao, H.; Tian, Q. Extraction and Reuse of Pattern Configuration for Handicrafts Personalized Customization. J. Graph. 2019 , 40 , 583–590. [ Google Scholar ]
• Zhu, L.; Hu, X.; Fu, C.-W.; Qin, J.; Heng, P.-A. Saliency-Aware Texture Smoothing. IEEE Trans. Vis. Comput. Graph. 2020 , 26 , 2471–2484. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rother, C.; Kolmogorov, V.; Blake, A. “GrabCut”: Interactive Foreground Extraction Using Iterated Graph Cuts. ACM Trans. Graph. 2004 , 23 , 309–314. [ Google Scholar ] [ CrossRef ]
• Ntirogiannis, K.; Gatos, B.; Pratikakis, I. Performance Evaluation Methodology for Historical Document Image Binarization. IEEE Trans. Image Process. 2013 , 22 , 595–609. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Ni, M.; Huang, Q.; Ni, N.; Zhao, H.; Sun, B. Research on the Design of Zhuang Brocade Patterns Based on Automatic Pattern Generation. Appl. Sci. 2024 , 14 , 5375. https://doi.org/10.3390/app14135375

Ni M, Huang Q, Ni N, Zhao H, Sun B. Research on the Design of Zhuang Brocade Patterns Based on Automatic Pattern Generation. Applied Sciences . 2024; 14(13):5375. https://doi.org/10.3390/app14135375

Ni, Minna, Qingqing Huang, Ni Ni, Huiqin Zhao, and Bo Sun. 2024. "Research on the Design of Zhuang Brocade Patterns Based on Automatic Pattern Generation" Applied Sciences 14, no. 13: 5375. https://doi.org/10.3390/app14135375

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals