quasi experimental variables and designs

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Quasi-Experimental Design | Definition, Types & Examples

Quasi-Experimental Design | Definition, Types & Examples

Published on July 31, 2020 by Lauren Thomas . Revised on January 22, 2024.

Like a true experiment , a quasi-experimental design aims to establish a cause-and-effect relationship between an independent and dependent variable .

However, unlike a true experiment, a quasi-experiment does not rely on random assignment . Instead, subjects are assigned to groups based on non-random criteria.

Quasi-experimental design is a useful tool in situations where true experiments cannot be used for ethical or practical reasons.

Table of contents

Differences between quasi-experiments and true experiments, types of quasi-experimental designs, when to use quasi-experimental design, advantages and disadvantages, other interesting articles, frequently asked questions about quasi-experimental designs.

There are several common differences between true and quasi-experimental designs.

Example of a true experiment vs a quasi-experiment

However, for ethical reasons, the directors of the mental health clinic may not give you permission to randomly assign their patients to treatments. In this case, you cannot run a true experiment.

Instead, you can use a quasi-experimental design.

You can use these pre-existing groups to study the symptom progression of the patients treated with the new therapy versus those receiving the standard course of treatment.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Many types of quasi-experimental designs exist. Here we explain three of the most common types: nonequivalent groups design, regression discontinuity, and natural experiments.

Nonequivalent groups design

In nonequivalent group design, the researcher chooses existing groups that appear similar, but where only one of the groups experiences the treatment.

In a true experiment with random assignment , the control and treatment groups are considered equivalent in every way other than the treatment. But in a quasi-experiment where the groups are not random, they may differ in other ways—they are nonequivalent groups .

When using this kind of design, researchers try to account for any confounding variables by controlling for them in their analysis or by choosing groups that are as similar as possible.

This is the most common type of quasi-experimental design.

Regression discontinuity

Many potential treatments that researchers wish to study are designed around an essentially arbitrary cutoff, where those above the threshold receive the treatment and those below it do not.

Near this threshold, the differences between the two groups are often so minimal as to be nearly nonexistent. Therefore, researchers can use individuals just below the threshold as a control group and those just above as a treatment group.

However, since the exact cutoff score is arbitrary, the students near the threshold—those who just barely pass the exam and those who fail by a very small margin—tend to be very similar, with the small differences in their scores mostly due to random chance. You can therefore conclude that any outcome differences must come from the school they attended.

Natural experiments

In both laboratory and field experiments, researchers normally control which group the subjects are assigned to. In a natural experiment, an external event or situation (“nature”) results in the random or random-like assignment of subjects to the treatment group.

Even though some use random assignments, natural experiments are not considered to be true experiments because they are observational in nature.

Although the researchers have no control over the independent variable , they can exploit this event after the fact to study the effect of the treatment.

However, as they could not afford to cover everyone who they deemed eligible for the program, they instead allocated spots in the program based on a random lottery.

Although true experiments have higher internal validity , you might choose to use a quasi-experimental design for ethical or practical reasons.

Sometimes it would be unethical to provide or withhold a treatment on a random basis, so a true experiment is not feasible. In this case, a quasi-experiment can allow you to study the same causal relationship without the ethical issues.

The Oregon Health Study is a good example. It would be unethical to randomly provide some people with health insurance but purposely prevent others from receiving it solely for the purposes of research.

However, since the Oregon government faced financial constraints and decided to provide health insurance via lottery, studying this event after the fact is a much more ethical approach to studying the same problem.

True experimental design may be infeasible to implement or simply too expensive, particularly for researchers without access to large funding streams.

At other times, too much work is involved in recruiting and properly designing an experimental intervention for an adequate number of subjects to justify a true experiment.

In either case, quasi-experimental designs allow you to study the question by taking advantage of data that has previously been paid for or collected by others (often the government).

Quasi-experimental designs have various pros and cons compared to other types of studies.

• Higher external validity than most true experiments, because they often involve real-world interventions instead of artificial laboratory settings.
• Higher internal validity than other non-experimental types of research, because they allow you to better control for confounding variables than other types of studies do.
• Lower internal validity than true experiments—without randomization, it can be difficult to verify that all confounding variables have been accounted for.
• The use of retrospective data that has already been collected for other purposes can be inaccurate, incomplete or difficult to access.

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Ecological validity

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

A quasi-experiment is a type of research design that attempts to establish a cause-and-effect relationship. The main difference with a true experiment is that the groups are not randomly assigned.

In experimental research, random assignment is a way of placing participants from your sample into different groups using randomization. With this method, every member of the sample has a known or equal chance of being placed in a control group or an experimental group.

Quasi-experimental design is most useful in situations where it would be unethical or impractical to run a true experiment .

Quasi-experiments have lower internal validity than true experiments, but they often have higher external validity  as they can use real-world interventions instead of artificial laboratory settings.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Thomas, L. (2024, January 22). Quasi-Experimental Design | Definition, Types & Examples. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/methodology/quasi-experimental-design/

Is this article helpful?

Lauren Thomas

Other students also liked, guide to experimental design | overview, steps, & examples, random assignment in experiments | introduction & examples, control variables | what are they & why do they matter, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Quasi Experimental Design Overview & Examples

By Jim Frost Leave a Comment

What is a Quasi Experimental Design?

A quasi experimental design is a method for identifying causal relationships that does not randomly assign participants to the experimental groups. Instead, researchers use a non-random process. For example, they might use an eligibility cutoff score or preexisting groups to determine who receives the treatment.

Quasi-experimental research is a design that closely resembles experimental research but is different. The term “quasi” means “resembling,” so you can think of it as a cousin to actual experiments. In these studies, researchers can manipulate an independent variable — that is, they change one factor to see what effect it has. However, unlike true experimental research, participants are not randomly assigned to different groups.

Learn more about Experimental Designs: Definition & Types .

When to Use Quasi-Experimental Design

Researchers typically use a quasi-experimental design because they can’t randomize due to practical or ethical concerns. For example:

• Practical Constraints : A school interested in testing a new teaching method can only implement it in preexisting classes and cannot randomly assign students.
• Ethical Concerns : A medical study might not be able to randomly assign participants to a treatment group for an experimental medication when they are already taking a proven drug.

Quasi-experimental designs also come in handy when researchers want to study the effects of naturally occurring events, like policy changes or environmental shifts, where they can’t control who is exposed to the treatment.

Quasi-experimental designs occupy a unique position in the spectrum of research methodologies, sitting between observational studies and true experiments. This middle ground offers a blend of both worlds, addressing some limitations of purely observational studies while navigating the constraints often accompanying true experiments.

A significant advantage of quasi-experimental research over purely observational studies and correlational research is that it addresses the issue of directionality, determining which variable is the cause and which is the effect. In quasi-experiments, an intervention typically occurs during the investigation, and the researchers record outcomes before and after it, increasing the confidence that it causes the observed changes.

However, it’s crucial to recognize its limitations as well. Controlling confounding variables is a larger concern for a quasi-experimental design than a true experiment because it lacks random assignment.

In sum, quasi-experimental designs offer a valuable research approach when random assignment is not feasible, providing a more structured and controlled framework than observational studies while acknowledging and attempting to address potential confounders.

Types of Quasi-Experimental Designs and Examples

Quasi-experimental studies use various methods, depending on the scenario.

Natural Experiments

This design uses naturally occurring events or changes to create the treatment and control groups. Researchers compare outcomes between those whom the event affected and those it did not affect. Analysts use statistical controls to account for confounders that the researchers must also measure.

Natural experiments are related to observational studies, but they allow for a clearer causality inference because the external event or policy change provides both a form of quasi-random group assignment and a definite start date for the intervention.

For example, in a natural experiment utilizing a quasi-experimental design, researchers study the impact of a significant economic policy change on small business growth. The policy is implemented in one state but not in neighboring states. This scenario creates an unplanned experimental setup, where the state with the new policy serves as the treatment group, and the neighboring states act as the control group.

Researchers are primarily interested in small business growth rates but need to record various confounders that can impact growth rates. Hence, they record state economic indicators, investment levels, and employment figures. By recording these metrics across the states, they can include them in the model as covariates and control them statistically. This method allows researchers to estimate differences in small business growth due to the policy itself, separate from the various confounders.

Nonequivalent Groups Design

This method involves matching existing groups that are similar but not identical. Researchers attempt to find groups that are as equivalent as possible, particularly for factors likely to affect the outcome.

For instance, researchers use a nonequivalent groups quasi-experimental design to evaluate the effectiveness of a new teaching method in improving students’ mathematics performance. A school district considering the teaching method is planning the study. Students are already divided into schools, preventing random assignment.

The researchers matched two schools with similar demographics, baseline academic performance, and resources. The school using the traditional methodology is the control, while the other uses the new approach. Researchers are evaluating differences in educational outcomes between the two methods.

They perform a pretest to identify differences between the schools that might affect the outcome and include them as covariates to control for confounding. They also record outcomes before and after the intervention to have a larger context for the changes they observe.

Regression Discontinuity

This process assigns subjects to a treatment or control group based on a predetermined cutoff point (e.g., a test score). The analysis primarily focuses on participants near the cutoff point, as they are likely similar except for the treatment received. By comparing participants just above and below the cutoff, the design controls for confounders that vary smoothly around the cutoff.

For example, in a regression discontinuity quasi-experimental design focusing on a new medical treatment for depression, researchers use depression scores as the cutoff point. Individuals with depression scores just above a certain threshold are assigned to receive the latest treatment, while those just below the threshold do not receive it. This method creates two closely matched groups: one that barely qualifies for treatment and one that barely misses out.

By comparing the mental health outcomes of these two groups over time, researchers can assess the effectiveness of the new treatment. The assumption is that the only significant difference between the groups is whether they received the treatment, thereby isolating its impact on depression outcomes.

Controlling Confounders in a Quasi-Experimental Design

Accounting for confounding variables is a challenging but essential task for a quasi-experimental design.

In a true experiment, the random assignment process equalizes confounders across the groups to nullify their overall effect. It’s the gold standard because it works on all confounders, known and unknown.

Unfortunately, the lack of random assignment can allow differences between the groups to exist before the intervention. These confounding factors might ultimately explain the results rather than the intervention.

Consequently, researchers must use other methods to equalize the groups roughly using matching and cutoff values or statistically adjust for preexisting differences they measure to reduce the impact of confounders.

A key strength of quasi-experiments is their frequent use of “pre-post testing.” This approach involves conducting initial tests before collecting data to check for preexisting differences between groups that could impact the study’s outcome. By identifying these variables early on and including them as covariates, researchers can more effectively control potential confounders in their statistical analysis.

Additionally, researchers frequently track outcomes before and after the intervention to better understand the context for changes they observe.

Statisticians consider these methods to be less effective than randomization. Hence, quasi-experiments fall somewhere in the middle when it comes to internal validity , or how well the study can identify causal relationships versus mere correlation . They’re more conclusive than correlational studies but not as solid as true experiments.

In conclusion, quasi-experimental designs offer researchers a versatile and practical approach when random assignment is not feasible. This methodology bridges the gap between controlled experiments and observational studies, providing a valuable tool for investigating cause-and-effect relationships in real-world settings. Researchers can address ethical and logistical constraints by understanding and leveraging the different types of quasi-experimental designs while still obtaining insightful and meaningful results.

Cook, T. D., & Campbell, D. T. (1979).  Quasi-experimentation: Design & analysis issues in field settings . Boston, MA: Houghton Mifflin

Share this:

Reader Interactions

Comments and questions cancel reply.

• Privacy Policy

Home » Quasi-Experimental Research Design – Types, Methods

Quasi-Experimental Research Design – Types, Methods

Table of Contents

Quasi-Experimental Design

Quasi-experimental design is a research method that seeks to evaluate the causal relationships between variables, but without the full control over the independent variable(s) that is available in a true experimental design.

In a quasi-experimental design, the researcher uses an existing group of participants that is not randomly assigned to the experimental and control groups. Instead, the groups are selected based on pre-existing characteristics or conditions, such as age, gender, or the presence of a certain medical condition.

Types of Quasi-Experimental Design

There are several types of quasi-experimental designs that researchers use to study causal relationships between variables. Here are some of the most common types:

Non-Equivalent Control Group Design

This design involves selecting two groups of participants that are similar in every way except for the independent variable(s) that the researcher is testing. One group receives the treatment or intervention being studied, while the other group does not. The two groups are then compared to see if there are any significant differences in the outcomes.

Interrupted Time-Series Design

This design involves collecting data on the dependent variable(s) over a period of time, both before and after an intervention or event. The researcher can then determine whether there was a significant change in the dependent variable(s) following the intervention or event.

Pretest-Posttest Design

This design involves measuring the dependent variable(s) before and after an intervention or event, but without a control group. This design can be useful for determining whether the intervention or event had an effect, but it does not allow for control over other factors that may have influenced the outcomes.

Regression Discontinuity Design

This design involves selecting participants based on a specific cutoff point on a continuous variable, such as a test score. Participants on either side of the cutoff point are then compared to determine whether the intervention or event had an effect.

Natural Experiments

This design involves studying the effects of an intervention or event that occurs naturally, without the researcher’s intervention. For example, a researcher might study the effects of a new law or policy that affects certain groups of people. This design is useful when true experiments are not feasible or ethical.

Data Analysis Methods

Here are some data analysis methods that are commonly used in quasi-experimental designs:

Descriptive Statistics

This method involves summarizing the data collected during a study using measures such as mean, median, mode, range, and standard deviation. Descriptive statistics can help researchers identify trends or patterns in the data, and can also be useful for identifying outliers or anomalies.

Inferential Statistics

This method involves using statistical tests to determine whether the results of a study are statistically significant. Inferential statistics can help researchers make generalizations about a population based on the sample data collected during the study. Common statistical tests used in quasi-experimental designs include t-tests, ANOVA, and regression analysis.

Propensity Score Matching

This method is used to reduce bias in quasi-experimental designs by matching participants in the intervention group with participants in the control group who have similar characteristics. This can help to reduce the impact of confounding variables that may affect the study’s results.

Difference-in-differences Analysis

This method is used to compare the difference in outcomes between two groups over time. Researchers can use this method to determine whether a particular intervention has had an impact on the target population over time.

Interrupted Time Series Analysis

This method is used to examine the impact of an intervention or treatment over time by comparing data collected before and after the intervention or treatment. This method can help researchers determine whether an intervention had a significant impact on the target population.

Regression Discontinuity Analysis

This method is used to compare the outcomes of participants who fall on either side of a predetermined cutoff point. This method can help researchers determine whether an intervention had a significant impact on the target population.

Steps in Quasi-Experimental Design

Here are the general steps involved in conducting a quasi-experimental design:

• Identify the research question: Determine the research question and the variables that will be investigated.
• Choose the design: Choose the appropriate quasi-experimental design to address the research question. Examples include the pretest-posttest design, non-equivalent control group design, regression discontinuity design, and interrupted time series design.
• Select the participants: Select the participants who will be included in the study. Participants should be selected based on specific criteria relevant to the research question.
• Measure the variables: Measure the variables that are relevant to the research question. This may involve using surveys, questionnaires, tests, or other measures.
• Implement the intervention or treatment: Implement the intervention or treatment to the participants in the intervention group. This may involve training, education, counseling, or other interventions.
• Collect data: Collect data on the dependent variable(s) before and after the intervention. Data collection may also include collecting data on other variables that may impact the dependent variable(s).
• Analyze the data: Analyze the data collected to determine whether the intervention had a significant impact on the dependent variable(s).
• Draw conclusions: Draw conclusions about the relationship between the independent and dependent variables. If the results suggest a causal relationship, then appropriate recommendations may be made based on the findings.

Quasi-Experimental Design Examples

Here are some examples of real-time quasi-experimental designs:

• Evaluating the impact of a new teaching method: In this study, a group of students are taught using a new teaching method, while another group is taught using the traditional method. The test scores of both groups are compared before and after the intervention to determine whether the new teaching method had a significant impact on student performance.
• Assessing the effectiveness of a public health campaign: In this study, a public health campaign is launched to promote healthy eating habits among a targeted population. The behavior of the population is compared before and after the campaign to determine whether the intervention had a significant impact on the target behavior.
• Examining the impact of a new medication: In this study, a group of patients is given a new medication, while another group is given a placebo. The outcomes of both groups are compared to determine whether the new medication had a significant impact on the targeted health condition.
• Evaluating the effectiveness of a job training program : In this study, a group of unemployed individuals is enrolled in a job training program, while another group is not enrolled in any program. The employment rates of both groups are compared before and after the intervention to determine whether the training program had a significant impact on the employment rates of the participants.
• Assessing the impact of a new policy : In this study, a new policy is implemented in a particular area, while another area does not have the new policy. The outcomes of both areas are compared before and after the intervention to determine whether the new policy had a significant impact on the targeted behavior or outcome.

Applications of Quasi-Experimental Design

Here are some applications of quasi-experimental design:

• Educational research: Quasi-experimental designs are used to evaluate the effectiveness of educational interventions, such as new teaching methods, technology-based learning, or educational policies.
• Health research: Quasi-experimental designs are used to evaluate the effectiveness of health interventions, such as new medications, public health campaigns, or health policies.
• Social science research: Quasi-experimental designs are used to investigate the impact of social interventions, such as job training programs, welfare policies, or criminal justice programs.
• Business research: Quasi-experimental designs are used to evaluate the impact of business interventions, such as marketing campaigns, new products, or pricing strategies.
• Environmental research: Quasi-experimental designs are used to evaluate the impact of environmental interventions, such as conservation programs, pollution control policies, or renewable energy initiatives.

When to use Quasi-Experimental Design

Here are some situations where quasi-experimental designs may be appropriate:

• When the research question involves investigating the effectiveness of an intervention, policy, or program : In situations where it is not feasible or ethical to randomly assign participants to intervention and control groups, quasi-experimental designs can be used to evaluate the impact of the intervention on the targeted outcome.
• When the sample size is small: In situations where the sample size is small, it may be difficult to randomly assign participants to intervention and control groups. Quasi-experimental designs can be used to investigate the impact of an intervention without requiring a large sample size.
• When the research question involves investigating a naturally occurring event : In some situations, researchers may be interested in investigating the impact of a naturally occurring event, such as a natural disaster or a major policy change. Quasi-experimental designs can be used to evaluate the impact of the event on the targeted outcome.
• When the research question involves investigating a long-term intervention: In situations where the intervention or program is long-term, it may be difficult to randomly assign participants to intervention and control groups for the entire duration of the intervention. Quasi-experimental designs can be used to evaluate the impact of the intervention over time.
• When the research question involves investigating the impact of a variable that cannot be manipulated : In some situations, it may not be possible or ethical to manipulate a variable of interest. Quasi-experimental designs can be used to investigate the relationship between the variable and the targeted outcome.

Purpose of Quasi-Experimental Design

The purpose of quasi-experimental design is to investigate the causal relationship between two or more variables when it is not feasible or ethical to conduct a randomized controlled trial (RCT). Quasi-experimental designs attempt to emulate the randomized control trial by mimicking the control group and the intervention group as much as possible.

The key purpose of quasi-experimental design is to evaluate the impact of an intervention, policy, or program on a targeted outcome while controlling for potential confounding factors that may affect the outcome. Quasi-experimental designs aim to answer questions such as: Did the intervention cause the change in the outcome? Would the outcome have changed without the intervention? And was the intervention effective in achieving its intended goals?

Quasi-experimental designs are useful in situations where randomized controlled trials are not feasible or ethical. They provide researchers with an alternative method to evaluate the effectiveness of interventions, policies, and programs in real-life settings. Quasi-experimental designs can also help inform policy and practice by providing valuable insights into the causal relationships between variables.

Overall, the purpose of quasi-experimental design is to provide a rigorous method for evaluating the impact of interventions, policies, and programs while controlling for potential confounding factors that may affect the outcome.

Advantages of Quasi-Experimental Design

Quasi-experimental designs have several advantages over other research designs, such as:

• Greater external validity : Quasi-experimental designs are more likely to have greater external validity than laboratory experiments because they are conducted in naturalistic settings. This means that the results are more likely to generalize to real-world situations.
• Ethical considerations: Quasi-experimental designs often involve naturally occurring events, such as natural disasters or policy changes. This means that researchers do not need to manipulate variables, which can raise ethical concerns.
• More practical: Quasi-experimental designs are often more practical than experimental designs because they are less expensive and easier to conduct. They can also be used to evaluate programs or policies that have already been implemented, which can save time and resources.
• No random assignment: Quasi-experimental designs do not require random assignment, which can be difficult or impossible in some cases, such as when studying the effects of a natural disaster. This means that researchers can still make causal inferences, although they must use statistical techniques to control for potential confounding variables.
• Greater generalizability : Quasi-experimental designs are often more generalizable than experimental designs because they include a wider range of participants and conditions. This can make the results more applicable to different populations and settings.

Limitations of Quasi-Experimental Design

There are several limitations associated with quasi-experimental designs, which include:

• Lack of Randomization: Quasi-experimental designs do not involve randomization of participants into groups, which means that the groups being studied may differ in important ways that could affect the outcome of the study. This can lead to problems with internal validity and limit the ability to make causal inferences.
• Selection Bias: Quasi-experimental designs may suffer from selection bias because participants are not randomly assigned to groups. Participants may self-select into groups or be assigned based on pre-existing characteristics, which may introduce bias into the study.
• History and Maturation: Quasi-experimental designs are susceptible to history and maturation effects, where the passage of time or other events may influence the outcome of the study.
• Lack of Control: Quasi-experimental designs may lack control over extraneous variables that could influence the outcome of the study. This can limit the ability to draw causal inferences from the study.
• Limited Generalizability: Quasi-experimental designs may have limited generalizability because the results may only apply to the specific population and context being studied.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Phenomenology – Methods, Examples and Guide

Qualitative Research – Methods, Analysis Types...

Questionnaire – Definition, Types, and Examples

Explanatory Research – Types, Methods, Guide

Basic Research – Types, Methods and Examples

Mixed Methods Research – Types & Analysis

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

7.3 Quasi-Experimental Research

Learning objectives.

• Explain what quasi-experimental research is and distinguish it clearly from both experimental and correlational research.
• Describe three different types of quasi-experimental research designs (nonequivalent groups, pretest-posttest, and interrupted time series) and identify examples of each one.

The prefix quasi means “resembling.” Thus quasi-experimental research is research that resembles experimental research but is not true experimental research. Although the independent variable is manipulated, participants are not randomly assigned to conditions or orders of conditions (Cook & Campbell, 1979). Because the independent variable is manipulated before the dependent variable is measured, quasi-experimental research eliminates the directionality problem. But because participants are not randomly assigned—making it likely that there are other differences between conditions—quasi-experimental research does not eliminate the problem of confounding variables. In terms of internal validity, therefore, quasi-experiments are generally somewhere between correlational studies and true experiments.

Quasi-experiments are most likely to be conducted in field settings in which random assignment is difficult or impossible. They are often conducted to evaluate the effectiveness of a treatment—perhaps a type of psychotherapy or an educational intervention. There are many different kinds of quasi-experiments, but we will discuss just a few of the most common ones here.

Nonequivalent Groups Design

Recall that when participants in a between-subjects experiment are randomly assigned to conditions, the resulting groups are likely to be quite similar. In fact, researchers consider them to be equivalent. When participants are not randomly assigned to conditions, however, the resulting groups are likely to be dissimilar in some ways. For this reason, researchers consider them to be nonequivalent. A nonequivalent groups design , then, is a between-subjects design in which participants have not been randomly assigned to conditions.

Imagine, for example, a researcher who wants to evaluate a new method of teaching fractions to third graders. One way would be to conduct a study with a treatment group consisting of one class of third-grade students and a control group consisting of another class of third-grade students. This would be a nonequivalent groups design because the students are not randomly assigned to classes by the researcher, which means there could be important differences between them. For example, the parents of higher achieving or more motivated students might have been more likely to request that their children be assigned to Ms. Williams’s class. Or the principal might have assigned the “troublemakers” to Mr. Jones’s class because he is a stronger disciplinarian. Of course, the teachers’ styles, and even the classroom environments, might be very different and might cause different levels of achievement or motivation among the students. If at the end of the study there was a difference in the two classes’ knowledge of fractions, it might have been caused by the difference between the teaching methods—but it might have been caused by any of these confounding variables.

Of course, researchers using a nonequivalent groups design can take steps to ensure that their groups are as similar as possible. In the present example, the researcher could try to select two classes at the same school, where the students in the two classes have similar scores on a standardized math test and the teachers are the same sex, are close in age, and have similar teaching styles. Taking such steps would increase the internal validity of the study because it would eliminate some of the most important confounding variables. But without true random assignment of the students to conditions, there remains the possibility of other important confounding variables that the researcher was not able to control.

Pretest-Posttest Design

In a pretest-posttest design , the dependent variable is measured once before the treatment is implemented and once after it is implemented. Imagine, for example, a researcher who is interested in the effectiveness of an antidrug education program on elementary school students’ attitudes toward illegal drugs. The researcher could measure the attitudes of students at a particular elementary school during one week, implement the antidrug program during the next week, and finally, measure their attitudes again the following week. The pretest-posttest design is much like a within-subjects experiment in which each participant is tested first under the control condition and then under the treatment condition. It is unlike a within-subjects experiment, however, in that the order of conditions is not counterbalanced because it typically is not possible for a participant to be tested in the treatment condition first and then in an “untreated” control condition.

If the average posttest score is better than the average pretest score, then it makes sense to conclude that the treatment might be responsible for the improvement. Unfortunately, one often cannot conclude this with a high degree of certainty because there may be other explanations for why the posttest scores are better. One category of alternative explanations goes under the name of history . Other things might have happened between the pretest and the posttest. Perhaps an antidrug program aired on television and many of the students watched it, or perhaps a celebrity died of a drug overdose and many of the students heard about it. Another category of alternative explanations goes under the name of maturation . Participants might have changed between the pretest and the posttest in ways that they were going to anyway because they are growing and learning. If it were a yearlong program, participants might become less impulsive or better reasoners and this might be responsible for the change.

Another alternative explanation for a change in the dependent variable in a pretest-posttest design is regression to the mean . This refers to the statistical fact that an individual who scores extremely on a variable on one occasion will tend to score less extremely on the next occasion. For example, a bowler with a long-term average of 150 who suddenly bowls a 220 will almost certainly score lower in the next game. Her score will “regress” toward her mean score of 150. Regression to the mean can be a problem when participants are selected for further study because of their extreme scores. Imagine, for example, that only students who scored especially low on a test of fractions are given a special training program and then retested. Regression to the mean all but guarantees that their scores will be higher even if the training program has no effect. A closely related concept—and an extremely important one in psychological research—is spontaneous remission . This is the tendency for many medical and psychological problems to improve over time without any form of treatment. The common cold is a good example. If one were to measure symptom severity in 100 common cold sufferers today, give them a bowl of chicken soup every day, and then measure their symptom severity again in a week, they would probably be much improved. This does not mean that the chicken soup was responsible for the improvement, however, because they would have been much improved without any treatment at all. The same is true of many psychological problems. A group of severely depressed people today is likely to be less depressed on average in 6 months. In reviewing the results of several studies of treatments for depression, researchers Michael Posternak and Ivan Miller found that participants in waitlist control conditions improved an average of 10 to 15% before they received any treatment at all (Posternak & Miller, 2001). Thus one must generally be very cautious about inferring causality from pretest-posttest designs.

Does Psychotherapy Work?

Early studies on the effectiveness of psychotherapy tended to use pretest-posttest designs. In a classic 1952 article, researcher Hans Eysenck summarized the results of 24 such studies showing that about two thirds of patients improved between the pretest and the posttest (Eysenck, 1952). But Eysenck also compared these results with archival data from state hospital and insurance company records showing that similar patients recovered at about the same rate without receiving psychotherapy. This suggested to Eysenck that the improvement that patients showed in the pretest-posttest studies might be no more than spontaneous remission. Note that Eysenck did not conclude that psychotherapy was ineffective. He merely concluded that there was no evidence that it was, and he wrote of “the necessity of properly planned and executed experimental studies into this important field” (p. 323). You can read the entire article here:

http://psychclassics.yorku.ca/Eysenck/psychotherapy.htm

Fortunately, many other researchers took up Eysenck’s challenge, and by 1980 hundreds of experiments had been conducted in which participants were randomly assigned to treatment and control conditions, and the results were summarized in a classic book by Mary Lee Smith, Gene Glass, and Thomas Miller (Smith, Glass, & Miller, 1980). They found that overall psychotherapy was quite effective, with about 80% of treatment participants improving more than the average control participant. Subsequent research has focused more on the conditions under which different types of psychotherapy are more or less effective.

In a classic 1952 article, researcher Hans Eysenck pointed out the shortcomings of the simple pretest-posttest design for evaluating the effectiveness of psychotherapy.

Wikimedia Commons – CC BY-SA 3.0.

Interrupted Time Series Design

A variant of the pretest-posttest design is the interrupted time-series design . A time series is a set of measurements taken at intervals over a period of time. For example, a manufacturing company might measure its workers’ productivity each week for a year. In an interrupted time series-design, a time series like this is “interrupted” by a treatment. In one classic example, the treatment was the reduction of the work shifts in a factory from 10 hours to 8 hours (Cook & Campbell, 1979). Because productivity increased rather quickly after the shortening of the work shifts, and because it remained elevated for many months afterward, the researcher concluded that the shortening of the shifts caused the increase in productivity. Notice that the interrupted time-series design is like a pretest-posttest design in that it includes measurements of the dependent variable both before and after the treatment. It is unlike the pretest-posttest design, however, in that it includes multiple pretest and posttest measurements.

Figure 7.5 “A Hypothetical Interrupted Time-Series Design” shows data from a hypothetical interrupted time-series study. The dependent variable is the number of student absences per week in a research methods course. The treatment is that the instructor begins publicly taking attendance each day so that students know that the instructor is aware of who is present and who is absent. The top panel of Figure 7.5 “A Hypothetical Interrupted Time-Series Design” shows how the data might look if this treatment worked. There is a consistently high number of absences before the treatment, and there is an immediate and sustained drop in absences after the treatment. The bottom panel of Figure 7.5 “A Hypothetical Interrupted Time-Series Design” shows how the data might look if this treatment did not work. On average, the number of absences after the treatment is about the same as the number before. This figure also illustrates an advantage of the interrupted time-series design over a simpler pretest-posttest design. If there had been only one measurement of absences before the treatment at Week 7 and one afterward at Week 8, then it would have looked as though the treatment were responsible for the reduction. The multiple measurements both before and after the treatment suggest that the reduction between Weeks 7 and 8 is nothing more than normal week-to-week variation.

Figure 7.5 A Hypothetical Interrupted Time-Series Design

The top panel shows data that suggest that the treatment caused a reduction in absences. The bottom panel shows data that suggest that it did not.

Combination Designs

A type of quasi-experimental design that is generally better than either the nonequivalent groups design or the pretest-posttest design is one that combines elements of both. There is a treatment group that is given a pretest, receives a treatment, and then is given a posttest. But at the same time there is a control group that is given a pretest, does not receive the treatment, and then is given a posttest. The question, then, is not simply whether participants who receive the treatment improve but whether they improve more than participants who do not receive the treatment.

Imagine, for example, that students in one school are given a pretest on their attitudes toward drugs, then are exposed to an antidrug program, and finally are given a posttest. Students in a similar school are given the pretest, not exposed to an antidrug program, and finally are given a posttest. Again, if students in the treatment condition become more negative toward drugs, this could be an effect of the treatment, but it could also be a matter of history or maturation. If it really is an effect of the treatment, then students in the treatment condition should become more negative than students in the control condition. But if it is a matter of history (e.g., news of a celebrity drug overdose) or maturation (e.g., improved reasoning), then students in the two conditions would be likely to show similar amounts of change. This type of design does not completely eliminate the possibility of confounding variables, however. Something could occur at one of the schools but not the other (e.g., a student drug overdose), so students at the first school would be affected by it while students at the other school would not.

Finally, if participants in this kind of design are randomly assigned to conditions, it becomes a true experiment rather than a quasi experiment. In fact, it is the kind of experiment that Eysenck called for—and that has now been conducted many times—to demonstrate the effectiveness of psychotherapy.

Key Takeaways

• Quasi-experimental research involves the manipulation of an independent variable without the random assignment of participants to conditions or orders of conditions. Among the important types are nonequivalent groups designs, pretest-posttest, and interrupted time-series designs.
• Quasi-experimental research eliminates the directionality problem because it involves the manipulation of the independent variable. It does not eliminate the problem of confounding variables, however, because it does not involve random assignment to conditions. For these reasons, quasi-experimental research is generally higher in internal validity than correlational studies but lower than true experiments.
• Practice: Imagine that two college professors decide to test the effect of giving daily quizzes on student performance in a statistics course. They decide that Professor A will give quizzes but Professor B will not. They will then compare the performance of students in their two sections on a common final exam. List five other variables that might differ between the two sections that could affect the results.

Discussion: Imagine that a group of obese children is recruited for a study in which their weight is measured, then they participate for 3 months in a program that encourages them to be more active, and finally their weight is measured again. Explain how each of the following might affect the results:

• regression to the mean
• spontaneous remission

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings . Boston, MA: Houghton Mifflin.

Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16 , 319–324.

Posternak, M. A., & Miller, I. (2001). Untreated short-term course of major depression: A meta-analysis of studies using outcomes from studies using wait-list control groups. Journal of Affective Disorders, 66 , 139–146.

Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy . Baltimore, MD: Johns Hopkins University Press.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Instant insights, infinite possibilities

The use and interpretation of quasi-experimental design

Last updated

6 February 2023

Reviewed by

Miroslav Damyanov

Short on time? Get an AI generated summary of this article instead

• What is a quasi-experimental design?

Commonly used in medical informatics (a field that uses digital information to ensure better patient care), researchers generally use this design to evaluate the effectiveness of a treatment – perhaps a type of antibiotic or psychotherapy, or an educational or policy intervention.

Even though quasi-experimental design has been used for some time, relatively little is known about it. Read on to learn the ins and outs of this research design.

Make research less tedious

Dovetail streamlines research to help you uncover and share actionable insights

• When to use a quasi-experimental design

A quasi-experimental design is used when it's not logistically feasible or ethical to conduct randomized, controlled trials. As its name suggests, a quasi-experimental design is almost a true experiment. However, researchers don't randomly select elements or participants in this type of research.

Researchers prefer to apply quasi-experimental design when there are ethical or practical concerns. Let's look at these two reasons more closely.

Ethical reasons

In some situations, the use of randomly assigned elements can be unethical. For instance, providing public healthcare to one group and withholding it to another in research is unethical. A quasi-experimental design would examine the relationship between these two groups to avoid physical danger.

Practical reasons

Randomized controlled trials may not be the best approach in research. For instance, it's impractical to trawl through large sample sizes of participants without using a particular attribute to guide your data collection .

Recruiting participants and properly designing a data-collection attribute to make the research a true experiment requires a lot of time and effort, and can be expensive if you don’t have a large funding stream.

A quasi-experimental design allows researchers to take advantage of previously collected data and use it in their study.

• Examples of quasi-experimental designs

Quasi-experimental research design is common in medical research, but any researcher can use it for research that raises practical and ethical concerns. Here are a few examples of quasi-experimental designs used by different researchers:

Example 1: Determining the effectiveness of math apps in supplementing math classes

A school wanted to supplement its math classes with a math app. To select the best app, the school decided to conduct demo tests on two apps before selecting the one they will purchase.

Scope of the research

Since every grade had two math teachers, each teacher used one of the two apps for three months. They then gave the students the same math exams and compared the results to determine which app was most effective.

Reasons why this is a quasi-experimental study

This simple study is a quasi-experiment since the school didn't randomly assign its students to the applications. They used a pre-existing class structure to conduct the study since it was impractical to randomly assign the students to each app.

Example 2: Determining the effectiveness of teaching modern leadership techniques in start-up businesses

A hypothetical quasi-experimental study was conducted in an economically developing country in a mid-sized city.

Five start-ups in the textile industry and five in the tech industry participated in the study. The leaders attended a six-week workshop on leadership style, team management, and employee motivation.

After a year, the researchers assessed the performance of each start-up company to determine growth. The results indicated that the tech start-ups were further along in their growth than the textile companies.

The basis of quasi-experimental research is a non-randomized subject-selection process. This study didn't use specific aspects to determine which start-up companies should participate. Therefore, the results may seem straightforward, but several aspects may determine the growth of a specific company, apart from the variables used by the researchers.

Example 3: A study to determine the effects of policy reforms and of luring foreign investment on small businesses in two mid-size cities

In a study to determine the economic impact of government reforms in an economically developing country, the government decided to test whether creating reforms directed at small businesses or luring foreign investments would spur the most economic development.

The government selected two cities with similar population demographics and sizes. In one of the cities, they implemented specific policies that would directly impact small businesses, and in the other, they implemented policies to attract foreign investment.

After five years, they collected end-of-year economic growth data from both cities. They looked at elements like local GDP growth, unemployment rates, and housing sales.

The study used a non-randomized selection process to determine which city would participate in the research. Researchers left out certain variables that would play a crucial role in determining the growth of each city. They used pre-existing groups of people based on research conducted in each city, rather than random groups.

• Advantages of a quasi-experimental design

Some advantages of quasi-experimental designs are:

Researchers can manipulate variables to help them meet their study objectives.

It offers high external validity, making it suitable for real-world applications, specifically in social science experiments.

Integrating this methodology into other research designs is easier, especially in true experimental research. This cuts down on the time needed to determine your outcomes.

• Disadvantages of a quasi-experimental design

Despite the pros that come with a quasi-experimental design, there are several disadvantages associated with it, including the following:

It has a lower internal validity since researchers do not have full control over the comparison and intervention groups or between time periods because of differences in characteristics in people, places, or time involved. It may be challenging to determine whether all variables have been used or whether those used in the research impacted the results.

There is the risk of inaccurate data since the research design borrows information from other studies.

There is the possibility of bias since researchers select baseline elements and eligibility.

• What are the different quasi-experimental study designs?

There are three distinct types of quasi-experimental designs:

Nonequivalent

Regression discontinuity, natural experiment.

This is a hybrid of experimental and quasi-experimental methods and is used to leverage the best qualities of the two. Like the true experiment design, nonequivalent group design uses pre-existing groups believed to be comparable. However, it doesn't use randomization, the lack of which is a crucial element for quasi-experimental design.

Researchers usually ensure that no confounding variables impact them throughout the grouping process. This makes the groupings more comparable.

Example of a nonequivalent group design

A small study was conducted to determine whether after-school programs result in better grades. Researchers randomly selected two groups of students: one to implement the new program, the other not to. They then compared the results of the two groups.

This type of quasi-experimental research design calculates the impact of a specific treatment or intervention. It uses a criterion known as "cutoff" that assigns treatment according to eligibility.

Researchers often assign participants above the cutoff to the treatment group. This puts a negligible distinction between the two groups (treatment group and control group).

Example of regression discontinuity

Students must achieve a minimum score to be enrolled in specific US high schools. Since the cutoff score used to determine eligibility for enrollment is arbitrary, researchers can assume that the disparity between students who only just fail to achieve the cutoff point and those who barely pass is a small margin and is due to the difference in the schools that these students attend.

Researchers can then examine the long-term effects of these two groups of kids to determine the effect of attending certain schools. This information can be applied to increase the chances of students being enrolled in these high schools.

This research design is common in laboratory and field experiments where researchers control target subjects by assigning them to different groups. Researchers randomly assign subjects to a treatment group using nature or an external event or situation.

However, even with random assignment, this research design cannot be called a true experiment since nature aspects are observational. Researchers can also exploit these aspects despite having no control over the independent variables.

Example of the natural experiment approach

An example of a natural experiment is the 2008 Oregon Health Study.

Oregon intended to allow more low-income people to participate in Medicaid.

Since they couldn't afford to cover every person who qualified for the program, the state used a random lottery to allocate program slots.

Researchers assessed the program's effectiveness by assigning the selected subjects to a randomly assigned treatment group, while those that didn't win the lottery were considered the control group.

• Differences between quasi-experiments and true experiments

There are several differences between a quasi-experiment and a true experiment:

Participants in true experiments are randomly assigned to the treatment or control group, while participants in a quasi-experiment are not assigned randomly.

In a quasi-experimental design, the control and treatment groups differ in unknown or unknowable ways, apart from the experimental treatments that are carried out. Therefore, the researcher should try as much as possible to control these differences.

Quasi-experimental designs have several "competing hypotheses," which compete with experimental manipulation to explain the observed results.

Quasi-experiments tend to have lower internal validity (the degree of confidence in the research outcomes) than true experiments, but they may offer higher external validity (whether findings can be extended to other contexts) as they involve real-world interventions instead of controlled interventions in artificial laboratory settings.

Despite the distinct difference between true and quasi-experimental research designs, these two research methodologies share the following aspects:

Both study methods subject participants to some form of treatment or conditions.

Researchers have the freedom to measure some of the outcomes of interest.

Researchers can test whether the differences in the outcomes are associated with the treatment.

• An example comparing a true experiment and quasi-experiment

Imagine you wanted to study the effects of junk food on obese people. Here's how you would do this as a true experiment and a quasi-experiment:

How to carry out a true experiment

In a true experiment, some participants would eat junk foods, while the rest would be in the control group, adhering to a regular diet. At the end of the study, you would record the health and discomfort of each group.

This kind of experiment would raise ethical concerns since the participants assigned to the treatment group are required to eat junk food against their will throughout the experiment. This calls for a quasi-experimental design.

How to carry out a quasi-experiment

In quasi-experimental research, you would start by finding out which participants want to try junk food and which prefer to stick to a regular diet. This allows you to assign these two groups based on subject choice.

In this case, you didn't assign participants to a particular group, so you can confidently use the results from the study.

When is a quasi-experimental design used?

Quasi-experimental designs are used when researchers don’t want to use randomization when evaluating their intervention.

What are the characteristics of quasi-experimental designs?

Some of the characteristics of a quasi-experimental design are:

Researchers don't randomly assign participants into groups, but study their existing characteristics and assign them accordingly.

Researchers study the participants in pre- and post-testing to determine the progress of the groups.

Quasi-experimental design is ethical since it doesn’t involve offering or withholding treatment at random.

Quasi-experimental design encompasses a broad range of non-randomized intervention studies. This design is employed when it is not ethical or logistically feasible to conduct randomized controlled trials. Researchers typically employ it when evaluating policy or educational interventions, or in medical or therapy scenarios.

How do you analyze data in a quasi-experimental design?

You can use two-group tests, time-series analysis, and regression analysis to analyze data in a quasi-experiment design. Each option has specific assumptions, strengths, limitations, and data requirements.

Should you be using a customer insights hub?

Do you want to discover previous research faster?

Do you share your research findings with others?

Do you analyze research data?

Start for free today, add your research, and get to key insights faster

Editor’s picks

Last updated: 18 April 2023

Last updated: 27 February 2023

Last updated: 22 August 2024

Last updated: 5 February 2023

Last updated: 16 August 2024

Last updated: 9 March 2023

Last updated: 30 April 2024

Last updated: 12 December 2023

Last updated: 11 March 2024

Last updated: 4 July 2024

Last updated: 6 March 2024

Last updated: 5 March 2024

Last updated: 13 May 2024

Latest articles

Related topics, .css-je19u9{-webkit-align-items:flex-end;-webkit-box-align:flex-end;-ms-flex-align:flex-end;align-items:flex-end;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;-webkit-box-flex-wrap:wrap;-webkit-flex-wrap:wrap;-ms-flex-wrap:wrap;flex-wrap:wrap;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;row-gap:0;text-align:center;max-width:671px;}@media (max-width: 1079px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}}@media (max-width: 799px){.css-je19u9{max-width:400px;}.css-je19u9>span{white-space:pre;}} decide what to .css-1kiodld{max-height:56px;display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;}@media (max-width: 1079px){.css-1kiodld{display:none;}} build next, decide what to build next, log in or sign up.

Get started for free

• Link to facebook
• Link to linkedin
• Link to twitter
• Link to youtube
• Writing Tips

An Introduction to Quasi-Experimental Design

• 3-minute read
• 9th January 2022

If you’re a researcher or student in the sciences, you’ll probably come across the term “quasi-experimental design” at some point. But what exactly does it mean?

In this post, we’ll guide you through the different forms of quasi-experimental design and how it compares to true experiments.

What is Quasi-Experimental Design?

Quasi-experimental design (QED) is a research design method that’s useful when regular experimental conditions are impractical or unethical.

Both quasi-experimental designs and true experiments show a cause-and-effect relationship between a dependent and independent variable . Participants in a true experiment are randomly assigned to different treatment groups. The quasi-experimental design, on the other hand, assigns groups based on criteria instead of randomly.

Quasi-Experimental Design Vs. True Experimental Design

The main difference between a quasi-experimental and true experimental design is that in the former, groups aren’t randomly assigned. There are also some other key differences between these research methods.

True experimental design involves:

●     Having control as a researcher over the design of the treatment or program that participants receive (i.e., the independent variable)

●     Control variables as a necessary component

In contrast, a quasi-experimental design involves:

●     The researcher studying groups after they’ve received a treatment or program

●     Control variables as a common element but they aren’t necessary for the experiment to work

Examples of Experimental Design

Perhaps the easiest way to understand quasi-experimental design is to look at how it might be used in practice.

Let’s say you hypothesize that having access to free art lessons will improve the mental health of children from low-income families.

In a true experiment, you’d randomly assign participants to two groups: one that receives free art lessons and another that doesn’t.

However, it’s ethically questionable to deny one group of children access to something that might benefit them.

Find this useful?

Subscribe to our newsletter and get writing tips from our editors straight to your inbox.

Instead, you might decide to compare the data from a community that’s already offered free art classes to these children with that of a community that’s not yet done so.

This second example would be a quasi-experimental design.

Advantages and Disadvantages of Quasi-Experimental Design

Quasi-experimental design has some advantages and disadvantages you’ll need to consider when designing your research.

On the plus side, quasi-experimental design:

●     Has a higher external validity than true experimental design, as it usually involves real-world scenarios

●     Allows you to control for unexpected, confounding variables, resulting in a higher internal validity than other non-experimental methods of research

●     Enables the study of cause-and-effect relationships without the ethical issue of denying a treatment to those who may benefit from it

●     Does not require access to large-scale funding and other practical concerns, as the treatment has already been issued by others

The disadvantages of quasi-experimental design, however, include:

●     Lower internal validity than found in true experiments, as it’s more difficult to account for all confounding variables without using random assignment

●     The necessary data required for research potentially being inaccurate, outdated, or difficult to access

Expert Proofreading for Researchers

We hope our guide has helped you understand the basics of quasi-experimental design.

If you need help with your research paper , our expert proofreaders are available 24/7. Try us out by submitting a free sample document today.

Share this article:

Post A New Comment

Got content that needs a quick turnaround? Let us polish your work. Explore our editorial business services.

5-minute read

Free Email Newsletter Template

Promoting a brand means sharing valuable insights to connect more deeply with your audience, and...

6-minute read

How to Write a Nonprofit Grant Proposal

If you’re seeking funding to support your charitable endeavors as a nonprofit organization, you’ll need...

9-minute read

How to Use Infographics to Boost Your Presentation

Is your content getting noticed? Capturing and maintaining an audience’s attention is a challenge when...

8-minute read

Why Interactive PDFs Are Better for Engagement

Are you looking to enhance engagement and captivate your audience through your professional documents? Interactive...

7-minute read

Seven Key Strategies for Voice Search Optimization

Voice search optimization is rapidly shaping the digital landscape, requiring content professionals to adapt their...

4-minute read

Five Creative Ways to Showcase Your Digital Portfolio

Are you a creative freelancer looking to make a lasting impression on potential clients or...

Make sure your writing is the best it can be with our expert English proofreading and editing.

Research Methodologies Guide

• Action Research
• Bibliometrics
• Case Studies
• Content Analysis
• Digital Scholarship This link opens in a new window
• Documentary
• Ethnography
• Focus Groups
• Grounded Theory
• Life Histories/Autobiographies
• Longitudinal
• Participant Observation
• Qualitative Research (General)

Quasi-Experimental Design

• Usability Studies

Quasi-Experimental Design is a unique research methodology because it is characterized by what is lacks. For example, Abraham & MacDonald (2011) state:

" Quasi-experimental research is similar to experimental research in that there is manipulation of an independent variable. It differs from experimental research because either there is no control group, no random selection, no random assignment, and/or no active manipulation. "

This type of research is often performed in cases where a control group cannot be created or random selection cannot be performed. This is often the case in certain medical and psychological studies. 

For more information on quasi-experimental design, review the resources below: 

Where to Start

Below are listed a few tools and online guides that can help you start your Quasi-experimental research. These include free online resources and resources available only through ISU Library.

• Quasi-Experimental Research Designs by Bruce A. Thyer This pocket guide describes the logic, design, and conduct of the range of quasi-experimental designs, encompassing pre-experiments, quasi-experiments making use of a control or comparison group, and time-series designs. An introductory chapter describes the valuable role these types of studies have played in social work, from the 1930s to the present. Subsequent chapters delve into each design type's major features, the kinds of questions it is capable of answering, and its strengths and limitations.
• Experimental and Quasi-Experimental Designs for Research by Donald T. Campbell; Julian C. Stanley. Call Number: Q175 C152e Written 1967 but still used heavily today, this book examines research designs for experimental and quasi-experimental research, with examples and judgments about each design's validity.

Online Resources

• Quasi-Experimental Design From the Web Center for Social Research Methods, this is a very good overview of quasi-experimental design.
• Experimental and Quasi-Experimental Research From Colorado State University.
• Quasi-experimental design--Wikipedia, the free encyclopedia Wikipedia can be a useful place to start your research- check the citations at the bottom of the article for more information.
• << Previous: Qualitative Research (General)
• Next: Sampling >>
• Last Updated: Sep 11, 2024 11:05 AM
• URL: https://instr.iastate.libguides.com/researchmethods

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• HHS Author Manuscripts

Quasi-Experimental Designs for Causal Inference

When randomized experiments are infeasible, quasi-experimental designs can be exploited to evaluate causal treatment effects. The strongest quasi-experimental designs for causal inference are regression discontinuity designs, instrumental variable designs, matching and propensity score designs, and comparative interrupted time series designs. This article introduces for each design the basic rationale, discusses the assumptions required for identifying a causal effect, outlines methods for estimating the effect, and highlights potential validity threats and strategies for dealing with them. Causal estimands and identification results are formalized with the potential outcomes notations of the Rubin causal model.

Causal inference plays a central role in many social and behavioral sciences, including psychology and education. But drawing valid causal conclusions is challenging because they are warranted only if the study design meets a set of strong and frequently untestable assumptions. Thus, studies aiming at causal inference should employ designs and design elements that are able to rule out most plausible threats to validity. Randomized controlled trials (RCTs) are considered as the gold standard for causal inference because they rely on the fewest and weakest assumptions. But under certain conditions quasi-experimental designs that lack random assignment can also be as credible as RCTs ( Shadish, Cook, & Campbell, 2002 ).

This article discusses four of the strongest quasi-experimental designs for identifying causal effects: regression discontinuity design, instrumental variable design, matching and propensity score designs, and the comparative interrupted time series design. For each design we outline the strategy and assumptions for identifying a causal effect, address estimation methods, and discuss practical issues and suggestions for strengthening the basic designs. To highlight the design differences, throughout the article we use a hypothetical example with the following causal research question: What is the effect of attending a summer science camp on students’ science achievement?

POTENTIAL OUTCOMES AND RANDOMIZED CONTROLLED TRIAL

Before we discuss the four quasi-experimental designs, we introduce the potential outcomes notation of the Rubin causal model (RCM) and show how it is used in the context of an RCT. The RCM ( Holland, 1986 ) formalizes causal inference in terms of potential outcomes, which allow us to precisely define causal quantities of interest and to explicate the assumptions required for identifying them. RCM considers a potential outcome for each possible treatment condition. For a dichotomous treatment variable (i.e., a treatment and control condition), each subject i has a potential treatment outcome Y i (1), which we would observe if subject i receives the treatment ( Z i = 1), and a potential control outcome Y i (0), which we would observe if subject i receives the control condition ( Z i = 0). The difference in the two potential outcomes, Y i (1)− Y i (0), represents the individual causal effect.

Suppose we want to evaluate the effect of attending a summer science camp on students’ science achievement score. Then each student has two potential outcomes: a potential control score for not attending the science camp, and the potential treatment score for attending the camp. However, the individual causal effects of attending the camp cannot be inferred from data, because the two potential outcomes are never observed simultaneously. Instead, researchers typically focus on average causal effects. The average treatment effect (ATE) for the entire study population is defined as the difference in the expected potential outcomes, ATE = E [ Y i (1)] − E [ Y i (0)]. Similarly, we can also define the ATE for the treated subjects (ATT), ATT = E [ Y i (1) | Z i = 1] − E [ Y (0) | Z i =1]. Although the expectations of the potential outcomes are not directly observable because not all potential outcomes are observed, we nonetheless can identify ATE or ATT under some reasonable assumptions. In an RCT, random assignment establishes independence between the potential outcomes and the treatment status, which allows us to infer ATE. Suppose that students are randomly assigned to the science camp and that all students comply with the assigned condition. Then random assignment guarantees that the camp attendance indicator Z is independent of the potential achievement scores Y i (0) and Y i (1).

The independence assumption allows us to rewrite ATE in terms of observable expectations (i.e., with observed outcomes instead of potential outcomes). First, due to the independence (randomization), the unconditional expectations of the potential outcomes can be expressed as conditional expectations, E [ Y i (1)] = E [ Y i (1) | Z i = 1] and E [ Y i (0)] = E [ Y i (0) | Z i = 0] Second, because the potential treatment outcomes are actually observed for the treated, we can replace the potential treatment outcome with the observed outcome such that E [ Y i (1) | Z i = 1] = E [ Y i | Z i = 1] and, analogously, E [ Y i (0) | Z i = 0] = E [ Y i | Z i = 0] Thus, the ATE is expressible in terms of observable quantities rather than potential outcomes, ATE = E [ Y i (1)] − E [ Y i (0)] = E [ Y i | Z i = 1] – E [ Y i | Z i = 0], and we that say ATE is identified.

This derivation also rests on the stable-unit-treatment-value assumption (SUTVA; Imbens & Rubin, 2015 ). SUTVA is required to properly define the potential outcomes, that is, (a) the potential outcomes of a subject depend neither on the assignment mode nor on other subjects’ treatment assignment, and (b) there is only one unique treatment and one unique control condition. Without further mentioning, we assume SUTVA for all quasi-experimental designs discussed in this article.

REGRESSION DISCONTINUITY DESIGN

Due to ethical or budgetary reasons, random assignment is often infeasible in practice. Nonetheless, researchers may sometimes still retain full control over treatment assignment as in a regression discontinuity (RD) design where, based on a continuous assignment variable and a cutoff score, subjects are deterministically assigned to treatment conditions.

Suppose that the science camp is a remedial program and only students whose grade point average (GPA) score is less than or equal to 2.0 are eligible to participate. Figure 1 shows a scatterplot of hypothetical data where the x-axis represents the assignment variable ( GPA ) and the y -axis the outcome ( Science Score ). All subjects with a GPA score below the cutoff attended the camp (circles), whereas all subjects scoring above the cutoff do not attend (squares). Because all low-achieving students are in the treatment group and all high-achieving students in the control group, their respective GPA distributions do not overlap, not even at the cutoff. This lack of overlap complicates the identification of a causal effect because students in the treatment and control group are not comparable at all (i.e., they have a completely different distribution of the GPA scores).

A hypothetical example of regression discontinuity design. Note . GPA = grade point average.

One strategy of dealing with the lack of overlap is to rely on the linearity assumption of regression models and to extrapolate into areas of nonoverlap. However, if the linear models do not correctly specify the functional form, the resulting ATE estimate is biased. A safer strategy is to evaluate the treatment effect only at the cutoff score where treatment and control cases almost overlap, and thus functional form assumptions and extrapolation are almost no longer needed. Consider the treatment and control students that score right at the cutoff or just above it. Students with a GPA score of 2.0 participate in the science camp and students with a GPA score of 2.1 are in the control condition (the status quo condition or a different camp). The two groups of students are essentially equivalent because the difference in their GPA scores is negligibly small (2.1 − 2.0 = .1) and likely due to random chance (measurement error) rather than a real difference in ability. Thus, in the very close neighborhood around the cutoff score, the RD design is equivalent to an RCT; therefore, the ATE at the cutoff (ATEC) is identified.

CAUSAL ESTIMAND AND IDENTIFICATION

ATEC is defined as the difference in the expected potential treatment and control outcomes for the subjects scoring exactly at the cutoff: ATEC = E [ Y i (1) | A i = a c ] − E [ Y i (0) | A i = a c ], where A denotes assignment variable and a c the cutoff score. Because we observe only treatment subjects and not control subjects right at the cutoff, we need two assumptions in order to identify ATEC ( Hahn, Todd, & van Klaauw, 2001 ): (a) the conditional expectations of the potential treatment and control outcomes are continuous at the cutoff ( continuity ), and (b) all subjects comply with treatment assignment ( full compliance ).

The continuity assumption can be expressed in terms of limits as lim a ↓ a C E [ Y i ( 1 ) | A i = a ] = E [ Y i ( 1 ) | A i = a ] = lim a ↑ a C E [ Y i ( 1 ) | A i = a ] and lim a ↓ a C E [ Y i ( 0 ) | A i = a ] = E [ Y i ( 0 ) | A i = a ] = lim a ↑ a C E [ Y i ( 0 ) | A i = a ] . Thus, we can rewrite ATEC as the difference in limits, A T E C = lim a ↑ a C E [ Y i ( 1 ) | A i = a c ] − lim a ↓ a C E [ Y i ( 0 ) | A i = a c ] , which solves the issue that no control subjects are observed directly at the cutoff. Then, by the full compliance assumption, the potential treatment and control outcomes can be replaced with the observed outcomes such that A T E C = lim a ↑ a C E [ Y i | A i = a c ] − lim a ↓ a C E [ Y i | A i = a c ] is identified at the cutoff (i.e., ATEC is now expressed in terms of observable quantities). The difference in the limits represents the discontinuity in the mean outcomes exactly at the cutoff ( Figure 1 ).

Estimating ATEC

ATEC can be estimated with parametric or nonparametric regression methods. First, consider the parametric regression of the outcome Y on the treatment Z , the cutoff-centered assignment variable A − a c , and their interaction: Y = β 0 + β 1 Z + β 2 ( A − a c ) + β 3 ( Z × ( A − a c )) + e . If the model correctly specifies the functional form, then β ^ 1 is an unbiased estimator for ATEC. In practice, an appropriate model specification frequently involves also quadratic and cubic terms of the assignment variable plus their interactions with the treatment indicator.

To avoid overly strong functional form assumptions, semiparametric or nonparametric regression methods like generalized additive models or local linear kernel regression can be employed ( Imbens & Lemieux, 2008 ). These methods down-weight or even discard observations that are not in the close neighborhood around the cutoff. The R packages rdd ( Dimmery, 2013 ) and rdrobust ( Calonico, Cattaneo, & Titiunik, 2015 ), or the command rd in STATA ( Nichols, 2007 ) are useful for estimation and diagnostic purposes.

Practical Issues

A major validity threat for RD designs is the manipulation of the assignment score around the cutoff, which directly results in a violation of the continuity assumption ( Wong et al., 2012 ). For instance, if a teacher knows the assignment score in advance and he wants all his students to attend the science camp, the teacher could falsely report a GPA score of 2.0 or below for the students whose actual GPA score exceeds the cutoff value.

Another validity threat is noncompliance, meaning that subjects assigned to the control condition may cross over to the treatment and subjects assigned to the treatment do not show up. An RD design with noncompliance is called a fuzzy RD design (instead of a sharp RD design with full compliance). A fuzzy RD design still allows us to identify the intention-to-treat effect or the local average treatment effect at the cutoff (LATEC). The intention-to-treat effect refers to the effect of treatment assignment rather than the actual treatment receipt. LATEC estimates ATEC for the subjects who comply with treatment assignment. LATEC is identified if one uses the assignment status as an instrumental variable for treatment receipt (see the upcoming Instrumental Variable section).

Finally, generalizability and statistical power are often mentioned as major disadvantages of RD designs. Because RD designs identify the treatment effect only at the cutoff, ATEC estimates are not automatically generalizable to subjects scoring further away from the cutoff. Statistical power for detecting a significant effect is an issue because the lack of overlap on the assignment variable results in increased standard errors. With semi- or nonparametric regression methods, power further diminishes.

Strengthening RD Designs

To avoid systematic manipulations of the assignment variable, it is desirable to conceal the assignment rule from study participants and administrators. If the assignment rule is known to them, manipulations can hardly be ruled out, particularly when the stakes are high. Researchers can use the McCrary test ( McCrary, 2008 ) to check for potential manipulations. The test investigates whether there is a discontinuity in the distribution of the assignment variable right at the cutoff. Plotting baseline covariates against the assignment variable, and regressing the covariates on the assignment variable and the treatment indicator also help in detecting potential discontinuities at the cutoff.

The RD design’s validity can be increased by combining the basic RD design with other designs. An example is the tie-breaking RD design, which uses two cutoff scores. Subjects scoring between the two cutoff scores are randomly assigned to treatment conditions, whereas subjects scoring outside the cutoff interval receive the treatment or control condition according to the RD assignment rule ( Black, Galdo & Smith, 2007 ). This design combines an RD design with an RCT and is advantageous with respect to the correct specification of the functional form, generalizability, and statistical power. Similar benefits can be obtained by adding pretest measures of the outcome or nonequivalent comparison groups ( Wing & Cook, 2013 ).

Imbens and Lemieux (2008) and Lee and Lemieux (2010) provided comprehensive introductions to RD designs. Lee and Lemieux also summarized many applications from economics. Angrist and Lavy (1999) applied the design to investigate the effect of class size on student achievement.

INSTRUMENTAL VARIABLE DESIGN

In practice, researchers often have no or only partial control over treatment selection. In addition, they might also lack reliable knowledge of the selection process. Nonetheless, even with limited control and knowledge of the selection process it is still possible to identify a causal treatment effect if an instrumental variable (IV) is available. An IV is an exogenous variable that is related to the treatment but is completely unrelated to the outcome, except via treatment. An IV design requires researchers either to create an IV at the design stage (as in an encouragement design; see next) or to find an IV in the data set at hand or a related data base.

Consider the science camp example, but instead of random or deterministic treatment assignment, students decide on their own or together with their parents whether to attend the camp. Many factors may determine the decision, for instance, students’ science ability and motivation, parents’ socioeconomic status, or the availability of public transportation for the daily commute to the camp. Whereas the first three variables are presumably also related to the science outcome, public transportation might be unrelated to the science score (except via camp attendance). Thus, the availability of public transportation may qualify as an IV. Figure 2 illustrates such IV design: Public transportation (IV) directly affects camp attendance but has no direct or indirect effect on science achievement (outcome) other than through camp attendance (treatment). The question mark represents unknown or unobserved confounders, that is, variables that simultaneously affect both camp attendance and science achievement. The IV design allows us to identify a causal effect even if some or all confounders are unknown or unobserved.

A diagram of an example of instrumental variable design.

The strategy for identifying a causal effect is based on exploiting the variation in the treatment variable explained by IV. In Figure 2 , the total variation in the treatment consists of (a) the variation induced by the IV and (b) the variation induced by confounders (question mark) and other exogenous variables (not shown in the figure). The identification of the camp’s effect requires us to isolate the treatment variation that is related to public transportation (IV), and then to use the isolated variation to investigate the camp’s effect on the science score. Because we exploit the treatment variation exclusively induced by the IV but ignore the variation induced by unobserved or unknown confounders, the IV design identifies the ATE for the sub-population of compliers only. In our example, the compliers are the students who attend the camp because public transportation is available and do not attend because it is unavailable. For students whose parents always use their own car to drop them off and pick them up at the camp location, we cannot infer the causal effect, because their camp attendance is completely unrelated to the availability of public transportation.

Causal Estimand and Identification

The complier average treatment effect (CATE) is defined as the expected difference in potential outcomes for the sub-population of compliers: CATE = E [ Y i (1) | Complier ] − E [ Y i (0) | Complier ] = τ C .

Identification requires us to distinguish between four latent groups: compliers (C), who attend the camp if public transportation is available but do not attend if unavailable; always-takers (A), who always attend the camp regardless of whether or not public transportation is available; never-takers (N), who never attend the camp regardless of public transportation; and defiers (D), who do not attend if public transportation is available but attend if unavailable. Because group membership is unknown, it is impossible to directly infer CATE from the data of compliers. However, CATE is identified from the entire data set if (a) the IV is predictive of the treatment ( predictive first stage ), (b) the IV is unrelated to the outcome except via treatment ( exclusion restriction ), and (c) no defiers are present ( monotonicity ; Angrist, Imbens, & Rubin, 1996 ; see Steiner, Kim, Hall, & Su, 2015 , for a graphical explanation).

First, notice that the IV’s effects on the treatment (γ) and the outcome (δ) are directly identified from the observed data because the IV’s relation with the treatment and outcome is unconfounded. In our example ( Figure 2 ), γ denotes the effect of public transportation on camp attendance and δ the indirect effect of public transportation on the science score. Both effects can be written as weighted averages of the corresponding group-specific effects ( γ C , γ A , γ N , γ D and δ C , δ A , δ N , δ D for compliers, always-takers, never-takers, and defiers, respectively): γ = p ( C ) γ C + p ( A ) γA + p ( N ) γ N + p ( D ) γ D and δ = p ( C ) δ C + p ( A ) δ A + p ( N ) δ N + p ( D ) δ D where p (.) represents the portion of the respective latent group in the population and p ( C ) + p ( A ) + p ( N ) + p ( D ) = 1. Because the treatment choice of always-takers and never-takers is entirely unaffected by the instrument, the IV’s effect on the treatment is zero, γ A = γ N = .0, and together with the exclusion restriction , we also know δ A = δ N = 0, that is, the IV has no effect on the outcome. If no defiers are present, p ( D ) = 0 ( monotonicity ), then the IV’s effects on the treatment and outcome simplify to γ = p ( C ) γC and δ = p ( C ) δC , respectively. Because δ C = γ C τ C and γ ≠ 0 ( predictive first stage ), the ratio of the observable IV effects, γ and δ, identifies CATE: δ γ = p ( C ) γ C τ C p ( C ) γ C = τ C .

Estimating CATE

A two-stage least squares (2SLS) regression is typically used for estimating CATE. In the first stage, treatment Z is regressed on the IV, Z = β 0 + β 1 IV + e . The linear first-stage model applies with a dichotomous treatment variable (linear probability model). The second stage then regresses the outcome Y on the predicted values Z ^ from the first stage model, Y = π 0 + π 1 Z ^ + r , where π ^ 1 is the CATE estimator. The two stages are automatically performed by the 2SLS procedure, which also provides an appropriate standard error for the effect estimate. The STATA commands ivregress and ivreg2 ( Baum, Schaffer, & Stillman, 2007 ) or the sem package in R ( Fox, 2006 ) perform the 2SLS regression.

One challenge in implementing an IV design is to find a valid instrument that satisfies the assumptions just discussed. In particular, the exclusion restriction is untestable and frequently hard to defend in practice. In our example, if high-income families live in suburban areas with bad public transportation connections, then the availability of the public transportation is likely related to the science score via household income (or socioeconomic status). Although conditioning on the observed household income can transform public transportation into a conditional IV (see next), one can frequently come up with additional scenarios that explains why the IV is related to the outcome and thus violates the exclusion restriction.

Another issue arises from “weak” IVs that are only weakly related to treatment. Weak IVs cause efficiency problems ( Wooldridge, 2012 ). If the availability of public transportation barely affects camp attendance because most parents give their children a ride anyway, the IV’s effect on the treatment ( γ ) is close to zero. Because γ ^ is the denominator in the CATE estimator, τ ^ C = δ ^ / γ ^ , an imprecisely estimated γ ^ results in a considerable over- or underestimation of CATE. Moreover, standard errors will be large.

One also needs to keep in mind that the substantive meaning of CATE depends on the chosen IV. Consider two slightly different IVs with respect to public transportation: the availability of (a) a bus service and (b) subway service. For the first IV, the complier population consists of students who choose to (not) attend the camp depending on the availability of a bus service. For the second IV, the complier population refers to the availability of a subway service. Because the two complier populations are very likely different from each other (students who are willing to take the subway might not be willing to take the bus), the corresponding CATEs refer to different subpopulations.

Strengthening IV Designs

Given the challenges in identifying a valid instrument from observed data, researchers should consider creating an IV at the design stage of a study. Although it might be impossible to directly assign subjects to treatment conditions, one might still be able to encourage participants to take the treatment. Subjects are randomly encouraged to sign up for treatment, but whether they actually comply with the encouragement is entirely their own decision ( Imai et al., 2011 ). Random encouragement qualifies as an IV because it very likely meets the exclusion restriction. For example, instead of collecting data on public transportation, researchers may advertise and recommend the science camp in a letter to the parents of a randomly selected sample of students.

With observational data it is hard to identify a valid IV because covariates that strongly predict the treatment are usually also related to the outcome. However, these covariates can still qualify as an IV if they affect the outcome only indirectly via other observed variables. Such covariates can be used as conditional IVs, that is, they meet the IV requirements conditional on the observed variables ( Brito & Pearl, 2002 ). Assume the availability of public transportation (IV) is associated with the science score via household income. Then, controlling for the reliably measured household income in both stages of the 2SLS analysis blocks the IV’s relation to the science score and turns public transportation into a conditional IV. However, controlling for a large set of variables does not guarantee that the exclusion restriction is more likely met. It may even result in more bias as compared to an IV analysis with fewer covariates ( Ding & Miratrix, 2015 ; Steiner & Kim, in press ). The choice of a valid conditional IV requires researchers to carefully select the control variables based on subject-matter theory.

The seminal article by Angrist et al. (1996) provides a thorough discussion of the IV design, and Steiner, Kim, et al. (2015 ) proved the identification result using graphical models. Excellent introductions to IV designs can be found in Angrist and Pischke (2009 , 2015) . Angrist and Krueger (1992) is an example of a creative application of the design with birthday as the IV. For encouragement designs, see Holland (1988) and Imai et al. (2011) .

MATCHING AND PROPENSITY SCORE DESIGN

This section considers quasi-experimental designs in which researchers lack control over treatment selection but have good knowledge about the selection mechanism or at least the confounders that simultaneously determine the treatment selection and the outcome. Due to self or third-person selection of subjects into treatment, the resulting treatment and control groups typically differ in observed but also unobserved baseline covariates. If we have reliable measures of all confounding covariates, then matching or propensity score (PS) designs balance groups on observed baseline covariates and thus enable the identification of causal effects ( Imbens & Rubin, 2015 ). Regression analysis and the analysis of covariance can also remove the confounding bias, but because they rely on functional form assumptions and extrapolation we discuss only nonparametric matching and PS designs.

Suppose that students decide on their own whether to attend the science camp. Although many factors can affect students’ decision, teachers with several years of experience of running the camp may know that selection is mostly driven by students’ science ability, liking of science, and their parents’ socioeconomic status. If all the selection-relevant factors that also affect the outcome are known, the question mark in Figure 2 can be replaced by the known confounding covariates.

Given the set of confounding covariates, causal inference with matching or PS designs is straightforward, at least theoretically. The basic one-to-one matching design matches each treatment subject to a control subject that is equivalent or at least very similar in observed covariates. To illustrate the idea of matching, consider a camp attendee with baseline measures of 80 on the science pre-test, 6 on liking science, and 50 on the socioeconomic status. Then a multivariate matching strategy tries to find a nonattendee with exactly the same or at least very similar baseline measures. If we succeed in finding close matches for all camp attendee, the matched samples of attendees and nonattendees will have almost identical covariate distributions.

Although multivariate matching works well when the number of confounders is small and the pool of control subjects is large relative to the number of treatment subjects, it is usually difficult to find close matches with a large set of covariates or a small pool of control subjects. Matching on the PS helps to overcome this issue because the PS is a univariate score computed from the observed covariates ( Rosenbaum & Rubin, 1983 ). The PS is formally defined as the conditional probability of receiving the treatment given the set of observed covariates X : PS = Pr( Z = 1 | X ).

Matching and PS designs usually investigate ATE = E [ Y i (1)] − E [ Y i (0)] or ATT = E [ Y i (1) | Z i = 1] – E [ Y i (0) | Z i = 1]. Both causal effects are identified if (a) the potential outcomes are statistically independent of the treatment indicator given the set of observed confounders X , { Y (1), Y (0)}⊥ Z | X ( unconfoundedness ; ⊥ denotes independence), and (b) the treatment probability is strictly between zero and one, 0 < Pr( Z = 1 | X ) < 1 ( positivity ).

By the positivity assumption we get E [ Y i (1)] = E X [ E [ Y i (1) | X ]] and E [ Y i (0)] = E X [ E [ Y i (0) | X ]]. If the unconfoundedness assumption holds, we can write the inner expectations as E [ Y i (1) | X ] = E [ Y i (1) | Z i =1; X ] and E [ Y i (0) | X ] = E [ Y i (0) | Z i = 0; X ]. Finally, because the treatment (control) outcomes of the treatment (control) subjects are actually observed, ATE is identified because it can be expressed in terms of observable quantities: ATE = E X [ E [ Y i | Z i = 1; X ]] – E X [ E [ Y i | Z i = 0; X ]]. The same can be shown for ATT. The unconfoundedness and positivity assumption are frequently referred to jointly as the strong ignorability assumption. Rosenbaum and Rubin (1983) proved that if the assignment is strongly ignorable given X , then it is also strongly ignorable given the PS alone.

Estimating ATE and ATT

Matching designs use a distance measure for matching each treatment subject to the closest control subject. The Mahalanobis distance is usually used for multivariate matching and the Euclidean distance on the logit of the PS for PS matching. Matching strategies differ with respect to the matching ratio (one-to-one or one-to-many), replacement of matched subjects (with or without replacement), use of a caliper (treatment subjects that do not have a control subject within a certain threshold remain unmatched), and the matching algorithm (greedy, genetic, or optimal matching; Sekhon, 2011 ; Steiner & Cook, 2013 ). Because we try to find at least one control subject for each treatment subject, matching estimators typically estimate ATT. Once treatment and control subjects are matched, ATT is computed as the difference in the mean outcome of the treatment and control group. An alternative matching strategy that allows for estimating ATE is full matching, which stratifies all subjects into the maximum number of strata, where each stratum contains at least one treatment and one control subject ( Hansen, 2004 ).

The PS can also be used for PS stratification and inverse-propensity weighting. PS stratification stratifies the treatment and control subjects into at least five strata and estimates the treatment effect within each stratum. ATE or ATT is then obtained as the weighted average of the stratum-specific treatment effects. Inverse-propensity weighting follows the same logic as inverse-probability weighting in survey research ( Horvitz & Thompson, 1952 ) and requires the computation of weights that refer to either the overall population (ATE) or the population of treated subjects only (ATT). Given the inverse-propensity weights, ATE or ATT is usually estimated via weighted least squares regression.

Because the true PSs are unknown, they need to be estimated from the observed data. The most common method for estimating the PS is logistic regression, which regresses the binary treatment indicator Z on predictors of the observed covariates. The PS model is specified according to balance criteria (instead of goodness of fit criteria), that is, the estimated PSs should remove all baseline differences in observed covariates ( Imbens & Rubin, 2015 ). The predicted probabilities from the PS model represent the estimated PSs.

All three PS designs—matching, stratification, and weighting—can benefit from additional covariance adjustments in an outcome regression. That is, for the matched, stratified or weighted data, the outcome is regressed on the treatment indicator and the additional covariates. Combining the PS design with a covariance adjustment gives researchers two chances to remove the confounding bias, by correctly specifying either the PS model or the outcome model. These combined methods are said to be doubly robust because they are robust against either the misspecification of the PS model or the misspecification of the outcome model ( Robins & Rotnitzky, 1995 ). The R packages optmatch ( Hansen & Klopfer, 2006 ) and MatchIt ( Ho et al., 2011 ) and the STATA command teffects , in particular teffects psmatch ( StataCorp, 2015 ), can be useful for matching or PS analyses.

The most challenging issue with matching and PS designs is the selection of covariates for establishing unconfoundedness. Ideally, subject-matter theory about the selection process and the outcome-generating model is used for selecting a set of covariates that removes all the confounding ( Pearl, 2009 ). If strong subject-matter theories are not available, selecting the right covariates is difficult. In the hope to remove a major part of the confounding bias—if not all of it—a frequently applied strategy is to match on as many covariates as possible. However, recent literature shows that thoughtless inclusion of covariates may increase rather than reduce the confounding bias ( Pearl, 2010 ; Steiner & Kim, in press). The risk of increasing bias can be reduced if the observed covariates cover a broad range of heterogeneous construct domains, including at least one reliable pretest measure of the outcome ( Steiner, Cook, et al., 2015 ). Besides having the right covariates, they also need to be reliably measured. The unreliable measurement of confounding covariates has a similar effect as the omission of a confounder: It results in a violation of the unconfoundedness assumption and thus in a biased effect estimate ( Steiner, Cook, & Shadish, 2011 ; Steiner & Kim, in press ).

Even if the set of reliably measured covariates establishes unconfoundedness, we still need to correctly specify the functional form of the PS model. Although parametric models like logistic regression, including higher order terms, might frequently approximate the correct functional form, they still rely on the linearity assumption. The linearity assumption can be relaxed if one estimates the PS with statistical learning algorithms like classification trees, neural networks, or the LASSO ( Keller, Kim, & Steiner, 2015 ; McCaffrey, Ridgeway, & Morral, 2004 ).

Strengthening Matching and PS Designs

The credibility of matching and PS designs heavily relies on the unconfoundedness assumption. Although empirically untestable, there are indirect ways for assessing unconfoundedness. First, unaffected (nonequivalent) outcomes that are known to be unaffected by the treatment can be used ( Shadish et al., 2002 ). For instance, we may expect that attendance in the science camp does not significantly affect the reading score. Thus, if we observe a significant group difference in the reading score after the PS adjustment, bias due to unobserved confounders (e.g., general intelligence) is still likely. Second, adding a second but conceptually different control group allows for a similar test as with the unaffected outcome ( Rosenbaum, 2002 ).

Because researchers rarely know whether the unconfoundedness assumption is actually met with the data at hand, it is important to assess the effect estimate’s sensitivity to potentially unobserved confounders. Sensitivity analyses investigate how strongly an estimate’s magnitude and significance changes if a confounder of a certain strength would have been omitted from the analyses. Causal conclusions are much more credible if the effect’s direction, magnitude, and significance is rather insensitive to omitted confounders ( Rosenbaum, 2002 ). However, despite the value of sensitivity analyses, they are not informative about whether hidden bias is actually present.

Schafer and Kang (2008) and Steiner and Cook (2013) provided a comprehensive introduction. Rigorous formalization and technical details of PS designs can be found in Imbens and Rubin (2015) . Rosenbaum (2002) discussed many important design issues in these designs.

COMPARATIVE INTERRUPTED TIME SERIES DESIGN

The designs discussed so far require researchers to have either full control over treatment assignment or reliable knowledge of the exogenous (IV) or endogenous part of the selection mechanism (i.e., the confounders). If none of these requirements are met, a comparative interrupted time series (CITS) design might be a viable alternative if (a) multiple measurements of the outcome ( time series ) are available for both the treatment and a comparison group and (b) the treatment group’s time series has been interrupted by an intervention.

Suppose that all students of one class in a school (say, an advanced science class) attend the camp, whereas all students of another class in the same school do not attend. Also assume that monthly measures of science achievement before and after the science camp are available. Figure 3 illustrates such a scenario where the x -axis represents time in Months and the y -axis the Science Score (aggregated at the class level). The filled symbols indicate the treatment group (science camp), open symbols the comparison group (no science camp). The science camp intervention divides both time series into a preintervention time series (circles) and a postintervention time series (squares). The changes in the levels and slopes of the pre- and postintervention regression lines represent the camp’s impact but possibly also the effect of other events that co-occur with the intervention. The dashed lines extrapolate the preintervention growth curves into the postintervention period, and thus represent the counterfactual situation where the intervention but also other co-occurring events are absent.

A hypothetical example of comparative interrupted time series design.

The strength of a CITS design is its ability to discriminate between the intervention’s effect and the effects of co-occurring events. Such events might be other potentially competing interventions (history effects) or changes in the measurement of the outcome (instrumentation), for instance. If the co-occurring events affect the treatment and comparison group to the same extent, then subtracting the changes in the comparison group’s growth curve from the changes in the treatment group’s growth curve provides a valid estimate of the intervention’s impact. Because we investigate the difference in the changes (= differences) of the two growth curves, the CITS design is a special case of the difference-in-differences design ( Somers et al., 2013 ).

Assume that a daily TV series about Albert Einstein was broadcast in the evenings of the science camp week and that students of both classes were exposed to the same extent to the TV series. It follows that the comparison group’s change in the growth curve represents the TV series’ impact. The comparison group’s time series in Figure 3 indicates that the TV series might have had an immediate impact on the growth curve’s level but almost no effect on the slope. On the other hand, the treatment group’s change in the growth curve is due to both the science camp and the TV series. Thus, in differencing out the TV series’ effect (estimated from the comparison group) we can identify the camp effect.

Let t c denote the time point of the intervention, then the intervention’s effect on the treated (ATT) at a postintervention time point t ≥ t c is defined as τ t = E [ Y i t T ( 1 ) ] − E [ Y i t T ( 0 ) ] , where Y i t T ( 0 ) and Y i t T ( 1 ) are the potential control and treatment outcomes of subject i in the treatment group ( T ) at time point t . The time series of the expected potential outcomes can be formalized as sum of nonparametric but additive time-dependent functions. The treatment group’s expected potential control outcome can be represented as E [ Y i t T ( 0 ) ] = f 0 T ( t ) + f E T ( t ) , where the control function f 0 T ( t ) generates the expected potential control outcomes in absence of any interventions ( I ) or co-occurring events ( E ), and the event function f E T ( t ) adds the effects of co-occurring events. Similarly, the expected potential treatment outcome can be written as E [ Y i t T ( 1 ) ] = f 0 T ( t ) + f E T ( t ) + f I T ( t ) , which adds the intervention’s effect τ t = f I T ( t ) to the control and event function. In the absence of a comparison group, we can try to identify the impact of the intervention by comparing the observable postintervention outcomes to the extrapolated outcomes from the preintervention time series (dashed line in Figure 3 ). Extrapolation is necessary because we do not observe any potential control outcomes in the postintervention period (only potential treatment outcomes are observed). Let f ^ 0 T ( t ) denote the parametric extrapolation of the preintervention control function f 0 T ( t ) , then the observable pre–post-intervention difference ( PP T ) in the expected control outcome is P P t T = f 0 T ( t ) + f E T ( t ) + f I T ( t ) − f ^ 0 T ( t ) = f I T ( t ) + ( f 0 T ( t ) − f ^ 0 T ( t ) ) + f E T ( t ) . Thus, in the absence of a comparison group, ATT is identified (i.e., P P t T = f I T ( t ) = τ t ) only if the control function is correctly specified ( f 0 T ( t ) = f ^ 0 T ( t ) ) and if no co-occurring events are present ( f E T ( t ) = 0 ).

The comparison group in a CITS design allows us to relax both of these identifying assumptions. In order to see this, we first define the expected control outcomes of the comparison group ( C ) as a sum of two time-dependent functions as before: E [ Y i t C ( 0 ) ] = f 0 C ( t ) + f E C ( t ) . Then, in extrapolating the comparison group’s preintervention function into the postintervention period, f ^ 0 C ( t ) , we can compute the pre–post-intervention difference for the comparison group: P P t C = f 0 C ( t ) + f E C ( t ) − f ^ 0 C ( t ) = f E C ( t ) + ( f 0 C ( t ) − f ^ 0 C ( t ) ) If the control function is correctly specified f 0 C ( t ) = f ^ 0 C ( t ) , the effect of co-occurring events is identified P P t C = f E C ( t ) . However, we do not necessarily need a correctly specified control function, because in a CITS design we focus on the difference in the treatment and comparison group’s pre–post-intervention differences, that is, P P t T − P P t C = f I T ( t ) + { ( f 0 T ( t ) − f ^ 0 T ( t ) ) − ( f 0 C ( t ) − f ^ 0 C ( t ) ) } + { f E T ( t ) − f E C ( t ) } . Thus, ATT is identified, P P t T − P P t C = f I T ( t ) = τ t , if (a) both control functions are either correctly specified or misspecified to the same additive extent such that ( f 0 T ( t ) − f ^ 0 T ( t ) ) = ( f 0 C ( t ) − f ^ 0 C ( t ) ) ( no differential misspecification ) and (b) the effect of co-occurring events is identical in the treatment and comparison group, f E T ( t ) = f E C ( t ) ( no differential event effects ).

Estimating ATT

CITS designs are typically analyzed with linear regression models that regress the outcome Y on the centered time variable ( T – t c ), the intervention indicator Z ( Z = 0 if t < t c , otherwise Z = 1), the group indicator G ( G = 1 for the treatment group and G = 0 for the control group), and the corresponding two-way and three-way interactions:

Depending on the number of subjects in each group, fixed or random effects for the subjects are included as well (time fixed or random effect can also be considered). β ^ 5 estimates the intervention’s immediate effect at the onset of the intervention (change in intercept) and β ^ 7 the intervention’s effect on the growth rate (change in slope). The inclusion of dummy variables for each postintervention time point (plus their interaction with the intervention and group indicators) would allow for a direct estimation of the time-specific effects. If the time series are long enough (at least 100 time points), then a more careful modeling of the autocorrelation structure via time series models should be considered.

Compared to other designs, CITS designs heavily rely on extrapolation and thus on functional form assumptions. Therefore, it is crucial that the functional forms of the pre- and postintervention time series (including their extrapolations) are correctly specified or at least not differentially misspecified. With short time series or measurement points that inadequately capture periodical variations, the correct specification of the functional form is very challenging. Another specification aspect concerns serial dependencies among the data points. Failing to model serial dependencies can bias effect estimates and their standard errors such that significance tests might be misleading. Accounting for serial dependencies requires autoregressive models (e.g., ARIMA models), but the time series should have at least 100 time points ( West, Biesanz, & Pitts, 2000 ). Standard fixed effects or random effects models deal at least partially with the dependence structure. Robust standard errors (e.g., Huber-White corrected ones) or the bootstrap can also be used to account for dependency structures.

Events that co-occur with the intervention of interest, like history or instrumentation effects, are a major threat to the time series designs that lack a comparison group ( Shadish et al., 2002 ). CITS designs are rather robust to co-occurring events as long as the treatment and comparison groups are affected to the same additive extent. However, there is no guarantee that both groups are exposed to the same events and affected to the same extent. For example, if students who do not attend the camp are less likely to watch the TV series, its effect cannot be completely differenced out (unless the exposure to the TV series is measured). If one uses aggregated data like class or school averages of achievement scores, then differential compositional shifts over time can also invalidate the CITS design. Compositional shifts occur due to dropouts or incoming subjects over time.

Strengthening CITS Designs

If the treatment and comparison group’s preintervention time series are very different (different levels and slopes), then the assumption that history or instrumentation threats affect both groups to the same additive extent may not hold. Matching treatment and comparison subjects prior to the analysis can increase the plausibility of this assumption. Instead of using all nonparticipating students of the comparison class, we may select only those students who have a similar level and growth in the preintervention science scores as the students participating in the camp. We can also match on additional covariates like socioeconomic status or motivation levels. Multivariate or PS matching can be used for this purpose. If the two groups are similar, it is more likely that they are affected by co-occurring events to the same extent.

As with the matching and PS designs, using an unaffected outcome in CITS designs helps to probe the untestable assumptions ( Coryn & Hobson, 2011 ; Shadish et al., 2002 ). For instance, we might expect that attending the science camp does not affect students’ reading scores but that some validity threats (e.g., attrition) operate on both the reading and science outcome. If we find a significant camp effect on the reading score, the validity of the CITS design for evaluating the camp’s impact on the science score is in doubt.

Another strategy to avoid validity threats is to control the time point of the intervention if possible. Researchers can wait with the implementation of the treatment until they have enough preintervention measures for reliably estimating the functional form. They can also choose to intervene when threats to validity are less likely (avoiding the week of the TV series). Control over the intervention also allows researchers to introduce and remove the treatment in subsequent time intervals, maybe even with switching replications between two (or more) groups. If the treatment is effective, we expect that the pattern of the intervention scheme is directly reflected in the time series of the outcome (for more details, see Shadish et al., 2002 ; for the literature on single case designs, see Kazdin, 2011 ).

A comprehensive introduction to CITS design can be found in Shadish et al. (2002) , which also addresses many classical applications. For more technical details of its identification, refer to Lechner (2011) . Wong, Cook, and Steiner (2009) evaluated the effect of No Child Left Behind using a CITS design.

CONCLUDING REMARKS

This article discussed four of the strongest quasi-experimental designs for causal inference when randomized experiments are not feasible. For each design we highlighted the identification strategies and the required assumptions. In practice, it is crucial that the design assumptions are met, otherwise biased effect estimates result. Because most important assumptions like the exclusion restriction or the unconfoundedness assumption are not directly testable, researchers should always try to assess their plausibility via indirect tests and investigate the effect estimates’ sensitivity to violations of these assumptions.

Our discussion of RD, IV, PS, and CITS designs made it also very clear that, in comparison to RCTs, quasi-experimental designs rely on more or stronger assumptions. With prefect control over treatment assignment and treatment implementation (as in an RCT), causal inference is warranted by a minimal set of assumptions. But with limited control over and knowledge about treatment assignment and implementation, stronger assumptions are required and causal effects might be identifiable only for local subpopulations. Nonetheless, observational data sometimes meet the assumptions of a quasi-experimental design, at least approximately, such that causal conclusions are credible. If so, the estimates of quasi-experimental designs—which exploit naturally occurring selection processes and real-world implementations of the treatment—are frequently better generalizable than the results from a controlled laboratory experiment. Thus, if external validity is a major concern, the results of randomized experiments should always be complemented by findings from valid quasi-experiments.

• Angrist JD, Imbens GW, & Rubin DB (1996). Identification of causal effects using instrumental variables . Journal of the American Statistical Association , 91 , 444–455. [ Google Scholar ]
• Angrist JD, & Krueger AB (1992). The effect of age at school entry on educational attainment: An application of instrumental variables with moments from two samples . Journal of the American Statistical Association , 87 , 328–336. [ Google Scholar ]
• Angrist JD, & Lavy V (1999). Using Maimonides’ rule to estimate the effect of class size on scholastic achievment . Quarterly Journal of Economics , 114 , 533–575. [ Google Scholar ]
• Angrist JD, & Pischke JS (2009). Mostly harmless econometrics: An empiricist’s companion . Princeton, NJ: Princeton University Press. [ Google Scholar ]
• Angrist JD, & Pischke JS (2015). Mastering’metrics: The path from cause to effect . Princeton, NJ: Princeton University Press. [ Google Scholar ]
• Baum CF, Schaffer ME, & Stillman S (2007). Enhanced routines for instrumental variables/generalized method of moments estimation and testing . The Stata Journal , 7 , 465–506. [ Google Scholar ]
• Black D, Galdo J, & Smith JA (2007). Evaluating the bias of the regression discontinuity design using experimental data (Working paper) . Chicago, IL: University of Chicago. [ Google Scholar ]
• Brito C, & Pearl J (2002). Generalized instrumental variables In Darwiche A & Friedman N (Eds.), Uncertainty in artificial intelligence (pp. 85–93). San Francisco, CA: Morgan Kaufmann. [ Google Scholar ]
• Calonico S, Cattaneo MD, & Titiunik R (2015). rdrobust: Robust data-driven statistical inference in regression-discontinuity designs (R package ver. 0.80) . Retrieved from http://CRAN.R-project.org/package=rdrobust
• Coryn CLS, & Hobson KA (2011). Using nonequivalent dependent variables to reduce internal validity threats in quasi-experiments: Rationale, history, and examples from practice . New Directions for Evaluation , 131 , 31–39. [ Google Scholar ]
• Dimmery D (2013). rdd: Regression discontinuity estimation (R package ver. 0.56) . Retrieved from http://CRAN.R-project.org/package=rdd
• Ding P, & Miratrix LW (2015). To adjust or not to adjust? Sensitivity analysis of M-bias and butterfly-bias . Journal of Causal Inference , 3 ( 1 ), 41–57. [ Google Scholar ]
• Fox J (2006). Structural equation modeling with the sem package in R . Structural Equation Modeling , 13 , 465–486. [ Google Scholar ]
• Hahn J, Todd P, & Van der Klaauw W (2001). Identification and estimation of treatment effects with a regression–discontinuity design . Econometrica , 69 ( 1 ), 201–209. [ Google Scholar ]
• Hansen BB (2004). Full matching in an observational study of coaching for the SAT . Journal of the American Statistical Association , 99 , 609–618. [ Google Scholar ]
• Hansen BB, & Klopfer SO (2006). Optimal full matching and related designs via network flows . Journal of Computational and Graphical Statistics , 15 , 609–627. [ Google Scholar ]
• Ho D, Imai K, King G, & Stuart EA (2011). MatchIt: Nonparametric preprocessing for parametric causal inference . Journal of Statistical Software , 42 ( 8 ), 1–28. Retrieved from http://www.jstatsoft.org/v42/i08/ [ Google Scholar ]
• Holland PW (1986). Statistics and causal inference . Journal of the American Statistical Association , 81 , 945–960. [ Google Scholar ]
• Holland PW (1988). Causal inference, path analysis and recursive structural equations models . ETS Research Report Series . doi: 10.1002/j.2330-8516.1988.tb00270.x [ CrossRef ] [ Google Scholar ]
• Horvitz DG, & Thompson DJ (1952). A generalization of sampling without replacement from a finite universe . Journal of the American Statistical Association , 47 , 663–685. [ Google Scholar ]
• Imai K, Keele L, Tingley D, & Yamamoto T (2011). Unpacking the black box of causality: Learning about causal mechanisms from experimental and observational studies . American Political Science Review , 105 , 765–789. [ Google Scholar ]
• Imbens GW, & Lemieux T (2008). Regression discontinuity designs: A guide to practice . Journal of Econometrics , 142 , 615–635. [ Google Scholar ]
• Imbens GW, & Rubin DB (2015). Causal inference in statistics, social, and biomedical sciences . New York, NY: Cambridge University Press. [ Google Scholar ]
• Kazdin AE (2011). Single-case research designs: Methods for clinical and applied settings . New York, NY: Oxford University Press. [ Google Scholar ]
• Keller B, Kim JS, & Steiner PM (2015). Neural networks for propensity score estimation: Simulation results and recommendations In van der Ark LA, Bolt DM, Chow S-M, Douglas JA, & Wang W-C (Eds.), Quantitative psychology research (pp. 279–291). New York, NY: Springer. [ Google Scholar ]
• Lechner M (2011). The estimation of causal effects by difference-in-difference methods . Foundations and Trends in Econometrics , 4 , 165–224. [ Google Scholar ]
• Lee DS, & Lemieux T (2010). Regression discontinuity designs in economics . Journal of Economic Literature , 48 , 281–355. [ Google Scholar ]
• McCaffrey DF, Ridgeway G, & Morral AR (2004). Propensity score estimation with boosted regression for evaluating causal effects in observational studies . Psychological Methods , 9 , 403–425. [ PubMed ] [ Google Scholar ]
• McCrary J (2008). Manipulation of the running variable in the regression discontinuity design: A density test . Journal of Econometrics , 142 , 698–714. [ Google Scholar ]
• Nichols A (2007). rd: Stata modules for regression discontinuity estimation . Retrieved from http://ideas.repec.org/c/boc/bocode/s456888.html
• Pearl J (2009). C ausality: Models, reasoning, and inference (2nd ed.). New York, NY: Cambridge University Press. [ Google Scholar ]
• Pearl J (2010). On a class of bias-amplifying variables that endanger effect estimates In Proceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (pp. 425–432). Corvallis, OR: Association for Uncertainty in Artificial Intelligence. [ Google Scholar ]
• Robins JM, & Rotnitzky A (1995). Semiparametric efficiency in multivariate regression models with missing data . Journal of the American Statistical Association , 90 ( 429 ), 122–129. [ Google Scholar ]
• Rosenbaum PR (2002). Observational studies . New York, NY: Springer. [ Google Scholar ]
• Rosenbaum PR, & Rubin DB (1983). The central role of the propensity score in observational studies for causal effects . Biometrika , 70 ( 1 ), 41–55. [ Google Scholar ]
• Schafer JL, & Kang J (2008). Average causal effects from nonrandomized studies: A practical guide and simulated example . Psychological Methods , 13 , 279–313. [ PubMed ] [ Google Scholar ]
• Sekhon JS (2011). Multivariate and propensity score matching software with automated balance optimization: The matching package for R . Journal of Statistical Software , 42 ( 7 ), 1–52. [ Google Scholar ]
• Shadish WR, Cook TD, & Campbell DT (2002). Experimental and quasi-experimental designs for generalized causal inference . Boston, MA: Houghton-Mifflin. [ Google Scholar ]
• Somers M, Zhu P, Jacob R, & Bloom H (2013). The validity and precision of the comparative interrupted time series design and the difference-in-difference design in educational evaluation (MDRC working paper in research methodology) . New York, NY: MDRC. [ Google Scholar ]
• StataCorp. (2015). Stata treatment-effects reference manual: Potential outcomes/counterfactual outcomes . College Station, TX: Stata Press; Retrieved from http://www.stata.com/manuals14/te.pdf [ Google Scholar ]
• Steiner PM, & Cook D (2013). Matching and propensity scores In Little T (Ed.), The Oxford handbook of quantitative methods in psychology (Vol. 1 , pp. 237–259). New York, NY: Oxford University Press. [ Google Scholar ]
• Steiner PM, Cook TD, Li W, & Clark MH (2015). Bias reduction in quasi-experiments with little selection theory but many covariates . Journal of Research on Educational Effectiveness , 8 , 552–576. [ Google Scholar ]
• Steiner PM, Cook TD, & Shadish WR (2011). On the importance of reliable covariate measurement in selection bias adjustments using propensity scores . Journal of Educational and Behavioral Statistics , 36 , 213–236. [ Google Scholar ]
• Steiner PM, & Kim Y (in press). The mechanics of omitted variable bias: Bias amplification and cancellation of offsetting biases . Journal of Causal Inference . [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Steiner PM, Kim Y, Hall CE, & Su D (2015). Graphical models for quasi-experimental designs . Sociological Methods & Research. Advance online publication . doi: 10.1177/0049124115582272 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• West SG, Biesanz JC, & Pitts SC (2000). Causal inference and generalization in field settings: Experimental and quasi-experimental designs In Reis HT & Judd CM (Eds.), Handbook of research methods in social and personality psychology (pp. 40–84). New York, NY: Cambridge University Press. [ Google Scholar ]
• Wing C, & Cook TD (2013). Strengthening the regression discontinuity design using additional design elements: A within-study comparison . Journal of Policy Analysis and Management , 32 , 853–877. [ Google Scholar ]
• Wong M, Cook TD, & Steiner PM (2009). No Child Left Behind: An interim evaluation of its effects on learning using two interrupted time series each with its own non-equivalent comparison series (Working Paper No. WP-09–11) . Evanston, IL: Institute for Policy Research, Northwestern University. [ Google Scholar ]
• Wong VC, Wing C, Steiner PM, Wong M, & Cook TD (2012). Research designs for program evaluation . Handbook of Psychology , 2 , 316–341. [ Google Scholar ]
• Wooldridge J (2012). Introductory econometrics: A modern approach (5th ed.). Mason, OH: South-Western Cengage Learning. [ Google Scholar ]

A Modern Guide to Understanding and Conducting Research in Psychology

Chapter 7 quasi-experimental research, learning objectives.

• Explain what quasi-experimental research is and distinguish it clearly from both experimental and correlational research.
• Describe three different types of quasi-experimental research designs (nonequivalent groups, pretest-posttest, and interrupted time series) and identify examples of each one.

The prefix quasi means “resembling.” Thus quasi-experimental research is research that resembles experimental research but is not true experimental research. Although the independent variable is manipulated, participants are not randomly assigned to conditions or orders of conditions ( Cook et al., 1979 ) . Because the independent variable is manipulated before the dependent variable is measured, quasi-experimental research eliminates the directionality problem. But because participants are not randomly assigned—making it likely that there are other differences between conditions—quasi-experimental research does not eliminate the problem of confounding variables. In terms of internal validity, therefore, quasi-experiments are generally somewhere between correlational studies and true experiments.

Quasi-experiments are most likely to be conducted in field settings in which random assignment is difficult or impossible. They are often conducted to evaluate the effectiveness of a treatment—perhaps a type of psychotherapy or an educational intervention. There are many different kinds of quasi-experiments, but we will discuss just a few of the most common ones here, focusing first on nonequivalent groups, pretest-posttest, interrupted time series, and combination designs before turning to single subject designs (including reversal and multiple-baseline designs).

7.1 Nonequivalent Groups Design

Recall that when participants in a between-subjects experiment are randomly assigned to conditions, the resulting groups are likely to be quite similar. In fact, researchers consider them to be equivalent. When participants are not randomly assigned to conditions, however, the resulting groups are likely to be dissimilar in some ways. For this reason, researchers consider them to be nonequivalent. A nonequivalent groups design , then, is a between-subjects design in which participants have not been randomly assigned to conditions.

Imagine, for example, a researcher who wants to evaluate a new method of teaching fractions to third graders. One way would be to conduct a study with a treatment group consisting of one class of third-grade students and a control group consisting of another class of third-grade students. This would be a nonequivalent groups design because the students are not randomly assigned to classes by the researcher, which means there could be important differences between them. For example, the parents of higher achieving or more motivated students might have been more likely to request that their children be assigned to Ms. Williams’s class. Or the principal might have assigned the “troublemakers” to Mr. Jones’s class because he is a stronger disciplinarian. Of course, the teachers’ styles, and even the classroom environments, might be very different and might cause different levels of achievement or motivation among the students. If at the end of the study there was a difference in the two classes’ knowledge of fractions, it might have been caused by the difference between the teaching methods—but it might have been caused by any of these confounding variables.

Of course, researchers using a nonequivalent groups design can take steps to ensure that their groups are as similar as possible. In the present example, the researcher could try to select two classes at the same school, where the students in the two classes have similar scores on a standardized math test and the teachers are the same sex, are close in age, and have similar teaching styles. Taking such steps would increase the internal validity of the study because it would eliminate some of the most important confounding variables. But without true random assignment of the students to conditions, there remains the possibility of other important confounding variables that the researcher was not able to control.

7.2 Pretest-Posttest Design

In a pretest-posttest design , the dependent variable is measured once before the treatment is implemented and once after it is implemented. Imagine, for example, a researcher who is interested in the effectiveness of an STEM education program on elementary school students’ attitudes toward science, technology, engineering and math. The researcher could measure the attitudes of students at a particular elementary school during one week, implement the STEM program during the next week, and finally, measure their attitudes again the following week. The pretest-posttest design is much like a within-subjects experiment in which each participant is tested first under the control condition and then under the treatment condition. It is unlike a within-subjects experiment, however, in that the order of conditions is not counterbalanced because it typically is not possible for a participant to be tested in the treatment condition first and then in an “untreated” control condition.

If the average posttest score is better than the average pretest score, then it makes sense to conclude that the treatment might be responsible for the improvement. Unfortunately, one often cannot conclude this with a high degree of certainty because there may be other explanations for why the posttest scores are better. One category of alternative explanations goes under the name of history . Other things might have happened between the pretest and the posttest. Perhaps an science program aired on television and many of the students watched it, or perhaps a major scientific discover occured and many of the students heard about it. Another category of alternative explanations goes under the name of maturation . Participants might have changed between the pretest and the posttest in ways that they were going to anyway because they are growing and learning. If it were a yearlong program, participants might become more exposed to STEM subjects in class or better reasoners and this might be responsible for the change.

Another alternative explanation for a change in the dependent variable in a pretest-posttest design is regression to the mean . This refers to the statistical fact that an individual who scores extremely on a variable on one occasion will tend to score less extremely on the next occasion. For example, a bowler with a long-term average of 150 who suddenly bowls a 220 will almost certainly score lower in the next game. Her score will “regress” toward her mean score of 150. Regression to the mean can be a problem when participants are selected for further study because of their extreme scores. Imagine, for example, that only students who scored especially low on a test of fractions are given a special training program and then retested. Regression to the mean all but guarantees that their scores will be higher even if the training program has no effect. A closely related concept—and an extremely important one in psychological research—is spontaneous remission . This is the tendency for many medical and psychological problems to improve over time without any form of treatment. The common cold is a good example. If one were to measure symptom severity in 100 common cold sufferers today, give them a bowl of chicken soup every day, and then measure their symptom severity again in a week, they would probably be much improved. This does not mean that the chicken soup was responsible for the improvement, however, because they would have been much improved without any treatment at all. The same is true of many psychological problems. A group of severely depressed people today is likely to be less depressed on average in 6 months. In reviewing the results of several studies of treatments for depression, researchers Michael Posternak and Ivan Miller found that participants in waitlist control conditions improved an average of 10 to 15% before they received any treatment at all ( Posternak & Miller, 2001 ) . Thus one must generally be very cautious about inferring causality from pretest-posttest designs.

Finally, it is possible that the act of taking a pretest can sensitize participants to the measurement process or heighten their awareness of the variable under investigation. This heightened sensitivity, called a testing effect , can subsequently lead to changes in their posttest responses, even in the absence of any external intervention effect.

7.3 Interrupted Time Series Design

A variant of the pretest-posttest design is the interrupted time-series design . A time series is a set of measurements taken at intervals over a period of time. For example, a manufacturing company might measure its workers’ productivity each week for a year. In an interrupted time series-design, a time series like this is “interrupted” by a treatment. In a recent COVID-19 study, the intervention involved the implementation of state-issued mask mandates and restrictions on on-premises restaurant dining. The researchers examined the impact of these measures on COVID-19 cases and deaths ( Guy Jr et al., 2021 ) . Since there was a rapid reduction in daily case and death growth rates following the implementation of mask mandates, and this effect persisted for an extended period, the researchers concluded that the implementation of mask mandates was the cause of the decrease in COVID-19 transmission. This study employed an interrupted time series design, similar to a pretest-posttest design, as it involved measuring the outcomes before and after the intervention. However, unlike the pretest-posttest design, it incorporated multiple measurements before and after the intervention, providing a more comprehensive analysis of the policy impacts.

Figure 7.1 shows data from a hypothetical interrupted time-series study. The dependent variable is the number of student absences per week in a research methods course. The treatment is that the instructor begins publicly taking attendance each day so that students know that the instructor is aware of who is present and who is absent. The top panel of Figure 7.1 shows how the data might look if this treatment worked. There is a consistently high number of absences before the treatment, and there is an immediate and sustained drop in absences after the treatment. The bottom panel of Figure 7.1 shows how the data might look if this treatment did not work. On average, the number of absences after the treatment is about the same as the number before. This figure also illustrates an advantage of the interrupted time-series design over a simpler pretest-posttest design. If there had been only one measurement of absences before the treatment at Week 7 and one afterward at Week 8, then it would have looked as though the treatment were responsible for the reduction. The multiple measurements both before and after the treatment suggest that the reduction between Weeks 7 and 8 is nothing more than normal week-to-week variation.

Figure 7.1: Hypothetical interrupted time-series design. The top panel shows data that suggest that the treatment caused a reduction in absences. The bottom panel shows data that suggest that it did not.

7.4 Combination Designs

A type of quasi-experimental design that is generally better than either the nonequivalent groups design or the pretest-posttest design is one that combines elements of both. There is a treatment group that is given a pretest, receives a treatment, and then is given a posttest. But at the same time there is a control group that is given a pretest, does not receive the treatment, and then is given a posttest. The question, then, is not simply whether participants who receive the treatment improve but whether they improve more than participants who do not receive the treatment.

Imagine, for example, that students in one school are given a pretest on their current level of engagement in pro-environmental behaviors (i.e., recycling, eating less red meat, abstaining for single-use plastics, etc.), then are exposed to an pro-environmental program in which they learn about the effects of human caused climate change on the planet, and finally are given a posttest. Students in a similar school are given the pretest, not exposed to an pro-environmental program, and finally are given a posttest. Again, if students in the treatment condition become more involved in pro-environmental behaviors, this could be an effect of the treatment, but it could also be a matter of history or maturation. If it really is an effect of the treatment, then students in the treatment condition should become engage in more pro-environmental behaviors than students in the control condition. But if it is a matter of history (e.g., news of a forest fire or drought) or maturation (e.g., improved reasoning or sense of responsibility), then students in the two conditions would be likely to show similar amounts of change. This type of design does not completely eliminate the possibility of confounding variables, however. Something could occur at one of the schools but not the other (e.g., a local heat wave with record high temperatures), so students at the first school would be affected by it while students at the other school would not.

Finally, if participants in this kind of design are randomly assigned to conditions, it becomes a true experiment rather than a quasi experiment. In fact, this kind of design has now been conducted many times—to demonstrate the effectiveness of psychotherapy.

KEY TAKEAWAYS

• Quasi-experimental research involves the manipulation of an independent variable without the random assignment of participants to conditions or orders of conditions. Among the important types are nonequivalent groups designs, pretest-posttest, and interrupted time-series designs.
• Quasi-experimental research eliminates the directionality problem because it involves the manipulation of the independent variable. It does not eliminate the problem of confounding variables, however, because it does not involve random assignment to conditions. For these reasons, quasi-experimental research is generally higher in internal validity than correlational studies but lower than true experiments.
• Practice: Imagine that two college professors decide to test the effect of giving daily quizzes on student performance in a statistics course. They decide that Professor A will give quizzes but Professor B will not. They will then compare the performance of students in their two sections on a common final exam. List five other variables that might differ between the two sections that could affect the results.

regression to the mean

Spontaneous remission, 7.5 single-subject research.

• Explain what single-subject research is, including how it differs from other types of psychological research and who uses single-subject research and why.
• Design simple single-subject studies using reversal and multiple-baseline designs.
• Explain how single-subject research designs address the issue of internal validity.
• Interpret the results of simple single-subject studies based on the visual inspection of graphed data.
• Explain some of the points of disagreement between advocates of single-subject research and advocates of group research.

Researcher Vance Hall and his colleagues were faced with the challenge of increasing the extent to which six disruptive elementary school students stayed focused on their schoolwork ( Hall et al., 1968 ) . For each of several days, the researchers carefully recorded whether or not each student was doing schoolwork every 10 seconds during a 30-minute period. Once they had established this baseline, they introduced a treatment. The treatment was that when the student was doing schoolwork, the teacher gave him or her positive attention in the form of a comment like “good work” or a pat on the shoulder. The result was that all of the students dramatically increased their time spent on schoolwork and decreased their disruptive behavior during this treatment phase. For example, a student named Robbie originally spent 25% of his time on schoolwork and the other 75% “snapping rubber bands, playing with toys from his pocket, and talking and laughing with peers” (p. 3). During the treatment phase, however, he spent 71% of his time on schoolwork and only 29% on other activities. Finally, when the researchers had the teacher stop giving positive attention, the students all decreased their studying and increased their disruptive behavior. This was consistent with the claim that it was, in fact, the positive attention that was responsible for the increase in studying. This was one of the first studies to show that attending to positive behavior—and ignoring negative behavior—could be a quick and effective way to deal with problem behavior in an applied setting.

Figure 7.2: Single-subject research has shown that positive attention from a teacher for studying can increase studying and decrease disruptive behavior. Photo by Jerry Wang on Unsplash.

Most of this book is about what can be called group research, which typically involves studying a large number of participants and combining their data to draw general conclusions about human behavior. The study by Hall and his colleagues, in contrast, is an example of single-subject research, which typically involves studying a small number of participants and focusing closely on each individual. In this section, we consider this alternative approach. We begin with an overview of single-subject research, including some assumptions on which it is based, who conducts it, and why they do. We then look at some basic single-subject research designs and how the data from those designs are analyzed. Finally, we consider some of the strengths and weaknesses of single-subject research as compared with group research and see how these two approaches can complement each other.

Overview of Single-Subject Research

What is single-subject research.

Single-subject research is a type of quantitative, quasi-experimental research that involves studying in detail the behavior of each of a small number of participants. Note that the term single-subject does not mean that only one participant is studied; it is more typical for there to be somewhere between two and 10 participants. (This is why single-subject research designs are sometimes called small-n designs, where n is the statistical symbol for the sample size.) Single-subject research can be contrasted with group research , which typically involves studying large numbers of participants and examining their behavior primarily in terms of group means, standard deviations, and so on. The majority of this book is devoted to understanding group research, which is the most common approach in psychology. But single-subject research is an important alternative, and it is the primary approach in some areas of psychology.

Before continuing, it is important to distinguish single-subject research from two other approaches, both of which involve studying in detail a small number of participants. One is qualitative research, which focuses on understanding people’s subjective experience by collecting relatively unstructured data (e.g., detailed interviews) and analyzing those data using narrative rather than quantitative techniques (see. Single-subject research, in contrast, focuses on understanding objective behavior through experimental manipulation and control, collecting highly structured data, and analyzing those data quantitatively.

It is also important to distinguish single-subject research from case studies. A case study is a detailed description of an individual, which can include both qualitative and quantitative analyses. (Case studies that include only qualitative analyses can be considered a type of qualitative research.) The history of psychology is filled with influential cases studies, such as Sigmund Freud’s description of “Anna O.” (see box “The Case of ‘Anna O.’”) and John Watson and Rosalie Rayner’s description of Little Albert ( Watson & Rayner, 1920 ) who learned to fear a white rat—along with other furry objects—when the researchers made a loud noise while he was playing with the rat. Case studies can be useful for suggesting new research questions and for illustrating general principles. They can also help researchers understand rare phenomena, such as the effects of damage to a specific part of the human brain. As a general rule, however, case studies cannot substitute for carefully designed group or single-subject research studies. One reason is that case studies usually do not allow researchers to determine whether specific events are causally related, or even related at all. For example, if a patient is described in a case study as having been sexually abused as a child and then as having developed an eating disorder as a teenager, there is no way to determine whether these two events had anything to do with each other. A second reason is that an individual case can always be unusual in some way and therefore be unrepresentative of people more generally. Thus case studies have serious problems with both internal and external validity.

The Case of “Anna O.”

Sigmund Freud used the case of a young woman he called “Anna O.” to illustrate many principles of his theory of psychoanalysis ( Freud, 1957 ) . (Her real name was Bertha Pappenheim, and she was an early feminist who went on to make important contributions to the field of social work.) Anna had come to Freud’s colleague Josef Breuer around 1880 with a variety of odd physical and psychological symptoms. One of them was that for several weeks she was unable to drink any fluids. According to Freud,

She would take up the glass of water that she longed for, but as soon as it touched her lips she would push it away like someone suffering from hydrophobia.…She lived only on fruit, such as melons, etc., so as to lessen her tormenting thirst (p. 9).

But according to Freud, a breakthrough came one day while Anna was under hypnosis.

[S]he grumbled about her English “lady-companion,” whom she did not care for, and went on to describe, with every sign of disgust, how she had once gone into this lady’s room and how her little dog—horrid creature!—had drunk out of a glass there. The patient had said nothing, as she had wanted to be polite. After giving further energetic expression to the anger she had held back, she asked for something to drink, drank a large quantity of water without any difficulty, and awoke from her hypnosis with the glass at her lips; and thereupon the disturbance vanished, never to return.

Freud’s interpretation was that Anna had repressed the memory of this incident along with the emotion that it triggered and that this was what had caused her inability to drink. Furthermore, her recollection of the incident, along with her expression of the emotion she had repressed, caused the symptom to go away.

As an illustration of Freud’s theory, the case study of Anna O. is quite effective. As evidence for the theory, however, it is essentially worthless. The description provides no way of knowing whether Anna had really repressed the memory of the dog drinking from the glass, whether this repression had caused her inability to drink, or whether recalling this “trauma” relieved the symptom. It is also unclear from this case study how typical or atypical Anna’s experience was.

Figure 7.3: “Anna O.” was the subject of a famous case study used by Freud to illustrate the principles of psychoanalysis. Source: Wikimedia Commons

Assumptions of Single-Subject Research

Again, single-subject research involves studying a small number of participants and focusing intensively on the behavior of each one. But why take this approach instead of the group approach? There are two important assumptions underlying single-subject research, and it will help to consider them now.

First and foremost is the assumption that it is important to focus intensively on the behavior of individual participants. One reason for this is that group research can hide individual differences and generate results that do not represent the behavior of any individual. For example, a treatment that has a positive effect for half the people exposed to it but a negative effect for the other half would, on average, appear to have no effect at all. Single-subject research, however, would likely reveal these individual differences. A second reason to focus intensively on individuals is that sometimes it is the behavior of a particular individual that is primarily of interest. A school psychologist, for example, might be interested in changing the behavior of a particular disruptive student. Although previous published research (both single-subject and group research) is likely to provide some guidance on how to do this, conducting a study on this student would be more direct and probably more effective.

Another assumption of single-subject research is that it is important to study strong and consistent effects that have biological or social importance. Applied researchers, in particular, are interested in treatments that have substantial effects on important behaviors and that can be implemented reliably in the real-world contexts in which they occur. This is sometimes referred to as social validity ( Wolf, 1978 ) . The study by Hall and his colleagues, for example, had good social validity because it showed strong and consistent effects of positive teacher attention on a behavior that is of obvious importance to teachers, parents, and students. Furthermore, the teachers found the treatment easy to implement, even in their often chaotic elementary school classrooms.

Who Uses Single-Subject Research?

Single-subject research has been around as long as the field of psychology itself. In the late 1800s, one of psychology’s founders, Wilhelm Wundt, studied sensation and consciousness by focusing intensively on each of a small number of research participants. Herman Ebbinghaus’s research on memory and Ivan Pavlov’s research on classical conditioning are other early examples, both of which are still described in almost every introductory psychology textbook.

In the middle of the 20th century, B. F. Skinner clarified many of the assumptions underlying single-subject research and refined many of its techniques ( Skinner, 1938 ) . He and other researchers then used it to describe how rewards, punishments, and other external factors affect behavior over time. This work was carried out primarily using nonhuman subjects—mostly rats and pigeons. This approach, which Skinner called the experimental analysis of behavior —remains an important subfield of psychology and continues to rely almost exclusively on single-subject research. For examples of this work, look at any issue of the Journal of the Experimental Analysis of Behavior . By the 1960s, many researchers were interested in using this approach to conduct applied research primarily with humans—a subfield now called applied behavior analysis ( Baer et al., 1968 ) . Applied behavior analysis plays a significant role in contemporary research on developmental disabilities, education, organizational behavior, and health, among many other areas. Examples of this work (including the study by Hall and his colleagues) can be found in the Journal of Applied Behavior Analysis . The single-subject approach can also be used by clinicians who take any theoretical perspective—behavioral, cognitive, psychodynamic, or humanistic—to study processes of therapeutic change with individual clients and to document their clients’ improvement ( Kazdin, 2019 ) .

Single-Subject Research Designs

General features of single-subject designs.

Before looking at any specific single-subject research designs, it will be helpful to consider some features that are common to most of them. Many of these features are illustrated in Figure 7.4 , which shows the results of a generic single-subject study. First, the dependent variable (represented on the y-axis of the graph) is measured repeatedly over time (represented by the x-axis) at regular intervals. Second, the study is divided into distinct phases, and the participant is tested under one condition per phase. The conditions are often designated by capital letters: A, B, C, and so on. Thus Figure 7.4 represents a design in which the participant was tested first in one condition (A), then tested in another condition (B), and finally retested in the original condition (A). (This is called a reversal design and will be discussed in more detail shortly.)

Figure 7.4: Results of a generic single-subject study illustrating several principles of single-subject research.

Another important aspect of single-subject research is that the change from one condition to the next does not usually occur after a fixed amount of time or number of observations. Instead, it depends on the participant’s behavior. Specifically, the researcher waits until the participant’s behavior in one condition becomes fairly consistent from observation to observation before changing conditions. This is sometimes referred to as the steady state strategy ( Sidman, 1960 ) . The idea is that when the dependent variable has reached a steady state, then any change across conditions will be relatively easy to detect. Recall that we encountered this same principle when discussing experimental research more generally. The effect of an independent variable is easier to detect when the “noise” in the data is minimized.

Reversal Designs

The most basic single-subject research design is the reversal design , also called the ABA design . During the first phase, A, a baseline is established for the dependent variable. This is the level of responding before any treatment is introduced, and therefore the baseline phase is a kind of control condition. When steady state responding is reached, phase B begins as the researcher introduces the treatment. Again, the researcher waits until that dependent variable reaches a steady state so that it is clear whether and how much it has changed. Finally, the researcher removes the treatment and again waits until the dependent variable reaches a steady state. This basic reversal design can also be extended with the reintroduction of the treatment (ABAB), another return to baseline (ABABA), and so on. The study by Hall and his colleagues was an ABAB reversal design (Figure 7.5 ).

Figure 7.5: An approximation of the results for Hall and colleagues’ participant Robbie in their ABAB reversal design. The percentage of time he spent studying (the dependent variable) was low during the first baseline phase, increased during the first treatment phase until it leveled off, decreased during the second baseline phase, and again increased during the second treatment phase.

Why is the reversal—the removal of the treatment—considered to be necessary in this type of design? If the dependent variable changes after the treatment is introduced, it is not always clear that the treatment was responsible for the change. It is possible that something else changed at around the same time and that this extraneous variable is responsible for the change in the dependent variable. But if the dependent variable changes with the introduction of the treatment and then changes back with the removal of the treatment, it is much clearer that the treatment (and removal of the treatment) is the cause. In other words, the reversal greatly increases the internal validity of the study.

Multiple-Baseline Designs

There are two potential problems with the reversal design—both of which have to do with the removal of the treatment. One is that if a treatment is working, it may be unethical to remove it. For example, if a treatment seemed to reduce the incidence of self-injury in a developmentally disabled child, it would be unethical to remove that treatment just to show that the incidence of self-injury increases. The second problem is that the dependent variable may not return to baseline when the treatment is removed. For example, when positive attention for studying is removed, a student might continue to study at an increased rate. This could mean that the positive attention had a lasting effect on the student’s studying, which of course would be good, but it could also mean that the positive attention was not really the cause of the increased studying in the first place.

One solution to these problems is to use a multiple-baseline design , which is represented in Figure 7.6 . In one version of the design, a baseline is established for each of several participants, and the treatment is then introduced for each one. In essence, each participant is tested in an AB design. The key to this design is that the treatment is introduced at a different time for each participant. The idea is that if the dependent variable changes when the treatment is introduced for one participant, it might be a coincidence. But if the dependent variable changes when the treatment is introduced for multiple participants—especially when the treatment is introduced at different times for the different participants—then it is less likely to be a coincidence.

Figure 7.6: Results of a generic multiple-baseline study. The multiple baselines can be for different participants, dependent variables, or settings. The treatment is introduced at a different time on each baseline.

As an example, consider a study by Scott Ross and Robert Horner ( Ross et al., 2009 ) . They were interested in how a school-wide bullying prevention program affected the bullying behavior of particular problem students. At each of three different schools, the researchers studied two students who had regularly engaged in bullying. During the baseline phase, they observed the students for 10-minute periods each day during lunch recess and counted the number of aggressive behaviors they exhibited toward their peers. (The researchers used handheld computers to help record the data.) After 2 weeks, they implemented the program at one school. After 2 more weeks, they implemented it at the second school. And after 2 more weeks, they implemented it at the third school. They found that the number of aggressive behaviors exhibited by each student dropped shortly after the program was implemented at his or her school. Notice that if the researchers had only studied one school or if they had introduced the treatment at the same time at all three schools, then it would be unclear whether the reduction in aggressive behaviors was due to the bullying program or something else that happened at about the same time it was introduced (e.g., a holiday, a television program, a change in the weather). But with their multiple-baseline design, this kind of coincidence would have to happen three separate times—an unlikely occurrence—to explain their results.

Data Analysis in Single-Subject Research

In addition to its focus on individual participants, single-subject research differs from group research in the way the data are typically analyzed. As we have seen throughout the book, group research involves combining data across participants. Inferential statistics are used to help decide whether the result for the sample is likely to generalize to the population. Single-subject research, by contrast, relies heavily on a very different approach called visual inspection . This means plotting individual participants’ data as shown throughout this chapter, looking carefully at those data, and making judgments about whether and to what extent the independent variable had an effect on the dependent variable. Inferential statistics are typically not used.

In visually inspecting their data, single-subject researchers take several factors into account. One of them is changes in the level of the dependent variable from condition to condition. If the dependent variable is much higher or much lower in one condition than another, this suggests that the treatment had an effect. A second factor is trend , which refers to gradual increases or decreases in the dependent variable across observations. If the dependent variable begins increasing or decreasing with a change in conditions, then again this suggests that the treatment had an effect. It can be especially telling when a trend changes directions—for example, when an unwanted behavior is increasing during baseline but then begins to decrease with the introduction of the treatment. A third factor is latency , which is the time it takes for the dependent variable to begin changing after a change in conditions. In general, if a change in the dependent variable begins shortly after a change in conditions, this suggests that the treatment was responsible.

In the top panel of Figure 7.7 , there are fairly obvious changes in the level and trend of the dependent variable from condition to condition. Furthermore, the latencies of these changes are short; the change happens immediately. This pattern of results strongly suggests that the treatment was responsible for the changes in the dependent variable. In the bottom panel of Figure 7.7 , however, the changes in level are fairly small. And although there appears to be an increasing trend in the treatment condition, it looks as though it might be a continuation of a trend that had already begun during baseline. This pattern of results strongly suggests that the treatment was not responsible for any changes in the dependent variable—at least not to the extent that single-subject researchers typically hope to see.

Figure 7.7: Visual inspection of the data suggests an effective treatment in the top panel but an ineffective treatment in the bottom panel.

The results of single-subject research can also be analyzed using statistical procedures—and this is becoming more common. There are many different approaches, and single-subject researchers continue to debate which are the most useful. One approach parallels what is typically done in group research. The mean and standard deviation of each participant’s responses under each condition are computed and compared, and inferential statistical tests such as the t test or analysis of variance are applied ( Fisch, 2001 ) . (Note that averaging across participants is less common.) Another approach is to compute the percentage of nonoverlapping data (PND) for each participant ( Scruggs & Mastropieri, 2021 ) . This is the percentage of responses in the treatment condition that are more extreme than the most extreme response in a relevant control condition. In the study of Hall and his colleagues, for example, all measures of Robbie’s study time in the first treatment condition were greater than the highest measure in the first baseline, for a PND of 100%. The greater the percentage of nonoverlapping data, the stronger the treatment effect. Still, formal statistical approaches to data analysis in single-subject research are generally considered a supplement to visual inspection, not a replacement for it.

The Single-Subject Versus Group “Debate”

Single-subject research is similar to group research—especially experimental group research—in many ways. They are both quantitative approaches that try to establish causal relationships by manipulating an independent variable, measuring a dependent variable, and controlling extraneous variables. As we will see, single-subject research and group research are probably best conceptualized as complementary approaches.

Data Analysis

One set of disagreements revolves around the issue of data analysis. Some advocates of group research worry that visual inspection is inadequate for deciding whether and to what extent a treatment has affected a dependent variable. One specific concern is that visual inspection is not sensitive enough to detect weak effects. A second is that visual inspection can be unreliable, with different researchers reaching different conclusions about the same set of data ( Danov & Symons, 2008 ) . A third is that the results of visual inspection—an overall judgment of whether or not a treatment was effective—cannot be clearly and efficiently summarized or compared across studies (unlike the measures of relationship strength typically used in group research).

In general, single-subject researchers share these concerns. However, they also argue that their use of the steady state strategy, combined with their focus on strong and consistent effects, minimizes most of them. If the effect of a treatment is difficult to detect by visual inspection because the effect is weak or the data are noisy, then single-subject researchers look for ways to increase the strength of the effect or reduce the noise in the data by controlling extraneous variables (e.g., by administering the treatment more consistently). If the effect is still difficult to detect, then they are likely to consider it neither strong enough nor consistent enough to be of further interest. Many single-subject researchers also point out that statistical analysis is becoming increasingly common and that many of them are using it as a supplement to visual inspection—especially for the purpose of comparing results across studies ( Scruggs & Mastropieri, 2021 ) .

Turning the tables, some advocates of single-subject research worry about the way that group researchers analyze their data. Specifically, they point out that focusing on group means can be highly misleading. Again, imagine that a treatment has a strong positive effect on half the people exposed to it and an equally strong negative effect on the other half. In a traditional between-subjects experiment, the positive effect on half the participants in the treatment condition would be statistically cancelled out by the negative effect on the other half. The mean for the treatment group would then be the same as the mean for the control group, making it seem as though the treatment had no effect when in fact it had a strong effect on every single participant!

But again, group researchers share this concern. Although they do focus on group statistics, they also emphasize the importance of examining distributions of individual scores. For example, if some participants were positively affected by a treatment and others negatively affected by it, this would produce a bimodal distribution of scores and could be detected by looking at a histogram of the data. The use of within-subjects designs is another strategy that allows group researchers to observe effects at the individual level and even to specify what percentage of individuals exhibit strong, medium, weak, and even negative effects.

External Validity

The second issue about which single-subject and group researchers sometimes disagree has to do with external validity—the ability to generalize the results of a study beyond the people and situation actually studied. In particular, advocates of group research point out the difficulty in knowing whether results for just a few participants are likely to generalize to others in the population. Imagine, for example, that in a single-subject study, a treatment has been shown to reduce self-injury for each of two developmentally disabled children. Even if the effect is strong for these two children, how can one know whether this treatment is likely to work for other developmentally disabled children?

Again, single-subject researchers share this concern. In response, they note that the strong and consistent effects they are typically interested in—even when observed in small samples—are likely to generalize to others in the population. Single-subject researchers also note that they place a strong emphasis on replicating their research results. When they observe an effect with a small sample of participants, they typically try to replicate it with another small sample—perhaps with a slightly different type of participant or under slightly different conditions. Each time they observe similar results, they rightfully become more confident in the generality of those results. Single-subject researchers can also point to the fact that the principles of classical and operant conditioning—most of which were discovered using the single-subject approach—have been successfully generalized across an incredibly wide range of species and situations.

And again turning the tables, single-subject researchers have concerns of their own about the external validity of group research. One extremely important point they make is that studying large groups of participants does not entirely solve the problem of generalizing to other individuals. Imagine, for example, a treatment that has been shown to have a small positive effect on average in a large group study. It is likely that although many participants exhibited a small positive effect, others exhibited a large positive effect, and still others exhibited a small negative effect. When it comes to applying this treatment to another large group , we can be fairly sure that it will have a small effect on average. But when it comes to applying this treatment to another individual , we cannot be sure whether it will have a small, a large, or even a negative effect. Another point that single-subject researchers make is that group researchers also face a similar problem when they study a single situation and then generalize their results to other situations. For example, researchers who conduct a study on the effect of cell phone use on drivers on a closed oval track probably want to apply their results to drivers in many other real-world driving situations. But notice that this requires generalizing from a single situation to a population of situations. Thus the ability to generalize is based on much more than just the sheer number of participants one has studied. It requires a careful consideration of the similarity of the participants and situations studied to the population of participants and situations that one wants to generalize to ( Shadish et al., 2002 ) .

Single-Subject and Group Research as Complementary Methods

As with quantitative and qualitative research, it is probably best to conceptualize single-subject research and group research as complementary methods that have different strengths and weaknesses and that are appropriate for answering different kinds of research questions ( Kazdin, 2019 ) . Single-subject research is particularly good for testing the effectiveness of treatments on individuals when the focus is on strong, consistent, and biologically or socially important effects. It is especially useful when the behavior of particular individuals is of interest. Clinicians who work with only one individual at a time may find that it is their only option for doing systematic quantitative research.

Group research, on the other hand, is good for testing the effectiveness of treatments at the group level. Among the advantages of this approach is that it allows researchers to detect weak effects, which can be of interest for many reasons. For example, finding a weak treatment effect might lead to refinements of the treatment that eventually produce a larger and more meaningful effect. Group research is also good for studying interactions between treatments and participant characteristics. For example, if a treatment is effective for those who are high in motivation to change and ineffective for those who are low in motivation to change, then a group design can detect this much more efficiently than a single-subject design. Group research is also necessary to answer questions that cannot be addressed using the single-subject approach, including questions about independent variables that cannot be manipulated (e.g., number of siblings, extroversion, culture).

• Single-subject research—which involves testing a small number of participants and focusing intensively on the behavior of each individual—is an important alternative to group research in psychology.
• Single-subject studies must be distinguished from case studies, in which an individual case is described in detail. Case studies can be useful for generating new research questions, for studying rare phenomena, and for illustrating general principles. However, they cannot substitute for carefully controlled experimental or correlational studies because they are low in internal and external validity.
• Single-subject research designs typically involve measuring the dependent variable repeatedly over time and changing conditions (e.g., from baseline to treatment) when the dependent variable has reached a steady state. This approach allows the researcher to see whether changes in the independent variable are causing changes in the dependent variable.
• Single-subject researchers typically analyze their data by graphing them and making judgments about whether the independent variable is affecting the dependent variable based on level, trend, and latency.
• Differences between single-subject research and group research sometimes lead to disagreements between single-subject and group researchers. These disagreements center on the issues of data analysis and external validity (especially generalization to other people). Single-subject research and group research are probably best seen as complementary methods, with different strengths and weaknesses, that are appropriate for answering different kinds of research questions.
• Does positive attention from a parent increase a child’s toothbrushing behavior?
• Does self-testing while studying improve a student’s performance on weekly spelling tests?
• Does regular exercise help relieve depression?
• Practice: Create a graph that displays the hypothetical results for the study you designed in Exercise 1. Write a paragraph in which you describe what the results show. Be sure to comment on level, trend, and latency.
• Discussion: Imagine you have conducted a single-subject study showing a positive effect of a treatment on the behavior of a man with social anxiety disorder. Your research has been criticized on the grounds that it cannot be generalized to others. How could you respond to this criticism?
• Discussion: Imagine you have conducted a group study showing a positive effect of a treatment on the behavior of a group of people with social anxiety disorder, but your research has been criticized on the grounds that “average” effects cannot be generalized to individuals. How could you respond to this criticism?

7.6 Glossary

The simplest reversal design, in which there is a baseline condition (A), followed by a treatment condition (B), followed by a return to baseline (A).

applied behavior analysis

A subfield of psychology that uses single-subject research and applies the principles of behavior analysis to real-world problems in areas that include education, developmental disabilities, organizational behavior, and health behavior.

A condition in a single-subject research design in which the dependent variable is measured repeatedly in the absence of any treatment. Most designs begin with a baseline condition, and many return to the baseline condition at least once.

A detailed description of an individual case.

experimental analysis of behavior

A subfield of psychology founded by B. F. Skinner that uses single-subject research—often with nonhuman animals—to study relationships primarily between environmental conditions and objectively observable behaviors.

group research

A type of quantitative research that involves studying a large number of participants and examining their behavior in terms of means, standard deviations, and other group-level statistics.

interrupted time-series design

A research design in which a series of measurements of the dependent variable are taken both before and after a treatment.

item-order effect

The effect of responding to one survey item on responses to a later survey item.

Refers collectively to extraneous developmental changes in participants that can occur between a pretest and posttest or between the first and last measurements in a time series. It can provide an alternative explanation for an observed change in the dependent variable.

multiple-baseline design

A single-subject research design in which multiple baselines are established for different participants, different dependent variables, or different contexts and the treatment is introduced at a different time for each baseline.

naturalistic observation

An approach to data collection in which the behavior of interest is observed in the environment in which it typically occurs.

nonequivalent groups design

A between-subjects research design in which participants are not randomly assigned to conditions, usually because participants are in preexisting groups (e.g., students at different schools).

nonexperimental research

Research that lacks the manipulation of an independent variable or the random assignment of participants to conditions or orders of conditions.

open-ended item

A questionnaire item that asks a question and allows respondents to respond in whatever way they want.

percentage of nonoverlapping data

A statistic sometimes used in single-subject research. The percentage of observations in a treatment condition that are more extreme than the most extreme observation in a relevant baseline condition.

pretest-posttest design

A research design in which the dependent variable is measured (the pretest), a treatment is given, and the dependent variable is measured again (the posttest) to see if there is a change in the dependent variable from pretest to posttest.

quasi-experimental research

Research that involves the manipulation of an independent variable but lacks the random assignment of participants to conditions or orders of conditions. It is generally used in field settings to test the effectiveness of a treatment.

rating scale

An ordered set of response options to a closed-ended questionnaire item.

The statistical fact that an individual who scores extremely on one occasion will tend to score less extremely on the next occasion.

A term often used to refer to a participant in survey research.

reversal design

A single-subject research design that begins with a baseline condition with no treatment, followed by the introduction of a treatment, and after that a return to the baseline condition. It can include additional treatment conditions and returns to baseline.

single-subject research

A type of quantitative research that involves examining in detail the behavior of each of a small number of participants.

single-variable research

Research that focuses on a single variable rather than on a statistical relationship between variables.

social validity

The extent to which a single-subject study focuses on an intervention that has a substantial effect on an important behavior and can be implemented reliably in the real-world contexts (e.g., by teachers in a classroom) in which that behavior occurs.

Improvement in a psychological or medical problem over time without any treatment.

steady state strategy

In single-subject research, allowing behavior to become fairly consistent from one observation to the next before changing conditions. This makes any effect of the treatment easier to detect.

survey research

A quantitative research approach that uses self-report measures and large, carefully selected samples.

testing effect

A bias in participants’ responses in which scores on the posttest are influenced by simple exposure to the pretest

visual inspection

The primary approach to data analysis in single-subject research, which involves graphing the data and making a judgment as to whether and to what extent the independent variable affected the dependent variable.

Experimental vs Quasi-Experimental Design: Which to Choose?

Here’s a table that summarizes the similarities and differences between an experimental and a quasi-experimental study design:

What is a quasi-experimental design?

A quasi-experimental design is a non-randomized study design used to evaluate the effect of an intervention. The intervention can be a training program, a policy change or a medical treatment.

Unlike a true experiment, in a quasi-experimental study the choice of who gets the intervention and who doesn’t is not randomized. Instead, the intervention can be assigned to participants according to their choosing or that of the researcher, or by using any method other than randomness.

Having a control group is not required, but if present, it provides a higher level of evidence for the relationship between the intervention and the outcome.

(for more information, I recommend my other article: Understand Quasi-Experimental Design Through an Example ) .

Examples of quasi-experimental designs include:

• One-Group Posttest Only Design
• Static-Group Comparison Design
• One-Group Pretest-Posttest Design
• Separate-Sample Pretest-Posttest Design

What is an experimental design?

An experimental design is a randomized study design used to evaluate the effect of an intervention. In its simplest form, the participants will be randomly divided into 2 groups:

• A treatment group: where participants receive the new intervention which effect we want to study.
• A control or comparison group: where participants do not receive any intervention at all (or receive some standard intervention).

Randomization ensures that each participant has the same chance of receiving the intervention. Its objective is to equalize the 2 groups, and therefore, any observed difference in the study outcome afterwards will only be attributed to the intervention – i.e. it removes confounding.

(for more information, I recommend my other article: Purpose and Limitations of Random Assignment ).

Examples of experimental designs include:

• Posttest-Only Control Group Design
• Pretest-Posttest Control Group Design
• Solomon Four-Group Design
• Matched Pairs Design
• Randomized Block Design

When to choose an experimental design over a quasi-experimental design?

Although many statistical techniques can be used to deal with confounding in a quasi-experimental study, in practice, randomization is still the best tool we have to study causal relationships.

Another problem with quasi-experiments is the natural progression of the disease or the condition under study — When studying the effect of an intervention over time, one should consider natural changes because these can be mistaken with changes in outcome that are caused by the intervention. Having a well-chosen control group helps dealing with this issue.

So, if losing the element of randomness seems like an unwise step down in the hierarchy of evidence, why would we ever want to do it?

This is what we’re going to discuss next.

When to choose a quasi-experimental design over a true experiment?

The issue with randomness is that it cannot be always achievable.

So here are some cases where using a quasi-experimental design makes more sense than using an experimental one:

• If being in one group is believed to be harmful for the participants , either because the intervention is harmful (ex. randomizing people to smoking), or the intervention has a questionable efficacy, or on the contrary it is believed to be so beneficial that it would be malevolent to put people in the control group (ex. randomizing people to receiving an operation).
• In cases where interventions act on a group of people in a given location , it becomes difficult to adequately randomize subjects (ex. an intervention that reduces pollution in a given area).
• When working with small sample sizes , as randomized controlled trials require a large sample size to account for heterogeneity among subjects (i.e. to evenly distribute confounding variables between the intervention and control groups).

Further reading

• Statistical Software Popularity in 40,582 Research Papers
• Checking the Popularity of 125 Statistical Tests and Models
• Objectives of Epidemiology (With Examples)
• 12 Famous Epidemiologists and Why

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case AskWhy Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Home Market Research Research Tools and Apps

Quasi-experimental Research: What It Is, Types & Examples

Much like an actual experiment, quasi-experimental research tries to demonstrate a cause-and-effect link between a dependent and an independent variable. A quasi-experiment, on the other hand, does not depend on random assignment, unlike an actual experiment. The subjects are sorted into groups based on non-random variables.

What is Quasi-Experimental Research?

“Resemblance” is the definition of “quasi.” Individuals are not randomly allocated to conditions or orders of conditions, even though the regression analysis is changed. As a result, quasi-experimental research is research that appears to be experimental but is not.

The directionality problem is avoided in quasi-experimental research since the regression analysis is altered before the multiple regression is assessed. However, because individuals are not randomized at random, there are likely to be additional disparities across conditions in quasi-experimental research.

As a result, in terms of internal consistency, quasi-experiments fall somewhere between correlational research and actual experiments.

The key component of a true experiment is randomly allocated groups. This means that each person has an equivalent chance of being assigned to the experimental group or the control group, depending on whether they are manipulated or not.

Simply put, a quasi-experiment is not a real experiment. A quasi-experiment does not feature randomly allocated groups since the main component of a real experiment is randomly assigned groups. Why is it so crucial to have randomly allocated groups, given that they constitute the only distinction between quasi-experimental and actual  experimental research ?

Let’s use an example to illustrate our point. Let’s assume we want to discover how new psychological therapy affects depressed patients. In a genuine trial, you’d split half of the psych ward into treatment groups, With half getting the new psychotherapy therapy and the other half receiving standard  depression treatment .

And the physicians compare the outcomes of this treatment to the results of standard treatments to see if this treatment is more effective. Doctors, on the other hand, are unlikely to agree with this genuine experiment since they believe it is unethical to treat one group while leaving another untreated.

A quasi-experimental study will be useful in this case. Instead of allocating these patients at random, you uncover pre-existing psychotherapist groups in the hospitals. Clearly, there’ll be counselors who are eager to undertake these trials as well as others who prefer to stick to the old ways.

These pre-existing groups can be used to compare the symptom development of individuals who received the novel therapy with those who received the normal course of treatment, even though the groups weren’t chosen at random.

If any substantial variations between them can be well explained, you may be very assured that any differences are attributable to the treatment but not to other extraneous variables.

As we mentioned before, quasi-experimental research entails manipulating an independent variable by randomly assigning people to conditions or sequences of conditions. Non-equivalent group designs, pretest-posttest designs, and regression discontinuity designs are only a few of the essential types.

What are quasi-experimental research designs?

Quasi-experimental research designs are a type of research design that is similar to experimental designs but doesn’t give full control over the independent variable(s) like true experimental designs do.

In a quasi-experimental design, the researcher changes or watches an independent variable, but the participants are not put into groups at random. Instead, people are put into groups based on things they already have in common, like their age, gender, or how many times they have seen a certain stimulus.

Because the assignments are not random, it is harder to draw conclusions about cause and effect than in a real experiment. However, quasi-experimental designs are still useful when randomization is not possible or ethical.

The true experimental design may be impossible to accomplish or just too expensive, especially for researchers with few resources. Quasi-experimental designs enable you to investigate an issue by utilizing data that has already been paid for or gathered by others (often the government). 

Because they allow better control for confounding variables than other forms of studies, they have higher external validity than most genuine experiments and higher  internal validity  (less than true experiments) than other non-experimental research.

Is quasi-experimental research quantitative or qualitative?

Quasi-experimental research is a quantitative research method. It involves numerical data collection and statistical analysis. Quasi-experimental research compares groups with different circumstances or treatments to find cause-and-effect links. 

It draws statistical conclusions from quantitative data. Qualitative data can enhance quasi-experimental research by revealing participants’ experiences and opinions, but quantitative data is the method’s foundation.

Quasi-experimental research types

There are many different sorts of quasi-experimental designs. Three of the most popular varieties are described below: Design of non-equivalent groups, Discontinuity in regression, and Natural experiments.

Design of Non-equivalent Groups

Example: design of non-equivalent groups, discontinuity in regression, example: discontinuity in regression, natural experiments, example: natural experiments.

However, because they couldn’t afford to pay everyone who qualified for the program, they had to use a random lottery to distribute slots.

Experts were able to investigate the program’s impact by utilizing enrolled people as a treatment group and those who were qualified but did not play the jackpot as an experimental group.

How QuestionPro helps in quasi-experimental research?

QuestionPro can be a useful tool in quasi-experimental research because it includes features that can assist you in designing and analyzing your research study. Here are some ways in which QuestionPro can help in quasi-experimental research:

Design surveys

Randomize participants, collect data over time, analyze data, collaborate with your team.

With QuestionPro, you have access to the most mature market research platform and tool that helps you collect and analyze the insights that matter the most. By leveraging InsightsHub, the unified hub for data management, you can ​​leverage the consolidated platform to organize, explore, search, and discover your  research data  in one organized data repository . 

Optimize Your quasi-experimental research with QuestionPro. Get started now!

LEARN MORE         FREE TRIAL

MORE LIKE THIS

A guide to conducting agile qualitative research for rapid insights with Digsite 

Sep 11, 2024

Was The Experience Memorable? — Tuesday CX Thoughts

Sep 10, 2024

What Does a Data Analyst Do? Skills, Tools & Tips

Sep 9, 2024

Best Gallup Access Alternatives & Competitors in 2024

Sep 6, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Tuesday CX Thoughts (TCXT)
• Uncategorized
• What’s Coming Up
• Workforce Intelligence

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Chapter 7: Nonexperimental Research

Quasi-Experimental Research

Learning Objectives

• Explain what quasi-experimental research is and distinguish it clearly from both experimental and correlational research.
• Describe three different types of quasi-experimental research designs (nonequivalent groups, pretest-posttest, and interrupted time series) and identify examples of each one.

The prefix  quasi  means “resembling.” Thus quasi-experimental research is research that resembles experimental research but is not true experimental research. Although the independent variable is manipulated, participants are not randomly assigned to conditions or orders of conditions (Cook & Campbell, 1979). [1] Because the independent variable is manipulated before the dependent variable is measured, quasi-experimental research eliminates the directionality problem. But because participants are not randomly assigned—making it likely that there are other differences between conditions—quasi-experimental research does not eliminate the problem of confounding variables. In terms of internal validity, therefore, quasi-experiments are generally somewhere between correlational studies and true experiments.

Quasi-experiments are most likely to be conducted in field settings in which random assignment is difficult or impossible. They are often conducted to evaluate the effectiveness of a treatment—perhaps a type of psychotherapy or an educational intervention. There are many different kinds of quasi-experiments, but we will discuss just a few of the most common ones here.

Nonequivalent Groups Design

Recall that when participants in a between-subjects experiment are randomly assigned to conditions, the resulting groups are likely to be quite similar. In fact, researchers consider them to be equivalent. When participants are not randomly assigned to conditions, however, the resulting groups are likely to be dissimilar in some ways. For this reason, researchers consider them to be nonequivalent. A  nonequivalent groups design , then, is a between-subjects design in which participants have not been randomly assigned to conditions.

Imagine, for example, a researcher who wants to evaluate a new method of teaching fractions to third graders. One way would be to conduct a study with a treatment group consisting of one class of third-grade students and a control group consisting of another class of third-grade students. This design would be a nonequivalent groups design because the students are not randomly assigned to classes by the researcher, which means there could be important differences between them. For example, the parents of higher achieving or more motivated students might have been more likely to request that their children be assigned to Ms. Williams’s class. Or the principal might have assigned the “troublemakers” to Mr. Jones’s class because he is a stronger disciplinarian. Of course, the teachers’ styles, and even the classroom environments, might be very different and might cause different levels of achievement or motivation among the students. If at the end of the study there was a difference in the two classes’ knowledge of fractions, it might have been caused by the difference between the teaching methods—but it might have been caused by any of these confounding variables.

Of course, researchers using a nonequivalent groups design can take steps to ensure that their groups are as similar as possible. In the present example, the researcher could try to select two classes at the same school, where the students in the two classes have similar scores on a standardized math test and the teachers are the same sex, are close in age, and have similar teaching styles. Taking such steps would increase the internal validity of the study because it would eliminate some of the most important confounding variables. But without true random assignment of the students to conditions, there remains the possibility of other important confounding variables that the researcher was not able to control.

Pretest-Posttest Design

In a  pretest-posttest design , the dependent variable is measured once before the treatment is implemented and once after it is implemented. Imagine, for example, a researcher who is interested in the effectiveness of an antidrug education program on elementary school students’ attitudes toward illegal drugs. The researcher could measure the attitudes of students at a particular elementary school during one week, implement the antidrug program during the next week, and finally, measure their attitudes again the following week. The pretest-posttest design is much like a within-subjects experiment in which each participant is tested first under the control condition and then under the treatment condition. It is unlike a within-subjects experiment, however, in that the order of conditions is not counterbalanced because it typically is not possible for a participant to be tested in the treatment condition first and then in an “untreated” control condition.

If the average posttest score is better than the average pretest score, then it makes sense to conclude that the treatment might be responsible for the improvement. Unfortunately, one often cannot conclude this with a high degree of certainty because there may be other explanations for why the posttest scores are better. One category of alternative explanations goes under the name of  history . Other things might have happened between the pretest and the posttest. Perhaps an antidrug program aired on television and many of the students watched it, or perhaps a celebrity died of a drug overdose and many of the students heard about it. Another category of alternative explanations goes under the name of  maturation . Participants might have changed between the pretest and the posttest in ways that they were going to anyway because they are growing and learning. If it were a yearlong program, participants might become less impulsive or better reasoners and this might be responsible for the change.

Another alternative explanation for a change in the dependent variable in a pretest-posttest design is  regression to the mean . This refers to the statistical fact that an individual who scores extremely on a variable on one occasion will tend to score less extremely on the next occasion. For example, a bowler with a long-term average of 150 who suddenly bowls a 220 will almost certainly score lower in the next game. Her score will “regress” toward her mean score of 150. Regression to the mean can be a problem when participants are selected for further study  because  of their extreme scores. Imagine, for example, that only students who scored especially low on a test of fractions are given a special training program and then retested. Regression to the mean all but guarantees that their scores will be higher even if the training program has no effect. A closely related concept—and an extremely important one in psychological research—is  spontaneous remission . This is the tendency for many medical and psychological problems to improve over time without any form of treatment. The common cold is a good example. If one were to measure symptom severity in 100 common cold sufferers today, give them a bowl of chicken soup every day, and then measure their symptom severity again in a week, they would probably be much improved. This does not mean that the chicken soup was responsible for the improvement, however, because they would have been much improved without any treatment at all. The same is true of many psychological problems. A group of severely depressed people today is likely to be less depressed on average in 6 months. In reviewing the results of several studies of treatments for depression, researchers Michael Posternak and Ivan Miller found that participants in waitlist control conditions improved an average of 10 to 15% before they received any treatment at all (Posternak & Miller, 2001) [2] . Thus one must generally be very cautious about inferring causality from pretest-posttest designs.

Does Psychotherapy Work?

Early studies on the effectiveness of psychotherapy tended to use pretest-posttest designs. In a classic 1952 article, researcher Hans Eysenck summarized the results of 24 such studies showing that about two thirds of patients improved between the pretest and the posttest (Eysenck, 1952) [3] . But Eysenck also compared these results with archival data from state hospital and insurance company records showing that similar patients recovered at about the same rate  without  receiving psychotherapy. This parallel suggested to Eysenck that the improvement that patients showed in the pretest-posttest studies might be no more than spontaneous remission. Note that Eysenck did not conclude that psychotherapy was ineffective. He merely concluded that there was no evidence that it was, and he wrote of “the necessity of properly planned and executed experimental studies into this important field” (p. 323). You can read the entire article here: Classics in the History of Psychology .

Fortunately, many other researchers took up Eysenck’s challenge, and by 1980 hundreds of experiments had been conducted in which participants were randomly assigned to treatment and control conditions, and the results were summarized in a classic book by Mary Lee Smith, Gene Glass, and Thomas Miller (Smith, Glass, & Miller, 1980) [4] . They found that overall psychotherapy was quite effective, with about 80% of treatment participants improving more than the average control participant. Subsequent research has focused more on the conditions under which different types of psychotherapy are more or less effective.

Interrupted Time Series Design

A variant of the pretest-posttest design is the  interrupted time-series design . A time series is a set of measurements taken at intervals over a period of time. For example, a manufacturing company might measure its workers’ productivity each week for a year. In an interrupted time series-design, a time series like this one is “interrupted” by a treatment. In one classic example, the treatment was the reduction of the work shifts in a factory from 10 hours to 8 hours (Cook & Campbell, 1979) [5] . Because productivity increased rather quickly after the shortening of the work shifts, and because it remained elevated for many months afterward, the researcher concluded that the shortening of the shifts caused the increase in productivity. Notice that the interrupted time-series design is like a pretest-posttest design in that it includes measurements of the dependent variable both before and after the treatment. It is unlike the pretest-posttest design, however, in that it includes multiple pretest and posttest measurements.

Figure 7.3 shows data from a hypothetical interrupted time-series study. The dependent variable is the number of student absences per week in a research methods course. The treatment is that the instructor begins publicly taking attendance each day so that students know that the instructor is aware of who is present and who is absent. The top panel of  Figure 7.3 shows how the data might look if this treatment worked. There is a consistently high number of absences before the treatment, and there is an immediate and sustained drop in absences after the treatment. The bottom panel of  Figure 7.3 shows how the data might look if this treatment did not work. On average, the number of absences after the treatment is about the same as the number before. This figure also illustrates an advantage of the interrupted time-series design over a simpler pretest-posttest design. If there had been only one measurement of absences before the treatment at Week 7 and one afterward at Week 8, then it would have looked as though the treatment were responsible for the reduction. The multiple measurements both before and after the treatment suggest that the reduction between Weeks 7 and 8 is nothing more than normal week-to-week variation.

Combination Designs

A type of quasi-experimental design that is generally better than either the nonequivalent groups design or the pretest-posttest design is one that combines elements of both. There is a treatment group that is given a pretest, receives a treatment, and then is given a posttest. But at the same time there is a control group that is given a pretest, does  not  receive the treatment, and then is given a posttest. The question, then, is not simply whether participants who receive the treatment improve but whether they improve  more  than participants who do not receive the treatment.

Imagine, for example, that students in one school are given a pretest on their attitudes toward drugs, then are exposed to an antidrug program, and finally are given a posttest. Students in a similar school are given the pretest, not exposed to an antidrug program, and finally are given a posttest. Again, if students in the treatment condition become more negative toward drugs, this change in attitude could be an effect of the treatment, but it could also be a matter of history or maturation. If it really is an effect of the treatment, then students in the treatment condition should become more negative than students in the control condition. But if it is a matter of history (e.g., news of a celebrity drug overdose) or maturation (e.g., improved reasoning), then students in the two conditions would be likely to show similar amounts of change. This type of design does not completely eliminate the possibility of confounding variables, however. Something could occur at one of the schools but not the other (e.g., a student drug overdose), so students at the first school would be affected by it while students at the other school would not.

Finally, if participants in this kind of design are randomly assigned to conditions, it becomes a true experiment rather than a quasi experiment. In fact, it is the kind of experiment that Eysenck called for—and that has now been conducted many times—to demonstrate the effectiveness of psychotherapy.

Key Takeaways

• Quasi-experimental research involves the manipulation of an independent variable without the random assignment of participants to conditions or orders of conditions. Among the important types are nonequivalent groups designs, pretest-posttest, and interrupted time-series designs.
• Quasi-experimental research eliminates the directionality problem because it involves the manipulation of the independent variable. It does not eliminate the problem of confounding variables, however, because it does not involve random assignment to conditions. For these reasons, quasi-experimental research is generally higher in internal validity than correlational studies but lower than true experiments.
• Practice: Imagine that two professors decide to test the effect of giving daily quizzes on student performance in a statistics course. They decide that Professor A will give quizzes but Professor B will not. They will then compare the performance of students in their two sections on a common final exam. List five other variables that might differ between the two sections that could affect the results.
• regression to the mean
• spontaneous remission

Image Descriptions

Figure 7.3 image description: Two line graphs charting the number of absences per week over 14 weeks. The first 7 weeks are without treatment and the last 7 weeks are with treatment. In the first line graph, there are between 4 to 8 absences each week. After the treatment, the absences drop to 0 to 3 each week, which suggests the treatment worked. In the second line graph, there is no noticeable change in the number of absences per week after the treatment, which suggests the treatment did not work. [Return to Figure 7.3]

• Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design & analysis issues in field settings . Boston, MA: Houghton Mifflin. ↵
• Posternak, M. A., & Miller, I. (2001). Untreated short-term course of major depression: A meta-analysis of studies using outcomes from studies using wait-list control groups. Journal of Affective Disorders, 66 , 139–146. ↵
• Eysenck, H. J. (1952). The effects of psychotherapy: An evaluation. Journal of Consulting Psychology, 16 , 319–324. ↵
• Smith, M. L., Glass, G. V., & Miller, T. I. (1980). The benefits of psychotherapy . Baltimore, MD: Johns Hopkins University Press. ↵

A between-subjects design in which participants have not been randomly assigned to conditions.

The dependent variable is measured once before the treatment is implemented and once after it is implemented.

A category of alternative explanations for differences between scores such as events that happened between the pretest and posttest, unrelated to the study.

An alternative explanation that refers to how the participants might have changed between the pretest and posttest in ways that they were going to anyway because they are growing and learning.

The statistical fact that an individual who scores extremely on a variable on one occasion will tend to score less extremely on the next occasion.

The tendency for many medical and psychological problems to improve over time without any form of treatment.

A set of measurements taken at intervals over a period of time that are interrupted by a treatment.

Research Methods in Psychology - 2nd Canadian Edition Copyright © 2015 by Paul C. Price, Rajiv Jhangiani, & I-Chant A. Chiang is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

Share This Book

• Open access
• Published: 10 September 2024

Food waste reduction and its environmental consequences: a quasi-experimental study in a campus canteen

• Seyedeh Fatemeh Fatemi 1 ,
• Hassan Eini-Zinab 2 ,
• Fatemeh Manafi Anari 2 ,
• Mahdieh Amirolad 2 ,
• Zahra Babaei 2 &
• Seyyed Reza Sobhani   ORCID: orcid.org/0000-0002-7308-8504 1  

Agriculture & Food Security volume  13 , Article number:  37 ( 2024 ) Cite this article

Metrics details

Food waste is the third-largest contributor to greenhouse gas emissions, which has severe environmental and economic effects. This study presents a two-level intervention to estimate the quantity and environmental consequences of food waste at a campus canteen, offering innovative solutions to reduce food waste and its environmental footprint.

Methodology

This study involved 300 students and consisted of three main stages: initial food waste assessment, environmental and economic impact evaluation, and qualitative exploration of the causes of food waste through interviews with students. The assessment included direct measures and weighing of leftover food, and the environmental and economic impact was calculated. A two-level intervention was implemented for students and staff, and a re-assessment of food waste was conducted to evaluate the intervention’s impact. Statistical analysis was performed using SPSS.

The study monitored 26 meals, finding that the total amount of food waste in the university canteens was (mean = 60.65 g/person), and the intervention reduced food waste by 16.35% per meal (50.73 g/person). Moreover, after the intervention, the amount of food waste costs and total water waste were reduced by 30.14% and 16.66%, respectively. Grey water was reduced significantly by 12.5% ( p  = 0.033). Interviews with students identified low-quality meals, unpleasant taste, large portions, and a limited menu as the main causes of food waste.

Conclusions

It is possible to tackle food waste effectively with educational intervention, decreasing portion size, and improving the quality and variety of food.

Introduction

Climate change requires a global approach that prioritizes sustainability [ 1 ]. Coordinated initiatives to address climate change can also improve food security, land management, and nutrition, and help end hunger, while it is estimated that about 30–50% of all food produced is wasted within the food supply chain [ 2 , 3 ]. Reducing food loss and waste would lower greenhouse gas emissions and promote a sustainable diet [ 4 ]. Furthermore, reducing food losses at the earlier stages of the food supply chain, such as production, processing, and distribution, can enhance food security and reduce the strain on natural resources, such as land, water, and energy [ 5 ]. By reducing the waste in the food supply chain, there would be enough food for about one billion people, which would increase food security and resource efficiency [ 6 ]. However, reducing food waste is not sufficient to achieve sustainability; it is also necessary to find innovative solutions to utilize the wasted food and convert it into valuable products and services [ 7 ]. Another way to tackle this problem is a sharing economy, which is a way of sharing goods and services without owning them. A sharing economy can help reduce food waste by connecting food producers, retailers, and consumers, and by redistributing surplus food [ 8 ].

Food waste exacerbates several public health and environmental issues, including food security, water shortages, and greenhouse gas emissions [ 9 ]. Agriculture consumes about 70% of the freshwater supply, and more than 25% of the total freshwater use is wasted due to food waste [ 10 ]. A previous study estimated that 1533.5 tons of CO2 were emitted from farm to table for 1,990 food items, equivalent to the greenhouse gas emissions of 278.82 passenger vehicles in a year [ 11 ]. Considering the finite land and water resources, climate change, and the environmental impacts of food production and consumption [ 2 , 3 ], it is clear that food waste is a global public health and environmental issue that requires the joint efforts of governments, industry, and individuals [ 12 , 13 ]. One of the ways to address this issue is to apply food waste valorization methods, which aim to recover and utilize the valuable components of food waste, such as nutrients, bioactive compounds, and energy [ 14 ].

The main cause of food waste in underdeveloped countries is at the production stage, while in developed countries, it is at the retail and consumption stages [ 15 ]. The literature suggests that the main causes of food waste at the consumer level are consumer attitudes, low awareness of food waste and its environmental consequences, lack of planned shopping, and failure to follow a specific diet [ 16 , 17 ].

However, this can be significantly prevented by actions taken by governments, industry, and people. Implementing clear legislative frameworks, establishing economic instruments such as landfill tax, incineration tax, and ‘‘Pay-as-you-throw” schemes, and launching awareness campaigns can reduce the amount of wasted food by 61% and 45% [ 18 ]. Moreover, promoting sustainable energy policies in developing countries can also help reduce food waste and improve food security [ 19 ].

Measuring the amount of food waste enables us to understand which food items are being wasted and what behaviors cause it. It also allows us to estimate the cost and value of the wasted food [ 20 ]. Educating individuals and raising their awareness are key factors in reducing food waste [ 21 ]. Additionally, factors such as environmentally friendly behavior, economic awareness, domestic skills, and collaborative behaviors may make sharing practices more effective in reducing food waste [ 22 ]. Some studies have implemented educational interventions, such as holding nutrition classes in schools to raise students’ knowledge about the nutritional benefits of school lunches, as well as the social, economic, and nutritional consequences of wasting food and its negative impacts [ 21 ]. Some other examples of interventions include storing food for consuming at another time, promoting consumption, increasing the support from canteen staff, serving smaller meals [ 23 ], and reducing dining frequency [ 11 ].

Some studies conducted in university canteens have shown how avoidable waste can be reduced by simply raising students’ awareness of the topic of food waste. These strategies can be useful to improve behaviors and increase the sustainability of university canteens [ 24 ]. Moreover, another study indicated that the most important factors affecting university students’ food waste generation are multidimensional, such as individual-level and family-level characteristics, catering features, and regional locations [ 25 ]. According to these studies, universities have the potential to influence food consumption habits while sustainably managing existing resources. As a controlled environment, the university canteen provides an ideal behavioral laboratory for studying the consumption link in the food system. Students represent the younger generation of consumers; hence, influencing their food-related habits can enhance the sustainability of future food consumption patterns [ 26 , 27 , 28 ].

The aim of the present study was (1) to estimate the quantity and the environmental consequences of food waste at the canteen, (2) to study students’ opinions on the causes of food waste through semi-structured face-to-face interviews, and (3) to implement a two-level intervention for students and staff to reduce the food waste.

Method and material

Case study description.

This study involved 300 students (all students entering the canteen) from Shahid Beheshti University of Medical Sciences in Tehran (Iran), from November 2018 to January 2019. The faculty had an independent and separate campus from the main campus of the university. The restaurant served approximately 300 meals every day. The inclusion criteria were students who had lunch in the canteen, and who received and ate their food in the canteen after booking their meal in advance. The present study consisted of three different steps. During the first step, which lasted for 13 days, the leftover food from lunch at the school’s canteen was weighed every day after lunch. Since the canteen menu was designed for a 13-day period and then repeated, 13 days were chosen as the number of days for measuring food waste. The assessment of food waste was based on direct measures conducted in the kitchens and at schools by the staff. The staff separated the wasted food from other non-food waste. The foodservice staff received briefings on the quantification procedures before the study period, and they were given paper handbooks with detailed instructions. To avoid bias linked to potential changes in students’ food consumption during the data collection period, the foodservice staff knew the real reasons for the experiment.

Food waste assessment

The food waste was measured by direct weighing and sampling, which is one of the most common and accurate methods of measuring food waste. The food waste was divided into two categories: Wasted food consisted of the leftovers from the students’ meals and did not include waste generated during food preparation and/or distribution. The data collection process involved weighing the following components: (i) Plate waste, which was the food that the diners discarded after they received their servings and left on their plates; (ii) Intact food, which included the non-served food (i.e., excess servings not served to the canteen) and other food items completely rejected by the canteen (i.e., portions of bread and fruit not collected by the students from the serving trays). The school food service offered three main courses for lunch: the first course was primarily based on a carbohydrate-rich component (such as rice, pasta, or bread), the second course was primarily based on a protein-rich component (such as meat, fish, or eggs) with a side dish of vegetables, and the third course was a portion of bread and fruit. The quantities of prepared food, plate waste and intact food were weighed separately for each lunch meal using an electronic scale (SECA ® model WT50001NF, Canada) with accuracy to the nearest gram. The data gathered referred to all the students dining at each school. This quantification allowed us to calculate, for each meal course: (a) Served food (SF), determined by the difference between prepared food (PF) and intact food (IF): SF = PF−IF; (b) Non-consumed food (NCF), which was the sum of plate waste (PW) and intact food (IF): NCF = PW + IF (i.e., the portion of prepared food that diners did not consume during lunch); (c) Consumed food (CF), calculated as the difference between the quantity of food prepared for lunch (PF) and the non-consumed food fraction (NCF): CF = PF−NCF. The weights of the students' food waste were measured, and a one-kilogram sample was taken from the total waste daily. This sample was then analyzed for four separate components (total, rice, meat, bread, vegetables, legumes, potato). Subsequently, the components of the sample were separated and weighed using a digital scale with accuracy to the nearest gram (SECA ® model WT50001NF, Canada). Then, the average food waste per student was calculated by dividing the total food waste by the number of students eating on that day.

Assessing the environmental and economic impact

The amount of carbon dioxide emitted from the farm to the table to produce that quantity of waste, as well as the amount of green water, blue water, and grey water and total water were all calculated for wasted food.

The 'carbon footprint' method was applied to calculate the amount of carbon dioxide emitted during the food production process. "The term carbon footprint is defined as a measure of the total amount of carbon dioxide emission, both direct and indirect, released by an activity or accumulated over the life stages of a specific product" [ 29 ]. Our data regarding the carbon dioxide emissions of each food item were source from the “BCFN DOUBLE PYRAMID DATABASE” and Food Climate List from Swedish University of Agricultural Sciences [ 30 ].

We used the water footprint method to measure the amount of water used in the production process of food items. The definition of water footprint is "the total volume of freshwater used to produce the goods and services consumed by an individual or community". The unit for presenting of footprint data for each food item is usually water volume, measured in cubic meter per ton (m 3 /ton). The water footprint data were available for Iran [ 31 , 32 ]. The water footprint comprises three different components: blue, green, and grey, respectively. The blue water footprint is defined as the consumption of blue water resources (which consist of surface and ground water) during the production of a product and throughout its supply chain. The word 'consumption' in this context refers to the utilization of water from available ground-surface water sources within a catchment area. Losses occur in cases such as water evaporation, were being returned to another catchment area or the sea, or the water being turned into a product [ 33 ]. The term green water footprint is used to donate the consumption of green water resources, which refers to rainwater that does not become run-off. And finally, the term grey water footprint to pollution and is defined as the volume of freshwater required to absorb the load of pollutants, considering natural background concentrations and existing ambient water quality standards [ 33 ]. In the present research, we calculated water footprint data in term of water volume per gram (m 3 /gm). The amount of water required in the process of producing each food item was determined by multiplying the water footprint by the mean food waste of the sample [ 34 ]. For example: Water used for wasted rice = Water footprint of rice (m 3 /gm) * Average wasted rice (gm).

Also, the cost and calories of the amount of wasted food were calculated. We calculated the cost by multiplying the amount of food waste in grams by the price per gram of each item obtained from the Central Bank of Iran. To evaluate and calculate the nutrient and energy intakes, the software NUTRITIONIST-IV (version 7.0; Squared Computing, Salem, OR, USA) was used.

Regarding the fact that the students are present at the campus for 9 months a year, all the consequences were calculated for 9 months a year instead of 12 months.

Qualitative study

In the second phase, semi-structured face-to-face interviews were conducted with 21 students to explore the causes of food waste at the canteen. It had a flexible structure and a question guide (open-ended questions). The question guide was developed based on the literature review and the research objectives and was pre-tested and revised before the data collection. The interviews were conducted by two trained researchers, which were conducted in a quiet and comfortable room in the school and lasted for about 30 min each. The interviews were recorded with the consent of the participants, and transcribed verbatim for data analysis. Such interviews are well suited for the exploring the perceptions and opinions of respondents regarding complex and sometimes sensitive issues, allowing for probing more information and clarification of answers. Data collection ceased when data saturation occurred.

Interventions for reducing food waste

The third stage consists of two parts: intervention and re-measurement of food waste, each lasting 13 days, which is the same duration as the first stage of our study, where measured the baseline level of food waste. Based on the result of the qualitative study conducted in the second phase, which investigated the causes of food waste through semi-structured interviews, a two-level intervention was carried out for students and staff to increase their knowledge and change their attitudes towards food waste. For the students, a campaign was launched with the motto "This is enough for me". Students in the campaign were informed about the results of the initial 13-day study, which included data on food waste, environmental impact, and estimated cost. To inform the students, pamphlets and posters were used. The results of the study were posted on the faculty's main electronic board and published on the Telegram ® channel and the Instagram page of the campus ( www.instagram.com/nnftri ). Posters with the campaign motto were also displayed on the walls of the campus canteen. Additionally, a brief explanation of motto, encouraging individuals to take only what they needed, was placed next to the canteen buffet. Students were also informed that they could request smaller portions from the staff. To intervene at the staff level, intervention involved training them to serve smaller portions to those who requested less food. In addition, they were instructed to provide bread in smaller pieces, allowing people to take bread according to their needs and avoid excessive portions.

Food waste re-assessment

For the re-measurement of food waste, another 13-day sampling session was conducted. The second sampling session followed the same methodology as the first one, which means that we used the same tools, procedures, and criteria to collect and analyze the food waste data. For example, we used the same weighing scales, containers, labels, and software to measure and record the food waste. We also used the same sampling days, sampling frequency, and sampling locations to ensure consistency and comparability. This step was carried out immediately after the end of the intervention, as the menu remained the same every 13 days, which means that were offered the same types of food and drinks in the same quantities and qualities. Additionally, this was done to avoid any confounding factors or external influences that could affect the food waste behavior of the students and staff. It aimed to assess the impact of the intervention on the quantity and composition of the food waste in the school canteen. Then, we calculated the following indicators for each type of food waste: the total wasted food, the quantity of carbon dioxide (the amount of carbon dioxide emitted from production to consumption), the water footprint (the volume of green, blue, and grey water consumed or polluted), the economic cost, and the caloric value.

Statistical analysis

Data analysis was performed using the Statistical Package for Social Sciences (SPSS) version 16 (SPSS Inc, Chicago). The Kolmogorov–Smirnov test was employed to assess the normal distribution of variables. To statistically compare quantitative variables between the two phases of the study, we used the paired t-test and the paired samples Wilcoxon test. Additionally, P -values less than 0.05 were considered statistically significant.

Food waste quantification

This study involved 300 students, corresponding to 26 monitored meals. On average, 147.26 g/per person of food remained unconsumed at the end of the lunch. The mean quantity of avoidable food waste in total during the first and third phases of the study, each lasting 13 days, was 17.96 and 15.52 kg per day, respectively. Table 1 shows the mean and standard deviation of food waste per person before and after the intervention, as well as the percentage reduction and the p-value for different food items. Overall, the mean total food waste before and after the study was 60.65 g/per person and 50.73 g/per person, respectively. There was no significant difference between the averages of food waste before and after the study ( P  = 0.397). Moreover, the p-value was non-significant for all items. Although the p-value was not significant, there was a reduction in the waste of all the food items, with legume waste reduction being the largest.

Energy and price quantification

Table 2 shows the price and energy content of wasted food. The cost of wasted food decreased by 30.14%, and energy content declined by 31.65%. The food that was wasted over the nine months when students dined on campus could feed an average of 1663 people, each consuming 2000 cal. In the first phase of the study, 120 people could be fed with the food that was wasted, and this number decreased to 84 after the intervention, resulting in 36 meals saved from being discarded. According to the estimates, the cost of year-long food waste in this faculty, based on the pre-intervention pattern, is equivalent to the salary of a manual worker in Iran for 17 months (The average wage of a manual worker is assumed to be 20000000 IRR per month).

Environmental impact

Table 3 shows the environmental effects of food waste in the canteen, including the total amounts of CO2 emissions and water. The p-value is non-significant for all items except for grey water ( p  = 0.033). Moreover, the campus emitted an average of 615674.9 g CO2 equivalent greenhouse gas per day due to food waste. Apart from CO2 emissions, which increased slightly (0.06%), total water waste, blue water, grey water, and green water waste all decreased. The total amount of water waste as a result of food waste on the campus during the nine months when the students dined in the canteen from farm to table was 6661.24 m 3 .

Qualitative result

Based on the semi-structured interviews, the main causes of food waste in the canteens as perceived by the students ( N  = 20) were determined. They believe that low-quality meals, unpleasant taste, large portions, and a limited menu, respectively, are the causes of food waste. The main cause of food waste as perceived by the university system is beyond of the students’ control. (Fig.  1 ).

The main causes of food waste in the canteens based on student's interview ( N  = 20)

The main purpose of the present study was to measure the amount of food waste in the canteen and to implement an intervention to reduce it. We also hypothesized that the intervention would reduce plate-waste. The study results showed that food waste was relatively high in the canteen and had significant environmental and financial consequences.

The reduction in food waste observed between the baseline and post-intervention periods was not statistically significant. It seems that continuous interventions may be needed to achieve a significant reduction in plate-waste over time.

In the present study, the total amount of food waste was estimated to be (mean = 60.65 g/person). A study in Iran reported that each person wasted about 27.6 kg of edible food annually, and households with better food consumption management had a lower level of food waste [ 35 ]. Another study in the university canteens showed that the total food waste amount was 246/75 t/a [ 36 ]. Regarding the types of wasted food, bread was more frequently wasted than rice products. Mohammadi et al. suggested that this might be due to the preference for fresh bread in Iran, as the quality of the bread deteriorated upon cooling down [ 37 ]. Vegetables were the second most wasted food category. A pilot study conducted by Abadi et al. in Iran also confirmed that staple foods and vegetables were the most commonly wasted food items [ 38 ]. Food waste in Western universities had similar characteristics. Rajan et al. found that in Northern British Columbia University in Canada, grain-based waste accounted for the largest proportion of food waste (28%), followed by raw fruits and vegetables (20%) [ 39 ].

Our results also revealed that the campus emitted an average of 615674.9 g CO2 equivalent greenhouse gas due to food waste. The mean waste of livestock and poultry meat by university students accounted for only 13.60 g (per person) of food waste. Meat production and consumption have a significant impact on greenhouse gas emissions. Therefore, reducing meat consumption and waste and promoting a balanced diet are important strategies for fostering ecologically sustainable consumption and enhancing students’ nutritious diets [ 40 ]. Given that on average, it takes 3 kcal of fossil fuel energy to produce 1 kcal of food on the farm (before accounting for energy requirements for food processing and transportation) [ 41 ], the annual food waste in the faculty canteen is responsible for 9983821 kcal of fossil fuel energy. The intervention resulted in a 29.98% or 30% reduction in fossil fuel use (from 721053.6 kcal in the first phase to 504837.3 kcal in the third phase, which was after the intervention).

Avoidable food waste accounts for approximately 20% of the European environmental footprint. Reducing waste of all food items, such as bread and rice, has significant positive effects on the environment, mainly due to the large mass of the waste generated [ 42 ]. A study conducted in America, which examined the environmental impact of increased food waste, found that 25% of freshwater was wasted due to food waste [ 41 ]. Similarly, a study measuring the carbon footprint of supermarket food waste reported a total carbon dioxide waste of 2500 t CO2e [ 43 ]. Furthermore, a national-level study reported that the amount of (green plus blue) water footprint ranged from 5938 to 8508 km3/year and it would increase by up to 22% by 2090 [ 44 ].

According to a semi-structured interview, the low quality of food was a major factor in food waste, as it was not appetizing. The quality of raw materials, such as meat, legumes, and vegetables, was poor. Beans and meat were undercooked, and the food lacked visual appeal. It also had an unpleasant smell and taste. A study suggested that improving food quality and taste could reduce food waste [ 45 ].

Surprisingly, the intervention, which was mainly educational, did not reduce food waste significantly. However, there was a considerable decrease in the waste of all food items. The intervention also reduced the percentage of water waste, with only the decrease in grey water being statistically significant. CO2 emissions stayed relatively constant throughout the study.

Educational campaigns that focused on strengthening beliefs about the environmental consequences of food waste and empowering students have been shown to be effective in reducing food waste [ 46 ]. Therefore, such campaigns should be implemented on campus, especially targeting incoming freshmen. In a study involving 540 students living in residence halls and participating in a meal plan, broadcasting prompt-messages led to a 15% reduction in food waste among students [ 47 ]. This result is consistent with our results, which showed a 16.35% decrease in food waste. Although the overall reduction in food waste was not statistically significant, it led to a 30.14% reduction in expenses and a 16.66% reduction in total water waste. The reduction in food waste also saved 41.72% of calories. Moreover, it is worth noting that our intervention mainly used social media, posters, and electronic interaction, which might not be as effective as face-to-face interventions [ 48 ].

Although low food quality was a significant cause of food waste on the campus, there were several other contributing factors. For instance, aspects of food safety and family standards played a pivotal role in food consumption and waste behavior [ 49 ]. A study suggested that taste, smell, flavor, and texture of food were associated with food waste behavior [ 45 ]. he findings of our study aligned with these causes of food waste. The students’ personal feedback indicated that chicken had the lowest quality and the worst taste. This underscores the importance of food standards and sensory perceptions in influencing food waste behavior. While this issue was crucial, we could not address it fully because our intervention was mainly educational, and we did not intervene in the ingredients, the recipe, or cooking.

Despite the reduction in food waste, most of the changes in our study were not statistically significant, and there could be several reasons for this. One possible reason is that the intervention lasted for a relatively short time, which might not have been enough to lead to drastic changes in food waste behavior. A study conducted in primary schools in Porto found that short-term results from an educational intervention were only observed in children, whereas a significant decrease in food waste among teachers was observed in the long-term [ 21 ]. Since our target population was adults, expecting an immediate decrease in food waste might not have been realistic. Achieving a complete reduction of avoidable food waste is unlikely within a short-term or mid-term timeframe [ 42 ]. To reduce the amount of food waste in university canteens, canteens should offer smaller servings of the main meal and smaller dishes. Additionally, canteens could introduce tiered pricing for different portion sizes to discourage students from blindly choosing large portions when both large and small portions have the same price [ 50 ].

Large portion sizes can contribute to more food waste [ 51 ]. In the canteen, food is provided in large portions (approximately 555 g), which is a significant factor in food waste. Along with most meals, optional soups and salads are available. Although these are optional, many students choose to include them, often without the intention of consuming them, which further contributes to food waste. Our intervention included serving bread, which was previously wasted in large quantities, in smaller pieces, resulting in reduced bread waste. However, this did not mean that the students received smaller meals; they could still take as much bread as they wanted. This approach successfully reduced waste. Similar studies have also shown that reducing portion sizes can help decrease waste [ 23 , 52 ]. Additionally, extending the dining time to allow students more time to eat [ 53 , 54 ], offering a menu with various choices [ 55 , 56 ], and encouraging students to share food [ 45 ] can all help prevent food waste. One effective approach to reducing food waste is involving students in meal planning [ 21 ]. It is important to note that individuals who rely more on convenient food options and are less interested in food preparation are more likely to discard food. Therefore, campus food, which requires no preparation, is more likely to be wasted than meals prepared at home [ 17 ]. To effectively reduce food waste on the campus, it is essential to understand students’ food waste behavior during the ‘in-use’ phase [ 49 ].

One of the challenges we faced during the intervention was persuading the students and workers that food waste could be reduced and had significant environmental consequences, reflecting their indifferent attitude toward the issue of food waste. Similar to our study, people often ignore the environmental consequences and carbon footprint of wasting food or when asked why they should reduce food waste [ 57 ].

The present study had several limitations. First of all, the intervention was only done in the short- and medium-term, as it was difficult to implement for a longer duration. According to other authors, it is essential to measure the impact at least 6 months after the intervention to assess behavior change retention and draw conclusions about its effectiveness [ 58 ]. Other limitations resulted from the fact that the food service operated a 2-week cycle menu, which might have introduced a bias into our results. Moreover, the limited face-to-face interactions with the students and the inability to improve the quality of food were other limitations.

A particular strength of this work is that we measured the impact of interventions at two different periods by weighing food rejected, which is the most accurate method for measuring plate waste. We weighed both original servings and plate waste for each participant [ 59 ]. In addition, we weighed all individual servings, and not just a random sample of initial servings, as done by other authors [ 60 ]. Moreover, analyzing the same menus at two different time-points gave us information about the outcome of the intervention, regardless of the effect associated with the menu type. Furthermore, by measuring food waste, we estimated some environmental consequences, such as CO2 emissions and water waste. We measured food waste in the canteen of the campus, which might provide us with information about food waste that we could not observe at the household level. Wasting food in households occurs due to different factors, such as meal preparation, food storage, and others, but at university, students deal with food in the edible state. Therefore, we cannot directly compare food waste in these two settings [ 57 ]. No similar studies on food waste in canteens were previously carried out in Iran.

In conclusion, this study casts a spotlight on the significant issue of food waste within the campus canteen, elucidating its profound environmental implications such as CO2 emissions and water wastage, along with its economic consequences. Despite concerted efforts to mitigate food waste, our findings indicate that the reduction achieved was relatively modest. However, it is important to note that this study has yielded valuable insights into the underlying causes of food waste, including factors such as low-quality meals, an unpleasant taste, oversized portions, and limited menu options. Educational interventions emerged as a potential catalyst for reshaping student behavior and curbing food waste. By enhancing awareness, adjusting portion sizes, and improving the quality of food served, these interventions hold promise for affecting more substantial waste reduction. While the immediate impact of our study may appear limited, our study provides a useful baseline for future research on food waste in campus canteens, as it demonstrates the feasibility and challenges of conducting such research. We recommend that future studies use larger and more representative samples, cover longer periods of time, and test different types of interventions, such as incentives, feedback, nudges, or social norms. By doing so, we hope to generate more robust and reliable evidence on how to reduce food waste and its negative consequences.

Availability of data and materials

The data used to support the findings of this study are available from the corresponding.

Mbow H-OP, Reisinger A, Canadell J, O’Brien P. Special Report on climate change, desertification, land degradation, sustainable land management, food security, and greenhouse gas fluxes in terrestrial ecosystems (SR2). Ginevra, IPCC 2017; 650.

Cuéllar AD, Webber ME. Wasted food, wasted energy: the embedded energy in food waste in the United States. Environ Sci Technol. 2010;44(16):6464–9.

Article   PubMed   PubMed Central   Google Scholar  

Palmer S. Paying the high price of food waste. Environ Nutr. 2010; 33(1).

García-González Á, Achón M, Carretero Krug A, Varela-Moreiras G, Alonso-Aperte E. Food sustainability knowledge and attitudes in the Spanish adult population: a cross-sectional study. Nutrients. 2020;12(10):3154.

Garcia-Herrero I, Hoehn D, Margallo M, Laso J, Bala A, Batlle-Bayer L, Fullana P, Vazquez-Rowe I, Gonzalez M, Durá M. On the estimation of potential food waste reduction to support sustainable production and consumption policies. Food Policy. 2018;80:24–38.

Article   Google Scholar  

Bahar NH, Lo M, Sanjaya M, Van Vianen J, Alexander P, Ickowitz A, Sunderland T. Meeting the food security challenge for nine billion people in 2050: What impact on forests. Glob Environ Chang. 2020;62:102056.

Morone P, Falcone PM, Tartiu VE. Food waste valorisation: assessing the effectiveness of collaborative research networks through the lenses of a COST action. J Clean Prod. 2019;238:117868.

Falcone PM, Imbert E. Bringing a sharing economy approach into the food sector: the potential of food sharing for reducing food waste. In: Morone P, Papendiek F, Tartiu V, editors. Food waste reduction and valorisation: sustainability assessment and policy analysis. Cham: Springer; 2017. p. 197–214.

Chapter   Google Scholar  

Rajan J, Fredeen AL, Booth AL, Watson M. Measuring food waste and creating diversion opportunities at Canada’s Green University TM. J Hunger Environ Nutr. 2018;13(4):573–86.

Kummu M, De Moel H, Porkka M, Siebert S, Varis O, Ward PJ. Lost food, wasted resources: global food supply chain losses and their impacts on freshwater, cropland, and fertiliser use. Sci Total Environ. 2012;438:477–89.

Article   CAS   PubMed   Google Scholar  

Babich R, Smith S. “Cradle to grave”: an analysis of sustainable food systems in a university setting. J Culin Sci Technol. 2010;8(4):180–90.

Reynolds C, Thompson K, Boland J, Dawson D. Climate change on the menu? A retrospective look at the development of South Australian municipal food waste policy. Int J Clim Change Impacts Responses. 2014;3:3–101.

Google Scholar  

Thyberg KL, Tonjes DJ. Drivers of food waste and their implications for sustainable policy development. Resour Conserv Recycl. 2016;106:110–23.

Otles S, Kartal C. Food waste valorization. In sustainable food systems from agriculture to industry. Amsterdam: Elsevier; 2018. p. 371–99.

Food and Agriculture Organization of the United Nations. Key facts on food loss and waste you should know! 2019. https://www.fao.org/save-food/resources/keyfindings/en/ . Accessed 28 Oct 2019

Farr-Wharton G, Foth M, Choi JHJ. Identifying factors that promote consumer behaviours causing expired domestic food waste. J Consum Behav. 2014;13(6):393–402.

Mallinson LJ, Russell JM, Barker ME. Attitudes and behaviour towards convenience food and food waste in the United Kingdom. Appetite. 2016;103:17–28.

Article   PubMed   Google Scholar  

Xevgenos D, Papadaskalopoulou C, Panaretou V, Moustakas K, Malamis D. Success stories for recycling of MSW at municipal level: a review. Waste Biomass Valorization. 2015;6:657–84.

Falcone PM. Sustainable energy policies in developing countries: a review of challenges and opportunities. Energies. 2023;16(18):6682.

Article   CAS   Google Scholar  

Reynolds CJ, Mirosa M, Clothier B. New Zealand’s food waste: estimating the tonnes, value, calories and resources wasted. Agriculture. 2016;6(1):9.

Martins ML, Rodrigues SS, Cunha LM, Rocha A. Strategies to reduce plate waste in primary schools–experimental evaluation. Public Health Nutr. 2016;19(8):1517–25.

Morone P, Falcone PM, Imbert E, Morone A. Does food sharing lead to food waste reduction? An experimental analysis to assess challenges and opportunities of a new consumption model. J Clean Prod. 2018;185:749–60.

Blondin SA, Djang HC, Metayer N, Anzman-Frasca S, Economos CD. ‘It’s just so much waste’. A qualitative investigation of food waste in a universal free School Breakfast Program. Public Health Nutr. 2015;18(9):1565–77.

Pinto RS, dos Santos Pinto RM, Melo FFS, Campos SS, Cordovil CMDS. A simple awareness campaign to promote food waste reduction in a University canteen. Waste Manage. 2018;76:28–38.

Qian L, Li F, Cao B, Wang L, Jin S. Determinants of food waste generation in Chinese university canteens: evidence from 9192 university students. Resour Conserv Recycl. 2021;167:105410.

Balzaretti C, Ventura V, Ratti S, Ferrazzi G, Spallina A, Carruba M, Castrica M. Improving the overall sustainability of the school meal chain: the role of portion sizes. Eat Weight Disord Stud Anorexia, Bulimia Obes. 2020;25(1):107–16.

Derqui B, Fayos T, Fernandez V. Towards a more sustainable food supply chain: opening up invisible waste in food service. Sustainability. 2016;8(7):693.

Wyse R, Yoong SL, Dodds P, Campbell L, Delaney T, Nathan N, Janssen L, Reilly K, Sutherland R, Wiggers J. Online canteens: awareness, use, barriers to use, and the acceptability of potential online strategies to improve public health nutrition in primary schools. Health Promot J Austr. 2017;28(1):67–71.

Wiedmann T, Minx J. A definition of ‘carbon footprint.’ Ecol Econ Res Trends. 2008;1:1–11.

Ruini LF, Ciati R, Pratesi CA, Marino M, Principato L, Vannuzzi E. Working toward healthy and sustainable diets: the “Double Pyramid Model” developed by the Barilla Center for food and nutrition to raise awareness about the environmental and nutritional impact of foods. Front Nutr. 2015;2:9.

Hoekstra AY, Chapagain A, Martinez-Aldaya M, Mekonnen M. Water footprint manual: state of the art 2009. Water Footprint Network, Enschede, The Netherlands. 2009.

Mekonnen MM, Hoekstra AY. The green, blue and grey water footprint of crops and derived crop products. Hydrol Earth Syst Sci. 2011;15(5):1577–600.

Mekonnen MM, Hoekstra AY. A global assessment of the water footprint of farm animal products. Ecosystems. 2012;15(3):401–15.

Sobhani SR, Rezazadeh A, Omidvar N, Eini-Zinab H. Healthy diet: a step toward a sustainable diet by reducing water footprint. J Sci Food Agric. 2019;99(8):3769–75.

Fami HS, Aramyan LH, Sijtsema SJ, Alambaigi A. Determinants of household food waste behavior in Tehran city: a structural model. Resour Conserv Recycl. 2019;143:154–66.

Li J, Li W, Wang L, Jin B. Environmental and cost impacts of food waste in university canteen from a life cycle perspective. Energies. 2021;14(18):5907.

Mohammadi IM. Factors influencing wheat, flour, and bread waste in Iran. J New Seeds. 2007;8(4):67–78.

Abadi B, Mahdavian S, Fattahi F. The waste management of fruit and vegetable in wholesale markets: intention and behavior analysis using path analysis. J Clean Prod. 2021;279:123802.

Wu Y, Tian X, Li X, Yuan H, Liu G. Characteristics, influencing factors, and environmental effects of plate waste at university canteens in Beijing, China. Resour Conserv Recycl. 2019;149:151–9.

Xue L, Prass N, Gollnow S, Davis J, Scherhaufer S, Östergren K, Cheng S, Liu G. Efficiency and carbon footprint of the German meat supply chain. Environ Sci Technol. 2019;53(9):5133–42.

Hall KD, Guo J, Dore M, Chow CC. The progressive increase of food waste in America and its environmental impact. PLoS ONE. 2009;4(11):e7940.

Usubiaga A, Butnar I, Schepelmann P. Wasting food, wasting resources: potential environmental savings through food waste reductions. J Ind Ecol. 2018;22(3):574–84.

Scholz K, Eriksson M, Strid I. Carbon footprint of supermarket food waste. Resour Conserv Recycl. 2015;94:56–65.

Mekonnen MM, Gerbens-Leenes W. The water footprint of global food production. Water. 2020;12(10):2696.

Lazell J. Consumer food waste behaviour in universities: sharing as a means of prevention. J Consum Behav. 2016;15(5):430–9.

Campbell-Arvai V. Food-related environmental beliefs and behaviours among university undergraduates: a mixed-methods study. Int J Sustain High Educ. 2015. https://doi.org/10.1108/IJSHE-06-2013-0071 .

Whitehair KJ, Shanklin CW, Brannon LA. Written messages improve edible food waste behaviors in a university dining facility. J Acad Nutr Diet. 2013;113(1):63–9.

Young W, Russell SV, Robinson CA, Barkemeyer R. Can social media be a tool for reducing consumers’ food waste? A behaviour change experiment by a UK retailer. Resour Conserv Recycl. 2017;117:195–203.

Quested TE, Marsh E, Stunell D, Parry AD. Spaghetti soup: the complex world of food waste behaviours. Resour Conserv Recycl. 2013;79:43–51.

Boschini M, Falasconi L, Cicatiello C, Franco S. Why the waste? A large-scale study on the causes of food waste at school canteens. J Clean Prod. 2020;246:118994.

Landry C, Smith TA, Turner D. Food waste and food retail density. J Food Prod Marketing. 2018;24(5):632–53.

Freedman MR, Brochado C. Reducing portion size reduces food intake and plate waste. Obesity. 2010;18(9):1864–6.

Bergman EA, Buergel NS, Englund TF, Femrite A. The relationship between the length of the lunch period and nutrient consumption in the elementary school lunch setting. J Child Nutr Manage. 2004;28(2):1.

Deliens T, Clarys P, De Bourdeaudhuij I, Deforche B. Determinants of eating behaviour in university students: a qualitative study using focus group discussions. BMC Public Health. 2014;14(1):1–12.

Guthrie HA. Effect of a flavored milk option in a school lunch program. J Am Diet Assoc. 1977;71(1):35–40.

Adams MA, Pelletier RL, Zive MM, Sallis JF. Salad bars and fruit and vegetable consumption in elementary schools: a plate waste study. J Am Diet Assoc. 2005;105(11):1789–92.

Watson M, Meah A. Food, waste and safety: negotiating conflicting social anxieties into the practices of domestic provisioning. Sociol Rev. 2012;60:102–20.

de Sousa D, Fogel A, Azevedo J, Padrão P. The effectiveness of web-based interventions to promote health behaviour change in adolescents: a systematic review. Nutrients. 2022;14(6):1258.

Visschers VH, Gundlach D, Beretta C. Smaller servings vs. information provision: results of two interventions to reduce plate waste in two university canteens. Waste Manag. 2020;103:323–33.

Cohen JF, Richardson S, Austin SB, Economos CD, Rimm EB. School lunch waste among middle school students: nutrients consumed and costs. Am J Prev Med. 2013;44(2):114–21.

Download references

Acknowledgements

We thank authorities of the International Campus of Shahid beheshti University of Medical Sciences and the department of Community Nutrition for their support during the design of this study. We also extend our profound gratitude to all study participants who made it possible for us to obtain data for this study.

Not applicable.

Author information

Authors and affiliations.

Department of Nutrition, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

Seyedeh Fatemeh Fatemi & Seyyed Reza Sobhani

Department of Community Nutrition, Faculty of Nutrition Sciences & Food Technology, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Hassan Eini-Zinab, Fatemeh Manafi Anari, Mahdieh Amirolad & Zahra Babaei

You can also search for this author in PubMed   Google Scholar

Contributions

S.R.S and H.E.Z.: conceptualization, methodology, data curation and supervision. F.M.A, M.A, Z.B: data collection. M.A, S.F.F and S.R.S.: writing-original draft preparation, methodology, software and investigation. S.R.S: supervision, validation, writing-reviewing and editing.

Corresponding author

Correspondence to Seyyed Reza Sobhani .

Ethics declarations

Ethics approval and consent to participate.

This study was conducted according to the guidelines laid down in the Declaration of Helsinki, and all procedures involving human subjects were approved by the Ethical Committees of the NNFTRI and the Faculty of Nutrition Sciences and Food Technology, Shahid Beheshti University of Medical Sciences. Written informed consent was obtained from all subjects.

Consent for publication

All authors read and are in support of article submission and publication.

Competing interests

We declare that none of us has a personal or financial competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Fatemi, S.F., Eini-Zinab, H., Anari, F.M. et al. Food waste reduction and its environmental consequences: a quasi-experimental study in a campus canteen. Agric & Food Secur 13 , 37 (2024). https://doi.org/10.1186/s40066-024-00488-y

Download citation

Received : 16 June 2023

Accepted : 05 June 2024

Published : 10 September 2024

DOI : https://doi.org/10.1186/s40066-024-00488-y

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Campus canteen
• Environmental effects
• Nutritional behavior

Agriculture & Food Security

ISSN: 2048-7010

• General enquiries: [email protected]

Quasi-Experimenteller Aufbau: Definition, Arten, Beispiele

Appinio Research · 11.09.2024 · 37min Lesezeit

Haben Sie sich schon einmal gefragt, wie Forscher Ursache-Wirkungs-Beziehungen in der realen Welt aufdecken, wo kontrollierte Experimente oft nicht möglich sind? Die quasi-experimentelle Planung ist der Schlüssel dazu. In diesem Leitfaden werden wir die Feinheiten des quasi-experimentellen Designs enträtseln und seine Definition, seinen Zweck und seine Anwendungen in verschiedenen Bereichen beleuchten. Egal, ob Sie studieren, berufstätig sind oder einfach nur neugierig auf die Methoden hinter aussagekräftigen Forschungsergebnissen sind, begleiten Sie uns, wenn wir in die Welt der quasi-experimentellen Planung eintauchen, um komplexe Konzepte zu vereinfachen und uns auf eine Reise des Wissens und der Entdeckung zu begeben.

Was ist ein quasi-experimentelles Design?

Ein quasi-experimentelles Design ist eine Forschungsmethode, mit der die Auswirkungen unabhängiger Variablen auf abhängige Variablen untersucht werden, wenn eine vollständige experimentelle Kontrolle nicht möglich oder ethisch nicht vertretbar ist. Sie liegt zwischen kontrollierten Experimenten, bei denen die Variablen streng kontrolliert werden, und reinen Beobachtungsstudien, bei denen die Forscher kaum Kontrolle über die Variablen haben. Das quasi-experimentelle Design ahmt einige Aspekte der experimentellen Forschung nach, verfügt aber nicht über eine Randomisierung.

Der Hauptzweck des quasi-experimentellen Designs besteht darin, Ursache-Wirkungs-Beziehungen zwischen Variablen in realen Umgebungen zu untersuchen. Forscher nutzen diesen Ansatz, um Forschungsfragen zu beantworten, Hypothesen zu testen und die Auswirkungen von Maßnahmen oder Behandlungen zu untersuchen, wenn sie keine traditionellen experimentellen Methoden anwenden können. Quasi-experimentelle Studien zielen darauf ab, die interne Validität zu maximieren und aussagekräftige Schlüsse zu ziehen, wobei praktische Einschränkungen und ethische Erwägungen berücksichtigt werden.

Quasi-experimentelles vs. experimentelles Design

Es ist wichtig, die Unterschiede zwischen quasi-experimentellem und experimentellem Design zu verstehen, um die einzigartigen Merkmale jedes Ansatzes zu verstehen:

• Randomisierung: Bei der experimentellen Planung ist die zufällige Zuordnung der Teilnehmer zu den Gruppen ein entscheidendes Merkmal. Bei der quasi-experimentellen Planung hingegen wird aufgrund praktischer Einschränkungen oder ethischer Erwägungen auf eine Randomisierung verzichtet.
• Kontrollgruppen : Experimentelle Studien umfassen in der Regel Kontrollgruppen, die keiner Behandlung oder einem Placebo unterzogen werden. Das quasi-experimentelle Design kann Vergleichsgruppen haben, hat aber nicht das gleiche Maß an Kontrolle.
• Manipulation der IV: Die experimentelle Planung beinhaltet die absichtliche Manipulation der unabhängigen Variablen. Bei quasi-experimentellen Versuchsplänen geht es oft um natürlich vorkommende unabhängige Variablen.
• Kausalschluss: Die experimentelle Planung ermöglicht aufgrund der Randomisierung und Kontrolle stärkere Kausalschlüsse. Quasi-experimentelles Design erlaubt kausale Schlüsse, allerdings mit einigen Einschränkungen.

Wann sollte man ein quasi-experimentelles Design verwenden?

Ein quasi-experimentelles Design ist in verschiedenen Situationen besonders wertvoll:

• Ethische Einschränkungen: Wenn die Manipulation der unabhängigen Variable ethisch nicht vertretbar oder unpraktisch ist, bietet ein quasi-experimentelles Design eine Alternative zur Untersuchung natürlich vorkommender Variablen.
• Reale Umgebungen: Wenn Forscher Phänomene in realen Kontexten untersuchen wollen, ermöglicht ihnen das quasi-experimentelle Design, dies ohne künstliche Laborumgebungen zu tun.
• Begrenzte Ressourcen: In Fällen, in denen die Ressourcen begrenzt sind und die Durchführung eines kontrollierten Experiments zu kostspielig ist, kann ein quasi-experimentelles Design wertvolle Erkenntnisse liefern.
• Politik- und Programmbewertung: Quasi-experimentelles Design wird häufig zur Bewertung der Wirksamkeit von Maßnahmen, Interventionen oder Programmen verwendet, die den Teilnehmern nicht zufällig zugewiesen werden können.

Bedeutung des quasi-experimentellen Designs in der Forschung

Quasi-experimentelles Design spielt in der Forschung aus mehreren Gründen eine wichtige Rolle:

• Komplexität der realen Welt: Sie ermöglicht es den Forschern, komplexe Probleme der realen Welt anzugehen, bei denen kontrollierte Experimente nicht durchführbar sind. Damit wird die Lücke zwischen kontrollierten Experimenten und reinen Beobachtungsstudien geschlossen.
• Ethische Forschung: Sie bietet einen ehrlichen Ansatz, wenn die Manipulation von Variablen oder die Zuweisung von Behandlungen den Teilnehmern schaden oder ethische Standards verletzen könnte.
• Auswirkungen auf Politik und Praxis: Quasi-experimentelle Studien führen zu Erkenntnissen, die direkt in die Politikgestaltung einfließen und praktische Lösungen in Bereichen wie Bildung, Gesundheitswesen und Sozialwissenschaften bieten.
• Verbesserte externe Validität: Ergebnisse aus quasi-experimenteller Forschung haben oft eine hohe externe Validität, wodurch sie besser auf breitere Bevölkerungsgruppen und Kontexte anwendbar sind.

Indem sie die Herausforderungen und Chancen des quasi-experimentellen Designs annehmen, können Forscher wertvolle Erkenntnisse zu ihrem jeweiligen Fachgebiet beitragen und positive Veränderungen in der realen Welt vorantreiben.

Schlüsselkonzepte im quasi-experimentellen Design

Beim quasi-experimentellen Design ist es wichtig, die grundlegenden Konzepte zu verstehen, die dieser Forschungsmethodik zugrunde liegen. Lassen Sie uns diese Schlüsselkonzepte im Detail untersuchen.

Unabhängige Variable

Die unabhängige Variable (IV) ist der Faktor, den Sie in Ihrer Forschung untersuchen oder manipulieren möchten. Im Gegensatz zu kontrollierten Experimenten, bei denen Sie die IV direkt beeinflussen können, geht es bei quasi-experimentellen Versuchsplänen oft um natürlich vorkommende Variablen. Wenn Sie zum Beispiel die Auswirkungen einer neuen Lehrmethode auf die Schülerleistungen untersuchen, ist die Lehrmethode Ihre unabhängige Variable.

Abhängige Variable

Die abhängige Variable (DV) ist das Ergebnis oder die Reaktion, die Sie messen, um die Auswirkungen von Veränderungen der unabhängigen Variable zu bewerten. Um beim Beispiel der Lehrmethode zu bleiben, wäre die abhängige Variable die akademische Leistung der Schüler, die in der Regel anhand von Testergebnissen, Noten oder anderen relevanten Messgrößen gemessen wird.

Kontrollgruppen vs. Vergleichsgruppen

Auch wenn bei quasi-experimentellen Studien der Luxus fehlt, die Teilnehmer nach dem Zufallsprinzip einer Kontroll- und einer Versuchsgruppe zuzuordnen, können Sie dennoch Vergleichsgruppen bilden, um aussagekräftige Schlüsse zu ziehen. Kontrollgruppen bestehen aus Personen, die die Behandlung nicht erhalten, während Vergleichsgruppen verschiedenen Stufen oder Varianten der Behandlung ausgesetzt sind. Diese Gruppen helfen den Forschern, die Wirkung der unabhängigen Variable zu messen.

Pre-Test und Post-Test Maßnahmen

Bei quasi-experimentellen Studien ist es üblich, Daten sowohl vor als auch nach der Einführung der unabhängigen Variable zu erheben. Die anfänglichen Daten (Pre-Test) dienen als Ausgangsbasis, so dass Sie die Veränderungen im Laufe der Zeit (Post-Test) messen können. Auf diese Weise lassen sich die Auswirkungen der unabhängigen Variable genauer beurteilen. Wenn Sie zum Beispiel die Wirksamkeit eines neuen Medikaments untersuchen, messen Sie den Gesundheitszustand der Patienten vor der Verabreichung des Medikaments (Pre-Test) und danach (Post-Test).

Bedrohungen der internen Validität

Die interne Validität ist entscheidend für den Nachweis einer Ursache-Wirkungs-Beziehung zwischen den unabhängigen und abhängigen Variablen. Bei einem quasi-experimentellen Design kann die interne Validität jedoch durch mehrere Faktoren beeinträchtigt werden. Zu diesen Bedrohungen gehören:

• Selektionsverzerrung : Wenn sich nicht randomisierte Gruppen systematisch in einer Weise unterscheiden, die das Ergebnis der Studie beeinflusst.
• Historische Effekte: Externe Ereignisse oder Veränderungen im Laufe der Zeit, die die Ergebnisse beeinflussen.
• Reifungseffekte: Natürliche Veränderungen oder Entwicklungen, die bei den Teilnehmern während der Studie auftreten.
• Regression zum Mittelwert: Die Tendenz der extremen Werte einer Variablen, sich bei einer erneuten Prüfung dem Mittelwert anzunähern.
• Attrition und Mortalität: Der Verlust von Teilnehmern im Laufe der Zeit, der die Ergebnisse möglicherweise verzerrt.
• Testeffekte: Der bloße Akt des Testens oder Beurteilens von Teilnehmern kann sich auf deren spätere Leistung auswirken.

Das Verständnis dieser Risiken ist für die Planung und Durchführung von quasi-experimentellen Studien, die gültige und zuverlässige Ergebnisse liefern, unerlässlich.

Randomisierung und Nicht-Randomisierung

In traditionellen experimentellen Studien ist die Randomisierung ein wirksames Mittel, um sicherzustellen, dass die Gruppen zu Beginn der Studie gleichwertig sind. Bei quasi-experimentellen Versuchsplänen wird jedoch aufgrund der Art der Untersuchung häufig auf eine Randomisierung verzichtet. Das bedeutet, dass die Teilnehmer nicht nach dem Zufallsprinzip den Behandlungs- und Kontrollgruppen zugewiesen werden. Stattdessen müssen die Forscher verschiedene Techniken anwenden, um Verzerrungen zu minimieren und sicherzustellen, dass die Gruppen so ähnlich wie möglich sind.

Wenn Sie beispielsweise eine Studie über die Auswirkungen einer neuen Lehrmethode in einem realen Klassenzimmer durchführen, können Sie die Schüler nicht nach dem Zufallsprinzip den Behandlungs- und Kontrollgruppen zuordnen. Stattdessen können Sie statistische Methoden verwenden, um die Schüler anhand relevanter Merkmale wie früherer akademischer Leistungen oder des sozioökonomischen Status zu vergleichen. Dieser Abgleich hilft bei der Kontrolle potenzieller Störvariablen und erhöht die Validität Ihrer Studie.

Arten von quasi-experimentellen Designs

Bei quasi-experimentellen Versuchsplänen verwenden Forscher verschiedene Ansätze, um kausale Zusammenhänge zu untersuchen und die Auswirkungen unabhängiger Variablen zu untersuchen, wenn eine vollständige experimentelle Kontrolle schwierig ist. Im Folgenden werden diese Arten von quasi-experimentellen Versuchsplänen erläutert.

Ein-Gruppen-Posttest-Only-Versuch

Der Ein-Gruppen-Posttest-Only-Versuch ist eine der einfachsten Formen des quasi-experimentellen Versuchsplans. Bei diesem Design wird eine einzige Gruppe der unabhängigen Variable ausgesetzt, und die Daten werden erst nach der Intervention erhoben. Im Gegensatz zu kontrollierten Experimenten gibt es keine Vergleichsgruppe. Dieses Design ist nützlich, wenn die Forscher keinen Vortest durchführen können oder wenn dies logistisch schwierig ist.

Beispiel : Angenommen, Sie möchten die Wirksamkeit eines neuen Zeitmanagement-Seminars bewerten. Sie bieten das Seminar einer Gruppe von Mitarbeitern an und messen unmittelbar danach deren Produktivität, um festzustellen, ob es eine erkennbare Wirkung gibt.

Ein-Gruppen-Pretest-Posttest-Design

Ähnlich wie beim Nur-Posttest-Design für eine Gruppe umfasst dieser Ansatz zusätzlich zum Posttest eine Pretest-Messung. Die Forscher erheben Daten sowohl vor als auch nach der Intervention. Durch den Vergleich der Pretest- und Posttest-Ergebnisse innerhalb derselben Gruppe erhalten Sie ein besseres Verständnis für die Veränderungen, die durch die unabhängige Variable auftreten.

Beispiel : Wenn Sie die Auswirkungen eines Stressbewältigungsprogramms auf das Stressniveau der Teilnehmer untersuchen, würden Sie deren Stressniveau vor dem Programm (Vortest) und nach Abschluss des Programms (Nachtest) messen, um etwaige Veränderungen zu bewerten.

Nicht-äquivalentes Gruppendesign

Das Design der nicht-äquivalenten Gruppen umfasst mehrere Gruppen, die jedoch nicht zufällig zugewiesen werden. Stattdessen müssen die Forscher die relevanten Variablen sorgfältig abgleichen oder kontrollieren, um Verzerrungen zu minimieren. Dieses Design ist besonders nützlich, wenn eine zufällige Zuweisung nicht möglich oder ethisch nicht vertretbar ist.

Beispiel : Stellen Sie sich vor, Sie untersuchen die Wirksamkeit von zwei Lehrmethoden an zwei verschiedenen Schulen. Sie können die Schüler nicht nach dem Zufallsprinzip den Schulen zuweisen, aber Sie können sie anhand von Faktoren wie Alter, frühere akademische Leistungen und sozioökonomischer Status sorgfältig zuordnen, um gleichwertige Gruppen zu bilden.

Zeitreihen-Design

Das Zeitreihendesign ist ein Ansatz, bei dem Daten zu mehreren Zeitpunkten vor und nach der Intervention erfasst werden. Dieses Design ermöglicht es den Forschern, Trends und Muster im Laufe der Zeit zu analysieren, was wertvolle Einblicke in die anhaltenden Auswirkungen der unabhängigen Variable liefert.

Beispiel : Wenn Sie die Auswirkungen einer neuen Marketingkampagne auf die Produktverkäufe untersuchen, würden Sie die Verkaufsdaten in regelmäßigen Abständen (z. B. monatlich) vor und nach dem Start der Kampagne erfassen, um langfristige Trends zu beobachten.

Regressions-Diskontinuitäts-Design

Das Regressions-Diskontinuitäts-Design wird eingesetzt, wenn die Teilnehmer auf der Grundlage eines bestimmten Grenzwerts oder Schwellenwerts verschiedenen Gruppen zugeordnet werden. Dieses Design wird häufig in der Bildungs- und Politikforschung verwendet, um die Auswirkungen von Interventionen in der Nähe eines Grenzwertes zu bewerten.

Beispiel : Angenommen, Sie bewerten die Auswirkungen eines Stipendienprogramms auf die akademischen Leistungen der Schüler. Studierende, die knapp über oder unter einem bestimmten GPA-Schwellenwert liegen, werden dem Programm unterschiedlich zugewiesen. Dieses Design hilft bei der Bewertung der Wirksamkeit des Programms am Grenzwert.

Propensity Score Matching

Propensity Score Matching ist eine Technik, mit der in nicht-randomisierten Studien vergleichbare Behandlungs- und Kontrollgruppen gebildet werden. Die Forscher berechnen Propensity Scores auf der Grundlage der Merkmale der Teilnehmer und gleichen die Personen in der Behandlungsgruppe mit denen in der Kontrollgruppe ab, die ähnliche Werte aufweisen.

Beispiel : Wenn Sie die Auswirkungen eines neuen Medikaments auf die Ergebnisse von Patienten untersuchen, würden Sie Propensity Scores verwenden, um Patienten, die das Medikament erhalten haben, mit denen abzugleichen, die es nicht erhalten haben, aber ähnliche Gesundheitsprofile aufweisen.

Unterbrochenes Zeitreihendesign

Beim unterbrochenen Zeitreihendesign werden Daten zu mehreren Zeitpunkten vor und nach der Einführung einer Intervention erhoben. Bei diesem Design findet die Intervention jedoch zu einem bestimmten Zeitpunkt statt, so dass die Forscher ihre unmittelbare Wirkung bewerten können.

Beispiel : Angenommen, Sie analysieren die Auswirkungen eines neuen Verkehrsmanagementsystems auf die Verkehrsunfälle. Sie erheben Unfalldaten vor und nach der Einführung des Systems, um etwaige abrupte Veränderungen direkt nach der Einführung zu beobachten.

Jedes dieser quasi-experimentellen Designs bietet einzigartige Vorteile und ist für bestimmte Forschungsfragen und Szenarien am besten geeignet. Die Wahl des richtigen Designs ist entscheidend für die Durchführung robuster und informativer Studien.

Vor- und Nachteile des quasi-experimentellen Designs

Das quasi-experimentelle Design bietet einen wertvollen Forschungsansatz, aber wie jede Methode hat auch diese ihre eigenen Vor- und Nachteile. Lassen Sie uns diese im Detail untersuchen.

Vorteile des quasi-experimentellen Designs

Das quasi-experimentelle Design bietet mehrere Vorteile, die es zu einem wertvollen Instrument in der Forschung machen:

• Anwendbarkeit in der realen Welt: Quasi-experimentelle Studien finden oft in realen Umgebungen statt, wodurch die Ergebnisse besser auf praktische Situationen anwendbar sind. Forscher können die Auswirkungen von Interventionen oder Variablen in dem Kontext untersuchen, in dem sie natürlicherweise auftreten.
• Ethische Erwägungen: In Situationen, in denen es unethisch wäre, die unabhängige Variable in einem kontrollierten Experiment zu manipulieren, bietet das quasi-experimentelle Design eine ethische Alternative. So wäre es beispielsweise unethisch, Teilnehmer für eine Studie über die gesundheitlichen Auswirkungen des Rauchens dem Rauchen zuzuweisen, aber Sie können natürlich vorkommende Gruppen von Rauchern und Nichtrauchern untersuchen.
• Kosteneffizienz: Die Durchführung quasi-experimenteller Forschung ist oft kostengünstiger als die Durchführung kontrollierter Experimente. Der Verzicht auf kontrollierte Umgebungen und umfangreiche Manipulationen kann sowohl Zeit als auch Ressourcen einsparen.

Diese Vorteile machen das quasi-experimentelle Design zu einer attraktiven Wahl für Forscher, die bei ihren Studien mit praktischen oder ethischen Beschränkungen konfrontiert sind.

Nachteile des quasi-experimentellen Designs

Das quasi-experimentelle Design birgt jedoch auch einige Herausforderungen und Nachteile in sich:

• Begrenzte Kontrolle: Im Gegensatz zu kontrollierten Experimenten, bei denen die Forscher die volle Kontrolle über die Variablen haben, fehlt bei quasi-experimentellen Versuchsplänen das gleiche Maß an Kontrolle. Diese eingeschränkte Kontrolle kann zu Störvariablen führen, die es schwierig machen, Kausalität nachzuweisen.
• Gefahren für die interne Validität: Verschiedene Gefahren für die interne Validität, wie z. B. Selektionsverzerrungen, historische Effekte und Reifungseffekte, können die Genauigkeit der kausalen Schlussfolgerungen beeinträchtigen. Die Forscher müssen diese Gefahren sorgfältig berücksichtigen, um die Gültigkeit ihrer Ergebnisse zu gewährleisten.
• Herausforderungen beim Kausalitätsschluss: Der Nachweis der Kausalität kann bei quasi-experimentellen Designs aufgrund des Fehlens von Randomisierung und Kontrolle schwierig sein. Sie können zwar überzeugende Argumente für die Kausalität vorbringen, aber sie sind möglicherweise nicht so schlüssig wie bei kontrollierten Experimenten.
• Potenzielle Störvariablen: Bei einem quasi-experimentellen Design ist es oft schwierig, alle möglichen Störvariablen zu kontrollieren, die die abhängige Variable beeinflussen können. Dies kann dazu führen, dass Veränderungen nicht ausschließlich auf die unabhängige Variable zurückgeführt werden können.

Trotz dieser Nachteile bleibt das quasi-experimentelle Design ein wertvolles Forschungsinstrument, wenn es mit Bedacht und im Bewusstsein seiner Grenzen eingesetzt wird. Forscher sollten ihre Forschungsfragen und die praktischen Einschränkungen, mit denen sie konfrontiert sind, sorgfältig abwägen, bevor sie diesen Ansatz wählen.

Wie führt man eine quasi-experimentelle Studie durch?

Die Durchführung einer quasi-experimentellen Studie erfordert eine sorgfältige Planung und Durchführung, um die Gültigkeit Ihrer Forschung zu gewährleisten. Im Folgenden werden die wesentlichen Schritte erläutert, die Sie bei der Durchführung einer solchen Studie beachten müssen.

1. Definieren Sie Forschungsfragen und Zielsetzungen

Der erste Schritt eines jeden Forschungsvorhabens besteht darin, Ihre Forschungsfragen und -ziele klar zu definieren. Dazu müssen Sie die unabhängige Variable (IV) und die abhängige Variable (DV) bestimmen, die Sie untersuchen wollen. Welche Beziehung wollen Sie erforschen, und was wollen Sie mit Ihrer Forschung erreichen?

• Legen Sie Ihre Forschungsfragen fest: Formulieren Sie zunächst präzise Forschungsfragen, die Ihre Studie beantworten soll. Diese Fragen sollten klar, fokussiert und für Ihr Studiengebiet relevant sein.
• Identifizieren Sie die unabhängige Variable: Definieren Sie die Variable, die Sie in Ihrer Studie manipulieren oder untersuchen wollen. Bestimmen
• Sie die abhängige Variable: Bestimmen Sie die Ergebnis- oder Reaktionsvariable, die durch Änderungen der unabhängigen Variable beeinflusst wird.
• Stellen Sie Hypothesen auf (falls zutreffend): Wenn Sie spezifische Hypothesen über die Beziehung zwischen IV und DV haben, geben Sie diese klar an. Hypothesen bieten einen Rahmen für die Prüfung Ihrer Forschungsfragen.

2. Wählen Sie ein geeignetes quasi-experimentelles Design

Die Wahl des richtigen quasi-experimentellen Designs ist entscheidend für das Erreichen Ihrer Forschungsziele. Wählen Sie ein Design, das auf Ihre Forschungsfragen und die verfügbaren Daten abgestimmt ist. Berücksichtigen Sie Faktoren wie die Durchführbarkeit des Designs und die damit verbundenen ethischen Überlegungen.

• Bewerten Sie Ihre Forschungsziele: Bewerten Sie Ihre Forschungsfragen und -ziele, um zu bestimmen, welche Art von quasi-experimentellem Design am besten geeignet ist. Jedes Design hat seine Stärken und Grenzen, wählen Sie also eines, das Ihren Zielen entspricht.
• Berücksichtigen Sie ethische Einschränkungen: Berücksichtigen Sie alle ethischen Bedenken in Bezug auf Ihre Forschung. Je nach dem Kontext Ihrer Studie können einige Designs ethisch besser vertretbar sein als andere.
• Beurteilen Sie die Datenverfügbarkeit: Stellen Sie sicher, dass Sie Zugang zu den erforderlichen Daten für das von Ihnen gewählte Design haben. Einige Designs erfordern umfangreiche historische Daten, während andere auf Daten angewiesen sind, die während der Studie erhoben werden.

3. Identifizieren und rekrutieren Sie Teilnehmer

Die Auswahl der richtigen Teilnehmer ist ein entscheidender Aspekt der quasi-experimentellen Forschung. Die Teilnehmer sollten die Population repräsentieren, über die Sie Rückschlüsse ziehen wollen, und Sie müssen ethische Erwägungen berücksichtigen, einschließlich der informierten Zustimmung.

• Definieren Sie Ihre Zielpopulation: Bestimmen Sie die Population, auf die Ihre Studie verallgemeinert werden soll. Ihre Stichprobe sollte repräsentativ für diese Population sein.
• Rekrutierungsprozess: Entwickeln Sie einen Plan für die Rekrutierung von Teilnehmern. Je nach Design müssen Sie möglicherweise bestimmte Gruppen oder Institutionen ansprechen.
• Informierte Zustimmung: Stellen Sie sicher, dass Sie die informierte Zustimmung der Teilnehmer einholen. Erläutern Sie klar und deutlich die Art der Studie, mögliche Risiken und ihre Rechte als Teilnehmer.

4. Daten sammeln

Die Datenerhebung ist ein entscheidender Schritt in der quasi-experimentellen Forschung. Sie müssen einen konsistenten und systematischen Prozess einhalten, um relevante Informationen vor und nach der Intervention oder Behandlung zu sammeln.

• Pre-Test-Maßnahmen: Erheben Sie gegebenenfalls Daten, bevor Sie die unabhängige Variable einführen. Stellen Sie sicher, dass die Messungen vor dem Test standardisiert und zuverlässig sind.
• Maßnahmen nach dem Test: Erheben Sie nach der Intervention Daten nach dem Test, indem Sie die gleichen Maßnahmen wie vor dem Test verwenden. So können Sie Veränderungen im Laufe der Zeit beurteilen.
• Aufrechterhaltung der Datenkonsistenz: Stellen Sie sicher, dass die Datenerhebungsverfahren für alle Teilnehmer und Zeitpunkte konsistent sind, um Verzerrungen zu minimieren.

5. Daten auswerten

Sobald Sie Ihre Daten gesammelt haben, ist es an der Zeit, sie mit geeigneten statistischen Verfahren zu analysieren. Die Wahl der Analyse hängt von Ihren Forschungsfragen und der Art der gesammelten Daten ab.

• Statistische Analyse: Verwenden Sie Statistiksoftware, um Ihre Daten zu analysieren. Zu den gebräuchlichen Techniken gehören t-Tests , Varianzanalyse (ANOVA) , Regressionsanalyse und weitere, je nach Design und Variablen.
• Kontrolle von Störvariablen: Achten Sie auf mögliche Störvariablen und beziehen Sie sie als Kovariaten in Ihre Analyse ein, um genaue Ergebnisse zu gewährleisten.

Chi-Quadrat-Rechner :

t-Test-Rechner :

6. Ergebnisse interpretieren

Wenn die Analyse abgeschlossen ist, können Sie die Ergebnisse interpretieren, um aussagekräftige Schlussfolgerungen über die Beziehung zwischen den unabhängigen und abhängigen Variablen zu ziehen.

• Untersuchen Sie die Effektgrößen: Beurteilen Sie die Größenordnung der beobachteten Effekte, um ihre praktische Bedeutung zu bestimmen.
• Berücksichtigen Sie Signifikanzniveaus: Bestimmen Sie, ob die beobachteten Ergebnisse statistisch signifikant sind. Verstehen Sie die p-Werte und ihre Auswirkungen.
• Vergleichen Sie die Ergebnisse mit den Hypothesen: Bewerten Sie, ob Ihre Ergebnisse Ihre Hypothesen und Forschungsfragen unterstützen oder ablehnen.

7. Ziehen Sie Schlussfolgerungen

Ziehen Sie auf der Grundlage Ihrer Analyse und Interpretation der Ergebnisse Schlussfolgerungen zu den Forschungsfragen und -zielen, die Sie sich gesetzt haben.

• Kausale Schlussfolgerungen: Erläutern Sie, inwieweit Ihre Studie kausale Schlussfolgerungen zulässt. Machen Sie die Einschränkungen und möglichen alternativen Erklärungen für Ihre Ergebnisse transparent.
• Implikationen und Anwendungen: Berücksichtigen Sie die praktischen Implikationen Ihrer Forschung. Wie tragen Ihre Ergebnisse zum bestehenden Wissen bei und wie können sie in realen Kontexten angewendet werden?
• Zukünftige Forschung: Nennen Sie Bereiche für zukünftige Forschung und mögliche Verbesserungen im Studiendesign. Weisen Sie auf Einschränkungen oder Zwänge hin, die die Ergebnisse Ihrer Studie beeinflusst haben könnten.

Wenn Sie diese Schritte sorgfältig befolgen, können Sie eine strenge und informative quasi-experimentelle Studie durchführen, die das Wissen in Ihrem Forschungsbereich erweitert.

Beispiele für quasi-experimentelles Design

Quasi-experimentelles Design findet in einer Vielzahl von Forschungsbereichen Anwendung, einschließlich unternehmensbezogener und Marktforschungsszenarien. Im Folgenden finden Sie einige detaillierte Beispiele für die Anwendung dieser Forschungsmethode in der Praxis:

Beispiel 1: Bewertung der Auswirkungen einer neuen Marketingstrategie

Angenommen, ein Unternehmen möchte die Wirksamkeit einer neuen Marketingstrategie zur Steigerung des Absatzes bewerten. Die Durchführung eines kontrollierten Experiments ist aufgrund des bestehenden Kundenstamms des Unternehmens und der Schwierigkeit, die Kunden nach dem Zufallsprinzip verschiedenen Marketingansätzen zuzuordnen, möglicherweise nicht durchführbar. In diesem Fall kann ein quasi-experimentelles Design verwendet werden.

• Unabhängige Variable: Die neue Marketingstrategie.
• Abhängige Variable: Umsatzerlöse.
• Design: Das Unternehmen könnte die neue Strategie für eine Gruppe von Kunden umsetzen, während die bestehende Strategie für eine andere Gruppe beibehalten wird. Beide Gruppen werden auf der Grundlage ähnlicher demografischer Merkmale und Kaufhistorie ausgewählt, um eine Verzerrung der Auswahl zu vermeiden. Daten aus der Zeit vor der Umsetzung (Verkaufszahlen) können als Basis dienen, und nach der Umsetzung können Daten gesammelt werden, um die Auswirkungen der Strategie zu bewerten.

Beispiel 2: Bewertung der Effektivität von Mitarbeiterschulungsprogrammen

Im Zusammenhang mit der Personal- und Mitarbeiterentwicklung versuchen Unternehmen häufig, die Auswirkungen von Schulungsprogrammen zu bewerten. Eine randomisierte kontrollierte Studie (RCT) mit zufälliger Zuweisung ist möglicherweise nicht praktikabel oder ethisch vertretbar, da einige Mitarbeiter eine bestimmte Schulung mehr benötigen als andere. Stattdessen kann ein quasi-experimentelles Design verwendet werden.

• Unabhängige Variable: Schulungsprogramme für Mitarbeiter.
• Abhängige Variable: Leistungskennzahlen der Mitarbeiter, z. B. Produktivität oder Qualität der Arbeit.
• Design: Das Unternehmen kann Schulungsprogramme für Mitarbeiter anbieten, die Interesse bekunden oder einen bestimmten Bedarf aufweisen, und so eine selbst gewählte Behandlungsgruppe bilden. Eine vergleichbare Kontrollgruppe kann aus Mitarbeitern mit ähnlichen Aufgaben und Qualifikationen bestehen, die nicht an der Schulung teilgenommen haben. Die Leistungskennzahlen vor der Schulung können als Basis dienen, und nach der Schulung können Daten gesammelt werden, um die Auswirkungen der Schulungsprogramme zu bewerten.

Beispiel 3: Analyse der Auswirkungen einer Änderung der Steuerpolitik

In den Bereichen Wirtschaft und öffentliche Ordnung untersuchen Forscher häufig die Auswirkungen von Änderungen der Steuerpolitik auf das wirtschaftliche Verhalten. Die Durchführung eines kontrollierten Experiments ist in solchen Fällen praktisch unmöglich. Daher wird in der Regel ein quasi-experimentelles Design verwendet.

• Unabhängige Variable: Steuerpolitische Änderungen (z. B. Steuersatzanpassungen).
• Abhängige Variable: Wirtschaftsindikatoren, wie Verbraucherausgaben oder Unternehmensinvestitionen.
• Design: Die Forscher können Daten aus verschiedenen Regionen oder Gerichtsbarkeiten analysieren, in denen steuerpolitische Änderungen durchgeführt wurden. Eine Region könnte die Behandlungsgruppe (mit steuerpolitischen Änderungen) darstellen, während eine ähnliche Region ohne steuerpolitische Änderungen als Kontrollgruppe dient. Durch den Vergleich der Wirtschaftsdaten vor und nach der Änderung der Politik in beiden Gruppen können die Forscher die Auswirkungen der Änderungen der Steuerpolitik bewerten.

Diese Beispiele veranschaulichen, wie das quasi-experimentelle Design in verschiedenen Forschungskontexten angewandt werden kann und wertvolle Erkenntnisse über die Auswirkungen unabhängiger Variablen in realen Szenarien liefert, in denen kontrollierte Experimente nicht durchführbar oder ethisch vertretbar sind. Durch die sorgfältige Auswahl von Vergleichsgruppen und die Kontrolle potenzieller Verzerrungen können Forscher aussagekräftige Schlussfolgerungen ziehen und Entscheidungsprozesse beeinflussen.

Wie veröffentlicht man quasi-experimentelle Forschung?

Die Veröffentlichung Ihrer quasi-experimentellen Forschungsergebnisse ist ein entscheidender Schritt, um zum Wissen der wissenschaftlichen Gemeinschaft beizutragen. Wir werden die wesentlichen Aspekte einer effektiven Berichterstattung und Veröffentlichung Ihrer quasi-experimentellen Forschung untersuchen.

Strukturierung Ihrer Forschungsarbeit

Bei der Vorbereitung Ihrer Forschungsarbeit ist es wichtig, ein gut strukturiertes Format einzuhalten, um Klarheit und Verständlichkeit zu gewährleisten. Hier sind die wichtigsten Elemente, die Sie berücksichtigen sollten:

Titel und Zusammenfassung

• Titel: Verfassen Sie einen prägnanten und informativen Titel, der das Wesentliche Ihrer Studie widerspiegelt. Er sollte die wichtigste Forschungsfrage oder Hypothese enthalten.
• Zusammenfassung: Fassen Sie Ihre Forschungsarbeit in einer strukturierten Zusammenfassung zusammen, die den Zweck, die Methoden, die Ergebnisse und die Schlussfolgerungen enthält. Achten Sie darauf, dass sie einen klaren Überblick über Ihre Studie gibt.
• Hintergrund und Begründung: Stellen Sie den Kontext Ihrer Studie dar, indem Sie die Forschungslücke oder das Problem, das Ihre Studie behandelt, erläutern. Erläutern Sie, warum Ihre Forschung relevant und wichtig ist.
• Forschungsfragen oder Hypothesen: Geben Sie Ihre Forschungsfragen oder Hypothesen und deren Bedeutung klar an.

Literaturübersicht

• Übersicht über verwandte Arbeiten: Erörtern Sie relevante Literatur, die Ihre Forschung unterstützt. Heben Sie Studien mit ähnlichen Methoden oder Ergebnissen hervor und erklären Sie, wie Ihre Forschung in diesen Kontext passt.
• Teilnehmer: Beschreiben Sie die Teilnehmer Ihrer Studie, einschließlich ihrer Merkmale und wie Sie sie rekrutiert haben.
• Quasi-experimentelles Design: Erläutern Sie das von Ihnen gewählte Design im Detail, einschließlich der unabhängigen und abhängigen Variablen, der Verfahren und der getroffenen Kontrollmaßnahmen.
• Datenerhebung: Erläutern Sie die Methoden der Datenerhebung , die verwendeten Instrumente und alle Maßnahmen vor und nach dem Test.
• Datenanalyse: Beschreiben Sie die verwendeten statistischen Verfahren, einschließlich der Kontrolle von Störvariablen.
• Darstellung der Ergebnisse: Stellen Sie Ihre Ergebnisse übersichtlich dar, gegebenenfalls unter Verwendung von Tabellen, Diagrammen und deskriptiven Statistiken. Geben Sie ggf. p-Werte und Effektgrößen an.
• Interpretation der Ergebnisse: Erläutern Sie die Implikationen Ihrer Ergebnisse und wie sie sich auf Ihre Forschungsfragen oder Hypothesen beziehen.
• Interpretation und Implikationen: Analysieren Sie Ihre Ergebnisse im Kontext der vorhandenen Literatur und Theorien. Erörtern Sie die praktischen Implikationen Ihrer Ergebnisse.
• Beschränkungen: Gehen Sie auf die Beschränkungen Ihrer Studie ein, einschließlich möglicher Verzerrungen oder Gefährdungen der internen Validität.
• Künftige Forschung: Schlagen Sie Bereiche für künftige Forschung vor und erläutern Sie, wie Ihre Studie zum Fachgebiet beiträgt.

Ethische Erwägungen bei der Berichterstattung

Ethische Berichterstattung ist bei quasi-experimenteller Forschung von größter Bedeutung. Stellen Sie sicher, dass Sie die ethischen Standards einhalten, einschließlich:

• Informierte Zustimmung: Geben Sie klar an, dass die informierte Zustimmung aller Teilnehmer eingeholt wurde, und beschreiben Sie den Prozess der informierten Zustimmung.
• Schutz der Teilnehmer: Erläutern Sie, wie Sie die Rechte und das Wohlergehen Ihrer Teilnehmer während der gesamten Studie geschützt haben.
• Vertraulichkeit: Erläutern Sie, wie Sie die Privatsphäre und Anonymität gewahrt haben, insbesondere bei der Präsentation einzelner Daten.
• Offenlegung von Interessenkonflikten: Erklären Sie alle potenziellen Interessenkonflikte, die die Interpretation Ihrer Ergebnisse beeinflussen könnten.

Häufig zu vermeidende Fallstricke

Achten Sie bei der Berichterstattung über Ihre quasi-experimentelle Forschung auf häufige Fallstricke, die die Qualität und Wirkung Ihrer Arbeit beeinträchtigen können:

• Übergeneralisierung: Achten Sie darauf, dass Sie Ihre Ergebnisse nicht übergeneralisieren. Geben Sie klar die Grenzen Ihrer Studie und die Populationen an, auf die Ihre Ergebnisse angewendet werden können.
• Fehlinterpretation der Kausalität: Legen Sie klar die Grenzen des Kausalitätsschlusses in der quasi-experimentellen Forschung dar. Vermeiden Sie starke kausale Behauptungen, wenn sie nicht durch solide Beweise gestützt werden.
• Ignorieren ethischer Bedenken: Ethische Überlegungen sind von größter Bedeutung. Wenn Sie nicht über die informierte Zustimmung, die ethische Aufsicht und den Schutz der Teilnehmer berichten, kann dies die Glaubwürdigkeit Ihrer Studie untergraben.

Richtlinien für eine transparente Berichterstattung

Um die Transparenz und Reproduzierbarkeit Ihrer quasi-experimentellen Forschung zu verbessern, sollten Sie sich an etablierte Berichterstattungsrichtlinien halten, z. B:

• CONSORT-Erklärung: Wenn Ihre Studie Interventionen oder Behandlungen beinhaltet, befolgen Sie die CONSORT-Richtlinien für die transparente Berichterstattung über randomisierte kontrollierte Studien.
• STROBE-Erklärung: Für Beobachtungsstudien bietet die STROBE-Erklärung eine Anleitung für die Berichterstattung über wesentliche Elemente.
• PRISMA-Erklärung: Wenn Ihre Forschung systematische Übersichten oder Meta-Analysen umfasst, halten Sie sich an die PRISMA-Richtlinien.
• Transparent Reporting of Evaluations with Non-Randomized Designs (TREND): Die TREND-Richtlinien bieten spezifische Empfehlungen für die transparente Berichterstattung über nicht-randomisierte Designs, einschließlich quasi-experimenteller Forschung.

Indem Sie diese Berichterstattungsrichtlinien befolgen und die höchsten ethischen Standards einhalten, können Sie zum Wissenszuwachs in Ihrem Bereich beitragen und die Glaubwürdigkeit und Wirkung Ihrer quasi-experimentellen Forschungsergebnisse sicherstellen.

Quasi-Experimentelles Design - Herausforderungen

Die Durchführung einer quasi-experimentellen Studie kann mit Herausforderungen verbunden sein, die sich auf die Gültigkeit und Zuverlässigkeit der Ergebnisse auswirken können. Wir werfen einen Blick auf einige häufige Herausforderungen und stellen Strategien vor, wie Sie diese effektiv angehen können.

Auswahlverzerrung

Herausforderung: Selektionsverzerrungen treten auf, wenn sich nicht-randomisierte Gruppen systematisch in einer Weise unterscheiden, die das Ergebnis der Studie beeinflusst. Diese Verzerrung kann die Validität Ihrer Forschung untergraben, da sie impliziert, dass die Gruppen zu Beginn der Studie nicht gleichwertig sind.

Umgang mit Selektionsverzerrungen:

• Matching: Verwenden Sie Matching-Techniken, um vergleichbare Behandlungs- und Kontrollgruppen zu bilden. Passen Sie die Teilnehmer anhand relevanter Merkmale wie Alter, Geschlecht oder frühere Leistungen an, um ein Gleichgewicht zwischen den Gruppen herzustellen.
• Statistische Kontrollen: Verwenden Sie statistische Kontrollen, um die Unterschiede zwischen den Gruppen zu berücksichtigen. Beziehen Sie Kovariaten in Ihre Analyse ein, um mögliche Verzerrungen auszugleichen.
• Sensitivitätsanalyse: Führen Sie Sensitivitätsanalysen durch, um zu beurteilen, wie anfällig Ihre Ergebnisse für Selektionsverzerrungen sind. Untersuchen Sie verschiedene Szenarien, um die Auswirkungen einer möglichen Verzerrung auf Ihre Schlussfolgerungen zu verstehen.

Historische Effekte

Herausforderung: Historische Effekte beziehen sich auf externe Ereignisse oder Veränderungen im Laufe der Zeit, die die Ergebnisse der Studie beeinflussen. Diese externen Faktoren können Ihre Forschung vereiteln, indem sie Variablen einführen, die Sie nicht berücksichtigt haben.

Umgang mit historischen Effekten:

• Sammeln Sie historische Daten: Sammeln Sie umfangreiche historische Daten, um Trends und Muster zu verstehen, die Ihre Studie beeinflussen könnten. Wenn Sie über einen umfassenden historischen Kontext verfügen, können Sie historische Effekte besser erkennen und berücksichtigen.
• Kontrollgruppen: Beziehen Sie, wann immer möglich, Kontrollgruppen ein. Indem Sie die Ergebnisse der Behandlungsgruppe mit denen einer Kontrollgruppe vergleichen, können Sie externe Einflüsse berücksichtigen, die beide Gruppen gleichermaßen betreffen.
• Zeitreihenanalyse: Verwenden Sie gegebenenfalls eine Zeitreihenanalyse, um zeitliche Trends zu erkennen und zu berücksichtigen. Diese Methode hilft bei der Unterscheidung zwischen den Auswirkungen der unabhängigen Variable und externen Ereignissen.

Reifungseffekte

Herausforderung: Reifungseffekte treten auf, wenn sich die Teilnehmer während der Studie unabhängig von der Intervention natürlich verändern oder entwickeln. Diese Veränderungen können Ihre Ergebnisse verfälschen, so dass es schwierig ist, die beobachteten Effekte ausschließlich der unabhängigen Variable zuzuschreiben.

Umgang mit Reifungseffekten:

• Randomisierung: Verwenden Sie nach Möglichkeit eine Randomisierung, um die Reifungseffekte gleichmäßig auf die Behandlungs- und Kontrollgruppen zu verteilen. Die zufällige Zuweisung minimiert die Auswirkungen der Reifung als Störvariable.
• Abgestimmte Paare: Wenn eine Randomisierung nicht möglich ist, verwenden Sie abgestimmte Paare oder statistische Kontrollen, um sicherzustellen, dass beide Gruppen ähnliche Reifungseffekte erfahren.
• Kürzere Zeiträume: Begrenzen Sie die Dauer Ihrer Studie, um die Wahrscheinlichkeit signifikanter Reifungseffekte zu verringern. Kürzere Studien sind weniger anfällig für langfristige Reifungseffekte.

Regression zum Mittelwert

Herausforderung: Unter Regression zum Mittelwert versteht man die Tendenz, dass sich die Extremwerte einer Variablen bei einer erneuten Prüfung dem Mittelwert annähern. Dies kann den Eindruck erwecken, dass eine Intervention wirksam ist, obwohl es sich in Wirklichkeit um ein natürliches statistisches Phänomen handelt.

Umgang mit der Regression zum Mittelwert:

• Verwenden Sie Kontrollgruppen: Nehmen Sie Kontrollgruppen in Ihre Studie auf, um eine Ausgangsbasis für den Vergleich zu schaffen. Dies hilft, echte Interventionseffekte von einer Regression auf den Mittelwert zu unterscheiden.
• Mehrere Datenpunkte: Sammeln Sie zahlreiche Datenpunkte, um Muster und Trends zu erkennen. Wenn sich extreme Werte bei nachfolgenden Messungen auf den Mittelwert zurückbilden, kann dies eher auf eine Regression auf den Mittelwert als auf einen echten Interventionseffekt hindeuten.
• Statistische Analyse: Wenden Sie bei der Analyse Ihrer Daten statistische Verfahren an, die die Regression zum Mittelwert berücksichtigen. Techniken wie die Analyse der Kovarianz (ANCOVA) können dabei helfen, Unterschiede in der Ausgangslage zu kontrollieren.

Ausfälle und Sterblichkeit

Herausforderung: Unter Abwanderung versteht man den Verlust von Teilnehmern im Verlauf Ihrer Studie, während die Mortalität den dauerhaften Verlust von Teilnehmern bezeichnet. Hohe Schwundraten können zu Verzerrungen führen und die Repräsentativität Ihrer Stichprobe beeinträchtigen.

Umgang mit Abwanderung und Sterblichkeit:

• Sorgfältige Auswahl der Teilnehmer: Wählen Sie Teilnehmer aus, die wahrscheinlich während der gesamten Dauer der Studie engagiert bleiben. Berücksichtigen Sie Faktoren, die zur Abwanderung führen können, wie z. B. die Motivation und das Engagement der Teilnehmer.
• Anreize: Bieten Sie den Teilnehmern Anreize oder Entschädigungen, um sie zur weiteren Teilnahme zu bewegen.
• Follow-up-Strategien: Setzen Sie wirksame Follow-up-Strategien ein, um die Abwanderung zu verringern. Regelmäßige Kommunikation und Erinnerungen können helfen, die Teilnehmer bei der Stange zu halten.
• Sensitivitätsanalyse: Führen Sie Sensitivitätsanalysen durch, um die Auswirkungen von Abbruch und Sterblichkeit auf Ihre Ergebnisse zu bewerten. Vergleichen Sie die Merkmale der Teilnehmer, die die Studie abgebrochen haben, mit denen derjenigen, die die Studie abgeschlossen haben.

Effekte testen

Herausforderung: Testeffekte treten auf, wenn der bloße Akt des Testens oder Beurteilens von Teilnehmern deren spätere Leistung beeinflusst. Dieses Phänomen kann zu Veränderungen in der abhängigen Variable führen, die nicht mit der unabhängigen Variable zusammenhängen.

Umgang mit Testeffekten:

• Ausgewogene Tests: Wenn möglich, sollten Sie die Reihenfolge der Tests oder Beurteilungen zwischen Behandlungs- und Kontrollgruppe ausgleichen. Dies trägt dazu bei, die Testeffekte gleichmäßig auf die Gruppen zu verteilen.
• Kontrollgruppen: Führen Sie Kontrollgruppen ein, die denselben Test- oder Beurteilungsverfahren unterzogen werden wie die Behandlungsgruppe. Durch den Vergleich der beiden Gruppen können Sie feststellen, ob Testeffekte die Ergebnisse beeinflusst haben.
• Minimierung der Testhäufigkeit: Begrenzen Sie die Häufigkeit der Tests oder Beurteilungen, um die Wahrscheinlichkeit von Testeffekten zu verringern. Die Durchführung von weniger Beurteilungen kann die Auswirkungen wiederholter Tests auf die Teilnehmer abschwächen.

Wenn Sie diese allgemeinen Herausforderungen proaktiv angehen, können Sie die Validität und Zuverlässigkeit Ihrer quasi-experimentellen Studie verbessern und Ihre Ergebnisse robuster und vertrauenswürdiger machen.

Fazit zum Quasi-Experimentellen Aufbau

Das quasi-experimentelle Design ist ein leistungsfähiges Instrument, das Forschern hilft, Ursache-Wirkungs-Beziehungen in realen Situationen zu untersuchen , in denen eine strenge Kontrolle nicht immer möglich ist. Wenn Sie die Schlüsselkonzepte und die Arten von Versuchsplänen verstehen und wissen, wie Sie mit den Herausforderungen umgehen, können Sie solide Forschungsarbeiten durchführen und wertvolle Erkenntnisse zu Ihrem Fachgebiet beitragen. Denken Sie daran, dass das quasi-experimentelle Design die Lücke zwischen kontrollierten Experimenten und reinen Beobachtungsstudien schließt, was es zu einem unverzichtbaren Ansatz in verschiedenen Bereichen macht, von der Wirtschafts- und Marktforschung bis hin zur öffentlichen Politik und darüber hinaus. Ganz gleich, ob Sie Forscher, Student oder Entscheidungsträger sind, das Wissen über quasi-experimentelles Design befähigt Sie dazu, fundierte Entscheidungen zu treffen und positive Veränderungen in der Welt voranzutreiben.

Wie kann man quasi-experimentelles Design mit Echtzeit-Insights aufladen?

Wir stellen Ihnen Appinio vor, die Echtzeit-Marktforschungsplattform, die die Welt des quasi-experimentellen Designs verändert. Stellen Sie sich vor, Sie könnten Ihre eigene Marktforschung in wenigen Minuten durchführen und so verwertbare Erkenntnisse gewinnen, die Ihre datengestützten Entscheidungen unterstützen. Appinio kümmert sich um die Forschung und die technische Komplexität, damit Sie sich auf das konzentrieren können, was für Ihr Unternehmen wirklich wichtig ist.

Das sind die Gründe, warum Appinio so einzigartig ist:

• Blitzschnelle Erkenntnisse: Von der Formulierung der Fragen bis zur Aufdeckung der Erkenntnisse liefert Appinio Ergebnisse innerhalb von Minuten und stellt sicher, dass Sie die Antworten erhalten, die Sie brauchen, wenn Sie sie brauchen.
• Kein Forschungsdiplom erforderlich: Unsere intuitive Plattform ist für jedermann geeignet, so dass kein Forschungsdiplom erforderlich ist. Jeder kann sofort einsteigen und die Vorteile von Echtzeit-Konsumentenanalysen nutzen.
• Globale Reichweite, lokales Know-how: Mit Zugang zu über 90 Ländern und der Möglichkeit, präzise Zielgruppen auf der Grundlage von mehr als 1200 Merkmalen zu definieren, können Sie quasi-experimentelle Forschung auf globaler Ebene durchführen und gleichzeitig den lokalen Bezug wahren.

Kostenlosen Zugang zur Appinio Platform erhalten!

Direkt ins Postfach! 📫

Jetzt anmelden und regelmäßig Updates zu den neuesten Reports und/oder Produktneuheiten erhalten.

News und Updates zum Thema Marktforschung - Direkt in's Postfach! 💌

Weitere interessante Artikel

11.09.2024 | 36min Lesezeit

11.09.2024 | 44min Lesezeit

Was ist A/B-Testing? Leitfaden, Tools, Beispiele

10.09.2024 | 33min Lesezeit

Kausalforschung: Definition, Aufbau, Tipps, Beispiele

This week: the arXiv Accessibility Forum

Help | Advanced Search

Computer Science > Robotics

Title: design of a variable stiffness quasi-direct drive cable-actuated tensegrity robot.

Abstract: Tensegrity robots excel in tasks requiring extreme levels of deformability and robustness. However, there are challenges in state estimation and payload versatility due to their high number of degrees of freedom and unconventional shape. This paper introduces a modular three-bar tensegrity robot featuring a customizable payload design. Our tensegrity robot employs a novel Quasi-Direct Drive (QDD) cable actuator paired with low-stretch polymer cables to achieve accurate proprioception without the need for external force or torque sensors. The design allows for on-the-fly stiffness tuning for better environment and payload adaptability. In this paper, we present the design, fabrication, assembly, and experimental results of the robot. Experimental data demonstrates the high accuracy cable length estimation (<1% error relative to bar length) and variable stiffness control of the cable actuator up to 7 times the minimum stiffness for self support. The presented tensegrity robot serves as a platform for future advancements in autonomous operation and open-source module design.

Submission history

Access paper:.

• HTML (experimental)
• Other Formats

References & Citations

• Google Scholar
• Semantic Scholar

BibTeX formatted citation

Bibliographic and Citation Tools

Code, data and media associated with this article, recommenders and search tools.

• Institution

arXivLabs: experimental projects with community collaborators

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs .