experiment meaning in statistics

• Encyclopedia ›

Definition Experiment

Experiments investigate and attempt to demonstrate the cause and effect relationship between two variables. An example can be seen in the test phase of pharmaceutical drugs , i.e., whether drug X effectively combats disease Y.

In experiments, the subjects are usually divided into two groups ‒ one control and one experimental group. The experimental group actually receives the drug while the control group only proceeds with the standard treatment. A distinction is made between laboratory (controlled environment) and field experiments (in natural settings). Experiments must satisfy the scientific quality criteria of objectivity , reliability  , and validity .

Please note that the definitions in our statistics encyclopedia are simplified explanations of terms. Our goal is to make the definitions accessible for a broad audience; thus it is possible that some definitions do not adhere entirely to scientific standards.

• Extreme value
• Extrapolation
• Exogenous variable
• Endogenous variable
• Empirical probability
• Elementary event
• Ecological fallacy

• History & Society
• Science & Tech
• Biographies
• Animals & Nature
• Geography & Travel
• Arts & Culture
• Games & Quizzes
• On This Day
• One Good Fact
• New Articles
• Lifestyles & Social Issues
• Philosophy & Religion
• Politics, Law & Government
• World History
• Health & Medicine
• Browse Biographies
• Birds, Reptiles & Other Vertebrates
• Bugs, Mollusks & Other Invertebrates
• Environment
• Fossils & Geologic Time
• Entertainment & Pop Culture
• Sports & Recreation
• Visual Arts
• Demystified
• Image Galleries
• Infographics
• Top Questions
• Britannica Kids
• Saving Earth
• Space Next 50
• Student Center
• Introduction
• Tabular methods
• Graphical methods
• Exploratory data analysis
• Events and their probabilities
• Random variables and probability distributions
• The binomial distribution
• The Poisson distribution
• The normal distribution
• Sampling and sampling distributions
• Estimation of a population mean
• Estimation of other parameters
• Estimation procedures for two populations
• Hypothesis testing
• Bayesian methods

Analysis of variance and significance testing

Regression model, least squares method, analysis of variance and goodness of fit, significance testing.

• Residual analysis
• Model building
• Correlation
• Time series and forecasting
• Nonparametric methods
• Acceptance sampling
• Statistical process control
• Sample survey methods
• Decision analysis

Experimental design

Our editors will review what you’ve submitted and determine whether to revise the article.

• Arizona State University - Educational Outreach and Student Services - Basic Statistics
• Princeton University - Probability and Statistics
• Statistics LibreTexts - Introduction to Statistics
• University of North Carolina at Chapel Hill - The Writing Center - Statistics
• Corporate Finance Institute - Statistics
• statistics - Children's Encyclopedia (Ages 8-11)
• statistics - Student Encyclopedia (Ages 11 and up)
• Table Of Contents

Data for statistical studies are obtained by conducting either experiments or surveys. Experimental design is the branch of statistics that deals with the design and analysis of experiments. The methods of experimental design are widely used in the fields of agriculture, medicine , biology , marketing research, and industrial production.

In an experimental study, variables of interest are identified. One or more of these variables, referred to as the factors of the study , are controlled so that data may be obtained about how the factors influence another variable referred to as the response variable , or simply the response. As a case in point, consider an experiment designed to determine the effect of three different exercise programs on the cholesterol level of patients with elevated cholesterol. Each patient is referred to as an experimental unit , the response variable is the cholesterol level of the patient at the completion of the program, and the exercise program is the factor whose effect on cholesterol level is being investigated. Each of the three exercise programs is referred to as a treatment .

Three of the more widely used experimental designs are the completely randomized design, the randomized block design, and the factorial design. In a completely randomized experimental design, the treatments are randomly assigned to the experimental units. For instance, applying this design method to the cholesterol-level study, the three types of exercise program (treatment) would be randomly assigned to the experimental units (patients).

The use of a completely randomized design will yield less precise results when factors not accounted for by the experimenter affect the response variable. Consider, for example, an experiment designed to study the effect of two different gasoline additives on the fuel efficiency , measured in miles per gallon (mpg), of full-size automobiles produced by three manufacturers. Suppose that 30 automobiles, 10 from each manufacturer, were available for the experiment. In a completely randomized design the two gasoline additives (treatments) would be randomly assigned to the 30 automobiles, with each additive being assigned to 15 different cars. Suppose that manufacturer 1 has developed an engine that gives its full-size cars a higher fuel efficiency than those produced by manufacturers 2 and 3. A completely randomized design could, by chance , assign gasoline additive 1 to a larger proportion of cars from manufacturer 1. In such a case, gasoline additive 1 might be judged to be more fuel efficient when in fact the difference observed is actually due to the better engine design of automobiles produced by manufacturer 1. To prevent this from occurring, a statistician could design an experiment in which both gasoline additives are tested using five cars produced by each manufacturer; in this way, any effects due to the manufacturer would not affect the test for significant differences due to gasoline additive. In this revised experiment, each of the manufacturers is referred to as a block, and the experiment is called a randomized block design. In general, blocking is used in order to enable comparisons among the treatments to be made within blocks of homogeneous experimental units.

Factorial experiments are designed to draw conclusions about more than one factor, or variable. The term factorial is used to indicate that all possible combinations of the factors are considered. For instance, if there are two factors with a levels for factor 1 and b levels for factor 2, the experiment will involve collecting data on a b treatment combinations. The factorial design can be extended to experiments involving more than two factors and experiments involving partial factorial designs.

A computational procedure frequently used to analyze the data from an experimental study employs a statistical procedure known as the analysis of variance. For a single-factor experiment, this procedure uses a hypothesis test concerning equality of treatment means to determine if the factor has a statistically significant effect on the response variable. For experimental designs involving multiple factors, a test for the significance of each individual factor as well as interaction effects caused by one or more factors acting jointly can be made. Further discussion of the analysis of variance procedure is contained in the subsequent section.

Regression and correlation analysis

Regression analysis involves identifying the relationship between a dependent variable and one or more independent variables . A model of the relationship is hypothesized, and estimates of the parameter values are used to develop an estimated regression equation . Various tests are then employed to determine if the model is satisfactory. If the model is deemed satisfactory, the estimated regression equation can be used to predict the value of the dependent variable given values for the independent variables.

In simple linear regression , the model used to describe the relationship between a single dependent variable y and a single independent variable x is y = β 0 + β 1 x + ε. β 0 and β 1 are referred to as the model parameters, and ε is a probabilistic error term that accounts for the variability in y that cannot be explained by the linear relationship with x . If the error term were not present, the model would be deterministic; in that case, knowledge of the value of x would be sufficient to determine the value of y .

In multiple regression analysis , the model for simple linear regression is extended to account for the relationship between the dependent variable y and p independent variables x 1 , x 2 , . . ., x p . The general form of the multiple regression model is y = β 0 + β 1 x 1 + β 2 x 2 + . . . + β p x p + ε. The parameters of the model are the β 0 , β 1 , . . ., β p , and ε is the error term.

Either a simple or multiple regression model is initially posed as a hypothesis concerning the relationship among the dependent and independent variables. The least squares method is the most widely used procedure for developing estimates of the model parameters. For simple linear regression, the least squares estimates of the model parameters β 0 and β 1 are denoted b 0 and b 1 . Using these estimates, an estimated regression equation is constructed: ŷ = b 0 + b 1 x . The graph of the estimated regression equation for simple linear regression is a straight line approximation to the relationship between y and x .

As an illustration of regression analysis and the least squares method, suppose a university medical centre is investigating the relationship between stress and blood pressure . Assume that both a stress test score and a blood pressure reading have been recorded for a sample of 20 patients. The data are shown graphically in Figure 4 , called a scatter diagram . Values of the independent variable, stress test score, are given on the horizontal axis, and values of the dependent variable, blood pressure, are shown on the vertical axis. The line passing through the data points is the graph of the estimated regression equation: ŷ = 42.3 + 0.49 x . The parameter estimates, b 0 = 42.3 and b 1 = 0.49, were obtained using the least squares method.

A primary use of the estimated regression equation is to predict the value of the dependent variable when values for the independent variables are given. For instance, given a patient with a stress test score of 60, the predicted blood pressure is 42.3 + 0.49(60) = 71.7. The values predicted by the estimated regression equation are the points on the line in Figure 4 , and the actual blood pressure readings are represented by the points scattered about the line. The difference between the observed value of y and the value of y predicted by the estimated regression equation is called a residual . The least squares method chooses the parameter estimates such that the sum of the squared residuals is minimized.

A commonly used measure of the goodness of fit provided by the estimated regression equation is the coefficient of determination . Computation of this coefficient is based on the analysis of variance procedure that partitions the total variation in the dependent variable, denoted SST, into two parts: the part explained by the estimated regression equation, denoted SSR, and the part that remains unexplained, denoted SSE.

The measure of total variation, SST, is the sum of the squared deviations of the dependent variable about its mean: Σ( y − ȳ ) 2 . This quantity is known as the total sum of squares. The measure of unexplained variation, SSE, is referred to as the residual sum of squares. For the data in Figure 4 , SSE is the sum of the squared distances from each point in the scatter diagram (see Figure 4 ) to the estimated regression line: Σ( y − ŷ ) 2 . SSE is also commonly referred to as the error sum of squares. A key result in the analysis of variance is that SSR + SSE = SST.

The ratio r 2 = SSR/SST is called the coefficient of determination. If the data points are clustered closely about the estimated regression line, the value of SSE will be small and SSR/SST will be close to 1. Using r 2 , whose values lie between 0 and 1, provides a measure of goodness of fit; values closer to 1 imply a better fit. A value of r 2 = 0 implies that there is no linear relationship between the dependent and independent variables.

When expressed as a percentage , the coefficient of determination can be interpreted as the percentage of the total sum of squares that can be explained using the estimated regression equation. For the stress-level research study, the value of r 2 is 0.583; thus, 58.3% of the total sum of squares can be explained by the estimated regression equation ŷ = 42.3 + 0.49 x . For typical data found in the social sciences, values of r 2 as low as 0.25 are often considered useful. For data in the physical sciences, r 2 values of 0.60 or greater are frequently found.

In a regression study, hypothesis tests are usually conducted to assess the statistical significance of the overall relationship represented by the regression model and to test for the statistical significance of the individual parameters. The statistical tests used are based on the following assumptions concerning the error term: (1) ε is a random variable with an expected value of 0, (2) the variance of ε is the same for all values of x , (3) the values of ε are independent, and (4) ε is a normally distributed random variable.

The mean square due to regression, denoted MSR, is computed by dividing SSR by a number referred to as its degrees of freedom ; in a similar manner, the mean square due to error, MSE , is computed by dividing SSE by its degrees of freedom. An F-test based on the ratio MSR/MSE can be used to test the statistical significance of the overall relationship between the dependent variable and the set of independent variables. In general, large values of F = MSR/MSE support the conclusion that the overall relationship is statistically significant. If the overall model is deemed statistically significant, statisticians will usually conduct hypothesis tests on the individual parameters to determine if each independent variable makes a significant contribution to the model.

• Science Notes Posts
• Contact Science Notes
• Todd Helmenstine Biography
• Anne Helmenstine Biography
• Free Printable Periodic Tables (PDF and PNG)
• Periodic Table Wallpapers
• Interactive Periodic Table
• Periodic Table Posters
• Science Experiments for Kids
• How to Grow Crystals
• Chemistry Projects
• Fire and Flames Projects
• Holiday Science
• Chemistry Problems With Answers
• Physics Problems
• Unit Conversion Example Problems
• Chemistry Worksheets
• Biology Worksheets
• Periodic Table Worksheets
• Physical Science Worksheets
• Science Lab Worksheets
• My Amazon Books

Experiment Definition in Science – What Is a Science Experiment?

In science, an experiment is simply a test of a hypothesis in the scientific method . It is a controlled examination of cause and effect. Here is a look at what a science experiment is (and is not), the key factors in an experiment, examples, and types of experiments.

Experiment Definition in Science

By definition, an experiment is a procedure that tests a hypothesis. A hypothesis, in turn, is a prediction of cause and effect or the predicted outcome of changing one factor of a situation. Both the hypothesis and experiment are components of the scientific method. The steps of the scientific method are:

• Make observations.
• Ask a question or identify a problem.
• State a hypothesis.
• Perform an experiment that tests the hypothesis.
• Based on the results of the experiment, either accept or reject the hypothesis.
• Draw conclusions and report the outcome of the experiment.

Key Parts of an Experiment

The two key parts of an experiment are the independent and dependent variables. The independent variable is the one factor that you control or change in an experiment. The dependent variable is the factor that you measure that responds to the independent variable. An experiment often includes other types of variables , but at its heart, it’s all about the relationship between the independent and dependent variable.

Examples of Experiments

Fertilizer and plant size.

For example, you think a certain fertilizer helps plants grow better. You’ve watched your plants grow and they seem to do better when they have the fertilizer compared to when they don’t. But, observations are only the beginning of science. So, you state a hypothesis: Adding fertilizer increases plant size. Note, you could have stated the hypothesis in different ways. Maybe you think the fertilizer increases plant mass or fruit production, for example. However you state the hypothesis, it includes both the independent and dependent variables. In this case, the independent variable is the presence or absence of fertilizer. The dependent variable is the response to the independent variable, which is the size of the plants.

Now that you have a hypothesis, the next step is designing an experiment that tests it. Experimental design is very important because the way you conduct an experiment influences its outcome. For example, if you use too small of an amount of fertilizer you may see no effect from the treatment. Or, if you dump an entire container of fertilizer on a plant you could kill it! So, recording the steps of the experiment help you judge the outcome of the experiment and aid others who come after you and examine your work. Other factors that might influence your results might include the species of plant and duration of the treatment. Record any conditions that might affect the outcome. Ideally, you want the only difference between your two groups of plants to be whether or not they receive fertilizer. Then, measure the height of the plants and see if there is a difference between the two groups.

Salt and Cookies

You don’t need a lab for an experiment. For example, consider a baking experiment. Let’s say you like the flavor of salt in your cookies, but you’re pretty sure the batch you made using extra salt fell a bit flat. If you double the amount of salt in a recipe, will it affect their size? Here, the independent variable is the amount of salt in the recipe and the dependent variable is cookie size.

Test this hypothesis with an experiment. Bake cookies using the normal recipe (your control group ) and bake some using twice the salt (the experimental group). Make sure it’s the exact same recipe. Bake the cookies at the same temperature and for the same time. Only change the amount of salt in the recipe. Then measure the height or diameter of the cookies and decide whether to accept or reject the hypothesis.

Examples of Things That Are Not Experiments

Based on the examples of experiments, you should see what is not an experiment:

• Making observations does not constitute an experiment. Initial observations often lead to an experiment, but are not a substitute for one.
• Making a model is not an experiment.
• Neither is making a poster.
• Just trying something to see what happens is not an experiment. You need a hypothesis or prediction about the outcome.
• Changing a lot of things at once isn’t an experiment. You only have one independent and one dependent variable. However, in an experiment, you might suspect the independent variable has an effect on a separate. So, you design a new experiment to test this.

Types of Experiments

There are three main types of experiments: controlled experiments, natural experiments, and field experiments,

• Controlled experiment : A controlled experiment compares two groups of samples that differ only in independent variable. For example, a drug trial compares the effect of a group taking a placebo (control group) against those getting the drug (the treatment group). Experiments in a lab or home generally are controlled experiments
• Natural experiment : Another name for a natural experiment is a quasi-experiment. In this type of experiment, the researcher does not directly control the independent variable, plus there may be other variables at play. Here, the goal is establishing a correlation between the independent and dependent variable. For example, in the formation of new elements a scientist hypothesizes that a certain collision between particles creates a new atom. But, other outcomes may be possible. Or, perhaps only decay products are observed that indicate the element, and not the new atom itself. Many fields of science rely on natural experiments, since controlled experiments aren’t always possible.
• Field experiment : While a controlled experiments takes place in a lab or other controlled setting, a field experiment occurs in a natural setting. Some phenomena cannot be readily studied in a lab or else the setting exerts an influence that affects the results. So, a field experiment may have higher validity. However, since the setting is not controlled, it is also subject to external factors and potential contamination. For example, if you study whether a certain plumage color affects bird mate selection, a field experiment in a natural environment eliminates the stressors of an artificial environment. Yet, other factors that could be controlled in a lab may influence results. For example, nutrition and health are controlled in a lab, but not in the field.
• Bailey, R.A. (2008). Design of Comparative Experiments . Cambridge: Cambridge University Press. ISBN 9780521683579.
• di Francia, G. Toraldo (1981). The Investigation of the Physical World . Cambridge University Press. ISBN 0-521-29925-X.
• Hinkelmann, Klaus; Kempthorne, Oscar (2008). Design and Analysis of Experiments. Volume I: Introduction to Experimental Design (2nd ed.). Wiley. ISBN 978-0-471-72756-9.
• Holland, Paul W. (December 1986). “Statistics and Causal Inference”.  Journal of the American Statistical Association . 81 (396): 945–960. doi: 10.2307/2289064
• Stohr-Hunt, Patricia (1996). “An Analysis of Frequency of Hands-on Experience and Science Achievement”. Journal of Research in Science Teaching . 33 (1): 101–109. doi: 10.1002/(SICI)1098-2736(199601)33:1<101::AID-TEA6>3.0.CO;2-Z

Related Posts

1.1 Definitions of Statistics, Probability, and Key Terms

The science of statistics deals with the collection, analysis, interpretation, and presentation of data . We see and use data in our everyday lives.

Collaborative Exercise

In your classroom, try this exercise. Have class members write down the average time—in hours, to the nearest half-hour—they sleep per night. Your instructor will record the data. Then create a simple graph, called a dot plot, of the data. A dot plot consists of a number line and dots, or points, positioned above the number line. For example, consider the following data:

5, 5.5, 6, 6, 6, 6.5, 6.5, 6.5, 6.5, 7, 7, 8, 8, 9.

The dot plot for this data would be as follows:

Does your dot plot look the same as or different from the example? Why? If you did the same example in an English class with the same number of students, do you think the results would be the same? Why or why not?

Where do your data appear to cluster? How might you interpret the clustering?

The questions above ask you to analyze and interpret your data. With this example, you have begun your study of statistics.

In this course, you will learn how to organize and summarize data. Organizing and summarizing data is called descriptive statistics . Two ways to summarize data are by graphing and by using numbers, for example, finding an average. After you have studied probability and probability distributions, you will use formal methods for drawing conclusions from good data. The formal methods are called inferential statistics . Statistical inference uses probability to determine how confident we can be that our conclusions are correct.

Effective interpretation of data, or inference, is based on good procedures for producing data and thoughtful examination of the data. You will encounter what will seem to be too many mathematical formulas for interpreting data. The goal of statistics is not to perform numerous calculations using the formulas, but to gain an understanding of your data. The calculations can be done using a calculator or a computer. The understanding must come from you. If you can thoroughly grasp the basics of statistics, you can be more confident in the decisions you make in life.

Statistical Models

Statistics, like all other branches of mathematics, uses mathematical models to describe phenomena that occur in the real world. Some mathematical models are deterministic. These models can be used when one value is precisely determined from another value. Examples of deterministic models are the quadratic equations that describe the acceleration of a car from rest or the differential equations that describe the transfer of heat from a stove to a pot. These models are quite accurate and can be used to answer questions and make predictions with a high degree of precision. Space agencies, for example, use deterministic models to predict the exact amount of thrust that a rocket needs to break away from Earth’s gravity and achieve orbit.

However, life is not always precise. While scientists can predict to the minute the time that the sun will rise, they cannot say precisely where a hurricane will make landfall. Statistical models can be used to predict life’s more uncertain situations. These special forms of mathematical models or functions are based on the idea that one value affects another value. Some statistical models are mathematical functions that are more precise—one set of values can predict or determine another set of values. Or some statistical models are mathematical functions in which a set of values do not precisely determine other values. Statistical models are very useful because they can describe the probability or likelihood of an event occurring and provide alternative outcomes if the event does not occur. For example, weather forecasts are examples of statistical models. Meteorologists cannot predict tomorrow’s weather with certainty. However, they often use statistical models to tell you how likely it is to rain at any given time, and you can prepare yourself based on this probability.

Probability

Probability is a mathematical tool used to study randomness. It deals with the chance of an event occurring. For example, if you toss a fair coin four times, the outcomes may not be two heads and two tails. However, if you toss the same coin 4,000 times, the outcomes will be close to half heads and half tails. The expected theoretical probability of heads in any one toss is 1 2 1 2 or .5. Even though the outcomes of a few repetitions are uncertain, there is a regular pattern of outcomes when there are many repetitions. After reading about the English statistician Karl Pearson who tossed a coin 24,000 times with a result of 12,012 heads, one of the authors tossed a coin 2,000 times. The results were 996 heads. The fraction 996 2,000 996 2,000 is equal to .498 which is very close to .5, the expected probability.

The theory of probability began with the study of games of chance such as poker. Predictions take the form of probabilities. To predict the likelihood of an earthquake, of rain, or whether you will get an A in this course, we use probabilities. Doctors use probability to determine the chance of a vaccination causing the disease the vaccination is supposed to prevent. A stockbroker uses probability to determine the rate of return on a client's investments.

In statistics, we generally want to study a population . You can think of a population as a collection of persons, things, or objects under study. To study the population, we select a sample . The idea of sampling is to select a portion, or subset, of the larger population and study that portion—the sample—to gain information about the population. Data are the result of sampling from a population.

Because it takes a lot of time and money to examine an entire population, sampling is a very practical technique. If you wished to compute the overall grade point average at your school, it would make sense to select a sample of students who attend the school. The data collected from the sample would be the students' grade point averages. In presidential elections, opinion poll samples of 1,000–2,000 people are taken. The opinion poll is supposed to represent the views of the people in the entire country. Manufacturers of canned carbonated drinks take samples to determine if a 16-ounce can contains 16 ounces of carbonated drink.

From the sample data, we can calculate a statistic. A statistic is a number that represents a property of the sample. For example, if we consider one math class as a sample of the population of all math classes, then the average number of points earned by students in that one math class at the end of the term is an example of a statistic. Since we do not have the data for all math classes, that statistic is our best estimate of the average for the entire population of math classes. If we happen to have data for all math classes, we can find the population parameter. A parameter is a numerical characteristic of the whole population that can be estimated by a statistic. Since we considered all math classes to be the population, then the average number of points earned per student over all the math classes is an example of a parameter.

One of the main concerns in the field of statistics is how accurately a statistic estimates a parameter. In order to have an accurate sample, it must contain the characteristics of the population in order to be a representative sample . We are interested in both the sample statistic and the population parameter in inferential statistics. In a later chapter, we will use the sample statistic to test the validity of the established population parameter.

A variable , usually notated by capital letters such as X and Y , is a characteristic or measurement that can be determined for each member of a population. Variables may describe values like weight in pounds or favorite subject in school. Numerical variables take on values with equal units such as weight in pounds and time in hours. Categorical variables place the person or thing into a category. If we let X equal the number of points earned by one math student at the end of a term, then X is a numerical variable. If we let Y be a person's party affiliation, then some examples of Y include Republican, Democrat, and Independent. Y is a categorical variable. We could do some math with values of X —calculate the average number of points earned, for example—but it makes no sense to do math with values of Y —calculating an average party affiliation makes no sense.

Data are the actual values of the variable. They may be numbers or they may be words. Datum is a single value.

Two words that come up often in statistics are mean and proportion . If you were to take three exams in your math classes and obtain scores of 86, 75, and 92, you would calculate your mean score by adding the three exam scores and dividing by three. Your mean score would be 84.3 to one decimal place. If, in your math class, there are 40 students and 22 are males and 18 females, then the proportion of men students is 22 40 22 40 and the proportion of women students is 18 40 18 40 . Mean and proportion are discussed in more detail in later chapters.

The words mean and average are often used interchangeably. In this book, we use the term arithmetic mean for mean.

Example 1.1

Determine what the population, sample, parameter, statistic, variable, and data referred to in the following study.

We want to know the mean amount of extracurricular activities in which high school students participate. We randomly surveyed 100 high school students. Three of those students were in 2, 5, and 7 extracurricular activities, respectively.

The population is all high school students.

The sample is the 100 high school students interviewed.

The parameter is the mean amount of extracurricular activities in which all high school students participate.

The statistic is the mean amount of extracurricular activities in which the sample of high school students participate.

The variable could be the amount of extracurricular activities by one high school student. Let X = the amount of extracurricular activities by one high school student.

The data are the number of extracurricular activities in which the high school students participate. Examples of the data are 2, 5, 7.

Find an article online or in a newspaper or magazine that refers to a statistical study or poll. Identify what each of the key terms—population, sample, parameter, statistic, variable, and data—refers to in the study mentioned in the article. Does the article use the key terms correctly?

Example 1.2

Determine what the key terms refer to in the following study.

A study was conducted at a local high school to analyze the average cumulative GPAs of students who graduated last year. Fill in the letter of the phrase that best describes each of the items below.

1. Population ____ 2. Statistic ____ 3. Parameter ____ 4. Sample ____ 5. Variable ____ 6. Data ____

• a) all students who attended the high school last year
• b) the cumulative GPA of one student who graduated from the high school last year
• c) 3.65, 2.80, 1.50, 3.90
• d) a group of students who graduated from the high school last year, randomly selected
• e) the average cumulative GPA of students who graduated from the high school last year
• f) all students who graduated from the high school last year
• g) the average cumulative GPA of students in the study who graduated from the high school last year

1. f ; 2. g ; 3. e ; 4. d ; 5. b ; 6. c

Example 1.3

As part of a study designed to test the safety of automobiles, the National Transportation Safety Board collected and reviewed data about the effects of an automobile crash on test dummies (The Data and Story Library, n.d.). Here is the criterion they used.

Cars with dummies in the front seats were crashed into a wall at a speed of 35 miles per hour. We want to know the proportion of dummies in the driver’s seat that would have had head injuries, if they had been actual drivers. We start with a simple random sample of 75 cars.

The population is all cars containing dummies in the front seat.

The sample is the 75 cars, selected by a simple random sample.

The parameter is the proportion of driver dummies—if they had been real people—who would have suffered head injuries in the population.

The statistic is proportion of driver dummies—if they had been real people—who would have suffered head injuries in the sample.

The variable X = whether driver dummies—if they had been real people—would have suffered head injuries.

The data are either: yes, had head injury, or no, did not.

Example 1.4

An insurance company would like to determine the proportion of all medical doctors who have been involved in one or more malpractice lawsuits. The company selects 500 doctors at random from a professional directory and determines the number in the sample who have been involved in a malpractice lawsuit.

The population is all medical doctors listed in the professional directory.

The parameter is the proportion of medical doctors who have been involved in one or more malpractice suits in the population.

The sample is the 500 doctors selected at random from the professional directory.

The statistic is the proportion of medical doctors who have been involved in one or more malpractice suits in the sample.

The variable X records whether a doctor has or has not been involved in a malpractice suit.

The data are either: yes, was involved in one or more malpractice lawsuits; or no, was not.

Do the following exercise collaboratively with up to four people per group. Find a population, a sample, the parameter, the statistic, a variable, and data for the following study: You want to determine the average—mean—number of glasses of milk college students drink per day. Suppose yesterday, in your English class, you asked five students how many glasses of milk they drank the day before. The answers were 1, 0, 1, 3, and 4 glasses of milk.

This book may not be used in the training of large language models or otherwise be ingested into large language models or generative AI offerings without OpenStax's permission.

Want to cite, share, or modify this book? This book uses the Creative Commons Attribution License and you must attribute Texas Education Agency (TEA). The original material is available at: https://www.texasgateway.org/book/tea-statistics . Changes were made to the original material, including updates to art, structure, and other content updates.

Access for free at https://openstax.org/books/statistics/pages/1-introduction

• Authors: Barbara Illowsky, Susan Dean
• Publisher/website: OpenStax
• Book title: Statistics
• Publication date: Mar 27, 2020
• Location: Houston, Texas
• Book URL: https://openstax.org/books/statistics/pages/1-introduction
• Section URL: https://openstax.org/books/statistics/pages/1-1-definitions-of-statistics-probability-and-key-terms

© Apr 16, 2024 Texas Education Agency (TEA). The OpenStax name, OpenStax logo, OpenStax book covers, OpenStax CNX name, and OpenStax CNX logo are not subject to the Creative Commons license and may not be reproduced without the prior and express written consent of Rice University.

JMP | Statistical Discovery.™ From SAS.

Statistics Knowledge Portal

A free online introduction to statistics

Design of experiments

What is design of experiments.

Design of experiments (DOE) is a systematic, efficient method that enables scientists and engineers to study the relationship between multiple input variables (aka factors) and key output variables (aka responses). It is a structured approach for collecting data and making discoveries.

When to use DOE?

• To determine whether a factor, or a collection of factors, has an effect on the response.
• To determine whether factors interact in their effect on the response.
• To model the behavior of the response as a function of the factors.
• To optimize the response.

Ronald Fisher first introduced four enduring principles of DOE in 1926: the factorial principle, randomization, replication and blocking. Generating and analyzing these designs relied primarily on hand calculation in the past; until recently practitioners started using computer-generated designs for a more effective and efficient DOE.

Why use DOE?

DOE is useful:

• In driving knowledge of cause and effect between factors.
• To experiment with all factors at the same time.
• To run trials that span the potential experimental region for our factors.
• In enabling us to understand the combined effect of the factors.

To illustrate the importance of DOE, let’s look at what will happen if DOE does NOT exist.

Experiments are likely to be carried out via trial and error or one-factor-at-a-time (OFAT) method.

Trial-and-error method

Test different settings of two factors and see what the resulting yield is.

Say we want to determine the optimal temperature and time settings that will maximize yield through experiments.

How the experiment looks like using trial-and-error method:

1. Conduct a trial at starting values for the two variables and record the yield:

2. Adjust one or both values based on our results:

3. Repeat Step 2 until we think we've found the best set of values:

As you can tell, the  cons of trial-and-error  are:

• Inefficient, unstructured and ad hoc (worst if carried out without subject matter knowledge).
• Unlikely to find the optimum set of conditions across two or more factors.

One factor at a time (OFAT) method

Change the value of the one factor, then measure the response, repeat the process with another factor.

In the same experiment of searching optimal temperature and time to maximize yield, this is how the experiment looks using an OFAT method:

1. Start with temperature: Find the temperature resulting in the highest yield, between 50 and 120 degrees.

    1a. Run a total of eight trials. Each trial increases temperature by 10 degrees (i.e., 50, 60, 70 ... all the way to 120 degrees).

    1b. With time fixed at 20 hours as a controlled variable.

    1c. Measure yield for each batch.

2. Run the second experiment by varying time, to find the optimal value of time (between 4 and 24 hours).

    2a. Run a total of six trials. Each trial increases temperature by 4 hours (i.e., 4, 8, 12… up to 24 hours).

    2b. With temperature fixed at 90 degrees as a controlled variable.

    2c. Measure yield for each batch.

3. After a total of 14 trials, we’ve identified the max yield (86.7%) happens when:

• Temperature is at 90 degrees; Time is at 12 hours.

As you can already tell, OFAT is a more structured approach compared to trial and error.

But there’s one major problem with OFAT : What if the optimal temperature and time settings look more like this?

We would have missed out acquiring the optimal temperature and time settings based on our previous OFAT experiments.

Therefore,  OFAT’s con  is:

• We’re unlikely to find the optimum set of conditions across two or more factors.

How our trial and error and OFAT experiments look:

Notice that none of them has trials conducted at a low temperature and time AND near optimum conditions.

What went wrong in the experiments?

• We didn't simultaneously change the settings of both factors.
• We didn't conduct trials throughout the potential experimental region.

The result was a lack of understanding on the combined effect of the two variables on the response. The two factors did interact in their effect on the response!

A more effective and efficient approach to experimentation is to use statistically designed experiments (DOE).

Apply Full Factorial DOE on the same example

1. Experiment with two factors, each factor with two values. 

These four trials form the corners of the design space:

2. Run all possible combinations of factor levels, in random order to average out effects of lurking variables .

3. (Optional) Replicate entire design by running each treatment twice to find out experimental error :

4. Analyzing the results enable us to build a statistical model that estimates the individual effects (Temperature & Time), and also their interaction.

It enables us to visualize and explore the interaction between the factors. An illustration of what their interaction looks like at temperature = 120; time = 4:

You can visualize, explore your model and find the most desirable settings for your factors using the JMP Prediction Profiler .

Summary: DOE vs. OFAT/Trial-and-Error

• DOE requires fewer trials.
• DOE is more effective in finding the best settings to maximize yield.
• DOE enables us to derive a statistical model to predict results as a function of the two factors and their combined effect.

If you're seeing this message, it means we're having trouble loading external resources on our website.

If you're behind a web filter, please make sure that the domains *.kastatic.org and *.kasandbox.org are unblocked.

To log in and use all the features of Khan Academy, please enable JavaScript in your browser.

AP®︎/College Statistics

Course: ap®︎/college statistics   >   unit 6, introduction to experiment design.

• The language of experiments
• Principles of experiment design
• Matched pairs experiment design
• Experiment designs
• Experiment design considerations

Want to join the conversation?

• Upvote Button navigates to signup page
• Downvote Button navigates to signup page
• Flag Button navigates to signup page

Video transcript

Calcworkshop

Experimental Design in Statistics w/ 11 Examples!

// Last Updated: September 20, 2020 - Watch Video //

A proper experimental design is a critical skill in statistics.

Jenn, Founder Calcworkshop ® , 15+ Years Experience (Licensed & Certified Teacher)

Without proper controls and safeguards, unintended consequences can ruin our study and lead to wrong conclusions.

So let’s dive in to see what’s this is all about!

What’s the difference between an observational study and an experimental study?

An observational study is one in which investigators merely measure variables of interest without influencing the subjects.

And an experiment is a study in which investigators administer some form of treatment on one or more groups?

In other words, an observation is hands-off, whereas an experiment is hands-on.

So what’s the purpose of an experiment?

To establish causation (i.e., cause and effect).

All this means is that we wish to determine the effect an independent explanatory variable has on a dependent response variable.

The explanatory variable explains a response, similar to a child falling and skins their knee and starting to cry. The child is crying in response to falling and skinning their knee. So the explanatory variable is the fall, and the response variable is crying.

Explanatory Vs Response Variable In Everyday Life

Let’s look at another example. Suppose a medical journal describes two studies in which subjects who had a seizure were randomly assigned to two different treatments:

• No treatment.
• A high dose of vitamin C.

The subjects were observed for a year, and the number of seizures for each subject was recorded. Identify the explanatory variable (independent variable), response variable (dependent variable), and include the experimental units.

The explanatory variable is whether the subject received either no treatment or a high dose of vitamin C. The response variable is whether the subject had a seizure during the time of the study. The experimental units in this study are the subjects who recently had a seizure.

Okay, so using the example above, notice that one of the groups did not receive treatment. This group is called a control group and acts as a baseline to see how a new treatment differs from those who don’t receive treatment. Typically, the control group is given something called a placebo, a substance designed to resemble medicine but does not contain an active drug component. A placebo is a dummy treatment, and should not have a physical effect on a person.

Before we talk about the characteristics of a well-designed experiment, we need to discuss some things to look out for:

• Confounding
• Lurking variables

Confounding happens when two explanatory variables are both associated with a response variable and also associated with each other, causing the investigator not to be able to identify their effects and the response variable separately.

A lurking variable is usually unobserved at the time of the study, which influences the association between the two variables of interest. In essence, a lurking variable is a third variable that is not measured in the study but may change the response variable.

For example, a study reported a relationship between smoking and health. A study of 1430 women were asked whether they smoked. Ten years later, a follow-up survey observed whether each woman was still alive or deceased. The researchers studied the possible link between whether a woman smoked and whether she survived the 10-year study period. They reported that:

• 21% of the smokers died
• 32% of the nonsmokers died

So, is smoking beneficial to your health, or is there something that could explain how this happened?

Older women are less likely to be smokers, and older women are more likely to die. Because age is a variable that influences the explanatory and response variable, it is considered a confounding variable.

But does smoking cause death?

Notice that the lurking variable, age, can also be a contributing factor. While there is a correlation between smoking and mortality, and also a correlation between smoking and age, we aren’t 100% sure that they are the cause of the mortality rate in women.

Lurking – Confounding – Correlation – Causation Diagram

Now, something important to point out is that a lurking variable is one that is not measured in the study that could influence the results. Using the example above, some other possible lurking variables are:

• Stress Level.

These variables were not measured in the study but could influence smoking habits as well as mortality rates.

What is important to note about the difference between confounding and lurking variables is that a confounding variable is measured in a study, while a lurking variable is not.

Additionally, correlation does not imply causation!

Alright, so now it’s time to talk about blinding: single-blind, double-blind experiments, as well as the placebo effect.

A single-blind experiment is when the subjects are unaware of which treatment they are receiving, but the investigator measuring the responses knows what treatments are going to which subject. In other words, the researcher knows which individual gets the placebo and which ones receive the experimental treatment. One major pitfall for this type of design is that the researcher may consciously or unconsciously influence the subject since they know who is receiving treatment and who isn’t.

A double-blind experiment is when both the subjects and investigator do not know who receives the placebo and who receives the treatment. A double-blind model is considered the best model for clinical trials as it eliminates the possibility of bias on the part of the researcher and the possibility of producing a placebo effect from the subject.

The placebo effect is when a subject has an effect or response to a fake treatment because they “believe” that the result should occur as noted by Yale . For example, a person struggling with insomnia takes a placebo (sugar pill) but instantly falls asleep because they believe they are receiving a sleep aid like Ambien or Lunesta.

Placebo Effect – Real Life Example

So, what are the three primary requirements for a well-designed experiment?

• Randomization

In a controlled experiment , the researchers, or investigators, decide which subjects are assigned to a control group and which subjects are assigned to a treatment group. In doing so, we ensure that the control and treatment groups are as similar as possible, and limit possible confounding influences such as lurking variables. A replicated experiment that is repeated on many different subjects helps reduce the chance of variation on the results. And randomization means we randomly assign subjects into control and treatment groups.

When subjects are divided into control groups and treatment groups randomly, we can use probability to predict the differences we expect to observe. If the differences between the two groups are higher than what we would expect to see naturally (by chance), we say that the results are statistically significant.

For example, if it is surmised that a new medicine reduces the effects of illness from 72 hours to 71 hours, this would not be considered statistically significant. The difference from 72 hours to 71 hours is not substantial enough to support that the observed effect was due to something other than normal random variation.

Now there are two major types of designs:

• Completely-Randomized Design (CRD)
• Block Design

A completely randomized design is the process of assigning subjects to control and treatment groups using probability, as seen in the flow diagram below.

Completely Randomized Design Example

A block design is a research method that places subjects into groups of similar experimental units or conditions, like age or gender, and then assign subjects to control and treatment groups using probability, as shown below.

Randomized Block Design Example

Additionally, a useful and particular case of a blocking strategy is something called a matched-pair design . This is when two variables are paired to control for lurking variables.

For example, imagine we want to study if walking daily improved blood pressure. If the blood pressure for five subjects is measured at the beginning of the study and then again after participating in a walking program for one month, then the observations would be considered dependent samples because the same five subjects are used in the before and after observations; thus, a matched-pair design.

Please note that our video lesson will not focus on quasi-experiments. A quasi experimental design lacks random assignments; therefore, the independent variable can be manipulated prior to measuring the dependent variable, which may lead to confounding. For the sake of our lesson, and all future lessons, we will be using research methods where random sampling and experimental designs are used.

Together we will learn how to identify explanatory variables (independent variable) and response variables (dependent variables), understand and define confounding and lurking variables, see the effects of single-blind and double-blind experiments, and design randomized and block experiments.

Experimental Designs – Lesson & Examples (Video)

1 hr 06 min

• Introduction to Video: Experiments
• 00:00:29 – Observational Study vs Experimental Study and Response and Explanatory Variables (Examples #1-4)
• Exclusive Content for Members Only
• 00:09:15 – Identify the response and explanatory variables and the experimental units and treatment (Examples #5-6)
• 00:14:47 – Introduction of lurking variables and confounding with ice cream and homicide example
• 00:18:57 – Lurking variables, Confounding, Placebo Effect, Single Blind and Double Blind Experiments (Example #7)
• 00:27:20 – What was the placebo effect and was the experiment single or double blind? (Example #8)
• 00:30:36 – Characteristics of a well designed and constructed experiment that is statistically significant
• 00:35:08 – Overview of Complete Randomized Design, Block Design and Matched Pair Design
• 00:44:23 – Design and experiment using complete randomized design or a block design (Examples #9-10)
• 00:56:09 – Identify the response and explanatory variables, experimental units, lurking variables, and design an experiment to test a new drug (Example #11)
• Practice Problems with Step-by-Step Solutions
• Chapter Tests with Video Solutions

Get access to all the courses and over 450 HD videos with your subscription

Monthly and Yearly Plans Available

Get My Subscription Now

Still wondering if CalcWorkshop is right for you? Take a Tour and find out how a membership can take the struggle out of learning math.

Statistical Aid: A School of Statistics

Learn statistics and data analysis intuitively, an intuitive study of experimental design.

Experimental design is the formulation of a set of rules and principles according to which an experiment is to be conducted to collect appropriate data whose analysis will lead to valid inferences for the problem under investigation. More precisely, experimental design is a way to carefully plan experiments in advance to get valid and objective based result.

An experiment

An experiment is a well defined act or an investigation conducted to discover the underlying facts about a phenomenon, which are utilized to test some hypotheses of interest to verify the results of previous investigations. More precisely, an experiment is the process of data collection from a non-existent population to get answer to the certain problems under investigation. There are two types of experiment. They are-

• Absolute experiment: An absolute experiment is one in which the absolute value of some characteristic is determined. Sample survey belongs to absolute experiment.
• Comparative experiment: A comparative experiment is one where two or more varieties or treatments are compared to assess the significance of difference among the varieties. Such experimental studies are based on test of hypothesis and estimation of differences among the effect of the different treatments in order to recommend best treatment.

Steps in experimental design

Design of experiment consists of the following steps-

• Choosing a set of treatments for comparisons.
• Selection of experimental units to which chosen treatments will be applied.
• Specification of the number of experimental units for inclusion in the experiment.
• Specification of the method of allocating the treatments to the experimental units.
• Specification of the measurements to be obtained from each experimental unit.
• Specification of the grouping of experimental units to control extraneous sources of variation.

Purposes of experimental design

There are some important purposes of experimental design-

• The main purpose of design of experiment is to collect maximum amount of necessary information for the problem under consideration, at a minimum cost in terms of time and resources.
• Design of experiment is needed to ensure that the requisite assumptions for analysis and interpretation  of data are fulfilled.
• Design of experiment is essential to increase the accuracy of the results of an experiment.

Principles of experimental design

According to R.A. Fisher, there are three basic principles of experimental design. They are-

• Replication: Replication means repetition of basic treatments under investigation. Thus replication is the repetition of same treatment on several experimental units. Experience indicates that even if same treatment is assigned to all experimental units, yields will be differ substantially. So it is essential to replicate the treatments to study the variation in the yields of each variety. 
• Randomization: Randomization is the process of distributing the treatments to the experimental units purely by chance or probability mechanism in such a way that any experimental unit is equally likely to receive any treatment. Then randomization ensures that no treatment is unduly favored or handicapped in the experiment. That means, it eliminates bias from the result of an experiment.
• Local Control: Local control is the procedure of reducing and controlling error variation by arranging the experimental units in blocks. By blocking, variation among the blocks is eliminated from the experimental error. So, it increases the precision of the result.

Requirements of a good experiment

A good experiment should satisfy the following conditions-

• Absence of bias: It is essential to plan an experiment so that unbiased estimates of treatment differences and treatment effect can be obtained from the data of the experiment.
• Measure of experimental error: Since the treatments under comparison apparently produce different results, test of significance in needed to assess the nature of treatment differences.
• Precision: Precisions depend on the experimental error. If the error is less than the precision is high.
• Clearly defined objective: Every experiment should have clearly defined objective on which the design and analysis of data considerably depend.
• Simplicity: Experimental design should be very simple and consistent.
• Range of validity: The conclusion drawn from the experimental data should have a wide range of validity.

Basic experimental designs

There are some basic or commonly used experimental design-

• Completely Randomized Design (CRD): A completely randomized design is a design in which the selected treatments are allocated or distributed to the experimental units completely at random. This design is divided into two category as, balanced completely randomized design and unbalanced completely randomized design.
• Randomized Block Design (RBD): A randomized block design is a design in which the whole set of experimental units are arranged in several blocks which are internally homogeneous and externally heterogeneous. 
• Latin Square Design (LSD): Latin square design is a design in which experimental units are arranged in complete blocks in two different ways, called rows and columns and then the selected treatments are randomly allocated to treatment.
• Time series analysis
• Business forecasting
• Hypothesis testing
• Logistic Regression
• Correlation Analysis
• Data analysis using SPSS
• ← Probability Distributions in Statistics
• Statistical Hypothesis Testing step by step procedure →

You cannot copy content of this page

Stack Exchange Network

Stack Exchange network consists of 183 Q&A communities including Stack Overflow , the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Definition of an "Experiment" in Probability

One can define the fundamental concepts of probability theory (such as a probability measure, random variable, etc) in a purely axiomatic manner. However, when we teach probability, we start off with the notion of an "experiment", a concept it seems to me which is something akin to pornography: difficult to define, but you tend to know it when you see it.

So I am curious if there is a general definition of an experiment (or if it something really best regarded more as an explanatory construct). To try to define an experiment as a type of function seems difficult to me b/c it would require the notion of a "random function" of some type.

Thanks, Jack

• probability

4 Answers 4

I like the way it is defined in Mathematical Statistics By Wiebe R. Pestman:

"A probability experiment is an experiment which, when repeated under the same conditions, does not necessarily give the same results"

This is useful as well.

• $\begingroup$ My initial reaction is to say that I'm gobsmacked. I'll have to read the article though to understand the context (and his meaning) better. Is this what most statisticians take to be the definition, do you think? Thanks $\endgroup$ –  Jack Zega Commented Feb 20, 2013 at 23:47
• $\begingroup$ @JackZega By no means I'm a statistician. The above tries to convey difference between deterministic experiment versus probability experiments. For example, if you try to measure voltage $V$ through a certain conductor with resistance $R$, you will always end up with the value $IR$ no matter what (this is a deterministic experiment ). One advice, the deeper you go the more hair you lose :) $\endgroup$ –  jay-sun Commented Feb 20, 2013 at 23:55
• $\begingroup$ I think this definition, taken out of its context, is quite dangerous. A probabilistic experiment must be infinitely repeatable. By infinitely repeatable we mean that each time you repeat the experiment, the sample space $\Omega$ (and so everything that follows from it) is constant. So actually, an experiment is a procedure that, when repeated keeps the sample space constant. Therefore the possible results of an experiments are always the same, but the actual result of a repetition of an experiment, could be different, if the experiment is a random experiment and not a deterministic one. $\endgroup$ –  Euler_Salter Commented Aug 18, 2018 at 14:03

One would think that it would be the other way round - everyone understands what it means to roll a die, but the notion of a random variable is far less trivial.

To define an experiment, first define a "generator" - any physical or algorithmic method for producing $N$ numbers, such that $N$ tends to infinity, the numbers produced are distributed according to random variable $X$.

The production of any individual number using a generator is an experiment.

“Experiment is a systematic way of varying all the factors of interest and observing impact of these all factors on the desired output.”

While reading Grimmett & Welsh book, I found that an experiment is

Any procedure whose consequences are not predetermined.

This is quite a restrictive definition in my opinion, because it excludes "deterministic" experiments where we can determine the final result 100% precision (because there is actually only one possible result). So to me, it the following definition given by Wikipedia seems more precise:

Any procedure that is infinitely repeatable and whose outcomes are well-defined

This seems fine, but it doesn't seem tremendously mathematical. So my guess is that the definition above is correct, and that is the definition used to define a sample space, outcomes, events, event space, probability measure, and so on. However, once we've defined all those mathematical structures, we can go back and say: actually, an experiment can easily be represented by a probability space $(\Omega, \Sigma, \mathbb{P})$. This could be seen as a circular definition, but if you use the word "represented" instead of "defined" then you should be fine.

I just wrote an article (which I will extend soon, it's still under construction) in my website SimpleAI

You must log in to answer this question.

Not the answer you're looking for browse other questions tagged probability statistics definition ..

• Upcoming Events
• 2024 Community Moderator Election ends in 3 days
• Featured on Meta
• We've made changes to our Terms of Service & Privacy Policy - July 2024
• Bringing clarity to status tag usage on meta sites
• Upcoming Moderator Election
• 2024 Community Moderator Election

Hot Network Questions

• What is the purpose of toroidal magnetic field in tokamak fusion device?
• Why do these finite group Dedekind matrices seem to have integer spectrum when specialized to the order of group elements?
• Do temperature variations make trains on Mars impractical?
• Whats the purpose of slots in wings?
• Is there a law against biohacking your pet?
• What does it mean to have a truth value of a 'nothing' type instance?
• 90/180 day rule with student resident permit, how strict it is applied
• Dial “M” for murder
• Are there jurisdictions where an uninvolved party can appeal a court decision?
• What is the meaning of "Exit, pursued by a bear"?
• Enigmatic Puzzle 4: Three Leaf Clover
• Why was I was allowed to bring 1.5 liters of liquid through security at Frankfurt Airport?
• What is the legal status of the Avengers before Civil War and after Winter Soldier?
• Why HIMEM was implemented as a DOS driver and not a TSR
• Does each unique instance of Nietzsche's superman have the same virtues and values?
• Is “overaction” an Indian English word?
• What is the meaning of these Greek words ἵπποπείρην and ἐπεμβάτην?
• What's wrong with my app authentication scheme?
• Restarted my computer and the display resolution changed
• What was the reason for not personifying God's spirit in NABRE's translation of John 14:17?
• Discrete cops and robbers
• Does H3PO exist?
• Unknown tool. Not sure what this tool is, found it in a tin of odds and ends
• Garage door not closing when sunlight is bright

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

What is an Observational Study: Definition & Examples

By Jim Frost 10 Comments

What is an Observational Study?

An observational study uses sample data to find correlations in situations where the researchers do not control the treatment, or independent variable, that relates to the primary research question. The definition of an observational study hinges on the notion that the researchers only observe subjects and do not assign them to the control and treatment groups. That’s the key difference between an observational study vs experiment. These studies are also known as quasi-experiments and correlational studies .

True experiments assign subject to the experimental groups where the researchers can manipulate the conditions. Unfortunately, random assignment is not always possible. For these cases, you can conduct an observational study.

In this post, learn about the types of observational studies, why they are susceptible to confounding variables, and how they compare to experiments. I’ll close this post by reviewing a published observational study about vitamin supplement usage.

Observational Study Definition

In an observational study, the researchers only observe the subjects and do not interfere or try to influence the outcomes. In other words, the researchers do not control the treatments or assign subjects to experimental groups. Instead, they observe and measure variables of interest and look for relationships between them. Usually, researchers conduct observational studies when it is difficult, impossible, or unethical to assign study participants to the experimental groups randomly. If you can’t randomly assign subjects to the treatment and control groups, then you observe the subjects in their self-selected states.

Observational Study vs Experiment

Randomized experiments provide better results than observational studies. Consequently, you should always use a randomized experiment whenever possible. However, if randomization is not possible, science should not come to a halt. After all, we still want to learn things, discover relationships, and make discoveries. For these cases, observational studies are a good alternative to a true experiment. Let’s compare the differences between an observational study vs. an experiment.

Random assignment in an experiment reduces systematic differences between experimental groups at the beginning of the study, which increases your confidence that the treatments caused any differences between groups you observe at the end of the study. In contrast, an observational study uses self-formed groups that can have pre-existing differences, which introduces the problem of confounding variables. More on that later!

In a randomized experiment, randomization tends to equalize confounders between groups and, thereby, prevents problems. In my post about random assignment , I describe that process as an elegant solution for confounding variables. You don’t need to measure or even know which variables are confounders, and randomization will still mitigate their effects. Additionally, you can use control variables in an experiment to keep the conditions as consistent as possible. For more detail about the differences, read Observational Study vs. Experiment .

If you’re looking for a middle ground choice between observational studies vs experiments, consider using a quasi-experimental design. These methods don’t require you to randomly assign participants to the experimental groups and still allow you to draw better causal conclusions about an intervention than an observational study. Learn more about Quasi-Experimental Design Overview & Examples .

Related posts : Experimental Design: Definition and Examples , Randomized Controlled Trials (RCTs) , and Control Groups in Experiments

Observational Study Examples

Consider using an observational study when random assignment for an experiment is problematic. This approach allows us to proceed and draw conclusions about effects even though we can’t control the independent variables. The following observational study examples will help you understand when and why to use them.

For example, if you’re studying how depression affects performance of an activity, it’s impossible to assign subjects to the depression and control group randomly. However, you can have subjects with and without depression perform the activity and compare the results in an observational study.

Or imagine trying to assign subjects to cigarette smoking and non-smoking groups randomly?! However, you can observe people in both groups and assess the differences in health outcomes in an observational study.

Suppose you’re studying a treatment for a disease. Ideally, you recruit a group of patients who all have the disease, and then randomly assign them to the treatment and control group. However, it’s unethical to withhold the treatment, which rules out a control group. Instead, you can compare patients who voluntarily do not use the medicine to those who do use it.

In all these observational study examples, the researchers do not assign subjects to the experimental groups. Instead, they observe people who are already in these groups and compare the outcomes. Hence, the scientists must use an observational study vs. an experiment.

Types of Observational Studies

The observational study definition states that researchers only observe the outcomes and do not manipulate or control factors . Despite this limitation, there various types of observational studies.

The following experimental designs are three standard types of observational studies.

• Cohort Study : A longitudinal observational study that follows a group who share a defining characteristic. These studies frequently determine whether exposure to risk factor affects an outcome over time.
• Case-Control Study : A retrospective observational study that compares two existing groups—the case group with the condition and the control group without it. Researchers compare the groups looking for potential risk factors for the condition.
• Cross-Sectional Study : Takes a snapshot of a moment in time so researchers can understand the prevalence of outcomes and correlations between variables at that instant.

Qualitative research studies are usually observational in nature, but they collect non-numeric data and do not perform statistical analyses.

Retrospective studies must be observational.

Later in this post, we’ll closely examine a quantitative observational study example that assesses vitamin supplement consumption and how that affects the risk of death. It’s possible to use random assignment to place each subject in either the vitamin treatment group or the control group. However, the study assesses vitamin consumption in 40,000 participants over the course of two decades. It’s unrealistic to enforce the treatment and control protocols over such a long time for so many people!

Drawbacks of Observational Studies

While observational studies get around the inability to assign subjects randomly, this approach opens the door to the problem of confounding variables. A confounding variable, or confounder, correlates with both the experimental groups and the outcome variable. Because there is no random process that equalizes the experimental groups in an observational study, confounding variables can systematically differ between groups when the study begins. Consequently, confounders can be the actual cause for differences in outcome at the end of the study rather than the primary variable of interest. If an experiment does not account for confounding variables, confounders can bias the results and create spurious correlations .

Performing an observational study can decrease the internal validity of your study but increase the external validity. Learn more about internal and external validity .

Let’s see how this works. Imagine an observational study that compares people who take vitamin supplements to those who do not. People who use vitamin supplements voluntarily will tend to have other healthy habits that exist at the beginning of the study. These healthy habits are confounding variables. If there are differences in health outcomes at the end of the study, it’s possible that these healthy habits actually caused them rather than the vitamin consumption itself. In short, confounders confuse the results because they provide alternative explanations for the differences.

Despite the limitations, an observational study can be a valid approach. However, you must ensure that your research accounts for confounding variables. Fortunately, there are several methods for doing just that!

Learn more about Correlation vs. Causation: Understanding the Differences .

Accounting for Confounding Variables in an Observational Study

Because observational studies don’t use random assignment, confounders can be distributed disproportionately between conditions. Consequently, experimenters need to know which variables are confounders, measure them, and then use a method to account for them. It involves more work, and the additional measurements can increase the costs. And there’s always a chance that researchers will fail to identify a confounder, not account for it, and produce biased results. However, if randomization isn’t an option, then you probably need to consider an observational study.

Trait matching and statistically controlling confounders using multivariate procedures are two standard approaches for incorporating confounding variables.

Related post : Causation versus Correlation in Statistics

Matching in Observational Studies

Matching is a technique that involves selecting study participants with similar characteristics outside the variable of interest or treatment. Rather than using random assignment to equalize the experimental groups, the experimenters do it by matching observable characteristics. For every participant in the treatment group, the researchers find a participant with comparable traits to include in the control group. Matching subjects facilitates valid comparisons between those groups. The researchers use subject-area knowledge to identify characteristics that are critical to match.

For example, a vitamin supplement study using matching will select subjects who have similar health-related habits and attributes. The goal is that vitamin consumption will be the primary difference between the groups, which helps you attribute differences in health outcomes to vitamin consumption. However, the researchers are still observing participants who decide whether they consume supplements.

Matching has some drawbacks. The experimenters might not be aware of all the relevant characteristics they need to match. In other words, the groups might be different in an essential aspect that the researchers don’t recognize. For example, in the hypothetical vitamin study, there might be a healthy habit or attribute that affects the outcome that the researchers don’t measure and match. These unmatched characteristics might cause the observed differences in outcomes rather than vitamin consumption.

Learn more about Matched Pairs Design: Uses & Examples .

Using Multiple Regression in Observational Studies

Random assignment and matching use different methods to equalize the experimental groups in an observational study. However, statistical techniques, such as multiple regression analysis , don’t try to equalize the groups but instead use a model that accounts for confounding variables. These studies statistically control for confounding variables.

In multiple regression analysis, including a variable in the model holds it constant while you vary the variable/treatment of interest. For information about this property, read my post When Should I Use Regression Analysis?

As with matching, the challenge is to identify, measure, and include all confounders in the regression model. Failure to include a confounding variable in a regression model can cause omitted variable bias to distort your results.

Next, we’ll look at a published observational study that uses multiple regression to account for confounding variables.

Related post : Independent and Dependent Variables in a Regression Model

Vitamin Supplement Observational Study Example

Murso et al. (2011)* use a longitudinal observational study that ran 22 years to assess differences in death rates for subjects who used vitamin supplements regularly compared to those who did not use them. This study used surveys to record the characteristics of approximately 40,000 participants. The surveys asked questions about potential confounding variables such as demographic information, food intake, health details, physical activity, and, of course, supplement intake.

Because this is an observational study, the subjects decided for themselves whether they were taking vitamin supplements. Consequently, it’s safe to assume that supplement users and non-users might be different in other ways. From their article, the researchers found the following pre-existing differences between the two groups:

Supplement users had a lower prevalence of diabetes mellitus, high blood pressure, and smoking status; a lower BMI and waist to hip ratio, and were less likely to live on a farm. Supplement users had a higher educational level, were more physically active and were more likely to use estrogen replacement therapy. Also, supplement users were more likely to have a lower intake of energy, total fat, and monounsaturated fatty acids, saturated fatty acids and to have a higher intake of protein, carbohydrates, polyunsaturated fatty acids, alcohol, whole grain products, fruits, and vegetables.

Whew! That’s a long list of differences! Supplement users were different from non-users in a multitude of ways that are likely to affect their risk of dying. The researchers must account for these confounding variables when they compare supplement users to non-users. If they do not, their results can be biased.

This example illustrates a key difference between an observational study vs experiment. In a randomized experiment, the randomization would have equalized the characteristics of those the researchers assigned to the treatment and control groups. Instead, the study works with self-sorted groups that have numerous pre-existing differences!

Using Multiple Regression to Statistically Control for Confounders

To account for these initial differences in the vitamin supplement observational study, the researchers use regression analysis and include the confounding variables in the model.

The researchers present three regression models. The simplest model accounts only for age and caloric intake. Next, are two models that include additional confounding variables beyond age and calories. The first model adds various demographic information and seven health measures. The second model includes everything in the previous model and adds several more specific dietary intake measures. Using statistical significance as a guide for specifying the correct regression model , the researchers present the model with the most variables as the basis for their final results.

It’s instructive to compare the raw results and the final regression results.

Raw results

The raw differences in death risks for consumers of folic acid, vitamin B6, magnesium, zinc, copper, and multivitamins are NOT statistically significant. However, the raw results show a significant reduction in the death risk for users of B complex, C, calcium, D, and E.

However, those are the raw results for the observational study, and they do not control for the long list of differences between the groups that exist at the beginning of the study. After using the regression model to control for the confounding variables statistically, the results change dramatically.

Adjusted results

Of the 15 supplements that the study tracked in the observational study, researchers found consuming seven of these supplements were linked to a statistically significant INCREASE in death risk ( p-value < 0.05): multivitamins (increase in death risk 2.4%), vitamin B6 (4.1%), iron (3.9%), folic acid (5.9%), zinc (3.0%), magnesium (3.6%), and copper (18.0%). Only calcium was associated with a statistically significant reduction in death risk of 3.8%.

In short, the raw results suggest that those who consume supplements either have the same or lower death risks than non-consumers. However, these results do not account for the multitude of healthier habits and attributes in the group that uses supplements.

In fact, these confounders seem to produce most of the apparent benefits in the raw results because, after you statistically control the effects of these confounding variables, the results worsen for those who consume vitamin supplements. The adjusted results indicate that most vitamin supplements actually increase your death risk!

This research illustrates the differences between an observational study vs experiment. Namely how the pre-existing differences between the groups allow confounders to bias the raw results, making the vitamin consumption outcomes look better than they really are.

In conclusion, if you can’t randomly assign subjects to the experimental groups, an observational study might be right for you. However, be aware that you’ll need to identify, measure, and account for confounding variables in your experimental design.

Jaakko Mursu, PhD; Kim Robien, PhD; Lisa J. Harnack, DrPH, MPH; Kyong Park, PhD; David R. Jacobs Jr, PhD; Dietary Supplements and Mortality Rate in Older Women: The Iowa Women’s Health Study ; Arch Intern Med . 2011;171(18):1625-1633.

Share this:

Reader Interactions

December 30, 2023 at 5:05 am

I see, but our professor required us to indicate what year it was put into the article. May you tell me what year was this published originally? <3

December 30, 2023 at 3:40 pm

December 29, 2023 at 10:46 am

Hi, may I use your article as a citation for my thesis paper? If so, may I know the exact date you published this article? Thank you!

December 29, 2023 at 2:13 pm

Definitely feel free to cite this article! 🙂

When citing online resources, you typically use an “Accessed” date rather than a publication date because online content can change over time. For more information, read Purdue University’s Citing Electronic Resources .

November 18, 2021 at 10:09 pm

Love your content and has been very helpful!

Can you please advise the question below using an observational data set:

I have three years of observational GPS data collected on athletes (2019/2020/2021). Approximately 14-15 athletes per game and 8 games per year. The GPS software outputs 50+ variables for each athlete in each game, which we have narrowed down to 16 variables of interest from previous research.

2 factors 1) Period (first half, second half, and whole game), 2) Position (two groups with three subgroups in each – forwards (group 1, group 2, group 3) and backs (group 1, group 2, group 3))

16 variables of interest – all numerical and scale variables. Some of these are correlated, but not all.

My understanding is that I can use a oneway ANOVA for each year on it’s own, using one factor at a time (period or position) with post hoc analysis. This is fine, if data meets assumptions and is normally distributed. This tells me any significant interactions between variables of interest with chosen factor. For example, with position factor, do forwards in group 1 cover more total running distance than forwards in group 2 or backs in group 3.

However, I want to go deeper with my analysis. If I want to see if forwards in group 1 cover more total running distance in period 1 than backs in group 3 in the same period, I need an additional factor and the oneway ANOVA does not suit. Therefore I can use a twoway ANOVA instead of 2 oneway ANOVA’s and that solves the issue, correct?

This is complicated further by looking to compare 2019 to 2020 or 2019 to 2021 to identify changes over time, which would introduce a third independent variable.

I believe this would require a threeway ANOVA for this observational data set. 3 factors – Position, Period, and Year?

Are there any issues or concerns you see at first glance?

I appreciate your time and consideration.

April 12, 2021 at 2:02 pm

Could an observational study use a correlational design.

e.g. measuring effects of two variables on happiness, if you’re not intervening.

April 13, 2021 at 12:14 am

Typically, with observational studies, you’d want to include potential confounders, etc. Consequently, I’ve seen regression analysis used more frequently for observational studies to be able to control for other things because you’re not using randomization. You could use correlation to observe the relationship. However, you wouldn’t be controlling for potential confounding variables. Just something to consider.

April 11, 2021 at 1:28 pm

Hi, If I am to administer moderate doses of coffee for a hypothetical experiment, does it raise ethical concerns? Can I use random assignment for it?

April 11, 2021 at 4:06 pm

I don’t see any inherent ethical problems here as long as you describe the participant’s experience in the experiment including the coffee consumption. They key with human subjects is “informed consent.” They’re agreeing to participate based on a full and accurate understanding of what participation involves. Additionally, you as a researcher, understand the process well enough to be able to ensure their safety.

In your study, as long as subject know they’ll be drinking coffee and agree to that, I don’t see a problem. It’s a proven safe substance for the vast majority of people. If potential subjects are aware of the need to consume coffee, they can determine whether they are ok with that before agreeing to participate.

June 17, 2019 at 4:51 am

Really great article which explains observational and experimental study very well. It presents broad picture with the case study which helped a lot in understanding the core concepts. Thanks

Comments and Questions Cancel reply

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Types of Variables in Research & Statistics | Examples

Types of Variables in Research & Statistics | Examples

Published on September 19, 2022 by Rebecca Bevans . Revised on June 21, 2023.

In statistical research , a variable is defined as an attribute of an object of study. Choosing which variables to measure is central to good experimental design .

If you want to test whether some plant species are more salt-tolerant than others, some key variables you might measure include the amount of salt you add to the water, the species of plants being studied, and variables related to plant health like growth and wilting .

You need to know which types of variables you are working with in order to choose appropriate statistical tests and interpret the results of your study.

You can usually identify the type of variable by asking two questions:

• What type of data does the variable contain?
• What part of the experiment does the variable represent?

Table of contents

Types of data: quantitative vs categorical variables, parts of the experiment: independent vs dependent variables, other common types of variables, other interesting articles, frequently asked questions about variables.

Data is a specific measurement of a variable – it is the value you record in your data sheet. Data is generally divided into two categories:

• Quantitative data represents amounts
• Categorical data represents groupings

A variable that contains quantitative data is a quantitative variable ; a variable that contains categorical data is a categorical variable . Each of these types of variables can be broken down into further types.

Quantitative variables

When you collect quantitative data, the numbers you record represent real amounts that can be added, subtracted, divided, etc. There are two types of quantitative variables: discrete and continuous .

Categorical variables

Categorical variables represent groupings of some kind. They are sometimes recorded as numbers, but the numbers represent categories rather than actual amounts of things.

There are three types of categorical variables: binary , nominal , and ordinal variables .

*Note that sometimes a variable can work as more than one type! An ordinal variable can also be used as a quantitative variable if the scale is numeric and doesn’t need to be kept as discrete integers. For example, star ratings on product reviews are ordinal (1 to 5 stars), but the average star rating is quantitative.

Example data sheet

To keep track of your salt-tolerance experiment, you make a data sheet where you record information about the variables in the experiment, like salt addition and plant health.

To gather information about plant responses over time, you can fill out the same data sheet every few days until the end of the experiment. This example sheet is color-coded according to the type of variable: nominal , continuous , ordinal , and binary .

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Experiments are usually designed to find out what effect one variable has on another – in our example, the effect of salt addition on plant growth.

You manipulate the independent variable (the one you think might be the cause ) and then measure the dependent variable (the one you think might be the effect ) to find out what this effect might be.

You will probably also have variables that you hold constant ( control variables ) in order to focus on your experimental treatment.

In this experiment, we have one independent and three dependent variables.

The other variables in the sheet can’t be classified as independent or dependent, but they do contain data that you will need in order to interpret your dependent and independent variables.

What about correlational research?

When you do correlational research , the terms “dependent” and “independent” don’t apply, because you are not trying to establish a cause and effect relationship ( causation ).

However, there might be cases where one variable clearly precedes the other (for example, rainfall leads to mud, rather than the other way around). In these cases you may call the preceding variable (i.e., the rainfall) the predictor variable and the following variable (i.e. the mud) the outcome variable .

Once you have defined your independent and dependent variables and determined whether they are categorical or quantitative, you will be able to choose the correct statistical test .

But there are many other ways of describing variables that help with interpreting your results. Some useful types of variables are listed below.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Likert scale

Research bias

• Implicit bias
• Framing effect
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic

Prevent plagiarism. Run a free check.

You can think of independent and dependent variables in terms of cause and effect: an independent variable is the variable you think is the cause , while a dependent variable is the effect .

In an experiment, you manipulate the independent variable and measure the outcome in the dependent variable. For example, in an experiment about the effect of nutrients on crop growth:

• The  independent variable  is the amount of nutrients added to the crop field.
• The  dependent variable is the biomass of the crops at harvest time.

Defining your variables, and deciding how you will manipulate and measure them, is an important part of experimental design .

A confounding variable , also called a confounder or confounding factor, is a third variable in a study examining a potential cause-and-effect relationship.

A confounding variable is related to both the supposed cause and the supposed effect of the study. It can be difficult to separate the true effect of the independent variable from the effect of the confounding variable.

In your research design , it’s important to identify potential confounding variables and plan how you will reduce their impact.

Quantitative variables are any variables where the data represent amounts (e.g. height, weight, or age).

Categorical variables are any variables where the data represent groups. This includes rankings (e.g. finishing places in a race), classifications (e.g. brands of cereal), and binary outcomes (e.g. coin flips).

You need to know what type of variables you are working with to choose the right statistical test for your data and interpret your results .

Discrete and continuous variables are two types of quantitative variables :

• Discrete variables represent counts (e.g. the number of objects in a collection).
• Continuous variables represent measurable amounts (e.g. water volume or weight).

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bevans, R. (2023, June 21). Types of Variables in Research & Statistics | Examples. Scribbr. Retrieved August 14, 2024, from https://www.scribbr.com/methodology/types-of-variables/

Is this article helpful?

Rebecca Bevans

Other students also liked, independent vs. dependent variables | definition & examples, confounding variables | definition, examples & controls, control variables | what are they & why do they matter, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

• More from M-W
• To save this word, you'll need to log in. Log In

Definition of experiment

 (Entry 1 of 2)

Definition of experiment  (Entry 2 of 2)

intransitive verb

• experimentation

Examples of experiment in a Sentence

These examples are programmatically compiled from various online sources to illustrate current usage of the word 'experiment.' Any opinions expressed in the examples do not represent those of Merriam-Webster or its editors. Send us feedback about these examples.

Word History

Middle English, "testing, proof, remedy," borrowed from Anglo-French esperiment, borrowed from Latin experīmentum "testing, experience, proof," from experīrī "to put to the test, attempt, have experience of, undergo" + -mentum -ment — more at experience entry 1

verbal derivative of experiment entry 1

14th century, in the meaning defined at sense 1a

1787, in the meaning defined above

Phrases Containing experiment

• control experiment
• controlled experiment
• experiment station
• pre - experiment
• thought experiment

Articles Related to experiment

This is the Difference Between a...

This is the Difference Between a Hypothesis and a Theory

In scientific reasoning, they're two completely different things

Dictionary Entries Near experiment

experiential time

experimental

Cite this Entry

“Experiment.” Merriam-Webster.com Dictionary , Merriam-Webster, https://www.merriam-webster.com/dictionary/experiment. Accessed 18 Aug. 2024.

Kids Definition

Kids definition of experiment.

Kids Definition of experiment  (Entry 2 of 2)

Medical Definition

Medical definition of experiment.

Medical Definition of experiment  (Entry 2 of 2)

More from Merriam-Webster on experiment

Nglish: Translation of experiment for Spanish Speakers

Britannica English: Translation of experiment for Arabic Speakers

Subscribe to America's largest dictionary and get thousands more definitions and advanced search—ad free!

Can you solve 4 words at once?

Word of the day.

See Definitions and Examples »

Get Word of the Day daily email!

Popular in Grammar & Usage

Plural and possessive names: a guide, commonly misspelled words, how to use em dashes (—), en dashes (–) , and hyphens (-), absent letters that are heard anyway, how to use accents and diacritical marks, popular in wordplay, 8 words for lesser-known musical instruments, it's a scorcher words for the summer heat, 7 shakespearean insults to make life more interesting, 10 words from taylor swift songs (merriam's version), 9 superb owl words, games & quizzes.

Tell us whether you accept cookies

We would like to use cookies to collect information about how you use ons.gov.uk .

We use this information to make the website work as well as possible and improve our services.

You’ve accepted all cookies. You can change your cookie settings at any time. Hide

• Release calendar
• Methodology

Guide to experimental statistics

How we define experimental statistics and why we publish statistics that are in development.

13 December 2023

This page was superseded by the Guide to official statistics in development on 13/12/2023. In September 2023, the Office for Statistics Regulation (OSR) changed the name of "experimental statistics" to "official statistics in development". Read more about this change in the OSR guidance on official statistics in development .

In this section

• How to interpret experimental statistics
• Labelling experimental statistics
• Why we publish experimental statistics
• Experimental statistics evaluation

1. How to interpret experimental statistics

Experimental statistics are official statistics that are in the testing phase and not yet fully developed.

Users should be aware that experimental statistics will potentially have a wider degree of uncertainty. The limitations of the statistics will be clearly explained within the release.

2. Labelling experimental statistics

The experimental statistics label is typically used where:

the statistics remain subject to testing of quality, volatility and ability to meet user needs

new methods are being tested and are still subject to modification or further evaluation

there is partial coverage (for example, of subgroups, regions or industries) at that stage of the development

there may be potential modification following user feedback about their usefulness and credibility

3. Why we publish experimental statistics

The reasons include:

consultation - experimental statistics are published to involve potential users and stakeholders at an early stage in assessing their quality and suitability

acclimatisation - where the experimental statistics are alternative versions of existing official statistics, it can help users become familiar with and understand the impact of new methods and approaches

use - experimental statistics can provide useful information for users as long as their nature is well-explained and understood

4. Experimental statistics evaluation

Once the evaluation of the experimental statistics is completed the label may be removed and the statistics can be published as official statistics. This decision will consider factors such as:

when it is judged that statistical methods used are robust

when coverage reaches a good level

when user feedback indicates that these statistics are useful and credible

when the defined development phase has ended