experiment minimum sample size

Power & Sample Size Calculator

Use this advanced sample size calculator to calculate the sample size required for a one-sample statistic, or for differences between two proportions or means (two independent samples). More than two groups supported for binomial data. Calculate power given sample size, alpha, and the minimum detectable effect (MDE, minimum effect of interest).

Experimental design

Data parameters

Related calculators

• Using the power & sample size calculator

Parameters for sample size and power calculations

Calculator output.

• Why is sample size determination important?
• What is statistical power?

Post-hoc power (Observed power)

• Sample size formula
• Types of null and alternative hypotheses in significance tests
• Absolute versus relative difference and why it matters for sample size determination

    Using the power & sample size calculator

This calculator allows the evaluation of different statistical designs when planning an experiment (trial, test) which utilizes a Null-Hypothesis Statistical Test to make inferences. It can be used both as a sample size calculator and as a statistical power calculator . Usually one would determine the sample size required given a particular power requirement, but in cases where there is a predetermined sample size one can instead calculate the power for a given effect size of interest.

1. Number of test groups. The sample size calculator supports experiments in which one is gathering data on a single sample in order to compare it to a general population or known reference value (one-sample), as well as ones where a control group is compared to one or more treatment groups ( two-sample, k-sample ) in order to detect differences between them. For comparing more than one treatment group to a control group the sample size adjustments based on the Dunnett's correction are applied. These are only approximately accurate and subject to the assumption of about equal effect size in all k groups, and can only support equal sample sizes in all groups and the control. Power calculations are not currently supported for more than one treatment group due to their complexity.

2. Type of outcome . The outcome of interest can be the absolute difference of two proportions (binomial data, e.g. conversion rate or event rate), the absolute difference of two means (continuous data, e.g. height, weight, speed, time, revenue, etc.), or the relative difference between two proportions or two means (percent difference, percent change, etc.). See Absolute versus relative difference for additional information. One can also calculate power and sample size for the mean of just a single group. The sample size and power calculator uses the Z-distribution (normal distribution) .

3. Baseline The baseline mean (mean under H 0 ) is the number one would expect to see if all experiment participants were assigned to the control group. It is the mean one expects to observe if the treatment has no effect whatsoever.

4. Minimum Detectable Effect . The minimum effect of interest, which is often called the minimum detectable effect ( MDE , but more accurately: MRDE, minimum reliably detectable effect) should be a difference one would not like to miss , if it existed. It can be entered as a proportion (e.g. 0.10) or as percentage (e.g. 10%). It is always relative to the mean/proportion under H 0 ± the superiority/non-inferiority or equivalence margin. For example, if the baseline mean is 10 and there is a superiority alternative hypothesis with a superiority margin of 1 and the minimum effect of interest relative to the baseline is 3, then enter an MDE of 2 , since the MDE plus the superiority margin will equal exactly 3. In this case the MDE (MRDE) is calculated relative to the baseline plus the superiority margin, as it is usually more intuitive to be interested in that value.

If entering means data, one needs to specify the mean under the null hypothesis (worst-case scenario for a composite null) and the standard deviation of the data (for a known population or estimated from a sample).

5. Type of alternative hypothesis . The calculator supports superiority , non-inferiority and equivalence alternative hypotheses. When the superiority or non-inferiority margin is zero, it becomes a classical left or right sided hypothesis, if it is larger than zero then it becomes a true superiority / non-inferiority design. The equivalence margin cannot be zero. See Types of null and alternative hypothesis below for an in-depth explanation.

6. Acceptable error rates . The type I error rate, α , should always be provided. Power, calculated as 1 - β , where β is the type II error rate, is only required when determining sample size. For an in-depth explanation of power see What is statistical power below. The type I error rate is equivalent to the significance threshold if one is doing p-value calculations and to the confidence level if using confidence intervals.

The sample size calculator will output the sample size of the single group or of all groups, as well as the total sample size required. If used to solve for power it will output the power as a proportion and as a percentage.

    Why is sample size determination important?

While this online software provides the means to determine the sample size of a test, it is of great importance to understand the context of the question, the "why" of it all.

Estimating the required sample size before running an experiment that will be judged by a statistical test (a test of significance, confidence interval, etc.) allows one to:

• determine the sample size needed to detect an effect of a given size with a given probability
• be aware of the magnitude of the effect that can be detected with a certain sample size and power
• calculate the power for a given sample size and effect size of interest

This is crucial information with regards to making the test cost-efficient. Having a proper sample size can even mean the difference between conducting the experiment or postponing it for when one can afford a sample of size that is large enough to ensure a high probability to detect an effect of practical significance.

For example, if a medical trial has low power, say less than 80% (β = 0.2) for a given minimum effect of interest, then it might be unethical to conduct it due to its low probability of rejecting the null hypothesis and establishing the effectiveness of the treatment. Similarly, for experiments in physics, psychology, economics, marketing, conversion rate optimization, etc. Balancing the risks and rewards and assuring the cost-effectiveness of an experiment is a task that requires juggling with the interests of many stakeholders which is well beyond the scope of this text.

    What is statistical power?

Statistical power is the probability of rejecting a false null hypothesis with a given level of statistical significance , against a particular alternative hypothesis. Alternatively, it can be said to be the probability to detect with a given level of significance a true effect of a certain magnitude. This is what one gets when using the tool in "power calculator" mode. Power is closely related with the type II error rate: β, and it is always equal to (1 - β). In a probability notation the type two error for a given point alternative can be expressed as [1] :

β(T α ; μ 1 ) = P(d(X) ≤ c α ; μ = μ 1 )

It should be understood that the type II error rate is calculated at a given point, signified by the presence of a parameter for the function of beta. Similarly, such a parameter is present in the expression for power since POW = 1 - β [1] :

POW(T α ; μ 1 ) = P(d(X) > c α ; μ = μ 1 )

In the equations above c α represents the critical value for rejecting the null (significance threshold), d(X) is a statistical function of the parameter of interest - usually a transformation to a standardized score, and μ 1 is a specific value from the space of the alternative hypothesis.

One can also calculate and plot the whole power function, getting an estimate of the power for many different alternative hypotheses. Due to the S-shape of the function, power quickly rises to nearly 100% for larger effect sizes, while it decreases more gradually to zero for smaller effect sizes. Such a power function plot is not yet supported by our statistical software, but one can calculate the power at a few key points (e.g. 10%, 20% ... 90%, 100%) and connect them for a rough approximation.

Statistical power is directly and inversely related to the significance threshold. At the zero effect point for a simple superiority alternative hypothesis power is exactly 1 - α as can be easily demonstrated with our power calculator. At the same time power is positively related to the number of observations, so increasing the sample size will increase the power for a given effect size, assuming all other parameters remain the same.

Power calculations can be useful even after a test has been completed since failing to reject the null can be used as an argument for the null and against particular alternative hypotheses to the extent to which the test had power to reject them. This is more explicitly defined in the severe testing concept proposed by Mayo & Spanos (2006).

Computing observed power is only useful if there was no rejection of the null hypothesis and one is interested in estimating how probative the test was towards the null . It is absolutely useless to compute post-hoc power for a test which resulted in a statistically significant effect being found [5] . If the effect is significant, then the test had enough power to detect it. In fact, there is a 1 to 1 inverse relationship between observed power and statistical significance, so one gains nothing from calculating post-hoc power, e.g. a test planned for α = 0.05 that passed with a p-value of just 0.0499 will have exactly 50% observed power (observed β = 0.5).

I strongly encourage using this power and sample size calculator to compute observed power in the former case, and strongly discourage it in the latter.

    Sample size formula

The formula for calculating the sample size of a test group in a one-sided test of absolute difference is:

where Z 1-α is the Z-score corresponding to the selected statistical significance threshold α , Z 1-β is the Z-score corresponding to the selected statistical power 1-β , σ is the known or estimated standard deviation, and δ is the minimum effect size of interest. The standard deviation is estimated analytically in calculations for proportions, and empirically from the raw data for other types of means.

The formula applies to single sample tests as well as to tests of absolute difference between two samples. A proprietary modification is employed when calculating the required sample size in a test of relative difference . This modification has been extensively tested under a variety of scenarios through simulations.

    Types of null and alternative hypotheses in significance tests

When doing sample size calculations, it is important that the null hypothesis (H 0 , the hypothesis being tested) and the alternative hypothesis is (H 1 ) are well thought out. The test can reject the null or it can fail to reject it. Strictly logically speaking it cannot lead to acceptance of the null or to acceptance of the alternative hypothesis. A null hypothesis can be a point one - hypothesizing that the true value is an exact point from the possible values, or a composite one: covering many possible values, usually from -∞ to some value or from some value to +∞. The alternative hypothesis can also be a point one or a composite one.

In a Neyman-Pearson framework of NHST (Null-Hypothesis Statistical Test) the alternative should exhaust all values that do not belong to the null, so it is usually composite. Below is an illustration of some possible combinations of null and alternative statistical hypotheses: superiority, non-inferiority, strong superiority (margin > 0), equivalence.

All of these are supported in our power and sample size calculator.

Careful consideration has to be made when deciding on a non-inferiority margin, superiority margin or an equivalence margin . Equivalence trials are sometimes used in clinical trials where a drug can be performing equally (within some bounds) to an existing drug but can still be preferred due to less or less severe side effects, cheaper manufacturing, or other benefits, however, non-inferiority designs are more common. Similar cases exist in disciplines such as conversion rate optimization [2] and other business applications where benefits not measured by the primary outcome of interest can influence the adoption of a given solution. For equivalence tests it is assumed that they will be evaluated using a two one-sided t-tests (TOST) or z-tests, or confidence intervals.

Note that our calculator does not support the schoolbook case of a point null and a point alternative, nor a point null and an alternative that covers all the remaining values. This is since such cases are non-existent in experimental practice [3][4] . The only two-sided calculation is for the equivalence alternative hypothesis, all other calculations are one-sided (one-tailed) .

    Absolute versus relative difference and why it matters for sample size determination

When using a sample size calculator it is important to know what kind of inference one is looking to make: about the absolute or about the relative difference, often called percent effect, percentage effect, relative change, percent lift, etc. Where the fist is μ 1 - μ the second is μ 1 -μ / μ or μ 1 -μ / μ x 100 (%). The division by μ is what adds more variance to such an estimate, since μ is just another variable with random error, therefore a test for relative difference will require larger sample size than a test for absolute difference. Consequently, if sample size is fixed, there will be less power for the relative change equivalent to any given absolute change.

For the above reason it is important to know and state beforehand if one is going to be interested in percentage change or if absolute change is of primary interest. Then it is just a matter of fliping a radio button.

    References

1 Mayo D.G., Spanos A. (2010) – "Error Statistics", in P. S. Bandyopadhyay & M. R. Forster (Eds.), Philosophy of Statistics, (7, 152–198). Handbook of the Philosophy of Science . The Netherlands: Elsevier.

2 Georgiev G.Z. (2017) "The Case for Non-Inferiority A/B Tests", [online] https://blog.analytics-toolkit.com/2017/case-non-inferiority-designs-ab-testing/ (accessed May 7, 2018)

3 Georgiev G.Z. (2017) "One-tailed vs Two-tailed Tests of Significance in A/B Testing", [online] https://blog.analytics-toolkit.com/2017/one-tailed-two-tailed-tests-significance-ab-testing/ (accessed May 7, 2018)

4 Hyun-Chul Cho Shuzo Abe (2013) "Is two-tailed testing for directional research hypotheses tests legitimate?", Journal of Business Research 66:1261-1266

5 Lakens D. (2014) "Observed power, and what to do if your editor asks for post-hoc power analyses" [online] http://daniellakens.blogspot.bg/2014/12/observed-power-and-what-to-do-if-your.html (accessed May 7, 2018)

Cite this calculator & page

If you'd like to cite this online calculator resource and information as provided on the page, you can use the following citation: Georgiev G.Z., "Sample Size Calculator" , [online] Available at: https://www.gigacalculator.com/calculators/power-sample-size-calculator.php URL [Accessed Date: 07 Sep, 2024].

Our statistical calculators have been featured in scientific papers and articles published in high-profile science journals by:

The author of this tool

     Statistical calculators

• Skip to secondary menu
• Skip to main content
• Skip to primary sidebar

Statistics By Jim

Making statistics intuitive

Sample Size Essentials: The Foundation of Reliable Statistics

By Jim Frost 2 Comments

What is Sample Size?

Sample size is the number of observations or data points collected in a study. It is a crucial element in any statistical analysis because it is the foundation for drawing inferences and conclusions about a larger population .

Imagine you’re tasting a new brand of cookies. Sampling just one cookie might not give you a true sense of the overall flavor—what if you picked the only burnt one? Similarly, in statistics, the sample size determines how well your study represents the larger group. A larger sample size can mean the difference between a snapshot and a panorama, providing a clearer, more accurate picture of the reality you’re studying.

In this blog post, learn why adequate sample sizes are not just a statistical nicety but a fundamental component of trustworthy research. However, large sample sizes can’t fix all problems. By understanding the impact of sample size on your results, you can make informed decisions about your research design and have more confidence in your findings .

Benefits of a Large Sample Size

A large sample size can significantly enhance the reliability and validity of study results. We’re primarily looking at how well representative samples reflect the populations from which the researchers drew them. Here are several key benefits.

Increased Precision

Larger samples tend to yield more precise estimates of the population  parameters . Larger samples reduce the effect of random fluctuations in the data, narrowing the margin of error around the estimated values.

Estimate precision refers to how closely the results obtained from a sample align with the actual population values. A larger sample size tends to yield more precise estimates because it reduces the effect of random variability within the sample. The more data points you have, the smaller the margin of error and the closer you are to capturing the correct value of the population parameter.

For example, estimating the average height of adults using a larger sample tends to give an estimate closer to the actual average than using a smaller sample.

Learn more about Statistics vs. Parameters , Margin of Error , and Confidence Intervals .

Greater Statistical Power

The power of a statistical test is its capability to detect an effect if there is one, such as a difference between groups or a correlation between variables. Larger samples increase the likelihood of detecting actual effects.

Statistical power is the probability that a study will detect an effect when one exists. The sample size directly influences it;  a larger sample size increases statistical power . Studies with more data are more likely to detect existing differences or relationships.

For instance, in testing whether a new drug is more effective than an existing one, a larger sample can more reliably detect small but real improvements in efficacy .

Better Generalizability

With a larger sample, there is a higher chance that the sample adequately represents the diversity of the population, improving the generalizability of the findings to the population.

Consider a national survey gauging public opinion on a policy. A larger sample captures a broader range of demographic groups and opinions.

Learn more about Representative Samples .

Reduced Impact of Outliers

In a large sample, outliers have less impact on the overall results because many observations dilute their influence. The numerous data points stabilize the averages and other statistical estimates, making them more representative of the general population.

If measuring income levels within a region, a few very high incomes will distort the average less in a larger sample than in a smaller one .

Learn more about 5 Ways to Identify Outliers .

The Limits of Larger Sample Sizes: A Cautionary Note

While larger sample sizes offer numerous advantages, such as increased precision and statistical power, it’s important to understand their limitations. They are not a panacea for all research challenges. Crucially, larger sample sizes do not automatically correct for biases in sampling methods , other forms of bias, or fundamental errors in study design. Ignoring these issues can lead to misleading conclusions, regardless of how many data points are collected.

Sampling Bias

Even a large sample is misleading if it’s not representative of the population. For instance, if a study on employee satisfaction only includes responses from headquarters staff but not remote workers, increasing the number of respondents won’t address the inherent bias in missing a significant segment of the workforce.

Learn more about Sampling Bias: Definition & Examples .

Other Forms of Bias

Biases related to data collection methods, survey question phrasing, or data analyst subjectivity can still skew results. If the underlying issues are not addressed, a larger sample size might magnify these biases instead of mitigating them.

Errors in Study Design

Simply adding more data points will not overcome a flawed experimental design . For example, increasing the sample size will not clarify the causal relationships if the design doesn’t control a confounding variable .

Large Sample Sizes are Expensive!

Additionally, it is possible to have too large a sample size. Larger sizes come with their own challenges, such as higher costs and logistical complexities. You get to a point of diminishing returns where you have a very large sample that will detect such small effects that they’re meaningless in a practical sense.

The takeaway here is that researchers must exercise caution and not rely solely on a large  sample size to safeguard the reliability and validity of their results. An adequate amount of data must be paired with an appropriate sampling method, a robust study design, and meticulous execution to truly understand and accurately represent the phenomena being studied .

Sample Size Calculation

Statisticians have devised quantitative ways to find a good sample size. You want a large enough sample to have a reasonable chance of detecting a meaningful effect when it exists but not too large to be overly expensive.

In general, these methods focus on using the population’s variability . More variable populations require larger samples to assess them. Let’s go back to the cookie example to see why.

If all cookies in a population are identical (zero variability), you only need to sample one cookie to know what the average cookie is like for the entire population. However, suppose there’s a little variability because some cookies are cooked perfectly while others are overcooked. You’ll need a larger sample size to understand the ratio of the perfect to overcooked cookies.

Now, instead of just those two types, you have an entire range of how much they are over and undercooked. And some use sweeter chocolate chips than others. You’ll need an even larger sample to understand the increased variability and know what an average cookie is really like.

Power and sample size analysis quantifies the population’s variability. Hence, you’ll often need a variability estimate to perform this type of analysis. These calculations also frequently factor in the smallest practically meaningful effect size you want to detect, so you’ll use a manageable sample size.

To learn more about determining how to find a sample size, read my following articles :

• How to Calculate Sample Size
• What is Power in Statistics?

Sample Size Summary

Understanding the implications of sample size is fundamental to conducting robust statistical analysis. While larger samples provide more reliable and precise estimates, smaller samples can compromise the validity of statistical inferences.

Always remember that the breadth of your sample profoundly influences the strength of your conclusions. So, whether conducting a simple survey or a complex experimental study, consider your sample size carefully. Your research’s integrity depends on it.

Consequently, the effort to achieve an adequate sample size is a worthwhile investment in the precision and credibility of your research .

Share this:

Reader Interactions

August 29, 2024 at 7:00 am

A problem which ought to be considered when running an opinion poll: Is the group of people who consent to answer strictly comparable to the group who do not consent?. If not, then there may be systematic bias

July 17, 2024 at 11:11 am

When I used survey data, we had a clear, conscious sampling method and the distinction made sense. However, with other types of data such as performance or sales data, I’m confused about the distinction. We have all the data of everyone who did the work, so by that understanding, we aren’t doing any sampling. However, is there a ‘hidden’ population of everyone who could potentially do that work? If we take a point in time, such as just first quarter performance, is that a sample or something else? I regularly see people just go ahead and apply the same statistics to both, suggesting that this is a ‘sample’, but I’m not sure what it’s a sample of or how!

Comments and Questions Cancel reply

Sample Size Calculator

Determines the minimum number of subjects for adequate study power, clincalc.com » statistics » sample size calculator, study group design.

Two independent study groups

One study group vs. population

Primary Endpoint

Dichotomous (yes/no)

Continuous (means)

Statistical Parameters

Dichotomous Endpoint, Two Independent Sample Study

About This Calculator

This calculator uses a number of different equations to determine the minimum number of subjects that need to be enrolled in a study in order to have sufficient statistical power to detect a treatment effect. 1

Before a study is conducted, investigators need to determine how many subjects should be included. By enrolling too few subjects, a study may not have enough statistical power to detect a difference (type II error). Enrolling too many patients can be unnecessarily costly or time-consuming.

Generally speaking, statistical power is determined by the following variables:

• Baseline Incidence: If an outcome occurs infrequently, many more patients are needed in order to detect a difference.
• Population Variance: The higher the variance (standard deviation), the more patients are needed to demonstrate a difference.
• Treatment Effect Size: If the difference between two treatments is small, more patients will be required to detect a difference.
• Alpha: The probability of a type-I error -- finding a difference when a difference does not exist. Most medical literature uses an alpha cut-off of 5% (0.05) -- indicating a 5% chance that a significant difference is actually due to chance and is not a true difference.
• Beta: The probability of a type-II error -- not detecting a difference when one actually exists. Beta is directly related to study power (Power = 1 - β). Most medical literature uses a beta cut-off of 20% (0.2) -- indicating a 20% chance that a significant difference is missed.

Post-Hoc Power Analysis

To calculate the post-hoc statistical power of an existing trial, please visit the post-hoc power analysis calculator .

References and Additional Reading

• Rosner B. Fundamentals of Biostatistics . 7th ed. Boston, MA: Brooks/Cole; 2011.

Related Calculators

• Post-hoc Power Calculator

Mailing List

New and popular, cite this page.

Show AMA citation

We've filled out some of the form to show you this clinical calculator in action. Click here to start from scratch and enter your own patient data.

tools4dev Practical tools for international development

How to choose a sample size (for the statistically challenged)

One of the most common questions I get asked by people doing surveys in international development is “how big should my sample size be?”. While there are many  sample size calculators and statistical guides available, those who never did statistics at university (or have forgotten it all) may find them intimidating or difficult to use.

If this sounds like you, then keep reading. This guide will explain how to choose a sample size for a basic survey without any of the complicated formulas. For more easy rules of thumb regarding sample sizes for other situations, I highly recommend  Sample size: A rough guide  by Ronán Conroy and   The Survey Research Handbook  by Pamela Alreck and Robert Settle.

This article is a short introduction to the topic for a more in-depth coverage of the topic consider enrolling in the free online course offered by University of Florida .

This advice is for:

• Basic surveys such as feedback forms, needs assessments, opinion surveys, etc. conducted as part of a program.
• Surveys that use random sampling.

This advice is NOT for:

• Research studies conducted by universities, research firms, etc.
• Complex or very large surveys, such as national household surveys.
• Surveys to compare between an intervention and control group or before and after a program (for this situation  Sample size: A rough guide ).
• Surveys that use non-random sampling, or a special type of sampling such as cluster or stratified sampling (for these situations see  Sample size: A rough guide  and the UN guidelines on household surveys ).
• Surveys where you plan to use fancy statistics to analyse the results, such as multivariate analysis (if you know how to do such fancy statistics then you should already know how to choose a sample size).

The minimum sample size is 100

Most statisticians agree that the minimum sample size to get any kind of meaningful result is 100. If your population is less than 100 then you really need to survey all of them.

A good maximum sample size is usually 10% as long as it does not exceed 1000

A good maximum sample size is usually around 10% of the population, as long as this does not exceed 1000. For example, in a population of 5000, 10% would be 500. In a population of 200,000, 10% would be 20,000. This exceeds 1000, so in this case the maximum would be 1000.

Even in a population of 200,000, sampling 1000 people will normally give a fairly accurate result. Sampling more than 1000 people won’t add much to the accuracy given the extra time and money it would cost.

Choose a number between the minimum and maximum depending on the situation

Suppose that you want to survey students at a school which has 6000 pupils enrolled. The minimum sample would be 100. This would give you a rough, but still useful, idea about their opinions. The maximum sample would be 600, which would give you a fairly accurate idea about their opinions.

Choose a number closer to the  minimum  if:

• You have limited time and money.
• You only need a rough estimate of the results.
• You don’t plan to divide the sample into different groups during the analysis, or you only plan to use a few large subgroups (e.g. males / females).
• You think most people will give similar answers.
• The decisions that will be made based on the results do not have significant consequences.

Choose a number closer to the  maximum  if:

• You have the time and money to do it.
• It is very important to get accurate results.
• You plan to divide the sample into many different groups during the analysis (e.g. different age groups, socio-economic levels, etc).
• You think people are likely to give very different answers.
• The decisions that will be made based on the results of the survey are important, expensive or have serious consequences.

In practice most people normally want the results to be as accurate as possible, so the limiting factor is usually time and money. In the example above, if you had the time and money to survey all 600 students then that will give you a fairly accurate result. If you don’t have enough time or money then just choose the largest number that you can manage, as long as it’s more than 100.

If you would like to learn more about Survey Data Collection consider taking the free course offered by University of Michigan and University of Maryland. Enroll here.

If you want to be a bit more scientific then use this table

While the previous rules of thumb are perfectly acceptable for most basic surveys, sometimes you need to sound more “scientific” in order to be taken seriously. In that case you can use the following table. Simply choose the column that most closely matches your population size. Then choose the row that matches the level of error you’re willing to accept in the results.

You will see on this table that the smallest samples are still around 100, and the biggest sample (for a population of more than 5000) is still around 1000. The same general principles apply as before – if you plan to divide the results into lots of sub-groups, or the decisions to be made are very important, you should pick a bigger sample.

Note: This table can only be used for basic surveys to measure what proportion of the population have a particular characteristic (e.g. what proportion of farmers are using fertiliser, what proportion of women believe myths about family planning, etc). It can’t be used if you are trying to compare two groups (e.g. control versus intervention) or two points in time (e.g. baseline and endline surveys). See  Sample size: A rough guide  for other tables that can be used in these cases.

Relax and stop worrying about the formulas

It’s a dirty little secret among statisticians that sample size formulas often require you to have information in advance that you don’t normally have. For example, you typically need to know (in numerical terms) how much the answers in the survey are likely to vary between individuals (if you knew that in advance then you wouldn’t be doing a survey!).

So even though it’s theoretically possible to calculate a sample size using a formula, in many cases experts still end up relying rules of thumb plus a good deal of common sense and pragmatism. That means you shouldn’t worry too much if you can’t use fancy maths to choose your sample size – you’re in good company.

Once you’ve chosen a sample size, don’t forget to write good survey questions ,  design the survey form properly  and   pre-test and pilot your questionnaire .

Photo by  James Cridland

Tags Monitoring & Evaluation

About Piroska Bisits Bullen

Related Articles

What can international development learn from tech start-ups?

13 May 2021

Social Enterprise Business Plan Template

12 May 2021

How to write an M&E framework – Free video tutorial & templates

10 September 2017

• Data Stories
• Integrations

CAPABILITIES

• Survey Analysis
• Data Visualization
• Dashboarding
• Automatic Updating
• PowerPoint Reporting
• Finding Data Stories
• Data Cleaning
• New Product Development
• Tracking Analysis
• Customer Feedback
• Segmentation
• Brand Analytics
• Pricing Research
• Advertising Research
• Statistical Testing
• Text Analysis
• Factor Analysis
• Driver Analysis
• Correspondence Analysis
• Cluster & Latent Class
• Success Stories
• Demo Videos
• Book a Demo
• Ebooks & Webinars
• Help Center
• Product Roadmap

UPCOMING WEBINAR

Register now

• Book a demo
• Choice Modeling/Conjoint Analysis
• Dimension Reduction
• Principal Component Analysis
• Machine Learning
• Linear Regression
• Cluster Analysis
• Latent Class Analysis
• Customer Feedback Surveys
• Dive Into Data
• Data Stories Tutorials
• Account Administration
• Beginner's guides
• Dashboard Best Practices
• Getting Started
• Reporting/Exporting
• Troubleshooting Common Issues
• JavaScript How To...
• JavaScript in Displayr
• R How To...
• R in Displayr
• Visualizations

How to calculate minimum sample size for a survey or experiment

A practical question when designing a customer feedback survey or experiment is to work out the required sample size. That is, what is the smallest number of data points required in the survey or experiment? There are three basic approaches: rules of thumb based on industry standards, working backwards from budget, and working backwards from confidence intervals. Each of these approaches is useful in some circumstances.

Working out sample size based on rules of thumb

Different industries have different rules of thumb when it comes to testing. These rules of thumb are not entirely made up; their logic relates to the confidence intervals analyses described later in this article.

Some examples of common rules of thumb are:

• Studies should involve sample sizes of at least 100 in each key group of interest. For example, if you are doing an AB test, then you would typically want a minimum sample size of 200, with 100 in each group. An exception to this is when testing anything where the actual rate being tested is small. For example, if testing conversion rates, where the typical conversion rate may be 2%, a substantially larger sample sizes are required.
• Sensory research studies, where people are given food products and asked to taste them, typically have a sample size of at least 60 per key group.
• In commercial market research, samples of less than 300 are usually considered to be too small. For strategically important studies, sample size of 1,000 are typically required. A minimum sample size of 200 per segment is considered safe for market segmentation studies (e.g., if you are doing a segmentation study and you are OK with having up to 6 segments, then a sample size of 1,200 is desirable).
• For nation-wide political polls, sample sizes of 1,000 or more are typically required. The logic of sampling means that whether a poll is nationwide or not should not be a consideration, but, in practice, it is usually taken into account.
• Early feasibility studies for medical devices have sample sizes of 10, traditional feasibility studies have sample sizes of 20 to 30. [1]
• If performing inspections, such as of outputs from a manufacturing plant or crops, it is common to apply a rule that sample size should be Sqrt(N) + 1 , where N is the population size.

Working out sample size from costs

A second common approach is to identify the budget and work backwards, using the following formula:

Sample size  = (Total budget - fixed costs)/cost per data point

This may sound crude, but the budget for a study is a way of working out the appetite for risk of the organization that has commissioned the study, and, as discussed in the next section, this is at the heart of determining sample size.

Working out sample size from confidence intervals

One of the reasons that the minimum sample sizes guidelines vary so much is that the true minimum sample size required for any study depends on the signal to noise ratio of the data. If the data intrinsically has a high level of noise in it, such as political polls and market research, then a large sample is required. In tightly controlled environments, such as those used in sensory studies, there is less noise and thus smaller sample sizes are acceptable. When testing medical devices, the outcome is to see if the device is problem free or not, rather than to estimate any specific rate, so an even smaller sample size is appropriate.

One formal method for working out sample sizes is to have researchers specify the required level of uncertainty they can deal with, expressed a confidence interval, and work out the sample size required to obtain this. For example, see here  and here for examples and discussions, respectively.

This is the textbook solution to working out sample size, and there are lots of nice theoretical concepts to help (e.g., power analysis). However, in practice, the approach only works when you have a good idea what the likely result will be and what the likely uncertainty will be (i.e., sampling error), and this is often not the case, outside of the world of clinical trials.

Learn how to statistically test Net Promoter Score in Displayr

[1] https://www.fda.gov/downloads/MedicalDevices/NewsEvents/WorkshopsConferences/UCM424735.pdf

Prepare to watch, play, learn, make, and discover!

Get access to all the premium content on displayr, last question, we promise, what type of survey data are you working with (select all that apply).

Market research Social research (commercial) Customer feedback Academic research Polling Employee research I don't have survey data

Sample Size Calculator

Find out the sample size.

This calculator computes the minimum number of necessary samples to meet the desired statistical constraints.

Find Out the Margin of Error

This calculator gives out the margin of error or confidence interval of observation or survey.

Related Standard Deviation Calculator | Probability Calculator

In statistics, information is often inferred about a population by studying a finite number of individuals from that population, i.e. the population is sampled, and it is assumed that characteristics of the sample are representative of the overall population. For the following, it is assumed that there is a population of individuals where some proportion, p , of the population is distinguishable from the other 1-p in some way; e.g., p may be the proportion of individuals who have brown hair, while the remaining 1-p have black, blond, red, etc. Thus, to estimate p in the population, a sample of n individuals could be taken from the population, and the sample proportion, p̂ , calculated for sampled individuals who have brown hair. Unfortunately, unless the full population is sampled, the estimate p̂ most likely won't equal the true value p , since p̂ suffers from sampling noise, i.e. it depends on the particular individuals that were sampled. However, sampling statistics can be used to calculate what are called confidence intervals, which are an indication of how close the estimate p̂ is to the true value p .

Statistics of a Random Sample

The uncertainty in a given random sample (namely that is expected that the proportion estimate, p̂ , is a good, but not perfect, approximation for the true proportion p ) can be summarized by saying that the estimate p̂ is normally distributed with mean p and variance p(1-p)/n . For an explanation of why the sample estimate is normally distributed, study the Central Limit Theorem . As defined below, confidence level, confidence intervals, and sample sizes are all calculated with respect to this sampling distribution. In short, the confidence interval gives an interval around p in which an estimate p̂ is "likely" to be. The confidence level gives just how "likely" this is – e.g., a 95% confidence level indicates that it is expected that an estimate p̂ lies in the confidence interval for 95% of the random samples that could be taken. The confidence interval depends on the sample size, n (the variance of the sample distribution is inversely proportional to n , meaning that the estimate gets closer to the true proportion as n increases); thus, an acceptable error rate in the estimate can also be set, called the margin of error, ε , and solved for the sample size required for the chosen confidence interval to be smaller than e ; a calculation known as "sample size calculation."

Confidence Level

The confidence level is a measure of certainty regarding how accurately a sample reflects the population being studied within a chosen confidence interval. The most commonly used confidence levels are 90%, 95%, and 99%, which each have their own corresponding z-scores (which can be found using an equation or widely available tables like the one provided below) based on the chosen confidence level. Note that using z-scores assumes that the sampling distribution is normally distributed, as described above in "Statistics of a Random Sample." Given that an experiment or survey is repeated many times, the confidence level essentially indicates the percentage of the time that the resulting interval found from repeated tests will contain the true result.

Confidence Interval

In statistics, a confidence interval is an estimated range of likely values for a population parameter, for example, 40 ± 2 or 40 ± 5%. Taking the commonly used 95% confidence level as an example, if the same population were sampled multiple times, and interval estimates made on each occasion, in approximately 95% of the cases, the true population parameter would be contained within the interval. Note that the 95% probability refers to the reliability of the estimation procedure and not to a specific interval. Once an interval is calculated, it either contains or does not contain the population parameter of interest. Some factors that affect the width of a confidence interval include: size of the sample, confidence level, and variability within the sample.

There are different equations that can be used to calculate confidence intervals depending on factors such as whether the standard deviation is known or smaller samples (n<30) are involved, among others. The calculator provided on this page calculates the confidence interval for a proportion and uses the following equations:

Within statistics, a population is a set of events or elements that have some relevance regarding a given question or experiment. It can refer to an existing group of objects, systems, or even a hypothetical group of objects. Most commonly, however, population is used to refer to a group of people, whether they are the number of employees in a company, number of people within a certain age group of some geographic area, or number of students in a university's library at any given time.

It is important to note that the equation needs to be adjusted when considering a finite population, as shown above. The (N-n)/(N-1) term in the finite population equation is referred to as the finite population correction factor, and is necessary because it cannot be assumed that all individuals in a sample are independent. For example, if the study population involves 10 people in a room with ages ranging from 1 to 100, and one of those chosen has an age of 100, the next person chosen is more likely to have a lower age. The finite population correction factor accounts for factors such as these. Refer below for an example of calculating a confidence interval with an unlimited population.

EX: Given that 120 people work at Company Q, 85 of which drink coffee daily, find the 99% confidence interval of the true proportion of people who drink coffee at Company Q on a daily basis.

Sample size: how many participants do I need in my research? *

Jeovany martínez-mesa.

1 Latin American Cooperative Oncology Group - Porto Alegre (RS), Brazil.

David Alejandro González-Chica

2 Universidade Federal de Santa Catarina (UFSC) - Florianópolis (SC), Brazil.

João Luiz Bastos

Renan rangel bonamigo.

3 Universidade Federal de Ciências da Saúde de Porto Alegre (UFCSPA) - Porto Alegre (RS), Brazil.

Rodrigo Pereira Duquia

The importance of estimating sample sizes is rarely understood by researchers, when planning a study. This paper aims to highlight the centrality of sample size estimations in health research. Examples that help in understanding the basic concepts involved in their calculation are presented. The scenarios covered are based more on the epidemiological reasoning and less on mathematical formulae. Proper calculation of the number of participants in a study diminishes the likelihood of errors, which are often associated with adverse consequences in terms of economic, ethical and health aspects.

INTRODUCTION

Investigations in the health field are oriented by research problems or questions, which should be clearly defined in the study project. Sample size calculation is an essential item to be included in the project to reduce the probability of error, respect ethical standards, define the logistics of the study and, last but not least, improve its success rates, when evaluated by funding agencies.

Let us imagine that a group of investigators decides to study the frequency of sunscreen use and how the use of this product is distributed in the "population". In order to carry out this task, the authors define two research questions, each of which involving a distinct sample size calculation: 1) What is the proportion of people that use sunscreen in the population?; and, 2) Are there differences in the use of sunscreen between men and women, or between individuals that are white or of another skin color group, or between the wealthiest and the poorest, or between people with more and less years of schooling? Before doing the calculations, it will be necessary to review a few fundamental concepts and identify which are the required parameters to determine them.

WHAT DO WE MEAN, WHEN WE TALK ABOUT POPULATIONS?

First of all, we must define what is a population . Population is the group of individuals restricted to a geographical region (neighborhood, city, state, country, continent etc.), or certain institutions (hospitals, schools, health centers etc.), that is, a set of individuals that have at least one characteristic in common. The target population corresponds to a portion of the previously mentioned population, about which one intends to draw conclusions, that is to say, it is a part of the population whose characteristics are an object of interest of the investigator. Finally, study population is that which will actually be part of the study, which will be evaluated and will allow conclusions to be drawn about the target population, as long as it is representative of the latter. Figure 1 demonstrates how these concepts are interrelated.

Graphic representation of the concepts of population, target population and study population

We will now separately consider the required parameters for sample size calculation in studies that aim at estimating the frequency of events (prevalence of health outcomes or behaviors, for example), to test associations between risk/protective factors and dichotomous health conditions (yes/no), as well as with health outcomes measured in numerical scales. 1 The formulas used for these calculations may be obtained from different sources - we recommend using the free online software OpenEpi ( www.openepi.com ). 2

WHICH PARAMETERS DOES SAMPLE SIZE CALCULATION DEPEND UPON FOR A STUDY THAT AIMS AT ESTIMATING THE FREQUENCY OF HEALTH OUTCOMES, BEHAVIORS OR CONDITIONS?

When approaching the first research question defined at the beginning of this article (What is the proportion of people that use sunscreen?), the investigators need to conduct a prevalence study. In order to do this, some parameters must be defined to calculate the sample size, as demonstrated in chart 1 .

Description of different parameters to be considered in the calculation of sample size for a study aiming at estimating the frequency of health ouctomes, behaviors or conditions

Chart 2 presents some sample size simulations, according to the outcome prevalence, sample error and the type of target population investigated. The same basic question was used in this table (prevalence of sunscreen use), but considering three different situations (at work, while doing sports or at the beach), as in the study by Duquia et al. conducted in the city of Pelotas, state of Rio Grande do Sul, in 2005. 3

Sample size calculation to estimate the frequency (prevalence) of sunscreen use in the population, considering different scenarios but keeping the significance level (95%) and the design effect (1.0) constant

p.p.= percentage points

The calculations show that, by holding the sample error and the significance level constant, the higher the expected prevalence, the larger will be the required sample size. However, when the expected prevalence surpasses 50%, the required sample size progressively diminishes - the sample size for an expected prevalence of 10% is the same as that for an expected prevalence of 90%.

The investigator should also define beforehand the precision level to be accepted for the investigated event (sample error) and the confidence level of this result (usually 95%). Chart 2 demonstrates that, holding the expected prevalence constant, the higher the precision (smaller sample error) and the higher the confidence level (in this case, 95% was considered for all calculations), the larger also will be the required sample size.

Chart 2 also demonstrates that there is a direct relationship between the target population size and the number of individuals to be included in the sample. Nevertheless, when the target population size is sufficiently large, that is, surpasses an arbitrary value (for example, one million individuals), the resulting sample size tends to stabilize. The smaller the target population, the larger the sample will be; in some cases, the sample may even correspond to the total number of individuals from the target population - in these cases, it may be more convenient to study the entire target population, carrying out a census survey, rather than a study based on a sample of the population.

SAMPLE CALCULATION TO TEST THE ASSOCIATION BETWEEN TWO VARIABLES: HYPOTHESES AND TYPES OF ERROR

When the study objective is to investigate whether there are differences in sunscreen use according to sociodemographic characteristics (such as, for example, between men and women), the existence of association between explanatory variables (exposure or independent variables, in this case sociodemographic variables) and a dependent or outcome variable (use of sunscreen) is what is under consideration.

In these cases, we need first to understand what the hypotheses are, as well as the types of error that may result from their acceptance or refutation. A hypothesis is a "supposition arrived at from observation or reflection, that leads to refutable predictions". 4 In other words, it is a statement that may be questioned or tested and that may be falsified in scientific studies.

In scientific studies, there are two types of hypothesis: the null hypothesis (H 0 ) or original supposition that we assume to be true for a given situation, and the alternative hypothesis (H A ) or additional explanation for the same situation, which we believe may replace the original supposition. In the health field, H 0 is frequently defined as the equality or absence of difference in the outcome of interest between the studied groups (for example, sunscreen use is equal in men and women). On the other hand, H A assumes the existence of difference between groups. H A is called two-tailed when it is expected that the difference between the groups will occur in any direction (men using more sunscreen than women or vice-versa). However, if the investigator expects to find that a specific group uses more sunscreen than the other, he will be testing a one-tailed H A .

In the sample investigated by Duquia et al., the frequency of sunscreen use at the beach was greater in men (32.7%) than in women (26.2%).3 Although this what was observed in the sample, that is, men do wear more sunscreen than women, the investigators must decide whether they refute or accept H 0 in the target population (which contends that there is no difference in sunscreen use according to sex). Given that the entire target population is hardly ever investigated to confirm or refute the difference observed in the sample, the authors have to be aware that, independently from their decision (accepting or refuting H 0 ), their conclusion may be wrong, as can be seen in figure 2 .

Types of possible results when performing a hypothesis test

In case the investigators conclude that both in the target population and in the sample sunscreen use is also different between men and women (rejecting H 0 ), they may be making a type I or Alpha error, which is the probability of rejecting H 0 based on sample results when, in the target population, H 0 is true (the difference between men and women regarding sunscreen use found in the sample is not observed in the target population). If the authors conclude that there are no differences between the groups (accepting H 0 ), the investigators may be making a type II or Beta error, which is the probability of accepting H 0 when, in the target population, H 0 is false (that is, H A is true) or, in other words, the probability of stating that the frequency of sunscreen use is equal between the sexes, when it is different in the same groups of the target population.

In order to accept or refute H 0 , the investigators need to previously define which is the maximum probability of type I and II errors that they are willing to incorporate into their results. In general, the type I error is fixed at a maximum value of 5% (0.05 or confidence level of 95%), since the consequences originated from this type of error are considered more harmful. For example, to state that an exposure/intervention affects a health condition, when this does not happen in the target population may bring about behaviors or actions (therapeutic changes, implementation of intervention programs etc.) with adverse consequences in ethical, economic and health terms. In the study conducted by Duquia et al., when the authors contend that the use of sunscreen was different according to sex, the p value presented (<0.001) indicates that the probability of not observing such difference in the target population is less that 0.1% (confidence level >99.9%). 3

Although the type II or Beta error is less harmful, it should also be avoided, since if a study contends that a given exposure/intervention does not affect the outcome, when this effect actually exists in the target population, the consequence may be that a new medication with better therapeutic effects is not administered or that some aspects related to the etiology of the damage are not considered. This is the reason why the value of the type II error is usually fixed at a maximum value of 20% (or 0.20). In publications, this value tends to be mentioned as the power of the study, which is the ability of the test to detect a difference, when in fact it exists in the target population (usually fixed at 80%, as a result of the 1-Beta calculation).

SAMPLE CALCULATION FOR STUDIES THAT AIM AT TESTING THE ASSOCIATION BETWEEN A RISK/PROTECTIVE FACTOR AND AN OUTCOME, EVALUATED DICHOTOMOUSLY

In cases where the exposure variables are dichotomous (intervention/control, man/woman, rich/poor etc.) and so is the outcome (negative/positive outcome, to use sunscreen or not), the required parameters to calculate sample size are those described in chart 3 . According to the previously mentioned example, it would be interesting to know whether sex, skin color, schooling level and income are associated with the use of sunscreen at work, while doing sports and at the beach. Thus, when the four exposure variables are crossed with the three outcomes, there would be 12 different questions to be answered and consequently an equal number of sample size calculations to be performed. Using the information in the article by Duquia et al. 3 for the prevalence of exposures and outcomes, a simulation of sample size calculations was used for each one of these situations ( Chart 4 ).

Ho - null hypothesis; Ha - alternative hypothesis

E=exposed group; NE=non-exposed group; r=NE/E relationship; PONE=prevalence of outcome in the non-exposed group (percentage of positives in non-exposed group), estimated based on formula from chart 3 , considering an PR of 1.50; PR=prevalence ratio/incidence or expected relative risk; n= minimum necessary sample size; ND=value could not be determined, as prevalence of outcome in the exposed would be above 100%, according to specified parameters.

Estimates show that studies with more power or that intend to find a difference of a lower magnitude in the frequency of the outcome (in this case, the prevalence rates) between exposed and non-exposed groups require larger sample sizes. For these reasons, in sample size calculations, an effect measure between 1.5 and 2.0 (for risk factors) or between 0.50 and 0.75 (for protective factors), and an 80% power are frequently used.

Considering the values in each column of chart 3 , we may conclude also that, when the nonexposed/exposed relationship moves away from one (similar proportions of exposed and non-exposed individuals in the sample), the sample size increases. For this reason, intervention studies usually work with the same proportion of individuals in the intervention and control groups. Upon analysis of the values on each line, it can be concluded that there is an inverse relationship between the prevalence of the outcome and the required sample size.

Based on these estimates, assuming that the authors intended to test all of these associations, it would be necessary to choose the largest estimated sample size (2,630 subjects). In case the required sample size is larger than the target population, the investigators may decide to perform a multicenter study, lengthen the period for data collection, modify the research question or face the possibility of not having sufficient power to draw valid conclusions.

Additional aspects need to be considered in the previous estimates to arrive at the final sample size, which may include the possibility of refusals and/or losses in the study (an additional 10-15%), the need for adjustments for confounding factors (an additional 10-20%, applicable to observational studies), the possibility of effect modification (which implies an analysis of subgroups and the need to duplicate or triplicate the sample size), as well as the existence of design effects (multiplication of sample size by 1.5 to 2.0) in case of cluster sampling.

SAMPLE CALCULATIONS FOR STUDIES THAT AIM AT TESTING THE ASSOCIATION BETWEEN A DICHOTOMOUS EXPOSURE AND A NUMERICAL OUTCOME

Suppose that the investigators intend to evaluate whether the daily quantity of sunscreen used (in grams), the time of daily exposure to sunlight (in minutes) or a laboratory parameter (such as vitamin D levels) differ according to the socio-demographic variables mentioned. In all of these cases, the outcomes are numerical variables (discrete or continuous) 1 , and the objective is to answer whether the mean outcome in the exposed/intervention group is different from the non-exposed/control group.

In this case, the first three parameters from chart 4 (alpha error, power of the study and relationship between non-exposed/exposed groups) are required, and the conclusions about their influences on the final sample size are also applicable. In addition to defining the expected outcome means in each group or the expected mean difference between nonexposed/exposed groups (usually at least 15% of the mean value in non-exposed group), they also need to define the standard deviation value for each group. There is a direct relationship between the standard deviation value and the sample size, the reason why in case of asymmetric variables the sample size would be overestimated. In such cases, the option may be to estimate sample sizes based on specific calculations for asymmetric variables, or the investigators may choose to use a percentage of the median value (for example, 25%) as a substitute for the standard deviation.

SAMPLE SIZE CALCULATIONS FOR OTHER TYPES OF STUDY

There are also specific calculations for some other quantitative studies, such as those aiming to assess correlations (exposure and outcome are numerical variables), time until the event (death, cure, relapse etc.) or the validity of diagnostic tests, but they are not described in this article, given that they were discussed elsewhere. 5

Sample size calculation is always an essential step during the planning of scientific studies. An insufficient or small sample size may not be able to demonstrate the desired difference, or estimate the frequency of the event of interest with acceptable precision. A very large sample may add to the complexity of the study, and its associated costs, rendering it unfeasible. Both situations are ethically unacceptable and should be avoided by the investigator.

Conflict of Interest: None

Financial Support: None

* Work carried out at the Latin American Cooperative Oncology Group (LACOG), Universidade Federal de Santa Catarina (UFSC), and Universidade Federal de Ciências da Saúde de Porto Alegre (UFCSPA), Brazil.

Como citar este artigo: Martínez-Mesa J, González-Chica DA, Bastos JL, Bonamigo RR, Duquia RP. Sample size: how many participants do I need in my research? An Bras Dermatol. 2014;89(4):609-15.

Root out friction in every digital experience, super-charge conversion rates, and optimize digital self-service

Uncover insights from any interaction, deliver AI-powered agent coaching, and reduce cost to serve

Increase revenue and loyalty with real-time insights and recommendations delivered to teams on the ground

Know how your people feel and empower managers to improve employee engagement, productivity, and retention

Take action in the moments that matter most along the employee journey and drive bottom line growth

Whatever they’re are saying, wherever they’re saying it, know exactly what’s going on with your people

Get faster, richer insights with qual and quant tools that make powerful market research available to everyone

Run concept tests, pricing studies, prototyping + more with fast, powerful studies designed by UX research experts

Track your brand performance 24/7 and act quickly to respond to opportunities and challenges in your market

Explore the platform powering Experience Management

• Free Account
• Product Demos
• For Digital
• For Customer Care
• For Human Resources
• For Researchers
• Financial Services
• All Industries

Popular Use Cases

• Customer Experience
• Employee Experience
• Net Promoter Score
• Voice of Customer
• Customer Success Hub
• Product Documentation
• Training & Certification
• XM Institute
• Popular Resources
• Customer Stories
• Artificial Intelligence

Market Research

• Partnerships
• Marketplace

The annual gathering of the experience leaders at the world’s iconic brands building breakthrough business results, live in Salt Lake City.

• English/AU & NZ
• Español/Europa
• Español/América Latina
• Português Brasileiro
• REQUEST DEMO

Sample size calculator

Updated December 8, 2023

Introduction to sample size

How can you calculate sample size, reduce the margin of error and produce surveys with statistically significant results? In this short guide, we explain how you can improve your surveys and showcase some of the tools and resources you can leverage in the process.

But first, when it comes to market research, how many people do you need to interview to get results representative of the target population with the level of confidence that you are willing to accept?

However, if all of this sounds new to you, let's start with what sample size is.

Free eBook: The complete guide to determining sample size

What is sample size?

Sample size is a term used in market research to define the number of subjects included in a survey, study, or experiment. In surveys with large populations, sample size is incredibly important. The reason for this is because it's unrealistic to get answers or results from everyone - instead, you can take a random sample of individuals that represent the population as a whole.

For example, we might want to compare the performance of long-distance runners that eat Weetabix for breakfast versus those who don't. Since it's impossible to track the dietary habits of every long-distance runner across the globe, we would have to focus on a segment of the survey population. This might mean selecting 1,000 runners for the study.

How can sample size influence results?

That said, no matter how diligent we are with our selection, there will always be some margin of error (also referred to as confidence interval) in the study results, that's because we can't speak to every long-distance runner or be confident of how Weetabix influences (in every possible scenario), the performance of long-distance runners. This is known as a "sampling error."

Larger sample sizes will help to mitigate the margin of error, helping to provide more statistically significant and meaningful results. In other words, a more accurate picture of how eating Weetabix can influence the performance of long-distance runners.

So what do you need to know when calculating the minimum sample size needed for a research project?

What you need to know to calculate survey sample size

Confidence interval (or margin of error).

The confidence interval is the plus-or-minus figure that represents the accuracy of the reported. Consider the following example:

A Canadian national sample showed "Who Canadians spend their money on for Mother's Day." Eighty-two percent of Canadians expect to buy gifts for their mom, compared to 20 percent for their wife and 15 percent for their mother-in-law. In terms of spending, Canadians expect to spend $93 on their wife this Mother's Day versus $58 on their mother. The national findings are accurate, plus or minus 2.75 percent, 19 times out of 20.

For example, if you use a confidence interval of 2.75 and 82% percent of your sample indicates they will "buy a gift for mom" you can be "confident (95% or 99%)" that if you had asked the question to ALL CANADIANS, somewhere between 79.25% (82%-2.75%) and 84.75% (82%+2.75%) would have picked that answer.

Confidence interval is also called the "margin of error." Are you needing to understand how the two calculations correlate?

Confidence level

The confidence level tells you how confident you are of this result. It is expressed as a percentage of times that different samples (if repeated samples were drawn) would produce this result. The 95% confidence level means that 19 times out of twenty that results would fall in this - + interval confidence interval. The 95% confidence level is the most commonly used.

When you put the confidence level and the confidence interval together, you can say that you are 95% (19 out of 20) sure that the true percentage of the population that will "buy a gift for mom" is between 79.25% and 84.75%.

Wider confidence intervals increase the certainty that the true answer is within the range specified. These wider confidence intervals come from smaller sample sizes. When the costs of an error is extremely high (a multi-million dollar decision is at stake) the confidence interval should be kept small. This can be done by increasing the sample size.

Population size

Population size is the total amount of people in the group you're trying to study. If you were taking a random sample of people across the U.K., then your population size would be just over 68 million (as of 09 August 2021).

Standard deviation

This refers to how much individual responses will vary between each other and the mean. If there's a low standard deviation, scores will be clustered near the mean with minimal variation. A higher standard deviation means that when plotted on a graph, responses will be more spread out.

Standard deviation is expressed as a decimal, and 0.5 is considered a "good" standard deviation to set to ensure a sample size that represents the population.

How can you calculate sample size?

After you've considered the four above variables, you should have everything required to calculate your sample size.

However, if you don't know your population size, you can still calculate your sample size. To do this, you need two pieces of information: a z-score and the sample size formula.

What is a z-score?

A z-score is simply the numerical representation of your desired confidence level. It tells you how many standard deviations from the mean your score is.

The most common percentages are 90%, 95%, and 99%.

z = (x – μ) / σ

As the formula shows, the z-score is simply the raw score minus the population mean and divided by the population's standard deviation.

Using a sample size calculation

Once you have your z-score, you can fill out your sample size formula, which is:

Is there an easier way to calculate sample size?

If you want an easier option, Qualtrics offers an online sample size calculator that can help you determine your ideal survey sample size in seconds. Just put in the confidence level, population size, margin of error, and the perfect sample size is calculated for you.

Best-practice tips for sample size

There are lots of variables to consider when it comes to generating a specific sample size. That said, there are a few best-practice tips (or rules) to ensure you get the best possible results:

1) Balance cost and confidence level

To increase confidence level or reduce the margin of error, you have to increase your sample size. Larger sizes almost invariably lead to higher costs. Take the time to consider what results you want from your surveys and what role it plays in your overall campaigns.

2) You don't always need statistically significant results

Depending on your target audience, you may not be able to get enough responses (or a large enough sample size) to achieve "statistically significant" results.

If it's for your own research and not a wider study, it might not be that much of a problem, but remember that any feedback you get from your surveys is important. It might not be statistically significant, but it can aid your activities going forward.

Ultimately, you should treat this on a case-by-case basis. Survey samples can still give you valuable answers without having sample sizes that represent the general population. But more on this in the section below.

3) Ask open-ended questions

Yes and no questions provide certainty, but open-ended questions provide insights you would have otherwise not received. To get the best results, it's worth having a mix of closed and open-ended questions. For a deeper dive into survey question types, check out our handbook.

Different types of surveys

From market research to customer satisfaction, there are plenty of different surveys that you can carry out to get the information you need, corresponding with your sample size.

The great thing about what we do at Qualtrics is that we offer a comprehensive collection of pre-made, customer, product, employee, and brand survey templates. This includes Net Promoter Score (NPS) surveys, manager feedback surveys, customer service surveys, and more.

The best part? You can access all of these templates for free. Each one is designed by our specialist team of subject matter experts and researchers so you can be sure that our best-practice question choices and clear designs will get more engagement and better quality data.

As well as offering free survey templates, you can check out our free survey builder. Trusted by over 11,000 brands and 99 of the top 100 business schools, our tool allows you to create, distribute and analyze surveys to find customer, employee, brand, product, and market research insights.

Drag-and-drop functionality means anyone can use it, and wherever you need to gather and analyze data, our platform can help.

Once you have determined your sample size , you’re ready for the next step in the research journey. market research.

Market research is the process of gathering information about consumers' needs and preferences, and it can provide incredible insights that help elevate your business (or your customers') to the next level.

If you want to learn more, we've got you covered. Just download our free guide and find out how you can:

• Identify use cases for market research
• Create and deliver effective market research campaigns
• Take action on research findings

Qualtrics // Experience Management

Qualtrics, the leader and creator of the experience management category, is a cloud-native software platform that empowers organizations to deliver exceptional experiences and build deep relationships with their customers and employees.

With insights from Qualtrics, organizations can identify and resolve the greatest friction points in their business, retain and engage top talent, and bring the right products and services to market. Nearly 20,000 organizations around the world use Qualtrics’ advanced AI to listen, understand, and take action. Qualtrics uses its vast universe of experience data to form the largest database of human sentiment in the world. Qualtrics is co-headquartered in Provo, Utah and Seattle.

Related Articles

May 20, 2024

Best strategy & research books to read in 2024

May 13, 2024

Experience Management

X4 2024 Strategy & Research Showcase: Introducing the future of insights generation

November 7, 2023

Brand Experience

The 4 market research trends redefining insights in 2024

June 27, 2023

The fresh insights people: Scaling research at Woolworths Group

June 20, 2023

Bank less, delight more: How Bankwest built an engine room for customer obsession

April 1, 2023

Academic Experience

How to write great survey questions (with examples)

November 18, 2022

Statistical analysis software: your complete guide to getting started

November 4, 2022

Create online surveys with our free online survey maker tool

Stay up to date with the latest xm thought leadership, tips and news., request demo.

Ready to learn more about Qualtrics?