negative control experiment results

Positive Control vs Negative Control: Differences & Examples

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

A positive control is designed to confirm a known response in an experimental design , while a negative control ensures there’s no effect, serving as a baseline for comparison.

The two terms are defined as below:

• Positive control refers to a group in an experiment that receives a procedure or treatment known to produce a positive result. It serves the purpose of affirming the experiment’s capability to produce a positive outcome.
• Negative control refers to a group that does not receive the procedure or treatment and is expected not to yield a positive result. Its role is to ensure that a positive result in the experiment is due to the treatment or procedure.

The experimental group is then compared to these control groups, which can help demonstrate efficacy of the experimental treatment in comparison to the positive and negative controls.

Positive Control vs Negative Control: Key Terms

Control groups.

A control group serves as a benchmark in an experiment. Typically, it is a subset of participants, subjects, or samples that do not receive the experimental treatment (as in negative control).

This could mean assigning a placebo to a human subject or leaving a sample unaltered in chemical experiments. By comparing the results obtained from the experimental group to the control, you can ascertain whether any differences are due to the treatment or random variability.

A well-configured experimental control is critical for drawing valid conclusions from an experiment. Correct use of control groups permits specificity of findings, ensuring the integrity of experimental data.

See More: Control Variables Examples

The Negative Control

Negative control is a group or condition in an experiment that ought to show no effect from the treatment.

It is useful in ensuring that the outcome isn’t accidental or influenced by an external cause. Imagine a medical test, for instance. You use distilled water, anticipating no reaction, as a negative control.

If a significant result occurs, it warns you of a possible contamination or malfunction during the testing. Failure of negative controls to stay ‘negative’ risks misinterpretation of the experiment’s result, and could undermine the validity of the findings.

The Positive Control

A positive control, on the other hand, affirms an experiment’s functionality by demonstrating a known reaction.

This might be a group or condition where the expected output is known to occur, which you include to ensure that the experiment can produce positive results when they are present. For instance, in testing an antibiotic, a well-known pathogen, susceptible to the medicine, could be the positive control.

Positive controls affirm that under appropriate conditions your experiment can produce a result. Without this reference, experiments could fail to detect true positive results, leading to false negatives. These two controls, used judiciously, are backbones of effective experimental practice.

Experimental Groups

Experimental groups are primarily characterized by their exposure to the examined variable.

That is, these are the test subjects that receive the treatment or intervention under investigation. The performance of the experimental group is then compared against the well-established markers – our positive and negative controls.

For example, an experimental group may consist of rats undergoing a pharmaceutical testing regime, or students learning under a new educational method. Fundamentally, this unit bears the brunt of the investigation and their response powers the outcomes.

However, without positive and negative controls, gauging the results of the experimental group could become erratic. Both control groups exist to highlight what outcomes are expected with and without the application of the variable in question. By comparing results, a clearer connection between the experiment variables and the observed changes surfaces, creating robust and indicative scientific conclusions.

Positive and Negative Control Examples

1. a comparative study of old and new pesticides’ effectiveness.

This hypothetical study aims to evaluate the effectiveness of a new pesticide by comparing its pest-killing potential with old pesticides and an untreated set. The investigation involves three groups: an untouched space (negative control), another treated with an established pesticide believed to kill pests (positive control), and a third area sprayed with the new pesticide (experimental group).

• Negative Control: This group consists of a plot of land infested by pests and not subjected to any pesticide treatment. It acts as the negative control. You expect no decline in pest populations in this area. Any unexpected decrease could signal external influences (i.e. confounding variables ) on the pests unrelated to pesticides, affecting the experiment’s validity.
• Positive Control: Another similar plot, this time treated with a well-established pesticide known to reduce pest populations, constitutes the positive control. A significant reduction in pests in this area would affirm that the experimental conditions are conducive to detect pest-killing effects when a pesticide is applied.
• Experimental Group: This group consists of the third plot impregnated with the new pesticide. Carefully monitoring the pest level in this research area against the backdrop of the control groups will reveal whether the new pesticide is effective or not. Through comparison with the other groups, any difference observed can be attributed to the new pesticide.

2. Evaluating the Effectiveness of a Newly Developed Weight Loss Pill

In this hypothetical study, the effectiveness of a newly formulated weight loss pill is scrutinized. The study involves three groups: a negative control group given a placebo with no weight-reducing effect, a positive control group provided with an approved weight loss pill known to cause a decrease in weight, and an experimental group given the newly developed pill.

• Negative Control: The negative control is comprised of participants who receive a placebo with no known weight loss effect. A significant reduction in weight in this group would indicate confounding factors such as dietary changes or increased physical activity, which may invalidate the study’s results.
• Positive Control: Participants in the positive control group receive an FDA-approved weight loss pill, anticipated to induce weight loss. The success of this control would prove that the experiment conditions are apt to detect the effects of weight loss pills.
• Experimental Group: This group contains individuals receiving the newly developed weight loss pill. Comparing the weight change in this group against both the positive and negative control, any difference observed would offer evidence about the effectiveness of the new pill.

3. Testing the Efficiency of a New Solar Panel Design

This hypothetical study focuses on assessing the efficiency of a new solar panel design. The study involves three sets of panels: a set that is shaded to yield no solar energy (negative control), a set with traditional solar panels that are known to produce an expected level of solar energy (positive control), and a set fitted with the new solar panel design (experimental group).

• Negative Control: The negative control involves a set of solar panels that are deliberately shaded, thus expecting no solar energy output. Any unexpected energy output from this group could point towards measurement errors, needed to be rectified for a valid experiment.
• Positive Control: The positive control set up involves traditional solar panels known to produce a specific amount of energy. If these panels produce the expected energy, it validates that the experiment conditions are capable of measuring solar energy effectively.
• Experimental Group: The experimental group features the new solar panel design. By comparing the energy output from this group against both the controls, any significant output variation would indicate the efficiency of the new design.

4. Investigating the Efficacy of a New Fertilizer on Plant Growth

This hypothetical study investigates the efficacy of a newly formulated fertilizer on plant growth. The study involves three sets of plants: a set without any fertilizer (negative control), a set treated with an established fertilizer known to promote plant growth (positive control), and a third set fed with the new fertilizer (experimental group).

• Negative Control: The negative control involves a set of plants not receiving any fertilizer. Lack of significant growth in this group will confirm that any observed growth in other groups is due to the applied fertilizer rather than other uncontrolled factors.
• Positive Control: The positive control involves another set of plants treated with a well-known fertilizer, expected to promote plant growth. Adequate growth in these plants will validate that the experimental conditions are suitable to detect the influence of a good fertilizer on plant growth.
• Experimental Group: The experimental group consists of the plants subjected to the newly formulated fertilizer. Investigating the growth in this group against the growth in the control groups will provide ascertained evidence whether the new fertilizer is efficient or not.

5. Evaluating the Impact of a New Teaching Method on Student Performance

This hypothetical study aims to evaluate the impact of a new teaching method on students’ performance. This study involves three groups, a group of students taught through traditional methods (negative control), another group taught through an established effective teaching strategy (positive control), and one more group of students taught through the new teaching method (experimental group).

• Negative Control: The negative control comprises students taught by standard teaching methods, where you expect satisfactory but not top-performing results. Any unexpected high results in this group could signal external factors such as private tutoring or independent study, which in turn may distort the experimental outcome.
• Positive Control: The positive control consists of students taught by a known efficient teaching strategy. High performance in this group would prove that the experimental conditions are competent to detect the efficiency of a teaching method.
• Experimental Group: This group consists of students receiving instruction via the new teaching method. By analyzing their performance against both control groups, any difference in results could be attributed to the new teaching method, determining its efficacy.

Table Summary

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 25 Number Games for Kids (Free and Easy)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 25 Word Games for Kids (Free and Easy)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 25 Outdoor Games for Kids
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 50 Incentives to Give to Students

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

Negative Control Outcomes : A Tool to Detect Bias in Randomized Trials

• 1 Division of Epidemiology, School of Public Health, University of California-Berkeley

Investigators have several design, measurement, and analytic tools to detect and reduce bias in epidemiological studies. One such approach, “negative controls,” has been used on an ad hoc basis for decades. A formal approach has recently been suggested for its use to detect confounding, selection, and measurement bias in epidemiological studies. 1 , 2 Negative controls in epidemiological studies are analogous to negative controls in laboratory experiments, in which investigators test for problems with the experimental method by leaving out an essential ingredient, inactivating the hypothesized active ingredient, or checking for an effect that would be impossible by the hypothesized mechanism. 1 A placebo treatment group in a randomized trial is an example of a negative control exposure (leaving out an essential ingredient) that helps remove bias that can result from participant or practitioner knowledge of an individual’s treatment assignment—the placebo treatment is susceptible to the same bias structure as the actual treatment but is causally unrelated to the outcome of interest.

Read More About

Arnold BF , Ercumen A. Negative Control Outcomes : A Tool to Detect Bias in Randomized Trials . JAMA. 2016;316(24):2597–2598. doi:10.1001/jama.2016.17700

Manage citations:

© 2024

Artificial Intelligence Resource Center

Cardiology in JAMA : Read the Latest

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

A Selective Review of Negative Control Methods in Epidemiology

• Epidemiologic Methods (P Howards, Section Editor)
• Published: 15 October 2020
• Volume 7 , pages 190–202, ( 2020 )

Cite this article

• Xu Shi   ORCID: orcid.org/0000-0001-8566-9552 1 ,
• Wang Miao 2 &
• Eric Tchetgen Tchetgen 3  

2526 Accesses

83 Citations

9 Altmetric

Explore all metrics

A Correction to this article was published on 08 May 2021

This article has been updated

Purpose of Review

Negative controls are a powerful tool to detect and adjust for bias in epidemiological research. This paper introduces negative controls to a broader audience and provides guidance on principled design and causal analysis based on a formal negative control framework.

Recent Findings

We review and summarize causal and statistical assumptions, practical strategies, and validation criteria that can be combined with subject-matter knowledge to perform negative control analyses. We also review existing statistical methodologies for the detection, reduction, and correction of confounding bias, and briefly discuss recent advances towards nonparametric identification of causal effects in a double-negative control design.

There is great potential for valid and accurate causal inference leveraging contemporary healthcare data in which negative controls are routinely available. Design and analysis of observational data leveraging negative controls is an area of growing interest in health and social sciences. Despite these developments, further effort is needed to disseminate these novel methods to ensure they are adopted by practicing epidemiologists.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Quality Control and Good Epidemiological Practice

Identification of causal effects in case-control studies

Methods for Analyzing Secondary Outcomes in Public Health Case–Control Studies

Change history, 08 may 2021.

A Correction to this paper has been published: https://doi.org/10.1007/s40471-021-00270-9

Papers of particular interest, published recently, have been highlighted as: • Of importance •• Of major importance

Ioannidis JPA. “Why most published research findings are false”. In: PLOS Medicine 2.8 (2005), pp. 696–701.

Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. In: Am J Epidemiol. 2016;183(8):758–64.

Google Scholar  

•• Lipsitch M, Tchetgen Tchetgen EJ, Cohen T. Negative controls: a tool for detecting confounding and bias in observational studies. In: Epidemiology. 2010;21.3:383–8 This paper is the first to formally define negative control exposure and outcome with conditions for bias detection as well as examples in epidemiology.

Arnold BF, Ercumen A, Benjamin-Chung J, Colford JM Jr. Brief report: negative controls to detect selection bias and measurement bias in epidemiologic studies. In: Epidemiology. 2016;27.5:637.

Arnold B, Ercumen A. Negative control outcomes: a tool to detect bias in randomized trials. In: J Am Med Assoc. 2016;316(24):2597–8.

Rosenbaum PR. The role of known effects in observational studies. In : Biometrics . 1989;45(2):557–69.

Weiss NS. Can the “specificity” of an association be rehabilitated as a basis for supporting a causal hypothesis? In: Epidemiology. 2002;13(1):6–8.

Glass DJ. Experimental Design for Biologists . Cold Spring Harbor Laboratory Press, 2014.

Cai Z and Kuroki M. “On identifying total effects in the presence of latent variables and selection bias”. In: Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence. 2008, pp. 62–69.

Liu L, Tchetgen Tchetgen EJ. “Regression-based negative control of homophily in dyadic peer effect analysis”. In: arXiv preprint arXiv:2002.06521 (2020).

Egami N. “Identification of Causal Diffusion Effects Under Structural Stationarity”. In: arXiv preprint arXiv:1810.07858 (2018).

• Miao W, Shi X, and Tchetgen Tchetgen EJ. “A Confounding Bridge Approach for Double Negative Control Inference on Causal Effects”. In: (2020). In progress, a prior version can be found at https://arxiv.org/abs/1808.04945 . This paper introduces the confounding bridge function that links primary and negative control outcome distributions for identification of the average treatment effect leveraging a negative control exposure.

Sofer T, Richardson DB, Colicino E, Schwartz J, Tchetgen Tchetgen EJ. On negative outcome control of unobserved confounding as a generalization of difference-in-differences. In: Stat Sci. 2016;31(3):348–61.

Jackson LA, Jackson ML, Nelson JC, Neuzil KM, Weiss NS. Evidence of bias in estimates of influenza vaccine effectiveness in seniors. In: Int J Epidemiol. 2006;35(2):337–44.

Splawa-Neyman J, Dabrowska DM, Speed TP. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. In: Stat Sci. 1990:465–72.

Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. In: Journal of Educational Psychology . 1974;66.5:688.

• Shi X, Miao W, Tchetgen Tchetgen EJ. Multiply robust causal inference with double negative control adjustment for categorical unmeasured confounding. In: J Royal Stat Soc: Series B (Statistical Methodology). 2020;82.2:521–40 This paper provides a general semiparametric framework for obtaining inferences about the average treatment effect under categorical unmeasured confounding and negative controls.

Article   Google Scholar  

Alan Brookhart M, Rassen JA, Schneeweiss S. Instrumental variable methods in comparative safety and effectiveness research. In: Pharmacoepidemiology and Drug Safety . 2010;19(6):537–54.

Angrist JD, Imbens GW, Rubin DB. Identification of causal effects using instrumental variables. In: J Am Stat Assoc. 1996;91(434):444–55.

Hernán MA and Robins JM. “Instruments for causal inference: an epidemiologist’s dream?” In: Epidemiology (2006), pp. 360–372.

Robins JM. Correcting for non-compliance in randomized trials using structural nested mean models. In: Commun Stat-Theory and methods. 1994;23(8):2379–412.

Wang L, Tchetgen Tchetgen EJ. Bounded, efficient and multiply robust estimation of average treatment effects using instrumental variables. In: J Royal Stat Soc: Series B (Statistical Methodology). 2018;80.3:531–50.

Prasad V, Jena AB. Prespecified falsification end points: can they validate true observational associations? In: J Am Med Assoc. 2013;309(3):241–2.

CAS   Google Scholar  

Markovitz AA, Hollingsworth JM, Ayanian JZ, Norton EC, Yan PL, Ryan AM. Performance in the Medicare shared savings program after accounting for nonrandom exit: an instrumental variable analysis. In: Ann Int Med. 2019;171(1):27–36.

Bijlsma MJ, Vansteelandt S, Janssen F, Hak E. The effect of adherence to statin therapy on cardiovascular mortality: quantification of unmeasured bias using falsification end-points. In: BMC Public Health. 2016;16.1:303.

Lin C-K, Lin R-T, Chen P-C, Wang P, De Marcellis-Warin N, Zigler C, et al. A global perspective on sulfur oxide controls in coal-fired power plants and cardiovascular disease. In: Sci Rep. 2018;8(1):1–9.

Dusetzina SB, Brookhart MA, Maciejewski ML. Control outcomes and exposures for improving internal validity of nonrandomized studies. In: Health Serv Res. 2015;50(5):1432–51.

Rosenbaum PR. Design of observational studies . New York, NY: Springer-Verlag, 2010.

Munaf̀o MR, Tilling K, Taylor AE, Evans DM, Smith GD. Collider scope: when selection bias can substantially influence observed associations. In: Int J Epidemiol. 2018;47(1):226–35.

Mealli F, Pacini B. Using secondary outcomes to sharpen inference in randomized experiments with noncompliance. In: J Am Stat Assoc. 2013;108(503):1120–31.

Rosenbaum PR. Detecting bias with confidence in observational studies. In: Biometrika. 1992;79(2):367–74.

Flanders WD, Klein M, Darrow LA, Strickland MJ, Sarnat SE, Sarnat JA, et al. A method for detection of residual confounding in time-series and other observational studies. In: Epidemiology. 2011;22.1:59.

• Flanders WD, Strickland MJ, Klein M. A new method for partial correction of residual confounding in time-series and other observational studies. In: Am J Epidemiol. 2017;185.10:941–9 This paper develops a regression-based method taking future air pollution as a negative control exposure to reduce residual confounding bias in a time-series study on air pollution effects.

de Luna X, Fowler P, Johansson P. Proxy variables and nonparametric identification of causal effects. In: Econ Lett. 2017;150:152–4.

Kuroki M, Pearl J. Measurement bias and effect restoration in causal inference. In: Biometrika. 2014;101(2):423–37.

•• Miao W, Geng Z, Tchetgen Tchetgen EJ. Identifying causal effects with proxy variables of an unmeasured confounder. In: Biometrika. 2018;105.4:987–93 This paper establishes sufficient conditions for nonparametric identification of the average treatment effect using double negative control.

• Madigan D, Stang PE, Berlin JA, Schuemie M, Overhage JM, Suchard MA, et al. A systematic statistical approach to evaluating evidence from observational studies. In: Annu Rev Stat Appl. 2014;1:11–39 This paper provides a systematic review of challenges in observational studies and describes a data-driven approach to calculating calibrated p values leveraging negative controls.

Schuemie MJ, Ryan PB, DuMouchel W, Suchard MA, Madigan D. Interpreting observational studies: why empirical calibration is needed to correct p-values. In: Stat Med. 2014;33(2):209–18.

Schuemie MJ, Hripcsak G, Ryan PB, Madigan D, Suchard MA. Robust empirical calibration of p-values using observational data. In: Statistics in Medicine . 2016;35.22:3883.

Schuemie MJ, Hripcsak G, Ryan PB, Madigan D, Suchard MA. Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data. In: Proc Natl Acad Sci. 2018;115(11):2571–7.

Schuemie MJ, Ryan PB, Hripcsak G, Madigan D, Suchard MA. Improving reproducibility by using high-throughput observational studies with empirical calibration. In: Philos Trans Royal Soc A: Math Phys Eng Sci. 2018;376.2128:20170356.

Yerushalmy J. The relationship of parents’ cigarette smoking to outcome of pregnancy– implications as to the problem of inferring causation from observed associations. In: Am J Epidemiol. 1971;93(6):443–56.

Mitchell EA, Ford RPK, Stewart AW, Taylor BJ, Becroft DMO, Thompson JMD, et al. Smoking and the sudden infant death syndrome. In: Pediatrics. 1993;91(5):893–6.

Howe LD, Matijasevich A, Tilling K, Brion M-J, Leary SD, Smith GD, Lawlor DA. Maternal smoking during pregnancy and off- spring trajectories of height and adiposity: comparing maternal and paternal associations. In: Int J Epidemiol. 2012;41(3):722–32.

Brion M-JA, Leary SD, Smith GD, Ness AR. Similar associations of parental prenatal smoking suggest child blood pressure is not influenced by intrauterine effects. In: Hypertension. 2007;49(6):1422–8.

Smith GD. Assessing intrauterine influences on offspring health outcomes: can epidemiological studies yield robust findings? In: Basic Clin Pharmacol Toxicol. 2008;102(2):245–56.

Brew BK, Gong T, Williams DM, Larsson H, Almqvist C. Using fathers as a negative control exposure to test the developmental origins of health and disease hypothesis: a case study on maternal distress and offspring asthma using Swedish register data. In: Scand J Public Health. 2017;45.17(suppl):36–40.

Taylor AE, Smith GD, Bares CB, Edwards AC, Munaf̀o MR. Partner smoking and maternal cotinine during pregnancy: implications for negative control methods. In: Drug Alcohol Depend. 2014;139:159–63.

Wang M, Tchetgen Tchetgen EJ. Invited commentary: bias attenuation and identification of causal effects with multiple negative controls. In: Am J Epidemiol. 2017;185(10):950–3.

Yu Y, Li H, Sun X, Liu X, Yang F, Hou L, et al. Identification and estimation of causal effects using a negative control exposure in time-series studies with applications to environmental epidemiology. Am J Epidemiol. kwaa172. https://doi.org/10.1093/aje/kwaa172 .

Lumley T, Sheppard L. Assessing seasonal confounding and model selection bias in air pollution epidemiology using positive and negative control analyses. In: Environmetrics. 2000;11(6):705–17.

Selby JV, Friedman GD, Quesenberry CP Jr, Weiss NS. A case–control study of screening sigmoidoscopy and mortality from colorectal cancer. In: N Engl J Med. 1992;326(10):653–7.

Zauber AG. The impact of screening on colorectal cancer mortality and incidence: has it really made a difference? In: Digest Dis Sci. 2015;60(3):681–91.

• Lousdal ML, Lash TL, Flanders WD, Brookhart MA, Kristiansen IS, Kalager M, et al. Negative controls to detect uncontrolled confounding in observational studies of mammographic screening comparing participants and non-participants. In: Int J Epidemiol. 2020; This paper uses both negative control exposure and negative control outcome to detect residual confounding in an observational study of mammographic screening comparing participants and non-participants.

Sheppard L, Levy D, Norris G, Larson TV, Koenig JQ. Effects of ambient air pollution on nonelderly asthma hospital admissions in Seattle, Washington, 1987–1994. In: Epidemiology. 1999:23–30.

Cuyler Hammond E, Horn D. The relationship between human smoking habits and death rates: a follow-up study of 187,766 men. In: J Am Med Assoc. 1954;155(15):1316–28.

Doll R, Bradford Hill A. The mortality of doctors in relation to their smoking habits. In: Br Med J. 1954;1(4877):1451–5.

Doll R, Hill BA. Lung cancer and other causes of death in relation to smoking. In: Br Med J. 1956;2(5001):1071–81.

Cornfield J, William H, Hammond EC, Lilienfeld AM, Shimkin MB, Wynder EL. Smoking and lung cancer: recent evidence and a discussion of some questions. In: J Natl Cancer Inst. 1959;22(1):173–203.

Trichopoulos D, Zavitsanos X, Katsouyanni K, Tzonou A, Dalla-Vorgia P. Psychological stress and fatal heart attack: the Athens (1981) earthquake natural experiment. In: Lancet. 1983;321(8322):441–4.

Smith GD. Negative control exposures in epidemiologic studies. Comments on “Negative controls: a tool for detecting confounding and bias in observational studies”. In: Epidemiology . 2012;23(2):350–1.

Weisskopf MG, Tchetgen Tchetgen EJ, Raz R. Commentary: on the use of imperfect negative control exposures in epidemiologic studies. In: Epidemiology. 2016;27(3):365–7.

Richardson DB, Keil A, Tchetgen Tchetgen EJ, Cooper GS. Negative control outcomes and the analysis of standardized mortality ratios. In: Epidemiology. 2015;26(5):727–32.

Richardson DB, Laurier D, Schubauer-Berigan MK, Tchetgen Tchetgen EJ, Cole SR. Assessment and indirect adjustment for confounding by smoking in cohort studies using relative hazards models. In: Am J Epidemiol. 2014;180(9):933–40.

Tchetgen Tchetgen EJ, Sofer T, and Richardson D. “Negative outcome control for unobserved confounding under a Cox proportional hazards model”. In: (2015). Available at https://biostats.bepress.com/harvardbiostat/paper192/ .

Glynn A, Ichino N. “Generalized nonlinear difference-in-difference-in-differences”. In: V-Dem Working Paper 90 (2019). Available at https:// papers . ssrn.com/sol3/papers.cfm?abstract_id=3410888 .

Tchetgen Tchetgen EJ. The control outcome calibration approach for causal inference with unobserved confounding. In: Am J Epidemiol. 2014;179(5):633–40.

Gagnon-Bartsch JA, Speed TP. Using control genes to correct for unwanted variation in microarray data. In: Biostatistics. 2012;13(3):539–52.

Jacob L, Gagnon-Bartsch JA, Speed TP. Correcting gene expression data when neither the unwanted variation nor the factor of interest are observed. In: Biostatistics. 2016;17(1):16–28.

• Wang J, Zhao Q, Hastie T, Owen AB. Confounder adjustment in multiple hypothesis testing. In: Ann Stat. 2017;45.5:1863–94 This paper unifies unmeasured confounding adjustment methods in multiple hypothesis testing and provides theoretical guarantees for these methods.

Newey WK, Powell JL. Instrumental variable estimation of nonparametric models. In: Econometrica. 2003;71(5):1565–78.

Hansen LP. Large sample properties of generalized method of moments estimators. In: Econometrica. 1982:1029–54.

Download references

Author information

Authors and affiliations.

Department of Biostatistics, University of Michigan, Ann Arbor, USA

Department of Probability and Statistics, Peking University, Beijing, China

Statistics Department, The Wharton School, University of Pennsylvania, Philadelphia, USA

Eric Tchetgen Tchetgen

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Xu Shi .

Ethics declarations

Conflict of interest.

The authors declare that they have no conflicts of interest.

Human and Animal Rights

This article does not contain any studies with human or animal subjects performed by any of the authors.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

The original online version of this article was revised: A number of corrections needs to be addressed.

Appendix 1. Examples of invalid negative controls that violates some assumption

Violation 1 : no arrow between U and W. There must be an arrow between U and W , because an NCO is a proxy of unmeasured confounder. It recovers the confounding bias by reflecting variation due to U .

Violation 2 : no arrow between U and Z and Z↛ A . The only scenario that Z does not need to be associated with U is when Z is an instrumental variable (see first cell of Table 3 of the Appendix ). In this case, A is a collider between Z and U , such that Z and U are marginally independent. Conditioning on a collider will create collider bias such that Z and U become conditionally dependent. The requirements about Z in Assumptions 5 and 7 are all made conditioning on A . Therefore, an instrumental variable is a valid NCE.

Violation 3 : Y  →  W. If the outcome causes the NCO, then the treatment directly causes the NCO via the path A→Y→W , which violates Assumption 3.

Violation 4 : Z → U ← W. The direction of the arrow between U and the negative control does not always matter. For example, we can have Z→U , U→Z , W→U , or U→W . However, if both Z and W cause U , then U is a collider in the path Z→U←W . In this case, conditional on U , Z and W will become associated. This violates Assumption 4.

Appendix 2. Example of causal graphs encoding the negative control assumptions

Below, we enumerate the possible relationships among Z , A , U and among Y , W , U in Appendix Table 3 . These partial graphs can be combined into a directed acyclic graph that encodes the negative control assumptions. Grey-colored graphs are invalid because of violation of key assumptions.

Rights and permissions

Reprints and permissions

About this article

Shi, X., Miao, W. & Tchetgen, E.T. A Selective Review of Negative Control Methods in Epidemiology. Curr Epidemiol Rep 7 , 190–202 (2020). https://doi.org/10.1007/s40471-020-00243-4

Download citation

Accepted : 31 August 2020

Published : 15 October 2020

Issue Date : December 2020

DOI : https://doi.org/10.1007/s40471-020-00243-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Bias correction
• Bias detection
• Bias reduction
• Negative control
• Unmeasured confounding

Advertisement

• Find a journal
• Publish with us
• Track your research

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 31 August 2023

Using negative controls to adjust for unmeasured confounding bias in time series studies

• Jie Kate Hu   ORCID: orcid.org/0000-0002-7987-8419 1 ,
• Eric J. Tchetgen Tchetgen 2 &
• Francesca Dominici   ORCID: orcid.org/0000-0002-9382-0141 1  

Nature Reviews Methods Primers volume  3 , Article number:  66 ( 2023 ) Cite this article

604 Accesses

1 Citations

2 Altmetric

Metrics details

• Environmental health
• Epidemiology

Unmeasured confounding threatens the validity of observational studies. Negative control variables (NCs) are variables that either do not cause the outcome of interest or are not caused by the exposure of interest and are increasingly available from emerging sensing technologies and digitized health records. Under appropriate assumptions, NCs can be used to adjust for unmeasured confounding bias. This Primer explains the assumptions and implementation of NCs for unmeasured confounding bias adjustment. Among the method’s broad applications in public health research, time series studies of environmental exposures — air pollution, wildfires and heat — and health outcomes are focused on. Three types of unmeasured confounding in time series studies are considered: time-invariant confounders with time-invariant confounding effects; time-invariant confounders with time-modified confounding effects; and time-varying confounders with immediate and/or lagged confounding effects. For each type of confounding, guidance is provided on how to select NCs using several case studies. Finally, challenges and opportunities are described, to help catalyse additional methodological developments.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 1 digital issues and online access to articles

92,52 € per year

only 92,52 € per issue

Buy this article

• Purchase on Springer Link
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Influence of exposure measurement errors on results from epidemiologic studies of different designs

Causal inference for time series

Improving the design stage of air pollution studies based on wind patterns

Polack, F. P. et al. Safety and efficacy of the BNT162b2 mRNA COVID-19 vaccine. N. Engl. J. Med. 383 , 2603–2615 (2020).

Article   Google Scholar  

Voysey, M. et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 397 , 99–111 (2021).

Baden, L. R. et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N. Engl. J. Med. 384 , 403–416 (2021).

Lopalco, P. L. & DeStefano, F. The complementary roles of phase 3 trials and post-licensure surveillance in the evaluation of new vaccines. Vaccine 33 , 1541–1548 (2015).

Vasileiou, E. et al. Interim findings from first-dose mass COVID-19 vaccination roll-out and COVID-19 hospital admissions in Scotland: a national prospective cohort study. Lancet 397 , 1646–1657 (2021).

Dean, N. E., Hogan, J. W. & Schnitzer, M. E. COVID-19 vaccine effectiveness and the test-negative design. N. Engl. J. Med. 385 , 1431–1433 (2021).

Lewnard, J. A. et al. Theoretical framework for retrospective studies of the effectiveness of SARS-CoV-2 vaccines. Epidemiology 32 , 508–517 (2021).

Shi, X., Miao, W. & Tchetgen, E. T. A selective review of negative control methods in epidemiology. Curr. Epidemiol. Rep. 7 , 190–202 (2020).

Zeger, S. L., Irizarry, R. & Peng, R. D. On time series analysis of public health and biomedical data. Annu. Rev. Public. Health 27 , 57–79 (2006). This article gives an overview of time series study designs and methods used in public health and biomedical research.

Samet, J. M., Dominici, F., Curriero, F. C., Coursac, I. & Zeger, S. L. Fine particulate air pollution and mortality in 20 U.S. cities, 1987–1994. N. Engl. J. Med. 343 , 1742–1749 (2000).

Dominici, F., McDermott, A. & Hastie, T. J. Improved semiparametric time series models of air pollution and mortality. J. Am. Stat. Assoc. 99 , 938–948 (2004).

Article   MathSciNet   MATH   Google Scholar  

Maclure, M. The case-crossover design: a method for studying transient effects on the risk of acute events. Am. J. Epidemiol. 133 , 144–153 (1991).

Neas, L. M., Schwartz, J. & Dockery, D. A case-crossover analysis of air pollution and mortality in Philadelphia. Environ. Health Perspect. 107 , 629–631 (1999).

Flanders, W. D., Strickland, M. J. & Klein, M. A new method for partial correction of residual confounding in time-series and other observational studies. Am. J. Epidemiol. 185 , 941–949 (2017).

Miao, W. & Tchetgen Tchetgen, E. Invited commentary: bias attenuation and identification of causal effects with multiple negative controls. Am. J. Epidemiol. 185 , 950–953 (2017).

Levintow, S. N. et al. Use of negative control outcomes to assess the comparability of patients initiating lipid-lowering therapies. Pharmacoepidemiol. Drug. Saf. 31 , 383–392 (2022).

McGrath, L. J. et al. Using negative control outcomes to assess the comparability of treatment groups among women with osteoporosis in the United States. Pharmacoepidemiol. Drug. Saf. 29 , 854–863 (2020).

Article   MathSciNet   Google Scholar  

Crabtree, B. F., Ray, S. C., Schmidt, P. M., O’Connor, P. T. & Schmidt, D. D. The individual over time: time series applications in health care research. J. Clin. Epidemiol. 43 , 241–260 (1990).

Miettinen, O. Confounding and effect-modification. Am. J. Epidemiol. 100 , 350–353 (1974).

Miettinen, O. S. & Cook, E. F. Confounding: essence and detection. Am. J. Epidemiol. 114 , 593–603 (1981).

Robins, J. M. & Greenland, S. The role of model selection in causal inference from nonexperimental data. Am. J. Epidemiol. 123 , 392–402 (1986).

Kleinbaum, D. G., Kupper, L. L. & Morgenstern, H. Epidemiologic Research: Principles and Quantitative Methods (Wiley, 1991).

Pearl, J. Causality (Cambridge Univ. Press, 2009). Chapters 1–3 of this book provide a systematic account of the concepts in causal inference, the relevant mathematical tools and the assumptions needed for drawing causal claims from data.

VanderWeele, T. J. & Shpitser, I. On the definition of a confounder. Ann. Stat. 41 , 196–220 (2013).

Pearl, J. Causal diagrams for empirical research. Biometrika 82 , 669–688 (1995).

Shrier, I. & Platt, R. W. Reducing bias through directed acyclic graphs. BMC Med. Res. Methodol. 8 , 70 (2008).

Lipsitch, M., Tchetgen Tchetgen, E. & Cohen, T. Negative controls: a tool for detecting confounding and bias in observational studies. Epidemiology 21 , 383–388 (2010). This article introduces and distinguishes two types of negative controls, exposure and outcome controls, and describes how to use them to detect confounding in epidemiological studies.

Angrist, J. D., Imbens, G. W. & Rubin, D. B. Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91 , 444–455 (1996).

Article   MATH   Google Scholar  

Schwartz, J., Bind, M.-A. & Koutrakis, P. Estimating causal effects of local air pollution on daily deaths: effect of low levels. Environ. Health Perspect. 125 , 23–29 (2017).

Lousdal, M. L. An introduction to instrumental variable assumptions, validation and estimation. Emerg. Themes Epidemiol. 15 , 1 (2018).

Hernán, M. A. & Robins, J. M. Instruments for causal inference: an epidemiologist’s dream? Epidemiology 17 , 360–372 (2006).

Meyer, B. D. Natural and quasi-experiments in economics. J. Bus. Economic Stat. 13 , 151–161 (1995).

Google Scholar  

Abadie, A. Semiparametric difference-in-differences estimators. Rev. Economic Stud. 72 , 1–19 (2005).

Sofer, T., Richardson, D. B., Colicino, E., Schwartz, J. & Tchetgen Tchetgen, E. J. On negative outcome control of unobserved confounding as a generalization of difference-in-differences. Stat. Sci. 31 , 348–361 (2016).

Carroll, R. J., Ruppert, D., Stefanski, L. A. & Crainiceanu, C. M. Measurement Error in Nonlinear Models: A Modern Perspective (Chapman and Hall/CRC, 2006).

Splawa-Neyman, J., Dabrowksa, D. M. & Speed, T. P. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Statist. Sci. 5 , 465–472 (1990).

Holland, P. W. Causal inference, path analysis, and recursive structural equations models. Sociological Methodol. 18 , 449–484 (1988).

Pearl, J. in Proc. Workshop on Causality: Objectives and Assessment at NIPS 2008 (eds Guyon, I., Janzing, D. & Schölkopf, B.) 39–58 (PMLR, 2010).

Rosenbaum, P. R. & Rubin, D. B. The central role of the propensity score in observational studies for causal effects. Biometrika 70 , 41–55 (1983).

Tchetgen, E. J. T., Ying, A., Cui, Y., Shi, X. & Miao, W. An introduction to proximal causal learning. Preprint at https://doi.org/10.48550/arXiv.2009.10982 (2020).

Miao, W., Geng, Z. & Tchetgen Tchetgen, E. Identifying causal effects with proxy variables of an unmeasured confounder. Biometrika 105 , 987–993 (2018). This article shows that with at least two independent proxy variables Z and W of an unmeasured confounder U , the causal effect is non-parametrically identified, satisfying certain conditions without identifying Pr( W  |  U ).

Armstrong, B. G. Effect of measurement error on epidemiological studies of environmental and occupational exposures. Occup. Environ. Med. 55 , 651–656 (1998).

Kuroki, M. & Pearl, J. Measurement bias and effect restoration in causal inference. Biometrika 101 , 423–437 (2014). This article shows that graphical techniques can be harnessed to address the problem of measurement errors, and discusses how, when only a proxy variable W of an unmeasured confounder U is observed, to estimate causal effects by identifying the error mechanism Pr( W  |  U ) from prior knowledge or from another proxy measurement of U .

Glynn, R. J., Knight, E. L., Levin, R. & Avorn, J. Paradoxical relations of drug treatment with mortality in older persons. Epidemiology 12 , 682–689 (2001).

Stürmer, T., Rothman, K. J., Avorn, J. & Glynn, R. J. Treatment effects in the presence of unmeasured confounding: dealing with observations in the tails of the propensity score distribution — a simulation study. Am. J. Epidemiol. 172 , 843–854 (2010).

Lawlor, D. A., Davey Smith, G., Kundu, D., Bruckdorfer, K. R. & Ebrahim, S. Those confounded vitamins: what can we learn from the differences between observational versus randomised trial evidence? Lancet 363 , 1724–1727 (2004).

Schuemie, M. J., Ryan, P. B., DuMouchel, W., Suchard, M. A. & Madigan, D. Interpreting observational studies: why empirical calibration is needed to correct P -values. Stat. Med. 33 , 209–218 (2014).

Dominici, F. et al. Fine particulate air pollution and hospital admission for cardiovascular and respiratory diseases. JAMA 295 , 1127–1134 (2006).

Bell, M. L. et al. Seasonal and regional short-term effects of fine particles on hospital admissions in 202 US counties, 1999–2005. Am. J. Epidemiol. 168 , 1301–1310 (2008).

Kloog, I. et al. Short term effects of particle exposure on hospital admissions in the Mid-Atlantic states: a population estimate. PLoS ONE 9 , e88578 (2014).

Article   ADS   Google Scholar  

Reid, J. S., Koppmann, R., Eck, T. F. & Eleuterio, D. P. A review of biomass burning emissions part II: intensive physical properties of biomass burning particles. Atmos. Chem. Phys. 5 , 799–825 (2005).

O’Dell, K. et al. Hazardous air pollutants in fresh and aged western US wildfire smoke and implications for long-term exposure. Environ. Sci. Technol. 54 , 11838–11847 (2020).

Lill, E. et al. Wildfire-driven changes in the abundance of gas-phase pollutants in the city of Boise, ID during summer 2018. Atmos. Pollut. Res. 13 , 101269 (2022).

Liu, J. C., Pereira, G., Uhl, S. A., Bravo, M. A. & Bell, M. L. A systematic review of the physical health impacts from non-occupational exposure to wildfire smoke. Environ. Res. 136 , 120–132 (2015).

Wood, L., Hooper, P., Foster, S. & Bull, F. Public green spaces and positive mental health — investigating the relationship between access, quantity and types of parks and mental wellbeing. Health Place. 48 , 63–71 (2017).

Cavanagh, J.-A. E., Zawar-Reza, P. & Wilson, J. G. Spatial attenuation of ambient particulate matter air pollution within an urbanised native forest patch. Urban. For. Urban Green. 8 , 21–30 (2009).

Trethewey, S. P. & Reynolds, E. K. M. Exposure to green spaces and all-cause mortality: limitations in measurement and definitions of exposure. Lancet Planet. Health 5 , e501 (2021).

Holland, I. et al. Measuring nature contact: a narrative review. Int. J. Env. Res. Public. Health 18 , 4092 (2021).

Gascon, M. et al. Mental health benefits of long-term exposure to residential green and blue spaces: a systematic review. Int. J. Environ. Res. Public. Health 12 , 4354–4379 (2015).

Tarpley, J. D., Schneider, S. R. & Money, R. L. Global vegetation indices from the NOAA-7 meteorological satellite. J. Appl. Meteorol. Climatol. 23 , 491–494 (1984).

Kogan, F. N. Droughts of the late 1980s in the United States as derived from NOAA polar-orbiting satellite data. Bull. Am. Meteorol. Soc. 76 , 655–668 (1995).

Clark, C. and Stansfeld, S. A. The effect of transportation noise on health and cognitive development: a review of recent evidence. Int. J. Comp. Psychol . 20 , 145–158 (2007).

Schneising, O., Buchwitz, M., Reuter, M., Bovensmann, H. & Burrows, J. P. Severe Californian wildfires in November 2018 observed from space: the carbon monoxide perspective. Atmos. Chem. Phys. 20 , 3317–3332 (2020).

Bobb, J. F., Obermeyer, Z., Wang, Y. & Dominici, F. Cause-specific risk of hospital admission related to extreme heat in older adults. JAMA 312 , 2659 (2014).

Simonson, A. et al. in Big Data Analytics in Earth, Atmospheric, and Ocean Sciences (eds Huang, T., Vance, T. & Lynnes, C.) 65–94 (American Geophysical Union (AGU), 2022).

Ramapriyan, H. K. The role and evolution of NASA’s earth science data systems. https://ntrs.nasa.gov/citations/20150018076 (NASA, 2015).

Flanders, W. D. et al. A method for detection of residual confounding in time-series and other observational studies. Epidemiology 22 , 59–67 (2011). This article pioneers the use of future exposure measurements in time series studies to detect unmeasured confounding — pointing out that in the absence of unmeasured confounding, future exposure should be independent of past outcome, but should not otherwise.

Goodman, S. N., Fanelli, D. & Ioannidis, J. P. A. What does research reproducibility mean? Sci. Transl. Med. 8 , 341ps12 (2016).

Peng, R. D. & Hicks, S. C. Reproducible research: a retrospective. Annu. Rev. Public. Health 42 , 79–93 (2021).

Murray-Rust, P. Open data in science. Nat. Prec. https://doi.org/10.1038/npre.2008.1526.1 (2008).

Ghassami, A., Ying, A., Shpitser, I. & Tchetgen, E. T. in Proc. 25th Int. Conf. Artificial Intelligence and Statistics (eds Camps-Valls, G., Ruiz, F. J. R. & Valera, I.) 7210–7239 (PMLR, 2022).

Mansournia, M. A., Etminan, M., Danaei, G., Kaufman, J. S. & Collins, G. Handling time varying confounding in observational research. BMJ 359 , j4587 (2017).

Ying, A., Miao, W., Shi, X. & Tchetgen Tchetgen, E. J. Proximal causal inference for complex longitudinal studies. J. R. Stat. Soc. Series B Stat. Methodol. 85 , 684–704 (2023).

Greenland, S. Confounding and exposure trends in case-crossover and case–time–control designs. Epidemiology 7 , 231–239 (1996).

Perkins, S. E. & Alexander, L. V. On the measurement of heat waves. J. Clim. 26 , 4500–4517 (2013).

Klompmaker, J. O. et al. Green space definition affects associations of green space with overweight and physical activity. Environ. Res. 160 , 531–540 (2018).

Faurot, K. R. et al. Using claims data to predict dependency in activities of daily living as a proxy for frailty. Pharmacoepidemiol. Drug. Saf. 24 , 59–66 (2015).

Download references

Acknowledgements

J.K.H. thanks the National Institute of Environmental Health Sciences (T32 ES 7069), Sloan Foundation (G-2020-13946) and Environmental Protection Agency (CR-83467701) for financial support. E.J.T.T. thanks the National Institutes of Health (NIH) (R01AG065276), National Cancer Institute (NCI) (R01CA222147), General Medical Sciences (R01GM139926) and National Institute of Allergy and Infectious Diseases (R01AI27271) for financial support. F.D. thanks the NIH (R01ES026217, R01MD012769, R01ES028033, 5R01AG060232-03, 1R01ES030616, 1R01AG066793, 1R01ES029950, 1R01ES 034373-01) and Sloan Foundation (G-2020-13946) for financial support.

Author information

Authors and affiliations.

Department of Biostatistics, Harvard T. H. Chan School of Public Health, Boston, MA, USA

Jie Kate Hu & Francesca Dominici

Department of Statistics and Data Science, Wharton School, University of Pennsylvania, Philadelphia, PA, USA

Eric J. Tchetgen Tchetgen

You can also search for this author in PubMed   Google Scholar

Contributions

Introduction (J.K.H. and F.D.); Experimentation (J.K.H., E.J.T.T. and F.D.); Results (J.K.H. and E.J.T.T.); Applications (J.K.H. and F.D.); Reproducibility and data deposition (J.K.H. and F.D.); Limitations and optimizations (J.K.H. and E.J.T.T.); Outlook (J.K.H., E.J.T.T. and F.D.); Overview of the Primer (all authors).

Corresponding author

Correspondence to Jie Kate Hu .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Peer review

Peer review information.

Nature Reviews Methods Primers thanks Sara Levintow; W. Dana Flanders; and William Henley, who co-reviewed with Sharlene Alauddin, for their contribution to the peer review of this work.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Related links

AirNow: https://gispub.epa.gov/airnow

GRIDMET: https://developers.google.com/earth-engine/datasets/catalog/IDAHO_EPSCOR_GRIDMET

Highest September temperatures in Napa: https://www.extremeweatherwatch.com/cities/napa/month-september/highest-temperatures

NASA: https://data.nasa.gov/

National Center of Health Statistics: https://data.cdc.gov/

NOAA: https://data.noaa.gov/

Supplementary information

Supplementary information.

An estimator for a parameter is asymptotically unbiased if its expectation converges to the true value of the parameter when the sample size is large enough.

A graphic test in which a set of variables U satisfies the backdoor criterion relative to an ordered pair of variables ( A , Y ) in a directed acyclic graph (DAG) if no node in U is a descendant of A ; and U blocks every path between A and Y that contains an arrow into A .

A characteristic that cannot be quantifiable. Categorical variables can be either nominal or ordinal.

The process of using data for uncovering causal relationships between variables.

(DAG). A graph contains a set of vertices (nodes) and a set of edges that connect some pairs of vertices. If every edge in a graph is an arrow that points from the first to the second vertex, we have a directed graph. A DAG is a graph that is directed and without directed cycles.

(NCE). A variable Z is an NCE if it is known a priori not to cause outcome Y , and the association between Z and Y is subject to the same unmeasured confounding mechanism as between exposure A and outcome Y .

(NCO). A variable W is an NCO if it is known a priori not to be caused by exposure A and the association between A and W is subject to the same unmeasured confounding mechanism as between exposure A and outcome Y .

The measurement error of a confounder is said to be non-differential if the measured confounder is conditionally independent of the exposure and outcome, given the true confounder.

The process of using a sample to make inferences about a population.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Cite this article.

Hu, J.K., Tchetgen Tchetgen, E.J. & Dominici, F. Using negative controls to adjust for unmeasured confounding bias in time series studies. Nat Rev Methods Primers 3 , 66 (2023). https://doi.org/10.1038/s43586-023-00249-4

Download citation

Accepted : 10 July 2023

Published : 31 August 2023

DOI : https://doi.org/10.1038/s43586-023-00249-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Home » Science » Chemistry » Biochemistry » Difference Between Positive and Negative Control

Difference Between Positive and Negative Control

Main difference – positive vs negative control.

Scientific control is a methodology that tests integrity in experiments by isolating variables as dictated by the scientific method in order to make a conclusion about such variables. It can be defined as an experiment that is designed to minimize the effect of variables other than the independent variables. (The things that are changing in an experiment are called variables). An experiment can be positively or negatively controlled. The main difference between positive and negative control is that positive control gives a response to the experiment whereas negative control does not give any response.

Key Areas Covered

1. What is Positive Control      – Definition, Process, Uses 2. What is Negative Control      – Definition, Process 3. What is the Difference Between Positive and Negative Control     – Comparison of Key Differences

Key Terms: Assay, Control, Experiment, Negative Control, Positive Control

What is Positive Control

A positive control is an experimental control that gives a positive result at the end of the experiment. This type of test always gives the result as a “yes”. It is a good indication to know if the test works. Hence, positive controls are used to evaluate the validity of a test.

The positive control is not exposed to the experimental test; it is done parallel to it. The positive control is used to get the expected result. This positive result ensures the success of the test. Once the positive result is given, the test can be used for the experimental treatment. If the positive control does not give the expected result, it should be done again and again (by varying different parameters) until a positive result is given.

Figure 1: ELISA experiment – An Enzyme Assy

There are many applications of positive control in biochemical experiments.

• To detect a disease
• To observe the growth of microorganisms
• To measure the amount of enzymes present after an enzyme assay is done (in positive control, the amount of enzyme after the purification should be a known amount)

What is Negative Control

A negative control is an experimental control that does not give a response to the test. The negative control is also not exposed to the experimental test directly. It is done parallel to the experiment as a control experiment.

The negative control is used to confirm that there is no response to the reagent or the microorganism (or any other parameter) used in the test. In order to get a good result from the negative control, one should ensure that there is no net response to the test. Hence, negative controls are helpful in identifying outside influences on the experiment. For example, the effect of contaminants on an experiment can be indicated.

Positive Control: A positive control is an experimental control that gives a positive result at the end of the experiment.

Negative Control: A negative control is an experimental control that does not give a response to the test.

Positive Control: Positive control gives positive result

Negative Control: Negative control gives a negative result.

Positive Control: Positive control gives a response to the experiment.

Negative Control: Negative control does not give any response.

Positive Control: Positive control ensures the success of the test.

Negative Control: Negative control is used to ensure that there is no response to the test.

Positive Control: Positive control is used to test the validity of an experiment.

Negative Control: Negative control is used to identify the influence of external factors on the test.

Positive control and negative control are two types of tests that give completely opposite responses in an experiment. The main difference between positive and negative control is that positive control gives a response to the experiment whereas negative control does not give any response.

 1. “Scientific Control.” The Titi Tudorancea Bulletin, Available here . 2. “Scientific control.” Wikipedia, Wikimedia Foundation, 24 Jan. 2018, Available here .

About the Author: Madhusha

Madhusha is a BSc (Hons) graduate in the field of Biological Sciences and is currently pursuing for her Masters in Industrial and Environmental Chemistry. Her interest areas for writing and research include Biochemistry and Environmental Chemistry.

​You May Also Like These

Leave a reply cancel reply.

Control Group vs Experimental Group

Julia Simkus

Editor at Simply Psychology

BA (Hons) Psychology, Princeton University

Julia Simkus is a graduate of Princeton University with a Bachelor of Arts in Psychology. She is currently studying for a Master's Degree in Counseling for Mental Health and Wellness in September 2023. Julia's research has been published in peer reviewed journals.

Learn about our Editorial Process

Saul McLeod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul McLeod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

In a controlled experiment , scientists compare a control group, and an experimental group is identical in all respects except for one difference – experimental manipulation.

Differences

Unlike the experimental group, the control group is not exposed to the independent variable under investigation. So, it provides a baseline against which any changes in the experimental group can be compared.

Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between the two are due to experimental manipulation rather than chance.

Almost all experimental studies are designed to include a control group and one or more experimental groups. In most cases, participants are randomly assigned to either a control or experimental group.

Because participants are randomly assigned to either group, we can assume that the groups are identical except for manipulating the independent variable in the experimental group.

It is important that every aspect of the experimental environment is the same and that the experimenters carry out the exact same procedures with both groups so researchers can confidently conclude that any differences between groups are actually due to the difference in treatments.

Control Group

A control group consists of participants who do not receive any experimental treatment. The control participants serve as a comparison group.

The control group is matched as closely as possible to the experimental group, including age, gender, social class, ethnicity, etc.

The difference between the control and experimental groups is that the control group is not exposed to the independent variable , which is thought to be the cause of the behavior being investigated.

Researchers will compare the individuals in the control group to those in the experimental group to isolate the independent variable and examine its impact.

The control group is important because it serves as a baseline, enabling researchers to see what impact changes to the independent variable produce and strengthening researchers’ ability to draw conclusions from a study.

Without the presence of a control group, a researcher cannot determine whether a particular treatment truly has an effect on an experimental group.

Control groups are critical to the scientific method as they help ensure the internal validity of a study.

Assume you want to test a new medication for ADHD . One group would receive the new medication, and the other group would receive a pill that looked exactly the same as the one that the others received, but it would be a placebo. The group that takes the placebo would be the control group.

Types of Control Groups

Positive control group.

• A positive control group is an experimental control that will produce a known response or the desired effect.
• A positive control is used to ensure a test’s success and confirm an experiment’s validity.
• For example, when testing for a new medication, an already commercially available medication could serve as the positive control.

Negative Control Group

• A negative control group is an experimental control that does not result in the desired outcome of the experiment.
• A negative control is used to ensure that there is no response to the treatment and help identify the influence of external factors on the test.
• An example of a negative control would be using a placebo when testing for a new medication.

Experimental Group

An experimental group consists of participants exposed to a particular manipulation of the independent variable. These are the participants who receive the treatment of interest.

Researchers will compare the responses of the experimental group to those of a control group to see if the independent variable impacted the participants.

An experiment must have at least one control group and one experimental group; however, a single experiment can include multiple experimental groups, which are all compared against the control group.

Having multiple experimental groups enables researchers to vary different levels of an experimental variable and compare the effects of these changes to the control group and among each other.

Assume you want to study to determine if listening to different types of music can help with focus while studying.

You randomly assign participants to one of three groups: one group that listens to music with lyrics, one group that listens to music without lyrics, and another group that listens to no music.

The group of participants listening to no music while studying is the control group, and the groups listening to music, whether with or without lyrics, are the two experimental groups.

Frequently Asked Questions

1. what is the difference between the control group and the experimental group in an experimental study.

Put simply; an experimental group is a group that receives the variable, or treatment, that the researchers are testing, whereas the control group does not. These two groups should be identical in all other aspects.

2. What is the purpose of a control group in an experiment

A control group is essential in experimental research because it:

Provides a baseline against which the effects of the manipulated variable (the independent variable) can be measured.

Helps to ensure that any changes observed in the experimental group are indeed due to the manipulation of the independent variable and not due to other extraneous or confounding factors.

Helps to account for the placebo effect, where participants’ beliefs about the treatment can influence their behavior or responses.

In essence, it increases the internal validity of the results and the confidence we can have in the conclusions.

3. Do experimental studies always need a control group?

Not all experiments require a control group, but a true “controlled experiment” does require at least one control group. For example, experiments that use a within-subjects design do not have a control group.

In  within-subjects designs , all participants experience every condition and are tested before and after being exposed to treatment.

These experimental designs tend to have weaker internal validity as it is more difficult for a researcher to be confident that the outcome was caused by the experimental treatment and not by a confounding variable.

4. Can a study include more than one control group?

Yes, studies can include multiple control groups. For example, if several distinct groups of subjects do not receive the treatment, these would be the control groups.

5. How is the control group treated differently from the experimental groups?

The control group and the experimental group(s) are treated identically except for one key difference: exposure to the independent variable, which is the factor being tested. The experimental group is subjected to the independent variable, whereas the control group is not.

This distinction allows researchers to measure the effect of the independent variable on the experimental group by comparing it to the control group, which serves as a baseline or standard.

Bailey, R. A. (2008). Design of Comparative Experiments. Cambridge University Press. ISBN 978-0-521-68357-9.

Hinkelmann, Klaus; Kempthorne, Oscar (2008). Design and Analysis of Experiments, Volume I: Introduction to Experimental Design (2nd ed.). Wiley. ISBN 978-0-471-72756-9.

Positive and Negative Controls

This is part of the NSW HSC science curriculum part of the Working Scientifically skills.

Positive and Negative Controls Explained

Introduction to Controls in Scientific Experiments

Controls are standard benchmarks used in experiments to ensure that the results are due to the factor being tested and not some external influence. There are two main types of controls: positive and negative. Controls play an important part in ensuring that the experimental results are valid.

Note that controls and controlled variables refer to different aspects of experiments.

Positive Controls

Positive controls are used in experiments to show what a positive result looks like. They ensure that the testing procedure is capable of producing results when the expected outcome is present.  They involve using a material or condition known to produce a positive result.

Positive controls confirm that the experimental setup can detect positive results and that all reagents and instruments are functioning correctly and as intended.

Negative Controls

Negative controls, on the other hand, are used to ensure that no change is observed when a change is not expected. They help confirm that any positive result in the experiment is truly due to the test condition and not due to external factors.

Why Do We Use Positive and Negative Controls?

Rule Out False Positives : Negative controls help in ruling out the possibility that external factors are causing the observed effect.

No Expected Outcome : These controls involve using a material or condition known not to produce the effect being tested.

Validity and Reliability : Positive and negative controls are crucial for establishing the validity and reliability of an experiment. They provide a way of checking whether the experimental method actually tests the what it's supposed to test, and a basis for comparison to the experimental group.

Error Identification : Controls can help identify errors in the experimental setup or procedure, ensuring that the results of an experiment are due to the variable being tested.

Interpretation of Results : Understanding what constitutes normal variation in an experiment is essential for accurately interpreting results.

Example of Controls in Chemistry

Experiment : Testing the Presence of Vitamin C in Fruit Juice

Aim:  To determine whether a particular fruit juice contains Vitamin C.

Positive Control : For this experiment, a known Vitamin C solution can be used. This solution should react positively with the testing reagent (like DCPIP, which changes colour in the presence of Vitamin C) to show that the test can indeed identify Vitamin C when it is present.

Negative Control : Distilled water serves as an effective negative control. It does not contain Vitamin C and should not react with the testing reagent. Any change in the negative control indicates contamination or an error in the experimental procedure.

Example of Controls in Physics

Experiment: Investigating Newton's Second Law of Motion

Aim : To verify Newton's Second Law of Motion, which states that the acceleration of an object is directly proportional to the net force acting on it and inversely proportional to its mass (`F = ma`).

Experimental Setup:

Students use a dynamic cart on a track, a set of known masses, a pulley system, and a force sensor or photogate timer to measure acceleration.

A photogate measures the velocity of cart by using how long the light beam within the gate has been obstructed by the opaque band mounted on the moving cart. By using the velocities measured by the two photogates and the time difference between the two, acceleration of the cart can be determined.

You can use the following simulation to familiarise with photogates.

• Positive Control: To ensure that the experimental setup e.g. photogate can correctly measure acceleration, use a cart with known mass and a predetermined force (e.g. weight force of a where `F = ma` can be accurately calculated. This setup should produce a predictable acceleration. When the experiment is conducted with these known values, the measured acceleration should closely match the theoretical acceleration calculated. This confirms that the equipment (force sensor, photogate timer, etc.) is functioning correctly and the experimental procedure is valid.
• Negative Control: To ensure that the measured acceleration is solely due to the applied force and not any other factors like friction or air resistance, conduct an experiment with no external force applied (other than the minimal force to overcome static friction). This can be done by using a dynamic cart on a level track without adding any additional weights or forces. The cart should exhibit minimal to no acceleration, indicating that any acceleration measured in the main experiment is due to the applied force and not inherent biases or errors in the setup.

RETURN TO WORKING SCIENTIFICALLY

• choosing a selection results in a full page refresh
• press the space key then arrow keys to make a selection

• Faculty Resource Center
• Biochemistry
• Bioengineering
• Cancer Research
• Developmental Biology
• Engineering
• Environment
• Immunology and Infection
• Neuroscience
• JoVE Journal
• JoVE Encyclopedia of Experiments
• JoVE Chrome Extension
• Environmental Sciences
• Pharmacology
• JoVE Science Education
• JoVE Lab Manual
• JoVE Business
• Videos Mapped to your Course
• High Schools
• Videos Mapped to Your Course

Chapter 12: Statistics in Practice

Back to chapter, controls in experiments, previous video 12.6: crossover experiments, next video 12.10: clinical trials.

Controls in an experiment are elements that are held constant and not affected by independent variables. Controls are essential for unbiased and accurate measurement of the dependent variables in response to the treatment.

For example, patients reporting in a hospital with high-grade fever, breathing difficulty, cough, cold, and severe body pain are suspected of COVID infection. But it is  also possible that other respiratory infection causes the same symptoms. So, the doctor recommends a COVID test.

The patient's nasal swabs are collected, and the  COVID test is performed. In addition, a control sample is maintained that does not have COVID viral RNA. This type of control is also called negative control. It helps to prevent false positive reports in patients' samples.

A positive control is another commonly used type of control in an experiment. Unlike the negative control, the positive control contains an actual sample – the viral RNA. This helps to match the presence of viral RNA in the test samples, and it validates the procedure and accuracy of the test.

When conducting an experiment, it is crucial to have control to reduce bias and accurately measure the dependent variables. It also marks the results more reliable. Controls are elements in an experiment that have the same characteristics as the treatment groups but are not affected by the independent variable. By sorting these data into control and experimental conditions, the relationship between the dependent and independent variables can be drawn. A randomized experiment always includes a control group that receives an inactive treatment but is otherwise managed exactly as the other groups. The control group helps researchers balance the effects of being in an experiment with the effects of the active treatments.

In clinical or diagnostic procedures, positive controls are included to validate the test results. The positive controls would show the expected result if the test had worked as expected. A negative control does not contain the main ingredient or treatment but includes everything else. For example, in a COVID RT-PCR test, a negative sample does not include the viral DNA. Experiments often use positive and negative controls to prevent or avoid false positives and false negative reports. In

This text is adapted from Openstax, Introductory Statistics, Section 1.4, Experimental Design and Ethics

Positive and Negative Control in Microbiology

Positive and Negative Control, Microbiology, bacteriology, virology, mycology, Positive vs Negative Control, different between Positive and Negative Control.

This discussion provides insights into positive controls, where an testing sample is tested, and negative controls, which aim to detect possible procedural errors.

Table of Contents

Basics of Positive Control in Microbiology

Positive control in microbiology: the basics, importance of positive control.

Positive controls offer a validation mechanism for microbiological research. They ensure that the experimental setup functions as intended and that any negative results are exact and not because the experiment itself failed. Because it contains known organisms that can successfully be grown, a positive control proves that the lab conditions, chemicals, and methods used in the experiment are effective. Therefore, should an experimental test fail to produce expected results, scientists will know to question the experimental procedures put in place.

Positive Control in Different Microbiology Areas

In various microbiology areas such as bacteriology , virology , and mycology , usage of positive controls is prevalent.

For example, in bacteriology, E. coli is often introduced as a positive control when testing for coliform bacteria in water systems. Because E. coli is easily identifiable and frequently present in contaminated water, it assists scientists in confirming that their methods for cultural, identification, and counting are adequate.

In the field of mycology, which deals with fungi, Aspergillus or Penicillium might be used as positive controls depending on the context of the experiment. The results from this control assist in validating the growth conditions and reagents employed in the experiment.

Negative Control in Microbiology: The Basics

By contrast, a negative control in microbiological tests is a parallel test setup using conditions known to give no response. This test is crucial because it demonstrates the absence of non-specific effects and validates the specificity of the results.

For instance, in an antimicrobial test, a negative control includes all the experimental setup elements but omits known antimicrobial substances. If growth is observed in this control, it helps establish that the test organism could grow under the test conditions, proving that any non-growth in the experiment demonstrates the antimicrobial’s effectiveness.

Similarly, in a polymerase chain reaction (PCR) testing, a negative control, such as sterile water, is included with each batch of reactions. The absence of amplification (no bands in gel electrophoresis) validates that there are no non-specific amplifications due to contaminants.

Hence, the negative control contributes to ensuring the reliability and the specificity of the experiment.

Understanding Negative Control in Microbiology

An introduction to negative control in microbiology.

The concept of negative control in microbiology forms a key part of research design, serving as a ‘benchmark’ or ‘norm’ for contrasting and evaluating the results of the experiment. Essentially, a negative control is a subset in a particular study not expected to yield a significant outcome, which verifies that the observed effects were not caused by the experimental process itself.

Significance of Negative Control in Microbiology Experiments

Functions of negative control: eliminating false positives and errors, examples of negative control in several microbiology fields.

In various fields of microbiology, negative controls are frequently used. For instance, in antibiotic susceptibility testing, a bacterial sample is cultured in the absence of antibiotics. If the bacteria still do not grow, it signifies a problem with the culture conditions or the bacteria itself, not necessarily the antibiotics’ effectiveness.

Understanding Positive Control in Microbiology

Interaction of positive and negative controls in microbiology.

The interplay between negative and positive controls is crucial for accurate, credible experimental outcomes. While negative controls help rule out extraneous effects or false positives, positive controls ensure the experiment is functioning as intended.

Essential Role of Controls in Microbiology Experiments

Difference between positive and negative controls, positive vs negative controls.

Positive and negative controls are cornerstones in microbiology experiments, functioning as instrumental tools in corroborating the dependability of the obtained results. While they follow a symbiotic relationship in the context of maintaining standards, their individual functions differ significantly.

Exploring Positive Controls

Delving into negative controls.

Conversely, a negative control does not involve any change. Typically, this sample encompasses a test environment, such as a microbiological growth medium, which does not contain any targeted bacteria or associated treatment. Negative controls play an integral role in confirming that any observed deviations originate from experimental procedures and not from extraneous or non-intentional factors including contamination.

Significance of Positive and Negative Controls

Together, positive and negative controls help develop faith in experimental outcomes by affirming the experiment’s validity. They also aid in capturing potential error sources.

In contrast, negative controls point out if non-intentional effects are introduced into the experimental setup. Discrepancies in the negative control signal towards possible contamination or equipment malfunction.

Appropriate Usage of Positive and Negative Controls

Positive and negative controls are employed throughout different stages of a microbiology experiment.

While the nature of these controls varies, their importance in assuring reliable, reproducible results that can be accepted by the scientific community remains constant regardless of the experimental setup.

Making easier to understand the concepts of positive and negative controls essentially boils down to their core roles in scientific experiments. Positive controls serve as a benchmark to confirm that the experimental setup is working as intended, while negative controls act as the litmus test for accidental errors, preventing the intrusion of false positives. Across bacteriology, virology, mycology, and more, these controls ensure that the scientific findings we base our decisions on are accurate and reliable.

FAQ – Positive and Negative Control in Microbiology

What is a positive control in microbiology, what is negative control test in microbiology.

A negative control test is an experiment in which the microbiologist knows that there will be a negative outcome. This is done to ensure that the test is not contaminated and that the results are accurate. For example, in a test for the presence of bacteria, a negative control would be a sterile solution. If the test detects bacteria in the negative control, then it is likely that the test is contaminated.

What is a positive and negative control example?

A positive control is an experiment that is expected to produce a positive outcome. For example, a positive control for a test for the presence of bacteria would be a known bacteria culture. A negative control is an experiment that is expected to produce a negative outcome. For example, a negative control for a test for the presence of bacteria would be a sterile solution.

Why use negative control in microbiology?

What is the difference between positive and negative control groups.

Positive control groups are exposed to a treatment that is known to produce a specific outcome. Negative control groups are not exposed to any treatment and are expected to have no change.

Leave a comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

• Call Us: +1 484.791.3823
• Sign In or Register
• Items My Cart
• Accept Quote

Positive and Negative Controls

To reduce variables in any type of experiment, it is recommended to include both positive and negative controls in the experimental design. Negative controls are particular samples included in the experiment that are treated the same as all the others but are not expected to change from any variable in the experiment. The positive control sample will show an expected result, helping the scientist understand that the experiment was performed properly. Some controls are specific to the type of experiment being performed, such as molecular weight standards used in protein or DNA gel electrophoresis, i.e. SDS-PAGE or agarose gel electrophoresis. The proper selection and use of controls ensures that experimental results are valid and can save valuable time.

Loading Control Antibodies

Loading control antibodies mostly recognize housekeeping proteins in cells used in a scientific experiment and allow the verification of equal protein loading between samples. Ideal loading controls are expressed constitutively and at high levels with low variability between cell lines and experimental conditions.

Loading controls are essential for the interpretation of assays. In Western blot assays, the loading control should be at a different molecular weight than the protein of interest, as this allows the protein to be visually distinguishable. Loading control antibodies not only allow the verification of equal protein loading between samples in Western blot assays, but they also allow for identification of certain cell compartmentalization or cellular localization in immunofluorescence microscopy (IF) and immunohistochemistry (IHC). Rockland’s loading control antibodies are suitable in assays including ELISA, FLISA, Western blot, IF, and IHC.

Western blot of pERK1/2, ERK1/2 . α-tubulin is used as a control to illustrate uniform protein loading.

Featured Loading Control Antibodies

Alpha-Tubulin Antibody

Beta Actin Antibody

GAPDH Antibody

Control cell lysates and nuclear extracts.

Rockland offers control cell lysates and nuclear extracts for use on SDS-PAGE as standalone samples or in combination with antibodies in Western blotting experiments. Our ready-to-use whole-cell lysates and nuclear extracts are derived from cell lines or tissues using highly advanced extraction protocols to ensure high quality, protein integrity, and lot-to-lot reproducibility.

Lysates are generated from either whole cells, which contain cell membrane, cytoplasmic, and nuclear proteins, or nuclear extracts, which are predominantly proteins that originate in the nucleus. Control lysates may be from cells that are stimulated with insulin, doxorubicin, etoposide, nocodozole, TNFa, or EGF. Lysates are also available from normal animal tissue derived from primary organs such as liver, heart, and brain. Additionally, Rockland offers a variety of lysates that contain over-expressed proteins (tagged and untagged) that can serve as positive controls for antibody reactivity. All extracts are tested by SDS-PAGE using 4–20% gradient gels and immunoblot analysis using antibodies to key cell signaling components to confirm the presence of both high molecular weight and low molecular weight proteins.

Featured Control Cell Lysates

Human Foreskin Fibroblast Whole Cell Lysate

A431 Whole Cell Lysate EGF Stimulated

E.coli HCP Control

12 Epitope Tag Control Lysate (GST)

Purified proteins.

Purified proteins or peptides are ideal as controls in flow cytometry, Western blot, and ELISA. Proteins can be used as loading controls in Western blot experiments or as titration agents in ELISA experiments. Rockland produces purified immunoglobulin proteins from a variety of species, often available by immunoglobulin class or as fragments of immunoglobulins. Peptides can be used to do competition assays or to be used in peptide arrays.

Featured Control Proteins

Hamster IgG

Low endotoxin controls.

Low endotoxin control proteins are IgG preparations of control serum purified by protein A chromatography using a low endotoxin methodology. These controls are ideal in biological assays like neutralization experiments, ELISA, flow cytometry, and other assays. For neutralization assays, where antibodies to cytokines, interleukins, infectious disease, and growth factors may be used to block bioactivity, our low endotoxin IgG serve as ideal control proteins. Rockland offers purified, low-endotoxin mouse and rabbit IgG.

Low Endotoxin Controls:

Related Resources

Tips for western blotting, tips for optimal sds-page separation, validating antibodies for western blotting, shopping cart.

Positive and Negative Controls

In chemistry, controls are a way to validate the results of your experiment.

A negative control contains all of the reagents used in the experiment, except for the material that is being detected. Therefore, the negative control should give a negative result.

A positive control contains the material that you are detecting, so the positive control should give a positive result.

A negative control validates the positive results, and a positive control validates the negative results. Without comparing our unknown samples to controls, we can’t be sure if our results are caused by the presence or absence of the compound we are testing for, or caused by an error in the procedure.

Example: When testing for reducing sugars, the negative control is water and the positive control is glucose. Water does not contain any reducing sugars so will give a negative result. Glucose is a reducing sugar so should give a positive result. If the controls do not give the expected results, we can conclude that there has been an error in the procedure.

Figure 1 - Results of the negative and positive controls for Fehling’s test for reducing sugars.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Sage Choice
• PMC10515451

Negative controls: Concepts and caveats

Bas bl penning de vries.

1 Julius Center for Health Sciences and Primary Care, Utrecht University Medical Center, Utrecht, The Netherlands

Rolf HH Groenwold

2 Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, The Netherlands

3 Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands

Associated Data

Supplemental material, sj-pdf-1-smm-10.1177_09622802231181230 for Negative controls: Concepts and caveats by Bas BL Penning de Vries and Rolf HH Groenwold in Statistical Methods in Medical Research

Unmeasured confounding is a well-known obstacle in causal inference. In recent years, negative controls have received increasing attention as a important tool to address concerns about the problem. The literature on the topic has expanded rapidly and several authors have advocated the more routine use of negative controls in epidemiological practice. In this article, we review concepts and methodologies based on negative controls for detection and correction of unmeasured confounding bias. We argue that negative controls may lack both specificity and sensitivity to detect unmeasured confounding and that proving the null hypothesis of a null negative control association is impossible. We focus our discussion on the control outcome calibration approach, the difference-in-difference approach, and the double-negative control approach as methods for confounding correction. For each of these methods, we highlight their assumptions and illustrate the potential impact of violations thereof. Given the potentially large impact of assumption violations, it may sometimes be desirable to replace strong conditions for exact identification with weaker, easily verifiable conditions, even when these imply at most partial identification of unmeasured confounding. Future research in this area may broaden the applicability of negative controls and in turn make them better suited for routine use in epidemiological practice. At present, however, the applicability of negative controls should be carefully judged on a case-by-case basis.

1. Introduction

In epidemiological research on causal effects, there are often concerns that one or more assumptions – such as exchangeability, no measurement error, or assumptions about missing data – are violated. In efforts to lessen these concerns, it has long been suggested that auxiliary variables be used that have a known (e.g. null) causal relation with the exposure or outcome of interest. 1 – 3 Observing an association that contradicts the belief in a causal null might alert the analyst to violations of the assumptions underlying the methods used in the study. Auxiliary variables known to be causally unrelated to the variables of primary interest are called negative controls and have the potential in bias detection as well as partial or complete bias correction in epidemiological research. 4

Applications of negative controls in epidemiological research are diverse. Dusetzina et al. 5 identified 11 studies that used a negative control exposure, negative control outcome, or both in studies on various topics, ranging from peri-operative beta-blocker use and the risk of acute myocardial infarction to proton-pump inhibitors and community-acquired pneumonia risk. Schuemie et al. 6 studied as many as 37 and 67 negative control exposures in two example studies on isoniazid use and acute liver injury and on selective serotonin reuptake inhibitor use and gastrointestinal bleeding, respectively. Increased attention for negative controls is exemplified by mention in, for example, the RECORD-PE reporting guideline for pharmacoepidemiological studies and the STROBE-MR guideline for Mendelian randomisation studies. 7 , 8

In recent years, negative controls have received increasing attention in the epidemiological and statistical literature. The literature on how to leverage negative controls in studies on causal effects has rapidly expanded and several authors have argued that negative controls should be more commonly employed. 2 , 9 , 4 This article aims to complement these efforts to increase the more routine implementation of negative controls with a discussion about a selection of caveats. Although we zoom in on the limitations of negative control methods, it should be noted that other methods (e.g. instrumental variable methods and conventional adjustment for a minimally sufficient set of covariates) are similarly subject to limitations and need not be universally preferred over negative controls. Focusing on the use of negative controls to address possible violations of the exchangeability assumption, that is, the assumption of no unmeasured confounding, we begin with a brief review of relevant definitions and discuss assumptions for bias detection. We then review methods for bias correction and study their sensitivity to assumption violations.

2. Negative controls

A negative control outcome (NCO) is a variable that is not causally affected by the exposure of interest A . 10 , 4 Likewise, a negative control exposure (NCE) is a variable that does not causally affect the outcome of interest Y , except possibly through the exposure of interest. 4 The causal directed acyclic graphs of Figure  1 (discussed later in this section) give examples of settings where a variable Z classifies as an NCO, an NCE or both. Given the absence of a direct causal effect of exposure A on an NCO Z or of NCE Z on outcome Y , any observed association between A and an Z , or between an Z and outcome Y given A , must be spurious. Leveraging negative controls involves translating information about such spurious associations into information about the spuriousness of associations between the primary exposure and outcome variables of interest.

Causal directed acyclic graphs of settings where Z is a negative control outcome (left), a negative control exposure (middle) or both (right). The absence of an arrow denotes the absence of a direct causal link. However, the presence of an arrow need not represent the presence of a direct causal link. Dashed double-headed arrows represent the marginal dependence of the (sets of) variable(s) that they connect, for example, through a common cause.

2.1. Negative controls for unmeasured confounding detection

Let Y ( a ) denote the outcome that would be realized had exposure A been set to a . Together with causal consistency (i.e. Y ( a ) = Y if A = a ) and positivity, epidemiologists often seek to invoke the exchangeability (or unmeasured confounding) condition Y ( a ) ⊥ ⊥ A (possibly within levels of a collection of observed variables) to establish identifiability of the effect of exposure A on outcome Y . 11 In observational studies, however, it is seldom evident that the exchangeability condition, E, for the exposure-outcome relation of interest is achieved. A key idea of negative controls is to find a ‘control’ statement, C, that translates into information about E and which is more easily verified or refuted.

Control statement C may refer to the absence of bias of a measure of the association between A and Y and the NCO or NCE variable, respectively. Knowing that any control association is noncausal renders the control statement empirically verifiable. If C implies E, then a null finding for the control statement would imply conditional exchangeability for the exposure–outcome relation of interest. Conversely, if E implies C, evidence of the bias of the control association corroborates the existence of unmeasured confounding.

2.2. Caveats in the use of negative controls to detect unmeasured confounding

There are a number of caveats concerning the use of negative controls for confounding detection. These caveats mainly concern the link between the control statement and exchangeability for the exposure–outcome relation of interest. Unfortunately, the extent to which one confers information about the other need not be evident. 12 A biased negative-control association need not imply unmeasured confounding for the exposure–outcome relation of interest and neither is the converse true generally.

First, while most applications of negative controls assume that confounding is the only source of bias, in reality, it may be one of potentially many sources of bias. A spurious negative control association could have resulted, at least in part, from collider stratification, measurement error, or violations of assumptions about missing data. 9 Even if unmeasured confounding for the negative control association implies unmeasured confounding for the exposure–outcome relation of interest, a biased negative control association need not be a reflection of unmeasured confounding. Conversely, a (near) null finding could be the result of opposing biases, masking the presence of unmeasured confounding. In other words, negative controls are a tool that may lack both specificity and sensitivity with respect to the type(s) of bias they are to detect.

Lipsitch et al. 2 suggested a principle for establishing a link that is based on the extent to which common causes of A and Y overlap with the common causes of the exposure or outcome and the negative control variable. Clearly, for an NCO, with complete overlap (e.g. V = U in Figure  1 ), the set of common causes of A and Y is empty if and only if the set of common causes of A and the NCO is empty. However, null values for certain measures of the effect of A on an NCO or of an NCE on Y need not imply that the set of unobserved common causes is empty, or, therefore, that there is conditional exchangeability for the primary exposure–outcome relation. Indeed, near null values may be the result of partially opposing confounding effects (or, more generally, opposing biases), and the relative effects may be different for the NCO versus the primary outcome Y .

With finite samples rather than complete knowledge of the theoretical or population distribution, sampling variability becomes relevant too, making it more important to acknowledge the distinction between absence of evidence and evidence of absence. 13 With finite samples, proving the null hypothesis of a null negative control association is impossible. Even if ‘highly’ powered studies cannot detect bias for the negative control relation, it may be injudicious to assume that the available data are sufficient to adequately control for confounding of the primary relation of interest, because a small degree of bias for the former relation may be associated with a substantial degree of bias for the latter. Sample size and power considerations are often ignored or left at secondary importance. While some papers have considered the power of negative control tests, 1 , 14 it is typically ignored how the negative control association relates to the extent of bias for the exposure–outcome relation of interest, yet high power to detect ‘small departures’ from exposure-NCO or NCE-outcome independence need not imply high power to detect small bias due to unmeasured confounding of the primary relation of interest. What are considered ‘small departures’ should therefore depend on the relationship between bias of the negative control association and the bias of the exposure–outcome relation of interest, or, likewise, depending on the link between the control statement C and the exchangeability condition E (as outlined in Section 2.1). Conversely, even if there is evidence of the contrary to the negative control null hypothesis, the bias due to uncontrolled confounding for the primary exposure–outcome relation may not be meaningful. In any case, it is important to consider the relative size of the biases in the negative control and primary exposure–outcome relations.

3. Negative control methods for uncontrolled confounding adjustment

The more recent literature on negative controls has considered how and under what conditions negative controls can be leveraged to partially or fully identify target causal quantities rather than merely the presence of bias. Lipsitch et al. 2 give conditions for valid inference about the direction of bias and thus for partial identification of the target causal quantity. These conditions are reviewed in Supplemental Appendix A. In what follows, we review three methods for full identification: the control outcome calibration approach (COCA), the (generalized) difference-in-difference (DiD) approach, and the double-negative control approach. Proofs of identification are given in Supplemental Appendix B for completeness. For each of the methods, we illustrate the potential impact of assumption violations on the identifiability of the targeted quantity. Throughout, departures from identification are termed bias.

3.1. Control outcome calibration approach

3.1.1. identification.

It may be tempting to regard the confounded association between the exposure of interest and an NCO as a direct measure of bias for the exposure–outcome effect of interest. However, it cannot generally be assumed that the direction or magnitude of bias is the same for the two relations. As an alternative to the restrictive and probably unrealistic ‘bias equivalence’ assumption, that is, the assumption of equality between the confounded negative control association and the bias due to unmeasured confounding of the exposure–outcome effect of interest, Tchetgen Tchetgen 10 proposed the COCA. The assumption of ‘bias equivalence’ would especially likely be violated if the NCO and primary outcome are measured on different scales and the bias is bounded differently depending on the scale, such as would be the case if the NCO was binary and the primary outcome continuous. The COCA leverages an NCO to adjust for unmeasured confounding without requiring that the NCO and primary outcome Y are measured on similar scales.

The next result, due to Tchetgen Tchetgen, 10 describes a regression-based approach to implementing the COCA, which – characteristically of the COCA – relies on the assumption that a (set of) counterfactual primary outcome(s) of interest is sufficient to render the NCO conditionally independent of the exposure of interest. Some intuition behind this approach may be obtained upon noting that the counterfactual outcomes may well capture information about baseline covariates and therefore serve as a proxy for unobserved pre-exposure variables that are predictive of the NCO. The reasoning rests on the assumption that the same covariates that explain the lack of exchangeability for the outcome of interest also explain the confounding of the exposure–NCO relation. However, even then it is not evident nor guaranteed that the counterfactual outcome proxy is sufficient to render the NCO and exposure conditionally independent.

A regression-based approach to implementing the COCA under rank preservation —

Suppose that the following conditions hold for all levels a of A :

∙ Consistency: Y ( a ) = Y if a = A .

∙ Rank preservation: for some constant θ , Y ( 0 ) = Y ( a ) − θ a .

∙ Exposure-NCO independence given counterfactual outcome: Z ⊥ ⊥ A | Y ( 0 ) .

∙ NCO model: for known one-to-one model link g , g ( E [ Z | A , Y ] ) = β 0 + β 1 A + β 2 Y , where β 0 , β 1 , β 2 are identified by a regression of Z on A and Y , and β 2 ≠ 0 .

Then, E [ Y ( a ) − Y ( a − 1 ) ] = θ is identified by − β 1 / β 2 .

Because counterfactual outcome Y ( 0 ) may not fully account for the unmeasured confounding between the exposure and NCO, it is important that the impact of assumption violations be gauged. To this end, Tchetgen Tchetgen described a sensitivity analysis, 10 given below in Theorem 2, for the special case of Theorem 1, where g is the identity link and A is a linear combination of Y ( 0 ) and an error term Δ . When the sensitivity parameter ( ρ ) is set to 0 , it is implicitly assumed that the NCO and exposure of interest are independent given counterfactual outcome Y ( 0 ) (because χ is independent of ( A , Y ) and therefore of Y ( 0 ) ) and, so, the result of Theorem 1 is recovered.

Sensitivity analysis for violations of Z ⊥ ⊥ A | Y ( 0 ) —

Suppose the following conditions hold for all levels a of  A :

∙ Conditional exposure-NCO independence: Z ⊥ ⊥ A | ( Y ( 0 ) , Δ ) .

∙ Exposure model: A = α 0 + α 1 Y ( 0 ) + Δ .

∙ NCO model: Z = β 0 + β 1 Y ( 0 ) + ρ Δ + χ , χ ⊥ ⊥ ( A , Y ) .

Then, E [ Z | A , Y ] = β 0 * + β 1 * A + β 2 * Y for some β 0 * , β 1 * , β 2 * , and if parameters β 1 * , β 2 * are identified (by a regression of Z on A and Y ) and β 2 * ≠ 0 , then θ = ( β 1 * − ρ ) / β 2 * .

Through the rank preservation assumption, Theorem 1 relies also on the strong assumption that all counterfactual outcomes of an individual are deterministically linked. A prerequisite of this assumption is that the within-person ranks of counterfactuals are the same for all individuals. In the next section, we consider violations of this assumption. However, as Theorem 3 states, in the special case where the outcome and exposure of interest are binary, there should be no concern about violations of this assumption as it can be dropped entirely. 10

COCA for binary primary outcome and exposure —

Suppose that the following conditions hold:

∙ Consistency: Y ( a ) = Y if a = A

∙ Positivity: 0 < Pr ( A = a , Y = y ) for y = 0 , 1 .

∙ Exposure-NCO independence given counterfactual outcome: Z ⊥ ⊥ A | Y ( a ) .

∙ Non-zero denominator: E [ Z | A = a , Y = 1 ] − E [ Z | A = a , Y = 0 ] ≠ 0 .

If the assumptions of Theorem 3 are met for a = 1 , the average treatment effect among the treated (ATT) E [ Y − Y ( 0 ) | A = 1 ] is identified. For identification of the average treatment effect (ATE) E [ Y ( 1 ) − Y ( 0 ) ] , the result requires that the assumptions are met for a = 0 , 1 . We will consider violations of these assumptions in the next section.

3.1.2. Sensitivity to assumption violations

In this subsection, we consider the sensitivity of the COCA to assumption violations. In particular, we illustrate the potential impact of deviating from rank preservation and of violating the assumption that counterfactual outcome Y ( 0 ) renders the exposure and NCO conditionally independent. While the classical measurement error in the outcome does not hamper inference in terms of bias in the classical linear regression setting, we also illustrate that this form of measurement error does result in bias of the COCA.

First, to illustrate the potential impact of deviating from rank preservation, consider the setting where A is binary and where the following models hold:

A standard implementation of the COCA as per Theorem 1 yields θ ^ = − β ^ 1 / β ^ 2 , where β ^ 1 and β ^ 2 are the coefficients for A and Y of an ordinary least squares regression of Z on A and Y .

Given a value of the ATE (i.e. E [ θ ] ), the parameter values are fully determined under models ( 1 ) by the joint distribution of the observed variables A , Y , Z (Supplemental Appendix C). In particular, given a fixed distribution of ( A , Y , Z ) , the variance of the individual effects Y ( 1 ) − Y ( 0 ) (i.e. Var ( θ ) = σ θ 2 ) and the ATE are linearly related via

(Supplemental Appendix C). For values of the ATE between − 4 and 2, we chose parameter values such that the distribution of ( A , Y , Z ) has marginal means E [ A ] = 0.25 , E [ Y ] = 0 and E [ Z ] = 0 , and covariance matrix

Figure  2 shows the bias of the COCA for the ATE. As shown, the magnitude of the bias is zero under rank preservation but increases linearly with an increasing variance of individual exposure–outcome effects.

Illustration of the effect of violating the rank preservation assumption on the difference between the quantity identified by the COCA and the ATE (bias of COCA; solid line) and the difference between E [ Y | A = 1 ] − E [ Y | A = 0 ] and the ATE (bias of crude analysis; dotted line). The dashed line depicts the relation between the variance of individual exposure–outcome effects Y ( 1 ) − Y ( 0 ) and the mean E [ Y ( 1 ) − Y ( 0 ) ] (the ATE) under a fixed observed data distribution; the solid line describes the relation between the ATE and the bias of the implementation of the COCA. COCA: control outcome calibration approach; ATE: average treatment effect.

In illustrating the sensitivity of the COCA against violations of rank preservation, it was assumed that the other assumptions were maintained. We now turn to the assumption of Exposure–NCO independence given counterfactual outcome Y ( 0 ) and likewise assume that all other assumptions, including rank preservation, are met. In particular, we consider the setting where Y ( 0 ) is the sum of two independent variables U 1 , U 2 . By assuming the following models, we also stipulate that some (albeit not necessarily the same) linear combination α 0 ′ + α 1 ′ U 1 + α 2 ′ U 2 is sufficient to render the exposure of interest and NCO conditionally independent:

Variables U 1 and U 2 can be viewed as common causes of the NCO and the exposure and outcome of interest. Again, the COCA identifies the quantity θ ^ = − β ^ 1 / β ^ 2 based on an ordinary least squares regression of NCO Z on A and Y , but this quantity is not generally equal to θ . Figure  3 shows the asymptotic bias (departure from identification of the ATE) of the COCA plotted against α 2 over the interval ( − 5 , 5 ) for the special case where U 1 and U 2 take the standard normal distribution and where α 0 , α 0 ′ , α 2 ′ = 0 , α 1 , σ A 2 , σ Z 2 = 1 and α 1 ′ = 2 . The bias is zero only when counterfactual outcome Y ( 0 ) is proportional to the linear combination of common causes U 1 and U 2 that renders the NCO and exposure of interest conditionally independent.

Illustration of the potential impact of violating the assumption that the NCO and exposure of interest are independent given counterfactual outcome Y ( 0 ) . The bias of the COCA (COCA − ATE) is given by the solid line; the bias of a crude analysis, Cov ( Y , A ) / Cov ( A ) , by the dashed line. NCO: negative control outcome; COCA: control outcome calibration approach; ATE: average treatment effect.

With α 2 , α 2 ′ = 0 , models ( 3 ) imply the same joint distribution of observed variables A , Y , Z as models ( 4 ):

An important difference between ( 3 ) and ( 4 ) is that the consistency assumption is violated (provided that Var ( U 2 ) > 0 ). The observed outcome Y is now the sum of the outcome of interest Y ( A ) and an independent mean-zero error term. Figure  3 therefore also illustrates that the validity of the COCA also critically rests on the absence of classical measurement error in the outcome. At α 2 = 0 , Figure  3 gives the bias of the COCA under ( 4 ) with the values for the parameters given above. Although ATE θ may not be identified in the presence of classical measurement error, in Supplemental Appendix C, partial identification bounds are derived for θ .

3.2. DiD approach

3.2.1. identification.

The DiD approach proposed by Sofer et al. 15 is an alternative approach to the COCA and does not assume rank preservation, nor does it require that the counterfactual outcome Y ( 0 ) renders the NCO and exposure of interest conditionally independent. Instead, the approach relies on bias equivalence for the primary exposure–outcome relation and the exposure–NCO relation. The simplest version of the DiD approach identifies the ATT under additive equi-confounding, as stated in Theorem 4, via the difference between the crude difference in primary outcome means and the bias of the exposure–NCO relation.

DiD approach for the ATT under additive equi-confounding —

Suppose that the following conditions hold for all levels a = 0 , 1 :

∙ Additive equi-confounding: E [ Y ( 0 ) | A = 1 ] − E [ Y ( 0 ) | A = 0 ] = E [ Z | A = 1 ] − E [ Z | A = 0 ] .

Then, E [ Y ( 1 ) − Y ( 0 ) | A = 1 ] = ( E [ Y | A = 1 ] − E [ Y | A = 0 ] ) − ( E [ Z | A = 1 ] − E [ Z | A = 0 ] ) .

Additive equi-confounding is relatively easy to interpret. However, the assumption may be particularly likely to be violated when primary outcome Y and NCO Z are measured on different scales (e.g. one is a binary variable, the other continuous). A generalized DiD approach still identifies the ATT under a different constraint on the dependence between Y ( 0 ) and A in relation to the dependence between Z and A . In particular, Theorem 5, based on Sofer et al., 15 relies on quantile–quantile equi-confounding, an example of which is depicted in Figure  4 .

Example of quantile–quantile equi-confounding. Dashed curves represents a = 1 , solid curves a = 0 . There is quantile–quantile equi-confounding because for every two points ( y 0 , p 0 ) and ( y 0 , p 1 ) on the solid and dashed curves, respectively, of the left panel, there exists z 0 such that ( z 0 , p 0 ) and ( z 0 , p 1 ) lie on the solid and dashed curves, respectively, of the right panel; quantiles y 0 and z 0 need not be the same.

Generalized DiD approach for the ATT under quantile–qualine equi-confounding —

∙ Quantile-quantile equi-confounding: F 0 ( F 1 − 1 ( p ) ) = G 0 ( G 1 − 1 ( p ) ) for all p ∈ [ 0 , 1 ] , where F a ( y ) = Pr ( Y ( 0 ) ≤ y | A = a ) , F a − 1 ( p ) = min { y : p ≤ F a ( y ) } , G a ( z ) = Pr ( Z ≤ z | A = a ) , G a − 1 ( p ) = min { z : p ≤ G a ( z ) } .

∙ F 1 is strictly increasing.

Then, E [ Y ( 1 ) − Y ( 0 ) | A = 1 ] = E [ Y | A = 1 ] − E [ F 0 − 1 ( G 0 ( G 1 − 1 ( V ) ) ) ] , where V ∼ Uniform [ 0 , 1 ] .

3.2.2. Sensitivity to assumption violations

We now give a simple setting where neither additive nor quantile–quantile equi-confounding is guaranteed to hold. The setting is characterized by two common causes U 1 , U 2 of the primary exposure and outcome and of the NCO. As before, we allow the relative effects of these common causes to differ between exposure, primary outcome and NCO, and we suppose that the following models hold:

Parameters α 1 , α 1 ′ , β 1 , β 2 control the dependence (confounding), through U 1 and U 2 , between A and Y ( 0 ) and between A and NCO Z ; in the special case where these parameters take the value 0, there is no confounding. The models of ( 5 ) imply

Implementing the DiD for the ATT θ would therefore identify, under ( 5 ), the quantity

with a bias of ( 1 − β 1 ) α 1 + ( 1 − β 2 ) α 1 ′ . The generalized DiD would instead identify

where G 1 − 1 is the quantile function associated with the distribution of Z | A = 1 , G 0 is the cumulative distribution function for Z | A = 0 and F 0 − 1 the quantile function of Y | A = 0 .

Figure  5 shows, for various parameter specifications, the bias of the (generalized) DiD for the ATT θ . Specifically, β 1 was varied over ( − 2 , 2 ) and α 1 ′ over { 0 , 1 } , while β 2 was set to 2 − β 1 , and p A = 0.5 , α 0 , α 0 ′ , β 0 , θ = 0 and α 1 , σ 1 2 , σ 2 2 , σ Y 2 , σ Z 2 = 1 . The figure illustrates that under additive and quantile–quantile equi-confounding the DiD and generalized DiD, respectively, identify the ATT. It also shows that both approaches are sensitive – albeit differently – to violations of their respective assumptions. Interestingly, even in the absence of additive equi-confounding the generalized DiD could be subject to considerable bias (Figure  5 , right panel, where the bias for the DiD is ( 1 − β 1 ) α 1 + ( 1 − β 2 ) α 2 ′ = 2 − ( β 1 + β 2 ) = 0 ). Beside the interpretability of its assumptions, an appealing property of the standard DiD approach is that the effects of common causes need not be the same for the NCO and the primary outcome of interest; if the net additive confounding is (close to) the same for the NCO and primary outcome, then the ATT may be (nearly) identified.

Illustrating of the potential impact of violating additive or quantile–quantile equi-confounding on the bias of the (generalized) difference-in-difference approach. Solid lines represent the difference-in-difference approach; dashed lines the generalized difference-in-difference; dotted lines the bias of a crude analysis, E [ Y | A = 1 ] − E [ Y | A = 0 ] .

3.3. Double-negative control approach

3.3.1. identification.

Recent developments in the use of negative controls to adjust for unmeasured confounding leverage multiple negative control variables or proxies of unmeasured common causes. 16 – 18 , 4 , 19 For example, the next result, due to Miao et al., 17 gives a set of conditions sufficient to identify the expected marginal counterfactual outcome E [ Y ( a ) ] by leveraging a pair of proxy variables B , Z of an unobserved variable U that renders the counterfactual outcomes independent of the exposure of interest (i.e. conditional exchangeability given U ).

The confounding bridge approach —

Suppose that for all levels a of A , the following conditions hold:

∙ Positivity: 0 < Pr ( A = a | B ) < 1 with probability 1.

∙ Latent ignorability: Y ( a ) ⊥ ⊥ ( A , B ) | U and Z ⊥ ⊥ ( A , B ) | U .

∙ Confounding bridge assumption: E [ Y | A = a , U ] = E [ h ( Z ) | A = a , U ] with probability 1 for some h .

∙ Completeness: for all g , if E [ g ( Z ) | A = a , B ] = 0 with probability 1, then Pr ( g ( Z ) = 0 | A = a ) = 1 .

Let H ( a ) be the collection of all h that satisfy E [ Y − h ( Z ) | A = a , B ] = 0 with probability 1. Then, H ( a ) is non-empty, and for all h ∈ H ( a ) , E [ Y ( a ) ] = E [ h ( Z ) ] .

Figure  6 shows a directed acyclic graph that is consistent with the assumptions of Theorem 6. The proxy variables can be seen to be negative control variables in the sense described by Shi et al., 4 thus making the confounding bridge approach a (double-)negative control approach. Like the primary exposure-outcome association, the exposure–NCO association is confounded by U . The function h is referred to as a confounding bridge because the confounding bridge assumption indicates that it links the Y - U association with the NCO- U association. The NCE is not part of this link but is meant to help identify it.

Causal directed acyclic graph with negative control pair satisfying the latent ignorability condition of Theorem 6.

The confounding bridge and completeness assumptions can be difficult to grasp. For categorical variables, however, the assumptions are subsumed under the conditions of the next result, due to Miao et al. 16 and Shi et al. 18

The proximal g-formula for categorical variables —

Let U , B , Z be discrete random variables with finite support such that U has no more categories than B or Z . Suppose that for all levels a of A , the following conditions hold:

∙ Positivity: 0 < Pr ( A = a , B = b ) for all categories b of B .

∙ Full rank: Pr ( Z | U ) and Pr ( U | A = a , B ) have rank equal to the number of levels of U .

Then, E [ Y ( a ) ] = h ( Z ) Pr ( Z ) , where h ( Z ) = E [ Y | A = a , B ] Pr ( Z | A = a , B ) − 1 .

Here, following Miao et al., 16 for any categorical variables X , Y , Z , Pr ( X | Y , Z ) denotes the matrix of probabilities Pr ( X = x | Y , Z ) with a one-to-one correspondence between rows and categories x of X and a one-to-one correspondence between columns and categories z of Z . Interestingly, the proximal g-formula can also be written as a weighted version of the standard g-formula:

with weights W ( a ) = ( diag Pr ( B ) ) − 1 Pr ( Z | A = a , B ) − 1 Pr ( Z ) and diag ( W ( a ) ) and diag ( B ) denoting the diagonal matrices with main diagonals W ( a ) and B , respectively. In the case that proxy variables B and Z are binary, the expression simplifies to

with weights

3.3.2. Sensitivity to assumption violations

Theorem 7 can accommodate any number of categories of U by taking proxy variables with sufficiently many categories, that is, by combining sufficiently many proxies. However, upon increasing the number of proxy variables, the latent ignorability assumption becomes more difficult to satisfy in the sense that Y ( a ) must be independent of increasingly many proxies given A and U . In this subsection, we consider the sensitivity of the proximal g-formula for violations of latent ignorability as well as of the assumption that U has no more categories than the proxy variables.

In particular, we consider the case where the variables A , Y of interest and the proxy variables B , Z are binary, where U is a pair ( U 1 , U 2 ) of independent binary variables, and where the following models hold:

where expit ( x ) = 1 / ( 1 + exp [ − x ] ) for all x . Intercepts α 0 , β 0 , γ 0 , θ 0 were chosen to ensure that Pr ( B = 1 ) = Pr ( A = 1 ) = 1 / 2 and Pr ( Z = 1 ) = Pr ( Y = 1 ) = 1 / 5 . We let ρ = 0 , β 1 = 1 , γ 1 = 0 , θ 1 = 0 by default. In scenario A, instead of taking β 1 = 1 , ρ = 0 , we vary β 1 over ( − 4 , 4 ) under ρ = 1 / 2 to violate the full rank assumption, which implies that U has no more categories than B or Z . In scenario B, instead of taking γ 1 = 0 , we violate the latent ignorability assumption by varying γ 1 over ( − 4 , 4 ) (i.e. Z is not a negative control outcome). In scenario C, we violate the same assumption, now by varying θ 1 over ( − 4 , 4 ) (i.e. B is not a negative control exposure) instead of taking θ 1 = 0 .

Figure  7 gives the bias of the proximal g-formula for the ATE E [ Y ( 1 ) − Y ( 0 ) ] for all scenarios. Also shown are the differences between the crude risk differences E [ Y | A = 1 ] − E [ Y | A = 0 ] and the ATE. The bias is zero under the default parameters, which are consistent with the assumptions of Theorem 7. The figure also illustrates that violations of these unverifiable assumptions can have a large impact on the validity of the double-negative control approach.

Bias of crude approach (dashed) and proximal g-formula (solid) under violations of the cardinality assumption (scenario A), negative control outcome condition (scenario B), or negative control exposure condition (scenario C).

In an other study, Vlassis et al. 20 found the bias of the crude risk difference to be consistently smaller than that of the proximal g-formula. Our results demonstrate that in some settings, the proximal g-formula results in considerably more bias than what would result from ignoring unmeasured confounding.

4. Conclusion

Negative controls have gained increasing interest in addressing concerns about the validity of a study. The literature on the topic has tended to consider increasingly ambitious tasks, from confounding detection to full identification of causal effects, typically at the cost of stronger and arguably more complex assumptions. Efforts have been made to introduce negative controls to a broader audience and ensure they are adopted in epidemiological practice. 4 However, little attention has yet been given to the methods’ assumptions and the potential impact of assumption violations. While the assumptions may be tenable enough in some specific cases to justify an application, in other situations substantial violations are possible. We have illustrated that assumption violations, some of which are likely even in very simple settings, may have a considerable impact on the validity of the negative control approach, thereby limiting its utility.

We stress the other methods commonly used to analyse observational data (e.g. covariate adjustment through regression analysis or instrumental variable methods) may also be sensitive to violations of their assumptions. However, a comparison between these methods and methods using negative controls is beyond the scope of this work. Researchers should decide on a case-by-case base of which methods the assumptions appear most plausible and thus which method appears most appropriate. Another aspect that should be considered on a case-by-case base is the magnitude that could arise due to violations of the assumptions underlying negative control methods. The illustrations presented here are based on arbitrary parameter values chosen such that they illustrate the relative bias contributions. However, we do not claim these are necessarily appropriate for a particular study. Considerations about the appropriateness and possible violation of the assumptions of negative control methods are, to a large extent, context-dependent.

Despite the possible abundance of negative controls, their routine use in epidemiological practice may fail to strengthen evidence about exposure–outcome effects unless it can be safely assumed that assumption violations are absent or else if the robustness against these violations is well understood. Given the potential impact of assumption violations, it may sometimes be desirable to replace strong conditions for identification with weaker conditions that are easier to verify, even when these weaker conditions imply at most partial identification. Future research in this area may broaden the applicability of negative controls and in turn make them more suited for routine use in epidemiological practice. When they are used, we advise that researches consider the results of their applications carefully and explicitly in light of the methods’ limitations and assumptions.

Supplemental Material

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The author(s) received no financial support for the research, authorship and/or publication of this article: RHHG was funded by the Netherlands Organization for Scientific Research (NWO-Vidi project 917.16.430). The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding bodies.

ORCID iD: Bas BL Penning de Vries https://orcid.org/0000-0001-9989-7732

Supplemental material: Supplemental material for this article is available online.

Physical Review Applied

• Collections
• Editorial Team

Machine-learning optimal control pulses in an optical quantum memory experiment

Elizabeth robertson, luisa esguerra, leon meßner, guillermo gallego, and janik wolters, phys. rev. applied 22 , 024026 – published 8 august 2024.

• No Citing Articles
• INTRODUCTION
• OPTICAL QUANTUM MEMORY OPTIMIZATION
• RESULTS AND DISCUSSION
• CONCLUSIONS
• ACKNOWLEDGMENTS

Efficient optical quantum memories are a milestone required for several quantum technologies, including repeater-based quantum key distribution and on-demand multiphoton generation. We present an efficiency optimization of an optical electromagnetically induced transparency (EIT) memory experiment in a warm cesium vapor using a genetic algorithm and analyze the resulting wave forms. The control pulse is represented either as a Gaussian or free-form pulse and the results from the optimization are compared. We see an improvement factor of 3(7)% when using optimized free-form pulses. By limiting the allowed pulse energy in a solution, we show an energy-based optimization giving a 30% reduction in energy, with minimal efficiency loss.

• Received 10 January 2024
• Revised 27 May 2024
• Accepted 8 July 2024

DOI: https://doi.org/10.1103/PhysRevApplied.22.024026

© 2024 American Physical Society

Physics Subject Headings (PhySH)

• Research Areas

Authors & Affiliations

• 1 Deutsches Zentrum für Luft- und Raumfahrt e.V. (DLR), Rutherfordstraße 2, 12489 Berlin, Germany
• 2 Technische Universität Berlin , Institute for Optics and Atomic Physics, Hardenbergstraße 36, 10623 Berlin, Germany
• 3 Technical University of Berlin, Straße des 17. Juni 135, 10623 Germany
• 4 Einstein Center Digital Future , Wilhelmstraße 67, Berlin 10117
• 5 Science of Intelligence Excellence Cluster, Marchstraße 23, Berlin 10587
• 6 Robotics Institute Germany, Straße des 17. Juni 135, Berlin 10623
• * Contact author: [email protected]

Article Text (Subscription Required)

References (subscription required).

Vol. 22, Iss. 2 — August 2024

Subject Areas

• Computational Physics
• Quantum Information

Access Options

• Buy Article »
• Log in with individual APS Journal Account »
• Log in with a username/password provided by your institution »
• Get access through a U.S. public or high school library »

Authorization Required

Other options.

• Buy Article »
• Find an Institution with the Article »

Download & Share

The experimental schematics. (a) An overview of the online genetic algorithm optimization setup. Possible solutions (control pulses) are generated by the genetic algorithm and are evaluated by carrying out a memory experiment. EOM, electro-optic modulator; AOM, acousto-optic modulator; AFG, arbitrary function generator; CP1, CP2, calcite prisms; PD, photodiode; Cs , cesium-vapor cell; magn. shield, magnetic shield. (b) The three-level Λ system used in the memory protocol. The signal (control) lasers are red detuned by 1 GHz from their resonant hyperfine transitions. A resonant pump laser populates the ground state | g ⟩ . (c) A typical memory experiment sequence. We denote t 0 = 0 as the center of the signal pulse to be stored and we read out such that the time between the signal in and the control read pulse is Δ t = 200 ns .

(a) Different gene representations for different solutions. Only three genes are used to encode the Gaussian; the amplitude, a , the FWHM, f , and the delay with respect to the signal pulse, d . Sixteen evenly distributed points, x 1 , … , x 16 , encode the free-form pulse. The signal pulse is shown using the dashed blue line. (b) An overview of a genetic algorithm.

The convergence of the free-form pulse, learned for a 18-ns-width signal. The fitness (light blue) and the related internal efficiency (dark blue) show the saturation of the learning process at generation 30. The pulse shapes for the best-performing pulse at each iteration are shown in purple; thus the final pulse is the end result. The green shows the number of new solutions tried per generation.

The efficiencies of the learned pulses, for varying values of the signal FWHM. The free-form pulses (purple) look to perform on average, slightly better than the Gaussian pulse (green) but this improvement is not statistically significant. The fits are modeled as η int = η 0 , t S / 1 + ( 4 ln  ( 2 ) / Δ t S Δ γ ) 2 , where η 0 , t S is the maximal achievable efficiency and γ is the bandwidth of the memory (for the derivation, see Ref. [ 37 ]). The theoretically achievable efficiency at 200 ns is plotted in red, for a calculated OD = 12 . We consider the measured 41-ns free-form efficiency to be an outlier with no physical meaning.

The free-form pulse optimization. We show the best-learned pulses for a signal pulse with (a) 8.9-ns, (b) 18-ns, (c) 31-ns, and (d) 41-ns FWHM. The distribution of values explored for each gene throughout the whole optimization process is shown in a violin plot, with the final chosen gene values indicated as purple dots. The corresponding electrical waveform generated from the final gene values is shown in purple and its corresponding optical signal in green. The variance of solutions that have a fitness within 10% of the best-learned solution is plotted in blue. The lines joining the points have no physical significance and are intended as a guide to the eye. The temporal overlap of the signal and control pulses shows a consistent trend in the closing of the coherence window (inset).

The efficiencies of the energy-restricted pulses, for (a) 31-ns and (c) 18-ns FWHM signal pulses. The corresponding learned pulses are plotted for the free-form pulses [plotted in the upper panels of (b) and (d)] and Gaussian pulses [plotted in the lower panels of (b) and (d)], with a color gradient from blue to red (in energy terms, low to high).

Sign up to receive regular email alerts from Physical Review Applied

• Forgot your username/password?
• Create an account

Article Lookup

Paste a citation or doi, enter a citation.

• Subscribe to journal Subscribe
• Get new issue alerts Get alerts

Secondary Logo

Journal logo.

Colleague's E-mail is Invalid

Your message has been successfully sent to your colleague.

Save my selection

Negative Controls

A tool for detecting confounding and bias in observational studies.

Lipsitch, Marc a,b,c ; Tchetgen Tchetgen, Eric a,c,d ; Cohen, Ted a,c,e

From the a Department of Epidemiology, b Department of Immunology and Infectious Diseases, c Center for Communicable Disease Dynamics, and d Department of Biostatistics, Harvard School of Public Health, Boston, MA; and e Division of Global Health Equity, Brigham and Women's Hospital, Boston, MA.

Submitted 19 March 2009; accepted 12 October 2009.

Supported by NIH 5U01GM076497 and 1U54GM088558 (Models of Infectious Disease Agent Study) to ML.

Supplemental digital content is available through direct URL citations in the HTML and PDF versions of this article ( www.epidem.com ).

Correspondence: Marc Lipsitch, Department of Epidemiology, Harvard School of Public Health, 677 Huntington Avenue, Boston, MA 02115. E-mail: [email protected] .

Noncausal associations between exposures and outcomes are a threat to validity of causal inference in observational studies. Many techniques have been developed for study design and analysis to identify and eliminate such errors. Such problems are not expected to compromise experimental studies, where careful standardization of conditions (for laboratory work) and randomization (for population studies) should, if applied properly, eliminate most such noncausal associations. We argue, however, that a routine precaution taken in the design of biologic laboratory experiments—the use of “negative controls”—is designed to detect both suspected and unsuspected sources of spurious causal inference. In epidemiology, analogous negative controls help to identify and resolve confounding as well as other sources of error, including recall bias or analytic flaws. We distinguish 2 types of negative controls (exposure controls and outcome controls), describe examples of each type from the epidemiologic literature, and identify the conditions for the use of such negative controls to detect confounding. We conclude that negative controls should be more commonly employed in observational studies, and that additional work is needed to specify the conditions under which negative controls will be sensitive detectors of other sources of error in observational studies.

Epidemiologists seek to distinguish the causal effect of exposure A on outcome Y from associations due to other mechanisms. Noncausal associations may be classified into 3 categories (in addition to chance) 1 : mismeasurement (eg, recall bias), confounding, and biased selection of individuals into the analysis.

In experimental biology, the manipulation of experimental conditions prevents many of the noncausal associations that arise in observational studies. Nonetheless, experimental biologists routinely question whether they have correctly inferred causal relationships from the results of their experiments. Biologists employ “negative controls” as a means of ruling out possible noncausal interpretations of their results. We describe the use of negative controls in experiments, highlight some examples of their use in epidemiologic studies, and define the conditions under which negative controls can detect confounding in epidemiologic studies. Although the particular threats to causal inference are different in experimental and observational sciences, the use of negative controls is a valuable means of identifying noncausal associations and can complement other epidemiologic methods for improving causal inference.

EXPERIMENTAL BIOLOGY: THREATS TO CAUSAL INFERENCE AND THE USE OF NEGATIVE CONTROLS

One might imagine that the experimental method would circumvent most threats to the validity of causal inference that occur in observational studies. For example, consider the hypothesis that a particular cytokine—a chemical involved in signaling in the immune system—enhances the killing of a species of bacteria by neutrophils, a class of white blood cells. 2 An experiment is devised in which neutrophils, bacteria, and growth medium are mixed together. In condition 1, the cytokine is added, and in condition 2, some inert substance such as saline solution is added. After incubation, the bacteria are enumerated and the number of live bacteria compared between conditions 1 and 2.

If the investigator finds fewer live bacteria in condition 1 than in condition 2, the finding is consistent with the hypothesis that the cytokine enhanced neutrophil-mediated killing. Nonetheless, concern remains that something other than cytokine-aided, neutrophil-mediated killing may be responsible. For example, perhaps there is a contaminant in the cytokine preparation that directly kills bacteria, or perhaps the cytokine itself kills bacteria, or perhaps some other unintended difference between the treated and untreated conditions (eg, temperature or pH) caused the differential survival of the bacteria.

Each of these unintended differences is broadly similar to a confounder—a characteristic associated with the exposure (presence or absence of the cytokine) and causes the outcome (differences in bacterial counts), thereby causing a spurious association between the presence of the cytokine and differences in bacterial counts.

Experimental biologists address such concerns in 2 ways. The first is to attempt to eliminate unwanted differences between the compared groups (in the design) and to measure and account for any unavoidable differences (in the analysis). For example, a researcher would make all conditions (dilution protocols, incubators, etc.) identical between the 2 conditions except for the variable of interest (ie, the presence/absence of the cytokine). Replication of the experiment reduces the likelihood that some chance factor was systematically different between the 2 experimental arms. Sometimes, experimental variation nonetheless remains. When experimental variation cannot be eliminated by these approaches, experimentalists may control for this variation by matching or statistical adjustment for the day on which an assay was performed. In experimental studies of population health outcomes (clinical trials), analogous precautions include randomization (to assure an expectation of baseline exchangeability between groups), 3 use of multiple individuals in each treatment group (replication), and analytic adjustment for measured confounders.

The second general approach is to perform negative controls: to repeat the experiment under conditions in which it is expected to produce a null result and verify that it does indeed produce a null result. Several strategies are employed to design negative controls, 2 such as:

• Leave out an essential ingredient. In the absence of neutrophils, there should be the same number of bacteria with or without the cytokine; if a contaminant (or the cytokine itself) is killing bacteria without involving neutrophils, this negative control should produce fewer bacteria with the cytokine than without.
• Inactivate the (hypothesized) active ingredient. Specific antibodies that neutralize the cytokine (but would have no effect on a contaminant) can be added to the preparation; killing should not occur if the cytokine is responsible for the effect.
• Check for an effect that would be impossible by the hypothesized mechanism. Suppose there were a species of bacteria that was completely impervious to the actions of neutrophils. The experiment could be repeated with this species, rather than the species of interest, to confirm there is no difference between conditions 1 and 2. This would help to rule out the possibility of some non-neutrophil-mediated effect of the cytokine preparation on bacteria.

As with the list of noncausal explanations for an experimental result, the list of possible negative controls is almost endless, and judgment is required to assess how many such noncausal explanations are plausible and which negative controls are of greatest value in ruling out key threats to valid inference. Peer reviewers of biologic experiments usually require some negative controls to validate experimental results.

EXAMPLES OF THE USE OF NEGATIVE CONTROLS IN EPIDEMIOLOGY

In an epidemiologic study to assess whether an association between a risk factor A and an outcome Y is likely to be causal, it is common to address the possibility of confounding by measured variables L by adjusting for them, using such techniques as restriction, stratification, multivariate modeling, matching, inverse-probability weighting, or g-estimation ( Fig. 1 ). 4

Epidemiologists also sometimes use negative controls to detect confounding and other sources of incorrect causal inference. This approach has been elegantly applied to the debate over the effects of vaccination of the elderly on “pneumonia or influenza hospitalization and on all-cause mortality. Observational studies in elderly persons have shown that vaccination against influenza is associated with a remarkably large reduction in one's risk of pneumonia/influenza hospitalization and also in one's risk of all-cause mortality in the following season, after adjustment for measured covariates that indicate health status. 5 However, older age is associated with a less robust immune response to influenza vaccination, and ecological data suggest that the benefits measured in observational studies far exceed the corresponding benefits expected at the population level when influenza vaccination rates have increased among the elderly. 6 Importantly, both outcomes are nonspecific, in the sense that they have unknown and time-varying contributions from influenza. This is obviously true for all-cause mortality, and it is also true for pneumonia/influenza hospitalization, because the cause of respiratory infection is often not ascertained, and many pneumonia cases are caused by agents other than influenza. The large degree of protection against these outcomes observed in individual level studies, combined with the lack of measurable vaccine effect in ecological studies, have led to a suspicion that uncontrolled confounding has exaggerated the impact of influenza vaccination on mortality and on pneumonia/influenza hospitalization in the elderly. 6-8

To test this hypothesis, Jackson et al 7 reproduced earlier estimates of the protective effect of influenza vaccination, but then repeated the analysis for 2 sets of negative control outcomes, and showed that the protective effect was observed even in circumstances where the vaccine could not have caused the protection. For the first negative control outcome, the authors 7 used the fact that vaccination often begins in autumn, while influenza transmission is often minimal until winter. Thus, they could assess the risk of pneumonia/influenza hospitalization and all-cause mortality among vaccinated versus unvaccinated persons before, during, and after influenza season. The only biologically plausible mechanism by which influenza vaccine could protect against mortality or pneumonia/influenza hospitalization is by preventing influenza or its consequences; therefore, Jackson et al 7 reasoned that if the effect measured in previous studies were causal, it should be most prominent during influenza season. If instead it were due to confounding, then the protective effect should be observable immediately after vaccination but before influenza season. In a cohort study analyzed with a Cox proportional hazards model, despite efforts to control for confounding, they observed that the protective effect was actually greatest before, intermediate during, and least after influenza season. They concluded that this is evidence that confounding, rather than protection against influenza, accounts for a substantial part of the observed “protection.” The use of this negative-control outcome approach is formally similar to the “leave-out-an-essential-ingredient” control described above, as influenza is essential in the proposed causal pathway.

Second, Jackson et al 7 postulated that the protective effects of influenza vaccination, if real, should be limited to outcomes plausibly linked to influenza. In contrast, if the relationship were due to an uncontrolled confounder, then the same “protection” might be observed for irrelevant outcomes. They repeated their analysis, but substituted hospitalization for injury or trauma as the end point. They found that influenza vaccination was also “protective” against injury or trauma hospitalization. This, too, was interpreted as evidence that some of the protection observed for pneumonia/influenza hospitalization or mortality was due to inadequately controlled confounding. This second negative control outcome is formally similar to the “check-for-an-effect-impossible-by-the-hypothesized-mechanism” approach described above.

Epidemiologists also sometimes use negative control exposures to examine whether observed associations are causal. An example is the inclusion in questionnaires of irrelevant variables, sometimes called “probe variables,” to assess if recall bias may be responsible for an observed association between a self-reported exposure and an outcome. A recent study 9 tested the association between multiple sclerosis (MS) and a variety of common childhood infections assessed by self-report. The investigators found statistically significant positive associations of MS with a recalled history of 5 different viral infections. Suspecting that cases may recall prior medical events more often or with more certainty than controls, the investigators' questionnaire also included several childhood medical events not plausibly associated with MS, such as broken limbs, tonsillectomy, and concussions. In the absence of a causal association, any measured association with these probe variables would suggest recall bias for the variables of interest. The authors found that the magnitude of association with these irrelevant exposures was comparable with the magnitude observed for each of the self-reported infections except one (infectious mononucleosis) that had a much stronger association. They concluded that, after accounting for recall bias, only infectious mononucleosis showed a specific association with MS.

Another application of negative controls has been to expose “immortal time bias,” a form of selection bias that produces spurious associations between observed variables. Suissa and Ernst 10 suspected that the reported benefits of nasal corticosteroids in preventing asthma resulted from this form of bias, in which exposed persons are credited with time at risk during which the event cannot occur, and thus exposed persons have an artificially low event rate. Inclusion of the “immortal” time is dependent on both being exposed during that time and on not having the outcome during that time 10 ; hence, a (negative) association is induced between exposure and outcome. To demonstrate such bias, the authors repeated prior analyses but restricted the exposed class to persons with a single annual dose of corticosteroids—a dose far too low to have plausible biologic effect (ie, a negative control exposure). They found that even this very modest exposure was associated with substantial protection against asthma, suggesting that the previous analytic approach was inappropriate. In this case, the investigators already suspected what form of bias was operating and used the analysis to prove their point. In principle, the original investigators could have done such an analysis to test for bias.

CHOICE OF NEGATIVE CONTROLS TO DETECT CONFOUNDING IN EPIDEMIOLOGY

Negative controls have been used to detect confounding (the influenza vaccine example 7 ), recall bias, (the MS example 9 ), and selection bias (the nasal corticosteroid example 10 ). Furthermore, it may be possible to specify how negative controls should be designed to aid in detecting biased causal inferences resulting from each of these mechanisms, and also perhaps to detect other forms of analytical errors. In this section, we focus on the conditions under which negative controls in epidemiology can detect confounding. 1

The essential purpose of a negative control is to reproduce a condition that cannot involve the hypothesized causal mechanism but is very likely to involve the same sources of bias that may have been present in the original association. If a contaminant (source of bias) was responsible for the effect of the cytokine on bacteria, it should have its effect even when the hypothesized mechanism of the effect (through neutrophils) is prevented through neutralization of the cytokine or through omission of neutrophils from the experiment. If an uncontrolled confounder (general good health or healthful practices) is responsible for the protection observed from influenza vaccine against mortality or pneumonia/influenza hospitalization, the same confounder might be associated with other outcomes that are not plausibly prevented by influenza vaccination.

This description suggests a general principle for the selection of negative controls to detect residual confounding. Ideally, a negative control outcome (N) should be an outcome such that the set of common causes of exposure A and outcome Y should be as identical as possible to the set of common causes of A and N ( Fig. 2 ). To the extent that the set of unobserved common causes of A and Y overlaps with the set of unobserved common causes (U) of A and N, we call the negative control outcome N “U-comparable” to Y. If N and Y are U-comparable outcomes (ie, with an identical set of common causes that are associated with A), and assuming that N is not caused by A, an association A-N when analyzed according to the same procedure used to analyze A-Y would indicate bias in the association A-Y. If N and Y are perfectly U-comparable and N is not caused by A, then a null finding of A-N implies that the A-Y association is not likely biased by the pathways examined through this negative control.

Negative control outcomes in practice will be only approximately U-comparable, at best. Thus, it is possible that the observed association between A and N is caused by some uncontrolled confounder U2, which is not a confounder of the A-Y association; hence, finding an unexpected association between A and N does not prove unequivocally that the A-Y association is biased. In the example of using death or hospitalization from injury as a negative control outcome for death or pneumonia/influenza hospitalization, one could argue that there may be some common causes of vaccination and injury that are not causes of all-cause death or pneumonia/influenza hospitalization. Such common causes (we cannot think of a plausible one) would create an association in the negative control analysis of vaccination and injury, even if the primary analyses of vaccination and death or pneumonia/influenza hospitalization were unconfounded—thus making the negative control detect bias even where none exists. However, if N is associated only with some, but not all, of the uncontrolled confounders of the association between A and Y, it is possible that A and N will appear unassociated despite the presence of uncontrolled confounding between A and Y. In the influenza vaccine example, one could argue that there are common causes of vaccination and death or pneumonia/influenza hospitalization—that are not causes of injury-related outcomes. Such a common cause (say, an aversion to vaccination that makes an individual less likely to get the pneumococcal vaccine) would be undetectable by this particular negative control. Despite these limitations, negative controls have value in alerting the analyst to possible residual confounding.

In principle, the measured confounders L of the A-Y relationship need not be causes of N as well, because a properly specified model that accounted for the confounding by L of A-Y would not be misled if such confounding were absent for A-N. In practice, the ideal negative control outcome should nonetheless be one with incoming arrows as similar as possible to those of Y, including the incoming arrows from L. This is true, first, because it is difficult in practice to imagine an outcome N that lacks association with known confounders L but has an association with uncontrolled (or even unknown) confounders similar to that of U-Y. In addition, because negative controls may be useful in detecting residual confounding by measured confounders L or analytic errors, it would be beneficial to have the L-N relationship be as similar as possible, quantitatively, to the L-Y relationship. In eAppendix 1 ( https://links.lww.com/EDE/A377 ), we describe the analytic basis for use of a U-comparable negative control outcome.

A negative control exposure B should be an exposure such that the common causes of A and Y are as nearly identical as possible to the common causes of B and Y ( Fig. 3 ). To the extent that the set of unobserved common causes U of A and Y overlaps with the set of unobserved common causes of B and Y, we call the negative control exposure B “U-comparable” to A. If A and B are perfectly U-comparable and B does not cause Y, then an association B-Y when analyzed according to the same model used to analyze A-Y would indicate bias in the association A-Y. If A and B are perfectly U-comparable and B does not cause Y, then a null finding of A-N means that the A-Y association is unbiased. We are not aware of an example of the use of a negative control exposure to detect confounding in this sense. In the influenza vaccination example, one might hypothesize that whatever residual confounders U (eg, poor health status) made one less likely to get influenza vaccine (A) and more likely to die of influenza or pneumonia (Y), might also make one less likely to get other vaccines, such as booster tetanus vaccine (B). Because tetanus does not cause pneumonia, tetanus vaccine receipt might be an appropriate negative control exposure for such a study. In the previous section, we mentioned the use of “probe variables” as negative controls to detect recall bias that might lead patients with MS to over-report a history of childhood infections. Recall bias, a form of reverse causation, has a different causal structure from confounding, 1 and we do not outline here the causal requirements for negative controls to detect reverse causation.

In observational settings, the comparability between exposure A and negative control exposure B will be only approximate. As in the case of negative control outcomes, this approximate comparability means that B and Y may be associated even when A-Y is unbiased; this would occur if there is some other confounder U2 linking B and Y that does not confound A-Y. Similarly, if A and B are only approximately comparable, it is possible for B and Y to show no association yet for A-Y to be biased, if the confounder biasing A-Y does not connect B to Y. An analytic basis for the use of negative control exposures is given in eAppendix 2 ( https://links.lww.com/EDE/A377 ).

In a cohort study, in which multiple exposures and outcomes are measured on each person, it is relatively straightforward to analyze negative control exposures and outcomes, assuming that suitable variables have been measured. In a case-control study, the use of negative control exposures is similarly straightforward because negative control exposures can be added to the set of exposure variables collected for each subject. If a case-control study is nested within a cohort, irrelevant outcomes can be selected and analyzed. A stand-alone case-control study presents some logistical problems for implementing negative-control outcomes. This might require a second case-control study in which “cases” include some irrelevant but comparable outcome to the cases in the main study. This difficulty is reduced if multiple control groups are used, as is occasionally done for other reasons. 11,12

A useful contrast can be drawn between variables that can serve as negative controls and those that can be used as instruments. 13-15 An instrumental variable is any variable that is connected causally to A but free of any of the confounding connections to Y from which A suffers. In contrast, a negative control outcome is connected to A through all possible confounding routes but not causally. Similarly, a negative control exposure is connected to Y through all possible confounding routes but not causally. Figure 3 depicts an instrumental variable Z that satisfies the necessary conditions of an instrument 16,17 while the variable B is an ideal negative exposure candidate.

We propose that negative controls should be applied more commonly in epidemiologic studies, as in laboratory experiments, and with the same goals: to detect uncontrolled confounding or other sources of bias that create a spurious causal inference. 1 The routine use of negative controls in experimental biology allows the detection of both suspected and unsuspected sources of bias. The challenge of deriving valid causal inference is at least as great in observational studies as in experiments. In other social sciences, negative control outcomes are sometimes recommended for use with observational as well as experimental studies, 18 to compensate for limited sample size and possible imbalance between treatment arms.

Hill 19 proposed specificity of association as one guideline for assessing causal inferences. Hill argued that causal inferences were more credible if the exposure (in his example, nickel mining) was associated with only certain types of outcomes (death from lung and nose cancer but not death from other cancers), and if the outcome was associated with one kind of exposure (nickel mining) but not many others. Hill himself, as well as more recent authors, 16,20,21 have been ambivalent about this particular guideline. Weiss 22 has argued that specificity of outcome and exposure may, in certain cases, lend credibility to causal inference, especially if there is a strong hypothesis of why the outcome (or exposure) should be specific to the cause. Both Hill's and Weiss's arguments are related to the ideas of negative controls; we suggest that informative tests of specificity of association are those that meet the criteria we have outlined for negative control exposures or outcomes. Their value will vary depending on the plausibility of the claim that the control considered is U-comparable to the exposure or outcome of interest.

Subject matter knowledge is required for the choice of negative controls, just as it is for the design of appropriate strategies to adjust for confounders. If an investigator identifies negative controls based on incorrect causal assumptions, the analysis involving negative controls may be misleading. If a causal association between 2 variables A-N is thought to be implausible and is used as a negative control for a study of some other association A-Y, then finding an association between A and N will erroneously suggest bias in the association A-Y.

A properly selected negative control is a sensitive, but blunt, tool to probe the credibility of a study. The “failure” of a negative control—the finding of an association that is judged not to be plausibly causal—does not identify what form of bias is operating. In particular, as we demonstrate in eAppendix 3 ( https://links.lww.com/EDE/A377 ), the magnitude of bias due to uncontrolled confounding cannot generally be inferred from the magnitude of a detected A-N (or B-Y) non-null association, without extra assumptions based on firm scientific understanding. Furthermore, such additional subject matter knowledge (or suspicion about the source of analytic errors) is necessary to determine where bias is likely to have arisen.

We have defined precisely the conditions under which negative controls are capable of detecting the existence and direction of bias due to uncontrolled confounders. We have argued by example that negative controls can also aid in detecting recall bias (reverse causation) or selection bias. Epidemiologists must weigh these potential benefits of employing negative controls against the increased cost associated with the measurement of additional variables, and the possibility that the assumptions under which the negative control variables were selected are faulty.

ACKNOWLEDGMENTS

We thank Murray Mittleman, Molly Franke, Justin O'Hagan, and Hsien-Ho Lin for helpful discussion.

Supplemental Digital Content

• EDE_2010_01_26_LIPSITCH_200784_SDC1.pdf; [PDF] (121 KB)
• + Favorites
• View in Gallery

Readers Of this Article Also Read

Brief report: negative controls to detect selection bias and measurement bias..., patterns of weight gain in middle-aged and older us adults, 1992–2010, negative control exposures in epidemiologic studies, can the “specificity” of an association be rehabilitated as a basis for..., the case-time-control design: further assumptions and conditions.

• Stocks in Action
• Stock Recommendation
• Result Analysis
• Across The Web
• Stock Market News
• Mojo Premium

Lakshmi Electrical Control Systems Reports Negative Financial Results for Q1 2024

Lakshmi Electrical Control Systems, a microcap company in the capital goods industry, recently announced its financial results for the quarter ending March 2024. The results, declared on August 9, 2024, have received a "Strong Sell" rating from MarketsMOJO. The company's financial performance for the quarter has been negative, with a score of -19 compared to -16 in the previous quarter. However, there are some positive aspects to the financials that are worth noting. One of the strengths of Lakshmi Electrical Control Systems is its strong operating cash flow, which has been consistently growing over the past three years. The company has also been able to settle its debtors faster, as seen in the half-yearly debtors turnover ratio, which has been increasing over the past five half-yearly periods. On the other hand, there are some areas of concern in the financials. The net sales for the quarter have fallen by -44.58% year on year, indicating a negative trend in the company's sales. The profit before tax less other income has also seen a significant decline of -96.41% year on year, and the net profit after tax has also decreased by -65.14% year on year. In addition, the company's operating profit, operating profit margin, and profit before tax less other income have all hit their lowest points in the last five quarters, indicating a negative trend in the company's efficiency and profitability. The non-operating income, which is 86.05% of the profit before tax, also raises concerns about the sustainability of the company's business model. Furthermore, the earnings per share for the quarter have also decreased, indicating a decline in profitability and lower returns for shareholders. Overall, Lakshmi Electrical Control Systems' financial performance for the quarter ending March 2024 has been negative, with some strengths and weaknesses. Investors should carefully consider these factors before making any investment decisions.

0"> {{ stock_short.product_icon_list.length-1 }} More