reference research meaning

• Privacy Policy

Home » References in Research – Types, Examples and Writing Guide

References in Research – Types, Examples and Writing Guide

Table of Contents

References in Research

Definition:

References in research are a list of sources that a researcher has consulted or cited while conducting their study. They are an essential component of any academic work, including research papers, theses, dissertations, and other scholarly publications.

Types of References

There are several types of references used in research, and the type of reference depends on the source of information being cited. The most common types of references include:

References to books typically include the author’s name, title of the book, publisher, publication date, and place of publication.

Example: Smith, J. (2018). The Art of Writing. Penguin Books.

Journal Articles

References to journal articles usually include the author’s name, title of the article, name of the journal, volume and issue number, page numbers, and publication date.

Example: Johnson, T. (2021). The Impact of Social Media on Mental Health. Journal of Psychology, 32(4), 87-94.

Web sources

References to web sources should include the author or organization responsible for the content, the title of the page, the URL, and the date accessed.

Example: World Health Organization. (2020). Coronavirus disease (COVID-19) advice for the public. Retrieved from https://www.who.int/emergencies/disease/novel-coronavirus-2019/advice-for-public

Conference Proceedings

References to conference proceedings should include the author’s name, title of the paper, name of the conference, location of the conference, date of the conference, and page numbers.

Example: Chen, S., & Li, J. (2019). The Future of AI in Education. Proceedings of the International Conference on Educational Technology, Beijing, China, July 15-17, pp. 67-78.

References to reports typically include the author or organization responsible for the report, title of the report, publication date, and publisher.

Example: United Nations. (2020). The Sustainable Development Goals Report. United Nations.

Formats of References

Some common Formates of References with their examples are as follows:

APA (American Psychological Association) Style

The APA (American Psychological Association) Style has specific guidelines for formatting references used in academic papers, articles, and books. Here are the different reference formats in APA style with examples:

Author, A. A. (Year of publication). Title of book. Publisher.

Example : Smith, J. K. (2005). The psychology of social interaction. Wiley-Blackwell.

Journal Article

Author, A. A., Author, B. B., & Author, C. C. (Year of publication). Title of article. Title of Journal, volume number(issue number), page numbers.

Example : Brown, L. M., Keating, J. G., & Jones, S. M. (2012). The role of social support in coping with stress among African American adolescents. Journal of Research on Adolescence, 22(1), 218-233.

Author, A. A. (Year of publication or last update). Title of page. Website name. URL.

Example : Centers for Disease Control and Prevention. (2020, December 11). COVID-19: How to protect yourself and others. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html

Magazine article

Author, A. A. (Year, Month Day of publication). Title of article. Title of Magazine, volume number(issue number), page numbers.

Example : Smith, M. (2019, March 11). The power of positive thinking. Psychology Today, 52(3), 60-65.

Newspaper article:

Author, A. A. (Year, Month Day of publication). Title of article. Title of Newspaper, page numbers.

Example: Johnson, B. (2021, February 15). New study shows benefits of exercise on mental health. The New York Times, A8.

Edited book

Editor, E. E. (Ed.). (Year of publication). Title of book. Publisher.

Example : Thompson, J. P. (Ed.). (2014). Social work in the 21st century. Sage Publications.

Chapter in an edited book:

Author, A. A. (Year of publication). Title of chapter. In E. E. Editor (Ed.), Title of book (pp. page numbers). Publisher.

Example : Johnson, K. S. (2018). The future of social work: Challenges and opportunities. In J. P. Thompson (Ed.), Social work in the 21st century (pp. 105-118). Sage Publications.

MLA (Modern Language Association) Style

The MLA (Modern Language Association) Style is a widely used style for writing academic papers and essays in the humanities. Here are the different reference formats in MLA style:

Author’s Last name, First name. Title of Book. Publisher, Publication year.

Example : Smith, John. The Psychology of Social Interaction. Wiley-Blackwell, 2005.

Journal article

Author’s Last name, First name. “Title of Article.” Title of Journal, volume number, issue number, Publication year, page numbers.

Example : Brown, Laura M., et al. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence, vol. 22, no. 1, 2012, pp. 218-233.

Author’s Last name, First name. “Title of Webpage.” Website Name, Publication date, URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC, 11 Dec. 2020, https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Publication date, page numbers.

Example : Smith, Mary. “The Power of Positive Thinking.” Psychology Today, Mar. 2019, pp. 60-65.

Newspaper article

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Publication date, page numbers.

Example : Johnson, Bob. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, 15 Feb. 2021, p. A8.

Editor’s Last name, First name, editor. Title of Book. Publisher, Publication year.

Example : Thompson, John P., editor. Social Work in the 21st Century. Sage Publications, 2014.

Chapter in an edited book

Author’s Last name, First name. “Title of Chapter.” Title of Book, edited by Editor’s First Name Last name, Publisher, Publication year, page numbers.

Example : Johnson, Karen S. “The Future of Social Work: Challenges and Opportunities.” Social Work in the 21st Century, edited by John P. Thompson, Sage Publications, 2014, pp. 105-118.

Chicago Manual of Style

The Chicago Manual of Style is a widely used style for writing academic papers, dissertations, and books in the humanities and social sciences. Here are the different reference formats in Chicago style:

Example : Smith, John K. The Psychology of Social Interaction. Wiley-Blackwell, 2005.

Author’s Last name, First name. “Title of Article.” Title of Journal volume number, no. issue number (Publication year): page numbers.

Example : Brown, Laura M., John G. Keating, and Sarah M. Jones. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence 22, no. 1 (2012): 218-233.

Author’s Last name, First name. “Title of Webpage.” Website Name. Publication date. URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC. December 11, 2020. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Publication date.

Example : Smith, Mary. “The Power of Positive Thinking.” Psychology Today, March 2019.

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Publication date.

Example : Johnson, Bob. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Example : Thompson, John P., ed. Social Work in the 21st Century. Sage Publications, 2014.

Author’s Last name, First name. “Title of Chapter.” In Title of Book, edited by Editor’s First Name Last Name, page numbers. Publisher, Publication year.

Example : Johnson, Karen S. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by John P. Thompson, 105-118. Sage Publications, 2014.

Harvard Style

The Harvard Style, also known as the Author-Date System, is a widely used style for writing academic papers and essays in the social sciences. Here are the different reference formats in Harvard Style:

Author’s Last name, First name. Year of publication. Title of Book. Place of publication: Publisher.

Example : Smith, John. 2005. The Psychology of Social Interaction. Oxford: Wiley-Blackwell.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Journal volume number (issue number): page numbers.

Example: Brown, Laura M., John G. Keating, and Sarah M. Jones. 2012. “The Role of Social Support in Coping with Stress among African American Adolescents.” Journal of Research on Adolescence 22 (1): 218-233.

Author’s Last name, First name. Year of publication. “Title of Webpage.” Website Name. URL. Accessed date.

Example : Centers for Disease Control and Prevention. 2020. “COVID-19: How to Protect Yourself and Others.” CDC. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html. Accessed April 1, 2023.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Magazine, month and date of publication.

Example : Smith, Mary. 2019. “The Power of Positive Thinking.” Psychology Today, March 2019.

Author’s Last name, First name. Year of publication. “Title of Article.” Title of Newspaper, month and date of publication.

Example : Johnson, Bob. 2021. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Editor’s Last name, First name, ed. Year of publication. Title of Book. Place of publication: Publisher.

Example : Thompson, John P., ed. 2014. Social Work in the 21st Century. Thousand Oaks, CA: Sage Publications.

Author’s Last name, First name. Year of publication. “Title of Chapter.” In Title of Book, edited by Editor’s First Name Last Name, page numbers. Place of publication: Publisher.

Example : Johnson, Karen S. 2014. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by John P. Thompson, 105-118. Thousand Oaks, CA: Sage Publications.

Vancouver Style

The Vancouver Style, also known as the Uniform Requirements for Manuscripts Submitted to Biomedical Journals, is a widely used style for writing academic papers in the biomedical sciences. Here are the different reference formats in Vancouver Style:

Author’s Last name, First name. Title of Book. Edition number. Place of publication: Publisher; Year of publication.

Example : Smith, John K. The Psychology of Social Interaction. 2nd ed. Oxford: Wiley-Blackwell; 2005.

Author’s Last name, First name. Title of Article. Abbreviated Journal Title. Year of publication; volume number(issue number):page numbers.

Example : Brown LM, Keating JG, Jones SM. The Role of Social Support in Coping with Stress among African American Adolescents. J Res Adolesc. 2012;22(1):218-233.

Author’s Last name, First name. Title of Webpage. Website Name [Internet]. Publication date. [cited date]. Available from: URL.

Example : Centers for Disease Control and Prevention. COVID-19: How to Protect Yourself and Others [Internet]. 2020 Dec 11. [cited 2023 Apr 1]. Available from: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. Title of Article. Title of Magazine. Year of publication; month and day of publication:page numbers.

Example : Smith M. The Power of Positive Thinking. Psychology Today. 2019 Mar 1:32-35.

Author’s Last name, First name. Title of Article. Title of Newspaper. Year of publication; month and day of publication:page numbers.

Example : Johnson B. New Study Shows Benefits of Exercise on Mental Health. The New York Times. 2021 Feb 15:A4.

Editor’s Last name, First name, editor. Title of Book. Edition number. Place of publication: Publisher; Year of publication.

Example: Thompson JP, editor. Social Work in the 21st Century. 1st ed. Thousand Oaks, CA: Sage Publications; 2014.

Author’s Last name, First name. Title of Chapter. In: Editor’s Last name, First name, editor. Title of Book. Edition number. Place of publication: Publisher; Year of publication. page numbers.

Example : Johnson KS. The Future of Social Work: Challenges and Opportunities. In: Thompson JP, editor. Social Work in the 21st Century. 1st ed. Thousand Oaks, CA: Sage Publications; 2014. p. 105-118.

Turabian Style

Turabian style is a variation of the Chicago style used in academic writing, particularly in the fields of history and humanities. Here are the different reference formats in Turabian style:

Author’s Last name, First name. Title of Book. Place of publication: Publisher, Year of publication.

Example : Smith, John K. The Psychology of Social Interaction. Oxford: Wiley-Blackwell, 2005.

Author’s Last name, First name. “Title of Article.” Title of Journal volume number, no. issue number (Year of publication): page numbers.

Example : Brown, LM, Keating, JG, Jones, SM. “The Role of Social Support in Coping with Stress among African American Adolescents.” J Res Adolesc 22, no. 1 (2012): 218-233.

Author’s Last name, First name. “Title of Webpage.” Name of Website. Publication date. Accessed date. URL.

Example : Centers for Disease Control and Prevention. “COVID-19: How to Protect Yourself and Others.” CDC. December 11, 2020. Accessed April 1, 2023. https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/prevention.html.

Author’s Last name, First name. “Title of Article.” Title of Magazine, Month Day, Year of publication, page numbers.

Example : Smith, M. “The Power of Positive Thinking.” Psychology Today, March 1, 2019, 32-35.

Author’s Last name, First name. “Title of Article.” Title of Newspaper, Month Day, Year of publication.

Example : Johnson, B. “New Study Shows Benefits of Exercise on Mental Health.” The New York Times, February 15, 2021.

Editor’s Last name, First name, ed. Title of Book. Place of publication: Publisher, Year of publication.

Example : Thompson, JP, ed. Social Work in the 21st Century. Thousand Oaks, CA: Sage Publications, 2014.

Author’s Last name, First name. “Title of Chapter.” In Title of Book, edited by Editor’s Last name, First name, page numbers. Place of publication: Publisher, Year of publication.

Example : Johnson, KS. “The Future of Social Work: Challenges and Opportunities.” In Social Work in the 21st Century, edited by Thompson, JP, 105-118. Thousand Oaks, CA: Sage Publications, 2014.

IEEE (Institute of Electrical and Electronics Engineers) Style

IEEE (Institute of Electrical and Electronics Engineers) style is commonly used in engineering, computer science, and other technical fields. Here are the different reference formats in IEEE style:

Author’s Last name, First name. Book Title. Place of Publication: Publisher, Year of publication.

Example : Oppenheim, A. V., & Schafer, R. W. Discrete-Time Signal Processing. Upper Saddle River, NJ: Prentice Hall, 2010.

Author’s Last name, First name. “Title of Article.” Abbreviated Journal Title, vol. number, no. issue number, pp. page numbers, Month year of publication.

Example: Shannon, C. E. “A Mathematical Theory of Communication.” Bell System Technical Journal, vol. 27, no. 3, pp. 379-423, July 1948.

Conference paper

Author’s Last name, First name. “Title of Paper.” In Title of Conference Proceedings, Place of Conference, Date of Conference, pp. page numbers, Year of publication.

Example: Gupta, S., & Kumar, P. “An Improved System of Linear Discriminant Analysis for Face Recognition.” In Proceedings of the 2011 International Conference on Computer Science and Network Technology, Harbin, China, Dec. 2011, pp. 144-147.

Author’s Last name, First name. “Title of Webpage.” Name of Website. Date of publication or last update. Accessed date. URL.

Example : National Aeronautics and Space Administration. “Apollo 11.” NASA. July 20, 1969. Accessed April 1, 2023. https://www.nasa.gov/mission_pages/apollo/apollo11.html.

Technical report

Author’s Last name, First name. “Title of Report.” Name of Institution or Organization, Report number, Year of publication.

Example : Smith, J. R. “Development of a New Solar Panel Technology.” National Renewable Energy Laboratory, NREL/TP-6A20-51645, 2011.

Author’s Last name, First name. “Title of Patent.” Patent number, Issue date.

Example : Suzuki, H. “Method of Producing Carbon Nanotubes.” US Patent 7,151,019, December 19, 2006.

Standard Title. Standard number, Publication date.

Example : IEEE Standard for Floating-Point Arithmetic. IEEE Std 754-2008, August 29, 2008

ACS (American Chemical Society) Style

ACS (American Chemical Society) style is commonly used in chemistry and related fields. Here are the different reference formats in ACS style:

Author’s Last name, First name; Author’s Last name, First name. Title of Article. Abbreviated Journal Title Year, Volume, Page Numbers.

Example : Wang, Y.; Zhao, X.; Cui, Y.; Ma, Y. Facile Preparation of Fe3O4/graphene Composites Using a Hydrothermal Method for High-Performance Lithium Ion Batteries. ACS Appl. Mater. Interfaces 2012, 4, 2715-2721.

Author’s Last name, First name. Book Title; Publisher: Place of Publication, Year of Publication.

Example : Carey, F. A. Organic Chemistry; McGraw-Hill: New York, 2008.

Author’s Last name, First name. Chapter Title. In Book Title; Editor’s Last name, First name, Ed.; Publisher: Place of Publication, Year of Publication; Volume number, Chapter number, Page Numbers.

Example : Grossman, R. B. Analytical Chemistry of Aerosols. In Aerosol Measurement: Principles, Techniques, and Applications; Baron, P. A.; Willeke, K., Eds.; Wiley-Interscience: New York, 2001; Chapter 10, pp 395-424.

Author’s Last name, First name. Title of Webpage. Website Name, URL (accessed date).

Example : National Institute of Standards and Technology. Atomic Spectra Database. https://www.nist.gov/pml/atomic-spectra-database (accessed April 1, 2023).

Author’s Last name, First name. Patent Number. Patent Date.

Example : Liu, Y.; Huang, H.; Chen, H.; Zhang, W. US Patent 9,999,999, December 31, 2022.

Author’s Last name, First name; Author’s Last name, First name. Title of Article. In Title of Conference Proceedings, Publisher: Place of Publication, Year of Publication; Volume Number, Page Numbers.

Example : Jia, H.; Xu, S.; Wu, Y.; Wu, Z.; Tang, Y.; Huang, X. Fast Adsorption of Organic Pollutants by Graphene Oxide. In Proceedings of the 15th International Conference on Environmental Science and Technology, American Chemical Society: Washington, DC, 2017; Volume 1, pp 223-228.

AMA (American Medical Association) Style

AMA (American Medical Association) style is commonly used in medical and scientific fields. Here are the different reference formats in AMA style:

Author’s Last name, First name. Article Title. Journal Abbreviation. Year; Volume(Issue):Page Numbers.

Example : Jones, R. A.; Smith, B. C. The Role of Vitamin D in Maintaining Bone Health. JAMA. 2019;321(17):1765-1773.

Author’s Last name, First name. Book Title. Edition number. Place of Publication: Publisher; Year.

Example : Guyton, A. C.; Hall, J. E. Textbook of Medical Physiology. 13th ed. Philadelphia, PA: Saunders; 2015.

Author’s Last name, First name. Chapter Title. In: Editor’s Last name, First name, ed. Book Title. Edition number. Place of Publication: Publisher; Year: Page Numbers.

Example: Rajakumar, K. Vitamin D and Bone Health. In: Holick, M. F., ed. Vitamin D: Physiology, Molecular Biology, and Clinical Applications. 2nd ed. New York, NY: Springer; 2010:211-222.

Author’s Last name, First name. Webpage Title. Website Name. URL. Published date. Updated date. Accessed date.

Example : National Cancer Institute. Breast Cancer Prevention (PDQ®)–Patient Version. National Cancer Institute. https://www.cancer.gov/types/breast/patient/breast-prevention-pdq. Published October 11, 2022. Accessed April 1, 2023.

Author’s Last name, First name. Conference presentation title. In: Conference Title; Conference Date; Place of Conference.

Example : Smith, J. R. Vitamin D and Bone Health: A Meta-Analysis. In: Proceedings of the Annual Meeting of the American Society for Bone and Mineral Research; September 20-23, 2022; San Diego, CA.

Thesis or dissertation

Author’s Last name, First name. Title of Thesis or Dissertation. Degree level [Doctoral dissertation or Master’s thesis]. University Name; Year.

Example : Wilson, S. A. The Effects of Vitamin D Supplementation on Bone Health in Postmenopausal Women [Doctoral dissertation]. University of California, Los Angeles; 2018.

ASCE (American Society of Civil Engineers) Style

The ASCE (American Society of Civil Engineers) style is commonly used in civil engineering fields. Here are the different reference formats in ASCE style:

Author’s Last name, First name. “Article Title.” Journal Title, volume number, issue number (year): page numbers. DOI or URL (if available).

Example : Smith, J. R. “Evaluation of the Effectiveness of Sustainable Drainage Systems in Urban Areas.” Journal of Environmental Engineering, vol. 146, no. 3 (2020): 04020010. https://doi.org/10.1061/(ASCE)EE.1943-7870.0001668.

Example : McCuen, R. H. Hydrologic Analysis and Design. 4th ed. Upper Saddle River, NJ: Pearson Education; 2013.

Author’s Last name, First name. “Chapter Title.” In: Editor’s Last name, First name, ed. Book Title. Edition number. Place of Publication: Publisher; Year: page numbers.

Example : Maidment, D. R. “Floodplain Management in the United States.” In: Shroder, J. F., ed. Treatise on Geomorphology. San Diego, CA: Academic Press; 2013: 447-460.

Author’s Last name, First name. “Paper Title.” In: Conference Title; Conference Date; Location. Place of Publication: Publisher; Year: page numbers.

Example: Smith, J. R. “Sustainable Drainage Systems for Urban Areas.” In: Proceedings of the ASCE International Conference on Sustainable Infrastructure; November 6-9, 2019; Los Angeles, CA. Reston, VA: American Society of Civil Engineers; 2019: 156-163.

Author’s Last name, First name. “Report Title.” Report number. Place of Publication: Publisher; Year.

Example : U.S. Army Corps of Engineers. “Hurricane Sandy Coastal Risk Reduction Program, New York and New Jersey.” Report No. P-15-001. Washington, DC: U.S. Army Corps of Engineers; 2015.

CSE (Council of Science Editors) Style

The CSE (Council of Science Editors) style is commonly used in the scientific and medical fields. Here are the different reference formats in CSE style:

Author’s Last name, First Initial. Middle Initial. “Article Title.” Journal Title. Year;Volume(Issue):Page numbers.

Example : Smith, J.R. “Evaluation of the Effectiveness of Sustainable Drainage Systems in Urban Areas.” Journal of Environmental Engineering. 2020;146(3):04020010.

Author’s Last name, First Initial. Middle Initial. Book Title. Edition number. Place of Publication: Publisher; Year.

Author’s Last name, First Initial. Middle Initial. “Chapter Title.” In: Editor’s Last name, First Initial. Middle Initial., ed. Book Title. Edition number. Place of Publication: Publisher; Year:Page numbers.

Author’s Last name, First Initial. Middle Initial. “Paper Title.” In: Conference Title; Conference Date; Location. Place of Publication: Publisher; Year.

Example : Smith, J.R. “Sustainable Drainage Systems for Urban Areas.” In: Proceedings of the ASCE International Conference on Sustainable Infrastructure; November 6-9, 2019; Los Angeles, CA. Reston, VA: American Society of Civil Engineers; 2019.

Author’s Last name, First Initial. Middle Initial. “Report Title.” Report number. Place of Publication: Publisher; Year.

Bluebook Style

The Bluebook style is commonly used in the legal field for citing legal documents and sources. Here are the different reference formats in Bluebook style:

Case citation

Case name, volume source page (Court year).

Example : Brown v. Board of Education, 347 U.S. 483 (1954).

Statute citation

Name of Act, volume source § section number (year).

Example : Clean Air Act, 42 U.S.C. § 7401 (1963).

Regulation citation

Name of regulation, volume source § section number (year).

Example: Clean Air Act, 40 C.F.R. § 52.01 (2019).

Book citation

Author’s Last name, First Initial. Middle Initial. Book Title. Edition number (if applicable). Place of Publication: Publisher; Year.

Example: Smith, J.R. Legal Writing and Analysis. 3rd ed. New York, NY: Aspen Publishers; 2015.

Journal article citation

Author’s Last name, First Initial. Middle Initial. “Article Title.” Journal Title. Volume number (year): first page-last page.

Example: Garcia, C. “The Right to Counsel: An International Comparison.” International Journal of Legal Information. 43 (2015): 63-94.

Website citation

Author’s Last name, First Initial. Middle Initial. “Page Title.” Website Title. URL (accessed month day, year).

Example : United Nations. “Universal Declaration of Human Rights.” United Nations. https://www.un.org/en/universal-declaration-human-rights/ (accessed January 3, 2023).

Oxford Style

The Oxford style, also known as the Oxford referencing system or the documentary-note citation system, is commonly used in the humanities, including literature, history, and philosophy. Here are the different reference formats in Oxford style:

Author’s Last name, First name. Book Title. Place of Publication: Publisher, Year of Publication.

Example : Smith, John. The Art of Writing. New York: Penguin, 2020.

Author’s Last name, First name. “Article Title.” Journal Title volume, no. issue (year): page range.

Example: Garcia, Carlos. “The Role of Ethics in Philosophy.” Philosophy Today 67, no. 3 (2019): 53-68.

Chapter in an edited book citation

Author’s Last name, First name. “Chapter Title.” In Book Title, edited by Editor’s Name, page range. Place of Publication: Publisher, Year of Publication.

Example : Lee, Mary. “Feminism in the 21st Century.” In The Oxford Handbook of Feminism, edited by Jane Smith, 51-69. Oxford: Oxford University Press, 2018.

Author’s Last name, First name. “Page Title.” Website Title. URL (accessed day month year).

Example : Jones, David. “The Importance of Learning Languages.” Oxford Language Center. https://www.oxfordlanguagecenter.com/importance-of-learning-languages/ (accessed 3 January 2023).

Dissertation or thesis citation

Author’s Last name, First name. “Title of Dissertation/Thesis.” PhD diss., University Name, Year of Publication.

Example : Brown, Susan. “The Art of Storytelling in American Literature.” PhD diss., University of Oxford, 2020.

Newspaper article citation

Author’s Last name, First name. “Article Title.” Newspaper Title, Month Day, Year.

Example : Robinson, Andrew. “New Developments in Climate Change Research.” The Guardian, September 15, 2022.

AAA (American Anthropological Association) Style

The American Anthropological Association (AAA) style is commonly used in anthropology research papers and journals. Here are the different reference formats in AAA style:

Author’s Last name, First name. Year of Publication. Book Title. Place of Publication: Publisher.

Example : Smith, John. 2019. The Anthropology of Food. New York: Routledge.

Author’s Last name, First name. Year of Publication. “Article Title.” Journal Title volume, no. issue: page range.

Example : Garcia, Carlos. 2021. “The Role of Ethics in Anthropology.” American Anthropologist 123, no. 2: 237-251.

Author’s Last name, First name. Year of Publication. “Chapter Title.” In Book Title, edited by Editor’s Name, page range. Place of Publication: Publisher.

Example: Lee, Mary. 2018. “Feminism in Anthropology.” In The Oxford Handbook of Feminism, edited by Jane Smith, 51-69. Oxford: Oxford University Press.

Author’s Last name, First name. Year of Publication. “Page Title.” Website Title. URL (accessed day month year).

Example : Jones, David. 2020. “The Importance of Learning Languages.” Oxford Language Center. https://www.oxfordlanguagecenter.com/importance-of-learning-languages/ (accessed January 3, 2023).

Author’s Last name, First name. Year of Publication. “Title of Dissertation/Thesis.” PhD diss., University Name.

Example : Brown, Susan. 2022. “The Art of Storytelling in Anthropology.” PhD diss., University of California, Berkeley.

Author’s Last name, First name. Year of Publication. “Article Title.” Newspaper Title, Month Day.

Example : Robinson, Andrew. 2021. “New Developments in Anthropology Research.” The Guardian, September 15.

AIP (American Institute of Physics) Style

The American Institute of Physics (AIP) style is commonly used in physics research papers and journals. Here are the different reference formats in AIP style:

Example : Johnson, S. D. 2021. “Quantum Computing and Information.” Journal of Applied Physics 129, no. 4: 043102.

Example : Feynman, Richard. 2018. The Feynman Lectures on Physics. New York: Basic Books.

Example : Jones, David. 2020. “The Future of Quantum Computing.” In The Handbook of Physics, edited by John Smith, 125-136. Oxford: Oxford University Press.

Conference proceedings citation

Author’s Last name, First name. Year of Publication. “Title of Paper.” Proceedings of Conference Name, date and location: page range. Place of Publication: Publisher.

Example : Chen, Wei. 2019. “The Applications of Nanotechnology in Solar Cells.” Proceedings of the 8th International Conference on Nanotechnology, July 15-17, Tokyo, Japan: 224-229. New York: AIP Publishing.

Example : American Institute of Physics. 2022. “About AIP Publishing.” AIP Publishing. https://publishing.aip.org/about-aip-publishing/ (accessed January 3, 2023).

Patent citation

Author’s Last name, First name. Year of Publication. Patent Number.

Example : Smith, John. 2018. US Patent 9,873,644.

References Writing Guide

Here are some general guidelines for writing references:

• Follow the citation style guidelines: Different disciplines and journals may require different citation styles (e.g., APA, MLA, Chicago). It is important to follow the specific guidelines for the citation style required.
• Include all necessary information : Each citation should include enough information for readers to locate the source. For example, a journal article citation should include the author(s), title of the article, journal title, volume number, issue number, page numbers, and publication year.
• Use proper formatting: Citation styles typically have specific formatting requirements for different types of sources. Make sure to follow the proper formatting for each citation.
• Order citations alphabetically: If listing multiple sources, they should be listed alphabetically by the author’s last name.
• Be consistent: Use the same citation style throughout the entire paper or project.
• Check for accuracy: Double-check all citations to ensure accuracy, including correct spelling of author names and publication information.
• Use reputable sources: When selecting sources to cite, choose reputable and authoritative sources. Avoid sources that are biased or unreliable.
• Include all sources: Make sure to include all sources used in the research, including those that were not directly quoted but still informed the work.
• Use online tools : There are online tools available (e.g., citation generators) that can help with formatting and organizing references.

Purpose of References in Research

References in research serve several purposes:

• To give credit to the original authors or sources of information used in the research. It is important to acknowledge the work of others and avoid plagiarism.
• To provide evidence for the claims made in the research. References can support the arguments, hypotheses, or conclusions presented in the research by citing relevant studies, data, or theories.
• To allow readers to find and verify the sources used in the research. References provide the necessary information for readers to locate and access the sources cited in the research, which allows them to evaluate the quality and reliability of the information presented.
• To situate the research within the broader context of the field. References can show how the research builds on or contributes to the existing body of knowledge, and can help readers to identify gaps in the literature that the research seeks to address.

Importance of References in Research

References play an important role in research for several reasons:

• Credibility : By citing authoritative sources, references lend credibility to the research and its claims. They provide evidence that the research is based on a sound foundation of knowledge and has been carefully researched.
• Avoidance of Plagiarism : References help researchers avoid plagiarism by giving credit to the original authors or sources of information. This is important for ethical reasons and also to avoid legal repercussions.
• Reproducibility : References allow others to reproduce the research by providing detailed information on the sources used. This is important for verification of the research and for others to build on the work.
• Context : References provide context for the research by situating it within the broader body of knowledge in the field. They help researchers to understand where their work fits in and how it builds on or contributes to existing knowledge.
• Evaluation : References provide a means for others to evaluate the research by allowing them to assess the quality and reliability of the sources used.

Advantages of References in Research

There are several advantages of including references in research:

• Acknowledgment of Sources: Including references gives credit to the authors or sources of information used in the research. This is important to acknowledge the original work and avoid plagiarism.
• Evidence and Support : References can provide evidence to support the arguments, hypotheses, or conclusions presented in the research. This can add credibility and strength to the research.
• Reproducibility : References provide the necessary information for others to reproduce the research. This is important for the verification of the research and for others to build on the work.
• Context : References can help to situate the research within the broader body of knowledge in the field. This helps researchers to understand where their work fits in and how it builds on or contributes to existing knowledge.
• Evaluation : Including references allows others to evaluate the research by providing a means to assess the quality and reliability of the sources used.
• Ongoing Conversation: References allow researchers to engage in ongoing conversations and debates within their fields. They can show how the research builds on or contributes to the existing body of knowledge.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Table of Contents – Types, Formats, Examples

Thesis – Structure, Example and Writing Guide

Research Paper – Structure, Examples and Writing...

Research Report – Example, Writing Guide and...

Research Methodology – Types, Examples and...

Scope of the Research – Writing Guide and...

• Study and research support
• Referencing

Referencing explained

Why and when to reference.

Referencing is an important part of academic work. It puts your work in context, demonstrates the breadth and depth of your research, and acknowledges other people’s work. You should reference whenever you use someone else’s idea.

View video using Microsoft Stream (link opens in a new window, available for University members only)

These webpages explain what referencing is, why it is important and give an overview of the main elements of how to reference. Our Referencing made simple tutorial opens in a new window and covers how to identify your source and create a reference with interactive examples.

Why reference?

Referencing correctly:

• helps you to avoid plagiarism by making it clear which ideas are your own and which are someone else’s
• shows your understanding of the topic
• gives supporting evidence for your ideas, arguments and opinions
• allows others to identify the sources you have used.

When to reference

Whenever you use an idea from someone else's work, for example from a journal article, textbook or website, you should cite the original author to make it clear where that idea came from. This is the case regardless of whether you have paraphrased, summarised or directly quoted their work. This is a key part of good practice in academic writing.

Read more on:

• academic integrity
• quoting, summarising, paraphrasing, and synthesising
• citing direct quotations in Leeds Harvard or citing direct quotations in Leeds Numeric styles.

University and school policies

The University referencing policy (PDF) sets out the referencing requirements that all taught students and tutors are expected to follow.

Each school in the University requires students to use a specific style of referencing. Check the referencing style used in your school before you begin.

All your citations and references should match the style you are using exactly, including any punctuation, capitalisation, italics and bold, and you should use the same referencing style throughout your assignment.

Skip to Content

Massey University

• Search OWLL
• Handouts (Printable)
• Pre-reading Service
• StudyUp Recordings
• StudyUp Postgraduate
• Academic writing
• Intro to academic writing
• What is academic writing?
• Writing objectively
• Writing concisely
• 1st vs. 3rd person
• Inclusive language
• Te Reo Māori
• Assignment planning
• Assignment planning calculator
• Interpreting the assignment question
• Command words
• Organising points
• Researching
• Identifying academic sources
• Evaluating source quality
• Editing & proofreading
• Apostrophes
• Other punctuation
• Active voice
• American vs. British spelling
• Conditionals
• Prepositions
• Pronoun Reference
• Sentence fragments
• Sentence Structure
• Subject-verb agreement
• Formatting and layout
• Word limits and assignment length
• Commonly confused words
• How assignments are marked
• Marking guides
• Getting an A
• Levels of assessment
• Using feedback
• Professional emails
• Forum posts
• Forum netiquette guidelines
• Sharing personal information
• Writing about personal experiences
• Assignment types
• What is an essay?
• Essay planning and structure
• Introduction
• Thesis statement
• Body paragraphs
• Essay revision
• Essay writing resources
• What is a report?
• Report structure
• Analysing issues for a report
• Business report
• What is a business report?
• Business report structure
• Inductive vs. deductive reports
• Other kinds of business communication
• Business report format and layout
• What is a lab report?
• Lab report structure
• Science lab report writing resources
• Psychology lab report writing resources
• Lab report body paragraphs
• Literature review
• What is a literature review?
• Writing a literature review
• Literature review structure
• Literature review writing resources
• Research proposal
• Writing a research proposal
• Research proposal structure
• Other types
• Article critique
• Book review
• Annotated bibliography
• Reflective writing
• Oral presentation
• Thesis / dissertation
• Article / conference paper
• Shorter responses
• PhD confirmation report
• Computer skills
• Microsoft Word
• Basic formatting
• Images, tables, & figures
• Long documents
• Microsoft Excel
• Basic spreadsheets
• Navigating & printing spreadsheets
• Charts / graphs & formulas
• Microsoft PowerPoint
• Basic skills
• Advanced skills
• Distance study
• Getting started
• How to study
• Online study techniques
• Distance support
• Reading & writing
• Reading strategies
• Writing strategies
• Grammar resources
• Listening & speaking
• Listening strategies
• Speaking strategies
• Maths & statistics
• Trigonometry
• Finance formulas
• Postgraduate study
• Intro to postgrad study
• Planning postgrad study
• Postgrad resources
• Postgrad assignment types
• Referencing
• Intro to referencing

What is referencing?

• Why reference?
• Common knowledge

Referencing styles

• What type of source is this?
• Reference list vs. bibliography
• Referencing software
• Quoting & paraphrasing
• Paraphrasing & summarising
• Paraphrasing techniques
• APA Interactive
• In-text citation
• Reference list
• Online material
• Other material
• Headings in APA
• Tables and Figures
• Referencing elements
• 5th vs. 6th edition
• 6th vs. 7th edition
• Chicago style
• Chicago Interactive
• About notes system
• Notes referencing elements
• Quoting and paraphrasing
• Author-date system
• MLA Interactive
• Abbreviations
• List of works cited
• Captions for images
• 8th vs 9th edition
• Oxford style
• Other styles
• Harvard style
• Vancouver style
• Legal citations
• Visual material
• Sample assignments
• Sample essay 1
• Sample essay 2
• Sample annotated bibliography
• Sample book review
• Study skills
• Time management
• Intro to time management
• Procrastination & perfectionism
• Goals & motivation
• Time management for internal students
• Time management for distance students
• Memory skills
• Principles of good memory
• Memory strategies
• Note-taking
• Note-taking methods
• Note-taking in lectures
• Note-taking while reading
• Digital note-taking
• Reading styles
• In-depth reading
• Reading comprehension
• Reading academic material
• Reading a journal article
• Reading an academic book
• Critical thinking
• What is critical thinking?
• Constructing an argument
• Critical reading
• Logical fallacies
• Tests & exams
• Exam & test study
• Planning exam study
• Gathering & sorting information
• Reviewing past exams
• Phases of revision
• Last-minute study strategies
• Question types
• Short answer
• Multi-choice
• Problem / computational
• Case-study / scenario
• Open book exam
• Open web exam or test
• Take home test
• In the exam
• Online exam
• Physical exam

Academic writing relies on more than just the ideas and experience of one author. It also uses the ideas and research of other sources: books, journal articles, websites, and so forth. These other sources may be used to support the author's ideas, or the author may be discussing, analysing, or critiquing other sources.

Referencing is used to tell the reader where ideas from other sources have been used in an assignment. There are many reasons why it is important to reference sources correctly:

• It shows the reader that you can find and use sources to create a solid argument
• It properly credits the originators of ideas, theories, and research findings
• It shows the reader how your argument relates to the big picture

(For a detailed discussion, see why reference? )

Failure to properly acknowledge sources is called plagiarism , and it can carry significant academic penalties. Fortunately, plagiarism is easy to avoid by following a few basic principles.

What needs to be referenced?

Whenever an assignment uses words, facts, ideas, theories, or interpretations from other sources, those sources must be referenced. Referencing is needed when:

• You have copied words from a book, article, or other source exactly ( quotation )
• You have used an idea or fact from an outside source, even if you haven't used their exact wording ( paraphrasing and summarising )

The only exception to this is when the information is common knowledge , which is something that anyone is likely to know. If you are uncertain whether to reference something or not, it is better to reference it.

Citations and references

There are two elements used in referencing:

• A citation in the text of the assignment (also known as in-text citations)
• An entry in a reference list at the end of the assignment

The citation contains only enough information for the reader to find the source in the reference list. Usually, this is the name of the source's author and the year the source was published. For example:

When testing the usability of a website, it is necessary to gather demographic information about the users (Lazar, 2006).

In this example, (Lazar, 2006) tells the reader that this information has come from a source written by Lazar, which was published in 2006. This is a signpost, pointing the reader to the reference list.

The reference list is a list of all the sources used (and cited) in an assignment. It is alphabetised according to the names of the authors. Each entry in the reference list contains detailed information about one source. This usually includes the author's name, the year of publication, the title of the source, and source location details (e.g., publisher’s name, URL). For example:

Durie, M. (2003). Ngā kāhui pou: Launching Māori futures . Huia.

Hazledine, T., & Quiggan, J. (2006). Public policy in Australia and New Zealand: The new global context. Australian Journal of Political Science, 41 (2), 131–143.

Lazar, J. (2006). Web usability: A user-centered design approach . Pearson Addison Wesley.

Ministry for Primary Industries. (2012).  Food safety . https://www.mpi.govt.nz/food-safety

If they wanted to, a reader could use this information to find these sources in a library or online.

Referencing is a formal system: there are rules and standards to follow when formatting citations and references. Many students find referencing quite intimidating at first. Like any skill, it takes time and patience to learn.

The examples above use APA style , a format created by the American Psychological Association. It is the most common referencing style used at Massey University.

Other styles include MLA style , Oxford style , Harvard style , and Chicago style . These styles are subtly different, and different colleges and departments may ask you to use different styles. Oxford style, for example, uses footnotes instead of in-text citations, and a bibliography instead of a reference list .

For more about the different referencing styles used at Massey University, see referencing styles .

Have a study or assignment writing question? Ask an expert at Academic Q+A forum

Get feedback on your draft assignment at Academic Writing and Learning Support

Need help with your assignment or referencing? Contact a consultant and receive online and/or face-to-face support

Page authorised by Director - Centre for Learner Success Last updated on 26 February, 2020

• Academic Q+A

Have a study or assignment writing question? Ask an expert at Academic Q+A

Live online workshops

• StudyUp (undergraduate)
• Campus workshops
• Albany (undergraduate)
• Albany (postgraduate)
• Albany (distance)
• Manawatu (undergraduate)
• Manawatu (postgraduate)

Upcoming events

• All upcoming events
• Academic writing and learning support
• 0800 MASSEY | (+64 6 350 5701)
• [email protected]
• Online form

• Locations and Hours
• UCLA Library
• Research Guides
• Research Tips and Tools

Articles, Books and . . . ? Understanding the Many Types of Information Found in Libraries

• Reference Sources

Encyclopedias

Dictionaries, almanacs and yearbooks, handbooks and manuals.

• Documents and Reports
• Non-Text Content
• Archival Materials

Reference sources are generally the place to begin your research, especially when you're starting out with an unfamiliar field. But they're also where you return when you need to look up formulas, facts, definitions, and other standard details; they tend to pack a lot of information into simple, easy-to-use packages.

Physical Media

Many reference works are available online and are accessible through links from the Library Catalog and from subject or course guides , but many valuable reference resources are still available only in print, and a few highly specialized tools are on microform or CD. Because print-only reference books are in high demand, they are kept in separate, non-circulating reference collections in most UCLA libraries.

Scholarly Sources

Reference sources are rarely peer-reviewed. In fact, because they mostly contain established, factual information, they're sometimes not even cited in academic works, unless directly quoted. Check your style manual for best guidelines.

Primary or Secondary Sources

As compilations of existing information, reference works are decisively in the category of secondary sources... to the point that some people call them tertiary sources .

Encyclopedias attempt to provide comprehensive summaries of knowledge in either a specific field (subject encyclopedias) or "everything" (general encyclopedias). Encyclopedias are typically divided into a collection of articles on discrete topics. Academically oriented encyclopedias will often include short bibliographies, making them a good resource for identifying key books and articles on a topic.

• Online Encyclopedias
• Finding Print Encyclopedias
• Subject dictionaries: define technical terms in specific fields, sometimes in as much detail as an encyclopedia
• Bilingual dictionaries: define words in a different language
• Thesauri: provide synonyms
• Rhyming dictionariess
• Major Online Dictionaries
• Finding Dictionaries (both online and in print)
• World Almanac and Book of Facts
• Statistical Yearbooks

Both "handbook" and "manual" refer to the traditional small size of the volumes, designed to fit in one hand for ease of use. Despite this origin, many modern handbooks are quite hefty!

• Databases containing online handbooks
• Find print handbooks in the Library Catalog Search "[subject] handbooks" within Subject List.
• << Previous: Home
• Next: Books >>
• Last Updated: May 16, 2024 10:30 AM
• URL: https://guides.library.ucla.edu/content-types

• Jump to menu
• Student Home
• Accept your offer
• How to enrol
• Student ID card
• Set up your IT
• Orientation Week
• Fees & payment
• Academic calendar
• Special consideration
• Transcripts
• The Nucleus: Student Hub
• Referencing
• Essay writing
• Learning abroad & exchange
• Professional development & UNSW Advantage
• Employability
• Financial assistance
• International students
• Equitable learning
• Postgraduate research
• Health Service
• Events & activities
• Emergencies
• Volunteering
• Clubs and societies
• Accommodation
• Health services
• Sport and gym
• Arc student organisation
• Security on campus
• Maps of campus
• Careers portal
• Change password

Why is Referencing Important?

Citations are not used simply to avoid plagiarism; they have other important roles too..

Referencing allows you to acknowledge the contribution of other writers and researchers in your work. Any university assignments that draw on the ideas, words or research of other writers must contain citations.

Referencing is also a way to give credit to the writers from whom you have borrowed words and ideas. By citing the work of a particular scholar, you acknowledge and respect the intellectual property rights of that researcher. As a student or academic, you can draw on any of the millions of ideas, insights and arguments published by other writers, many of whom have spent years researching and writing. All you need to do is acknowledge their contribution to your assignment.

Referencing is a way to provide evidence to support the assertions and claims in your own assignments. By citing experts in your field, you are showing your marker that you are aware of the field in which you are operating. Your citations map the space of your discipline and allow you to navigate your way through your chosen field of study, in the same way that sailors steer by the stars.

References should always be accurate, allowing your readers to trace the sources of information you have used. The best way to make sure you reference accurately is to keep a record of all the sources you used when reading and researching for an assignment.

Citations also make your writing more persuasive. 

Exercise: Look at the two paragraphs below: which one seems more authoritative?

The paragraphs are identical, except for the absence of citations from paragraph 1.

The first paragraph may be just as interesting as the second, but within an academic context, a context that requires you to show from where you have taken ideas, the second has far more authority, it is more persuasive. It shows that the ideas you are discussing are matters that are important to your particular academic community.

What kind of information do I need to reference?

Printed books are not the only sources that require acknowledgement. ANY words, ideas or information taken from ANY source requires a reference.

Reference when you are using words or ideas from:

• books and journal articles
• newspapers and magazines
• pamphlets or brochures
• films, documentaries, television programs or advertisements
• websites or electronic resources
• letters, emails, online discussion forums
• personal interviews
• lecturers or tutors. (Not always necessary but check with your lecturer or tutor about their preferences before you draw on their ideas.)

You also need to reference when you reprint any diagrams, illustrations, charts or pictures.

No need to reference when you are:

• writing your own observations or experiment results, for example, a report on a field trip
• writing about your own experiences, for example, a reflective journal
• writing your own thoughts, comments or conclusions in an assignment
• evaluating or offering your own analysis
• using 'common knowledge' (facts that can be found in numerous places and are likely to be known by a lot of people) or folklore
• using generally accepted facts or information. This will vary in different disciplines of study. If in doubt, ask your tutor.
• Academic Integrity & Plagiarism
• Academic integrity online module
• What is plagiarism?
• Intellectual insecurity
• Poor time management
• Lack of a clear argument
• Lack of critical and analytical skills
• Inadequate research
• Poor notetaking
• Why is referencing important?
• Underdeveloped writing skills
• Avoiding plagiarism
• Other plagiarism resources

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Referencing

A Quick Guide to Referencing | Cite Your Sources Correctly

Referencing means acknowledging the sources you have used in your writing. Including references helps you support your claims and ensures that you avoid plagiarism .

There are many referencing styles, but they usually consist of two things:

• A citation wherever you refer to a source in your text.
• A reference list or bibliography at the end listing full details of all your sources.

The most common method of referencing in UK universities is Harvard style , which uses author-date citations in the text. Our free Harvard Reference Generator automatically creates accurate references in this style.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Referencing styles, citing your sources with in-text citations, creating your reference list or bibliography, harvard referencing examples, frequently asked questions about referencing.

Each referencing style has different rules for presenting source information. For in-text citations, some use footnotes or endnotes , while others include the author’s surname and date of publication in brackets in the text.

The reference list or bibliography is presented differently in each style, with different rules for things like capitalisation, italics, and quotation marks in references.

Your university will usually tell you which referencing style to use; they may even have their own unique style. Always follow your university’s guidelines, and ask your tutor if you are unsure. The most common styles are summarised below.

Harvard referencing, the most commonly used style at UK universities, uses author–date in-text citations corresponding to an alphabetical bibliography or reference list at the end.

Harvard Referencing Guide

Vancouver referencing, used in biomedicine and other sciences, uses reference numbers in the text corresponding to a numbered reference list at the end.

Vancouver Referencing Guide

APA referencing, used in the social and behavioural sciences, uses author–date in-text citations corresponding to an alphabetical reference list at the end.

APA Referencing Guide APA Reference Generator

MHRA referencing, used in the humanities, uses footnotes in the text with source information, in addition to an alphabetised bibliography at the end.

MHRA Referencing Guide

OSCOLA referencing, used in law, uses footnotes in the text with source information, and an alphabetical bibliography at the end in longer texts.

OSCOLA Referencing Guide

The only proofreading tool specialized in correcting academic writing

The academic proofreading tool has been trained on 1000s of academic texts and by native English editors. Making it the most accurate and reliable proofreading tool for students.

Correct my document today

In-text citations should be used whenever you quote, paraphrase, or refer to information from a source (e.g. a book, article, image, website, or video).

Quoting and paraphrasing

Quoting is when you directly copy some text from a source and enclose it in quotation marks to indicate that it is not your own writing.

Paraphrasing is when you rephrase the original source into your own words. In this case, you don’t use quotation marks, but you still need to include a citation.

In most referencing styles, page numbers are included when you’re quoting or paraphrasing a particular passage. If you are referring to the text as a whole, no page number is needed.

In-text citations

In-text citations are quick references to your sources. In Harvard referencing, you use the author’s surname and the date of publication in brackets.

Up to three authors are included in a Harvard in-text citation. If the source has more than three authors, include the first author followed by ‘ et al. ‘

The point of these citations is to direct your reader to the alphabetised reference list, where you give full information about each source. For example, to find the source cited above, the reader would look under ‘J’ in your reference list to find the title and publication details of the source.

Placement of in-text citations

In-text citations should be placed directly after the quotation or information they refer to, usually before a comma or full stop. If a sentence is supported by multiple sources, you can combine them in one set of brackets, separated by a semicolon.

If you mention the author’s name in the text already, you don’t include it in the citation, and you can place the citation immediately after the name.

• Another researcher warns that the results of this method are ‘inconsistent’ (Singh, 2018, p. 13) .
• Previous research has frequently illustrated the pitfalls of this method (Singh, 2018; Jones, 2016) .
• Singh (2018, p. 13) warns that the results of this method are ‘inconsistent’.

The terms ‘bibliography’ and ‘reference list’ are sometimes used interchangeably. Both refer to a list that contains full information on all the sources cited in your text. Sometimes ‘bibliography’ is used to mean a more extensive list, also containing sources that you consulted but did not cite in the text.

A reference list or bibliography is usually mandatory, since in-text citations typically don’t provide full source information. For styles that already include full source information in footnotes (e.g. OSCOLA and Chicago Style ), the bibliography is optional, although your university may still require you to include one.

Format of the reference list

Reference lists are usually alphabetised by authors’ last names. Each entry in the list appears on a new line, and a hanging indent is applied if an entry extends onto multiple lines.

Different source information is included for different source types. Each style provides detailed guidelines for exactly what information should be included and how it should be presented.

Below are some examples of reference list entries for common source types in Harvard style.

• Chapter of a book
• Journal article

Your university should tell you which referencing style to follow. If you’re unsure, check with a supervisor. Commonly used styles include:

• Harvard referencing , the most commonly used style in UK universities.
• MHRA , used in humanities subjects.
• APA , used in the social sciences.
• Vancouver , used in biomedicine.
• OSCOLA , used in law.

Your university may have its own referencing style guide.

If you are allowed to choose which style to follow, we recommend Harvard referencing, as it is a straightforward and widely used style.

References should be included in your text whenever you use words, ideas, or information from a source. A source can be anything from a book or journal article to a website or YouTube video.

If you don’t acknowledge your sources, you can get in trouble for plagiarism .

To avoid plagiarism , always include a reference when you use words, ideas or information from a source. This shows that you are not trying to pass the work of others off as your own.

You must also properly quote or paraphrase the source. If you’re not sure whether you’ve done this correctly, you can use the Scribbr Plagiarism Checker to find and correct any mistakes.

Harvard referencing uses an author–date system. Sources are cited by the author’s last name and the publication year in brackets. Each Harvard in-text citation corresponds to an entry in the alphabetised reference list at the end of the paper.

Vancouver referencing uses a numerical system. Sources are cited by a number in parentheses or superscript. Each number corresponds to a full reference at the end of the paper.

Is this article helpful?

Other students also liked.

• A Quick Guide to Harvard Referencing | Citation Examples
• APA Referencing (7th Ed.) Quick Guide | In-text Citations & References

How to Avoid Plagiarism | Tips on Citing Sources

More interesting articles.

• A Quick Guide to OSCOLA Referencing | Rules & Examples
• Harvard In-Text Citation | A Complete Guide & Examples
• Harvard Referencing for Journal Articles | Templates & Examples
• Harvard Style Bibliography | Format & Examples
• MHRA Referencing | A Quick Guide & Citation Examples
• Reference a Website in Harvard Style | Templates & Examples
• Referencing Books in Harvard Style | Templates & Examples
• Vancouver Referencing | A Quick Guide & Reference Examples

Scribbr APA Citation Checker

An innovative new tool that checks your APA citations with AI software. Say goodbye to inaccurate citations!

Referencing

Referencing is one of the most important aspects of any academic research and poor or lack of referencing will not only diminish your marks, but such practices may also be perceived as plagiarism by your university and disciplinary actions may follow that may even result in expulsion from the course.

Difference between References and Bibliography

It is very important to be able to distinguish between References and Bibliography. Under References you list resources that you referred to within the body of the work that also include quotations.  For example,

It has been noted that “time and the management of time is an important issue, and the supply of time management products – books, articles, CDs, workshops, etc. – reflects the huge demand for these products” (Walsh, 2007, p.3).

Interchangeability of identical parts and a high level of straightforwardness of attaching these parts through the assembly line can be considered as revolutionary components of Fordism for the first part of the 20 th century (Nolan, 2008).

Under Bibliography, on the other hand, you need to list resources that you have read during the research process in order to widen your knowledge about the research area , but specific piece of information from these resources have not been used in your research in the direct manner. You do not need to refer to Bibliography within the body of the text.

There are various methods of referencing such as Harvard, APA and Vancouver referencing systems. You should check with your dissertation handbook for the exact type of referencing required and follow this requirement thoroughly.

John Dudovskiy

Reference Sources: What They Are and How to Use Them: Home

Subject index, search hiebert library.

• What is a Reference Source?
• What are Reference Sources good for?
• Is it acceptable to cite Reference Sources in a Research Paper?
• Where can Reference Sources be found?

"Reference sources" are used to locate general factual information on a particular topic.They usually are read selectively for specific pieces of information, rather than from beginning to end. Examples of reference sources include:

• Encyclopedias
• Dictionaries
• Bibliographies

We realize that the term "reference sources" used this way may be a bit confusing, since your professors might also talk about "references" as a way of describing  anything  that you might cite in a research paper. Always be sure to ask your professor if you aren't clear on how they're using the term.

What are reference sources good for?

Reference sources often provide an excellent starting point for a research project. They are a good place to find general background and introductory information, specialized terminology, and lists of references for further research. Think of them as a way to find your bearings in a particular subject area before digging into more detailed scholarly sources.

Is it acceptable to cite reference sources in a research paper?

You may have heard professors tell you not to use encyclopedias when writing a research paper. What they probably meant were general reference sources like the  Encyclopaedia Britannica  or  Wikipedia . While even general sources like these can contain useful information, they are not specialized enough to use for academic work. There are, however, other highly specialized reference sources (some of them even called "encyclopedias" or "dictionaries") that might be perfectly appropriate for use in a research project. Such specialized sources are written by experts in their fields and sometimes include quite detailed scholarly treatment of their topics. These can provide an excellent starting point for your research. Always check with your professor to find out whether it's acceptable to use such specialized reference sources as citations in a research paper.

Where can reference sources be found in Hiebert Library?

We've prepared lists of our most useful reference sources, arranged by subject area. Select the appropriate subject area from the list below to see the list for that topic.

The lists are divided by online and print resources. Online reference sources can be accessed simply by clicking on them. The library's print reference section is located near the main circulation desk (just to the right of the desk as you walk through the front gate). Items in the reference section cannot be checked out. They are intended for use in the library only.

• Anabaptist/Mennonite studies                                                 
• Biblical studies
• Biology & environmental science
• Business administration & Economics
• Christian ministries
• Church history
• Communication
• Criminal justice & criminology
• Mathematics
• Peacemaking & conflict studies
• Physical sciences
• Political science
• Pre-health, nursing, and pharmacy
• Psychology & Marriage/family therapy
• Social work
• Software engineering
• Spanish language & culture
• Last Updated: Mar 9, 2023 9:48 AM
• URL: https://fresno.libguides.com/reference

• Directories
• What are citations and why should I use them?
• When should I use a citation?
• Why are there so many citation styles?
• Which citation style should I use?
• Chicago Notes Style
• Chicago Author-Date Style
• AMA Style (medicine)
• Bluebook (law)
• Additional Citation Styles
• Built-in Citation Tools
• Quick Citation Generators
• Citation Management Software
• Start Your Research
• Research Guides
• University of Washington Libraries
• Library Guides
• UW Libraries
• Citing Sources

Citing Sources: What are citations and why should I use them?

What is a citation.

Citations are a way of giving credit when certain material in your work came from another source. It also gives your readers the information necessary to find that source again-- it provides an important roadmap to your research process. Whenever you use sources such as books, journals or websites in your research, you must give credit to the original author by citing the source. 

Why do researchers cite?

Scholarship is a conversation  and scholars use citations not only to  give credit  to original creators and thinkers, but also to  add strength and authority  to their own work.  By citing their sources, scholars are  placing their work in a specific context  to show where they “fit” within the larger conversation.  Citations are also a great way to  leave a trail  intended to help others who may want to explore the conversation or use the sources in their own work.

In short, citations

(1) give credit

(2) add strength and authority to your work

(3) place your work in a specific context

(4) leave a trail for other scholars

"Good citations should reveal your sources, not conceal them. They should honeslty reflect the research you conducted." (Lipson 4)

Lipson, Charles. "Why Cite?"  Cite Right: A Quick Guide to Citation Styles--MLA, APA, Chicago, the Sciences, Professions, and More . Chicago: U of Chicago, 2006. Print.

What does a citation look like?

Different subject disciplines call for citation information to be written in very specific order, capitalization, and punctuation. There are therefore many different style formats. Three popular citation formats are MLA Style (for humanities articles) and APA or Chicago (for social sciences articles).

MLA style (print journal article):  

Whisenant, Warren A. "How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX." Sex Roles Vol. 49.3 (2003): 179-182.

APA style (print journal article):

Whisenant, W. A. (2003) How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX. Sex Roles , 49 (3), 179-182.

Chicago style (print journal article):

Whisenant, Warren A. "How Women Have Fared as Interscholastic Athletic Administrators Since the Passage of Title IX." Sex Roles 49, no. 3 (2003): 179-182.

No matter which style you use, all citations require the same basic information:

• Author or Creator
• Container (e.g., Journal or magazine, website, edited book)
• Date of creation or publication
• Publisher 

You are most likely to have easy access to all of your citation information when you find it in the first place. Take note of this information up front, and it will be much easier to cite it effectively later.

• << Previous: Basics of Citing
• Next: When should I use a citation? >>
• Last Updated: May 1, 2024 12:48 PM
• URL: https://guides.lib.uw.edu/research/citations

Educational resources and simple solutions for your research journey

How to Write References in Research Papers: Navigating the Maze (Part 2)

You truly are navigating a maze when it comes to ci tations and the question of how to write references in research pa per s. In part 1 of this article, we touched upon citations, which are pointers embedded in the text of a research paper, to sources of information or to other research relevant to that being described in the research paper. Those pointers lead to references in research papers , which typically appear at the end of the text. Whereas citations merely point us to sources in research papers, references describe those sources in sufficient detail for readers (1) to know the title of each source, who is responsible for its content, and when it was published; (2) to look up those sources; and (3) to obtain the documents in question if required.  

Table of Contents

Types of references in research papers

In writing a research paper, a researcher draws upon many sources of information, knowledge, opinions, and so on. One of the the most common type s of reference s  in research papers is other research papers published in journals; other common sources include technical reports, handbooks, presentations at conferences, and books. Increasingly, the sources in research papers are digital and include web pages, databases, blog posts, and even tweets and emails.  

Not all sources are considered equally credible , and some may not be accessible to all because they are behind paywalls or available only to members of a network (company intranets, for example) or because they are personal exchanges.  

How to write references in research papers

If the citations follow the Harvard system, references in a research paper s are sorted alphabetically by the last name of the first author; if the citations follow the Vancouver system, the references are arranged by numbers: the reference corresponding to the first numbered citation is numbered 1, and so on. If a source is cited again, its allocated number does not change.  

Some additional conventions govern the alphabetic sorting of references in research papers . For instance, when authors have some papers in which they are the only author and others in which they have one or more co-authors or when the same author or authors have papers published in different years or even within the same year.  

Some publishers make even greater demands of references in research papers : authors are expected to sort the list of references alphabetically, as in the Harvard system; then number the sorted list serially; and then renumber all the citations within the text so that each corresponds to its new number!  

How to add references in a research paper: Key elements

For a source of information to be described accurately, some minimum details are required. Here’s one example of w rit ing references in research paper s – ‘ Nature 171 : 737’ is a code that, if you know how to decipher it, tells you that it means an article published in Nature (a weekly journal published from the UK) that begins on page 737 of volume 171 of that journal. However, it does not tell you what the article was about, who wrote it, when it was published, or even how long it is. A complete reference in research paper s (Fig. 1), however, tells you that the title of the article was ‘Molecular structure of nucleic acids: a structure for deoxyribose nucleic acid’, that it was written by J. D. Watson and F. H. C. Crick, that it was published in 1953, and that it ran to no more than two pages.  

When thinking about how to write research references , remember that the elements that make up a reference to an article published in a journal are different from those that make up a reference to a book (edition if not the first, the publisher, and the place of publication, although the last is no longer considered essential in today’s globalized publishing). The elements that make up a reference to a technical report include the name of the organization issuing that report and the report number, if any, and that to a conference presentation gives the title of the conference, the date(s) on which it was held and the place, the name of the organizer(s) of the conference, and so on.  

Note that journals or publishers differ in the elements they expect authors to include when they state how to put references in research papers ; for example, some journals give only minimal information and exclude the titles of articles and some use the ‘elided’ form of page numbers (737–38 instead of 737–738, for example).  

Then there is the question of abbreviated names of journals: some publishers abbreviate journal titles and some don’t ( Annals of Applied Biology or Ann. Appl. Biol.). And those who do, often disagree on the correct abbreviation—and on whether the abbreviations should end in dots (whether the word ‘Journal’ should be given as J. or J or Jnl or Jnl.).  

Sequence of the elements that make up reference s in research papers  

Publishers and journals also differ in the order or sequence in which they present the elements or components of reference s in research papers : usually, British and European publishers put the year of publication after the names of authors whereas US publishers move the year closer to the volume number of the journal.  

Even within an element, the sequence of references in research paper s can have subtle differences. In Harvard system, because the last name of the first author is using for sorting, the name is ‘inverted’, that is the last name is given first, followed by initials (Watson J D instead of J D Watson). However, some journals invert the names of all the authors whereas some invert the name of only the first author. In Vancouver system, the names are seldom inverted because the sequence is not alphabetical.  

Punctuation to separate the elements that make up references in research papers

The many exasperating details that go into formatting references include punctuation marks (or their absence). In giving the initials of authors, some journals use dots, some journals use space, some use both, and some use neither (Watson J.D. or Watson J D or Watson J. D. or Watson JD). Some use a comma between the last name and the initials whereas some reserve the comma only to separate one name from the next (Watson, J D and Crick, F H C or Watson J D, Crick F H C). Some use ‘and’ some don’t, even when there are only two authors, and some use ‘&’ instead which makes it even more confusing for those struggling with how to write references in a research paper.

When the place of publication was a required element in the case of books, some publishers used the colon and some used the comma (and also changed the order, as in New York: Harper & Row or Harper & Row, New York). Some publishers end each reference with a full stop (period) and some don’t.

Typography of references in research papers  

As if the variations mentioned above were not enough, when figuring out how to add references in a research paper , you also have to contend with the differences in typography as well: journal titles in italics or in normal type, volume numbers in bold or in normal type, hyphens or en dashes between page numbers (737-738 or 737–738), and so on.  

All is not lost, however, if you despair of ever getting the references in a research paper right. For example, some publishers now insist on correct formatting only after a paper has been accepted for publication. Also, ICMJE, the International Committee of Medical Journal Editors, recommends a set of uniform requirements for manuscripts (the requirements include the formatting of citations and references), and hundreds of medical journals ( www.icmje.org/journals-following-the-icmje-recommendations/ ) have agreed that as long as authors adhere to those recommendations on how to mention references for research papers , any changes to the formatting any journal wants to make will be made by the journal in question.  

Lastly, several software packages help authors to automate this mundane task of consistent formatting of references in research paper s—but that is another article and another day.  

The details involved in using citations and references correctly can be overwhelming for some of us. While this article covers the key tips to help you understand how to give reference s in research paper s , be sure to check out article 1 of this two-part series for more on what, when and how to cite in a research paper. One way to check whether these are handled correctly in your manuscript is to use Researcher.Life’s AI powered manuscript optimizer , which can flag any discrepancies, departures from standard style, and mismatches between citations and references in research paper s.  

R Discovery is a literature search and research reading platform that accelerates your research discovery journey by keeping you updated on the latest, most relevant scholarly content. With 250M+ research articles sourced from trusted aggregators like CrossRef, Unpaywall, PubMed, PubMed Central, Open Alex and top publishing houses like Springer Nature, JAMA, IOP, Taylor & Francis, NEJM, BMJ, Karger, SAGE, Emerald Publishing and more, R Discovery puts a world of research at your fingertips.  

Try R Discovery Prime FREE for 1 week or upgrade at just US$72 a year to access premium features that let you listen to research on the go, read in your language, collaborate with peers, auto sync with reference managers, and much more. Choose a simpler, smarter way to find and read research – Download the app and start your free 7-day trial today !  

Related Posts

What are Experimental Groups in Research

What is IMRaD Format in Research?

We apologize for any inconvenience as we update our site to a new look.

• Walden University
• Faculty Portal

Reference List: Common Reference List Examples

Article (with doi).

Alvarez, E., & Tippins, S. (2019). Socialization agents that Puerto Rican college students use to make financial decisions. Journal of Social Change , 11 (1), 75–85. https://doi.org/10.5590/JOSC.2019.11.1.07

Laplante, J. P., & Nolin, C. (2014). Consultas and socially responsible investing in Guatemala: A case study examining Maya perspectives on the Indigenous right to free, prior, and informed consent. Society & Natural Resources , 27 , 231–248. https://doi.org/10.1080/08941920.2013.861554

Use the DOI number for the source whenever one is available. DOI stands for "digital object identifier," a number specific to the article that can help others locate the source. In APA 7, format the DOI as a web address. Active hyperlinks for DOIs and URLs should be used for documents meant for screen reading. Present these hyperlinks in blue and underlined text (the default formatting in Microsoft Word), although plain black text is also acceptable. Be consistent in your formatting choice for DOIs and URLs throughout your reference list. Also see our Quick Answer FAQ, "Can I use the DOI format provided by library databases?"

Jerrentrup, A., Mueller, T., Glowalla, U., Herder, M., Henrichs, N., Neubauer, A., & Schaefer, J. R. (2018). Teaching medicine with the help of “Dr. House.” PLoS ONE , 13 (3), Article e0193972. https://doi.org/10.1371/journal.pone.0193972

For journal articles that are assigned article numbers rather than page ranges, include the article number in place of the page range.

For more on citing electronic resources, see  Electronic Sources References .

Article (Without DOI)

Found in a common academic research database or in print.

Casler , T. (2020). Improving the graduate nursing experience through support on a social media platform. MEDSURG Nursing , 29 (2), 83–87.

If an article does not have a DOI and you retrieved it from a common academic research database through the university library, there is no need to include any additional electronic retrieval information. The reference list entry looks like the entry for a print copy of the article. (This format differs from APA 6 guidelines that recommended including the URL of a journal's homepage when the DOI was not available.) Note that APA 7 has additional guidance on reference list entries for articles found only in specific databases or archives such as Cochrane Database of Systematic Reviews, UpToDate, ProQuest Dissertations and Theses Global, and university archives. See APA 7, Section 9.30 for more information.

Found on an Open Access Website

Eaton, T. V., & Akers, M. D. (2007). Whistleblowing and good governance. CPA Journal , 77 (6), 66–71. http://archives.cpajournal.com/2007/607/essentials/p58.htm

Provide the direct web address/URL to a journal article found on the open web, often on an open access journal's website. In APA 7, active hyperlinks for DOIs and URLs should be used for documents meant for screen reading. Present these hyperlinks in blue and underlined text (the default formatting in Microsoft Word), although plain black text is also acceptable. Be consistent in your formatting choice for DOIs and URLs throughout your reference list.

Weinstein, J. A. (2010).  Social change  (3rd ed.). Rowman & Littlefield.

If the book has an edition number, include it in parentheses after the title of the book. If the book does not list any edition information, do not include an edition number. The edition number is not italicized.

American Nurses Association. (2015). Nursing: Scope and standards of practice (3rd ed.).

If the author and publisher are the same, only include the author in its regular place and omit the publisher.

Lencioni, P. (2012). The advantage: Why organizational health trumps everything else in business . Jossey-Bass. https://amzn.to/343XPSJ

As a change from APA 6 to APA 7, it is no longer necessary to include the ebook format in the title. However, if you listened to an audiobook and the content differs from the text version (e.g., abridged content) or your discussion highlights elements of the audiobook (e.g., narrator's performance), then note that it is an audiobook in the title element in brackets. For ebooks and online audiobooks, also include the DOI number (if available) or nondatabase URL but leave out the electronic retrieval element if the ebook was found in a common academic research database, as with journal articles. APA 7 allows for the shortening of long DOIs and URLs, as shown in this example. See APA 7, Section 9.36 for more information.

Chapter in an Edited Book

Poe, M. (2017). Reframing race in teaching writing across the curriculum. In F. Condon & V. A. Young (Eds.), Performing antiracist pedagogy in rhetoric, writing, and communication (pp. 87–105). University Press of Colorado.

Include the page numbers of the chapter in parentheses after the book title.

Christensen, L. (2001). For my people: Celebrating community through poetry. In B. Bigelow, B. Harvey, S. Karp, & L. Miller (Eds.), Rethinking our classrooms: Teaching for equity and justice (Vol. 2, pp. 16–17). Rethinking Schools.

Also include the volume number or edition number in the parenthetical information after the book title when relevant.

Freud, S. (1961). The ego and the id. In J. Strachey (Ed.),  The standard edition of the complete psychological works of Sigmund Freud  (Vol. 19, pp. 3-66). Hogarth Press. (Original work published 1923)

When a text has been republished as part of an anthology collection, after the author’s name include the date of the version that was read. At the end of the entry, place the date of the original publication inside parenthesis along with the note “original work published.” For in-text citations of republished work, use both dates in the parenthetical citation, original date first with a slash separating the years, as in this example: Freud (1923/1961). For more information on reprinted or republished works, see APA 7, Sections 9.40-9.41.

Classroom Resources

Citing classroom resources.

If you need to cite content found in your online classroom, use the author (if there is one listed), the year of publication (if available), the title of the document, and the main URL of Walden classrooms. For example, you are citing study notes titled "Health Effects of Exposure to Forest Fires," but you do not know the author's name, your reference entry will look like this:

Health effects of exposure to forest fires [Lecture notes]. (2005). Walden University Canvas. https://waldenu.instructure.com

If you do know the author of the document, your reference will look like this:

Smith, A. (2005). Health effects of exposure to forest fires [PowerPoint slides]. Walden University Canvas. https://waldenu.instructure.com  

A few notes on citing course materials:

• [Lecture notes]
• [Course handout]
• [Study notes]
• It can be difficult to determine authorship of classroom documents. If an author is listed on the document, use that. If the resource is clearly a product of Walden (such as the course-based videos), use Walden University as the author. If you are unsure or if no author is indicated, place the title in the author spot, as above.
• If you cannot determine a date of publication, you can use n.d. (for "no date") in place of the year.

Note:  The web location for Walden course materials is not directly retrievable without a password, and therefore, following APA guidelines, use the main URL for the class sites: https://class.waldenu.edu.

Citing Tempo Classroom Resources

Clear author: 

Smith, A. (2005). Health effects of exposure to forest fires [PowerPoint slides]. Walden University Brightspace. https://mytempo.waldenu.edu

Unclear author:

Health effects of exposure to forest fires [Lecture notes]. (2005). Walden University Brightspace. https://mytempo.waldenu.edu

Conference Sessions and Presentations

Feinman, Y. (2018, July 27). Alternative to proctoring in introductory statistics community college courses [Poster presentation]. Walden University Research Symposium, Minneapolis, MN, United States. https://scholarworks.waldenu.edu/symposium2018/23/

Torgerson, K., Parrill, J., & Haas, A. (2019, April 5-9). Tutoring strategies for online students [Conference session]. The Higher Learning Commission Annual Conference, Chicago, IL, United States. http://onlinewritingcenters.org/scholarship/torgerson-parrill-haas-2019/

Dictionary Entry

Merriam-Webster. (n.d.). Leadership. In Merriam-Webster.com dictionary . Retrieved May 28, 2020, from https://www.merriam-webster.com/dictionary/leadership

When constructing a reference for an entry in a dictionary or other reference work that has no byline (i.e., no named individual authors), use the name of the group—the institution, company, or organization—as author (e.g., Merriam Webster, American Psychological Association, etc.). The name of the entry goes in the title position, followed by "In" and the italicized name of the reference work (e.g., Merriam-Webster.com dictionary , APA dictionary of psychology ). In this instance, APA 7 recommends including a retrieval date as well for this online source since the contents of the page change over time. End the reference entry with the specific URL for the defined word.

Discussion Board Post

Osborne, C. S. (2010, June 29). Re: Environmental responsibility [Discussion post]. Walden University Canvas.  https://waldenu.instructure.com  

Dissertations or Theses

Retrieved From a Database

Nalumango, K. (2019). Perceptions about the asylum-seeking process in the United States after 9/11 (Publication No. 13879844) [Doctoral dissertation, Walden University]. ProQuest Dissertations and Theses.

Retrieved From an Institutional or Personal Website

Evener. J. (2018). Organizational learning in libraries at for-profit colleges and universities [Doctoral dissertation, Walden University]. ScholarWorks. https://scholarworks.waldenu.edu/cgi/viewcontent.cgi?article=6606&context=dissertations

Unpublished Dissertation or Thesis

Kirwan, J. G. (2005). An experimental study of the effects of small-group, face-to-face facilitated dialogues on the development of self-actualization levels: A movement towards fully functional persons [Unpublished doctoral dissertation]. Saybrook Graduate School and Research Center.

For further examples and information, see APA 7, Section 10.6.

Legal Material

For legal references, APA follows the recommendations of The Bluebook: A Uniform System of Citation , so if you have any questions beyond the examples provided in APA, seek out that resource as well.

Court Decisions

Reference format:

Name v. Name, Volume Reporter Page (Court Date). URL

Sample reference entry:

Brown v. Board of Education, 347 U.S. 483 (1954). https://www.oyez.org/cases/1940-1955/347us483

Sample citation:

In Brown v. Board of Education (1954), the Supreme Court ruled racial segregation in schools unconstitutional.

Note: Italicize the case name when it appears in the text of your paper.

Name of Act, Title Source § Section Number (Year). URL

Sample reference entry for a federal statute:

Individuals With Disabilities Education Act, 20 U.S.C. § 1400 et seq. (2004). https://www.congress.gov/108/plaws/publ446/PLAW-108publ446.pdf

Sample reference entry for a state statute:

Minnesota Nurse Practice Act, Minn. Stat. §§ 148.171 et seq. (2019). https://www.revisor.mn.gov/statutes/cite/148.171

Sample citation: Minnesota nurses must maintain current registration in order to practice (Minnesota Nurse Practice Act, 2010).

Note: The § symbol stands for "section." Use §§ for sections (plural). To find this symbol in Microsoft Word, go to "Insert" and click on Symbol." Look in the Latin 1-Supplement subset. Note: U.S.C. stands for "United States Code." Note: The Latin abbreviation " et seq. " means "and what follows" and is used when the act includes the cited section and ones that follow. Note: List the chapter first followed by the section or range of sections.

Unenacted Bills and Resolutions

(Those that did not pass and become law)

Title [if there is one], bill or resolution number, xxx Cong. (year). URL

Sample reference entry for Senate bill:

Anti-Phishing Act, S. 472, 109th Cong. (2005). https://www.congress.gov/bill/109th-congress/senate-bill/472

Sample reference entry for House of Representatives resolution:

Anti-Phishing Act, H.R. 1099, 109th Cong. (2005). https://www.congress.gov/bill/109th-congress/house-bill/1099

The Anti-Phishing Act (2005) proposed up to 5 years prison time for people running Internet scams.

These are the three legal areas you may be most apt to cite in your scholarly work. For more examples and explanation, see APA 7, Chapter 11.

Magazine Article

Clay, R. (2008, June). Science vs. ideology: Psychologists fight back about the misuse of research. Monitor on Psychology , 39 (6). https://www.apa.org/monitor/2008/06/ideology

Note that for citations, include only the year: Clay (2008). For magazine articles retrieved from a common academic research database, leave out the URL. For magazine articles from an online news website that is not an online version of a print magazine, follow the format for a webpage reference list entry.

Newspaper Article (Retrieved Online)

Baker, A. (2014, May 7). Connecticut students show gains in national tests. New York Times . http://www.nytimes.com/2014/05/08/nyregion/national-assessment-of-educational-progress-results-in-Connecticut-and-New-Jersey.html

Include the full date in the format Year, Month Day. Do not include a retrieval date for periodical sources found on websites. Note that for citations, include only the year: Baker (2014). For newspaper articles retrieved from a common academic research database, leave out the URL. For newspaper articles from an online news website that is not an online version of a print newspaper, follow the format for a webpage reference list entry.

OASIS Resources

Oasis webpage.

OASIS. (n.d.). Common reference list examples . Walden University. https://academicguides.waldenu.edu/writingcenter/apa/references/examples

For all OASIS content, list OASIS as the author. Because OASIS webpages do not include publication dates, use “n.d.” for the year.

Interactive Guide

OASIS. (n.d.). Embrace iterative research and writing [Interactive guide]. Walden University. https://academics.waldenu.edu/oasis/iterative-research-writing-web

For OASIS multimedia resources, such as interactive guides, include a description of the resource in brackets after the title.

Online Video/Webcast

Walden University. (2013).  An overview of learning  [Video]. Walden University Canvas.  https://waldenu.instructure.com  

Use this format for online videos such as Walden videos in classrooms. Most of our classroom videos are produced by Walden University, which will be listed as the author in your reference and citation. Note: Some examples of audiovisual materials in the APA manual show the word “Producer” in parentheses after the producer/author area. In consultation with the editors of the APA manual, we have determined that parenthetical is not necessary for the videos in our courses. The manual itself is unclear on the matter, however, so either approach should be accepted. Note that the speaker in the video does not appear in the reference list entry, but you may want to mention that person in your text. For instance, if you are viewing a video where Tobias Ball is the speaker, you might write the following: Tobias Ball stated that APA guidelines ensure a consistent presentation of information in student papers (Walden University, 2013). For more information on citing the speaker in a video, see our page on Common Citation Errors .

Taylor, R. [taylorphd07]. (2014, February 27). Scales of measurement [Video]. YouTube. https://www.youtube.com/watch?v=PDsMUlexaMY

OASIS. (2020, April 15). One-way ANCOVA: Introduction [Video]. YouTube. https://youtu.be/_XnNDQ5CNW8

For videos from streaming sites, use the person or organization who uploaded the video in the author space to ensure retrievability, whether or not that person is the speaker in the video. A username can be provided in square brackets. As a change from APA 6 to APA 7, include the publisher after the title, and do not use "Retrieved from" before the URL. See APA 7, Section 10.12 for more information and examples.

See also reference list entry formats for TED Talks .

Technical and Research Reports

Edwards, C. (2015). Lighting levels for isolated intersections: Leading to safety improvements (Report No. MnDOT 2015-05). Center for Transportation Studies. http://www.cts.umn.edu/Publications/ResearchReports/reportdetail.html?id=2402

Technical and research reports by governmental agencies and other research institutions usually follow a different publication process than scholarly, peer-reviewed journals. However, they present original research and are often useful for research papers. Sometimes, researchers refer to these types of reports as gray literature , and white papers are a type of this literature. See APA 7, Section 10.4 for more information.

Reference list entires for TED Talks follow the usual guidelines for multimedia content found online. There are two common places to find TED talks online, with slightly different reference list entry formats for each.

TED Talk on the TED website

If you find the TED Talk on the TED website, follow the format for an online video on an organizational website:

Owusu-Kesse, K. (2020, June). 5 needs that any COVID-19 response should meet [Video]. TED Conferences. https://www.ted.com/talks/kwame_owusu_kesse_5_needs_that_any_covid_19_response_should_meet

The speaker is the author in the reference list entry if the video is posted on the TED website. For citations, use the speaker's surname.

TED Talk on YouTube

If you find the TED Talk on YouTube or another streaming video website, follow the usual format for streaming video sites:

TED. (2021, February 5). The shadow pandemic of domestic violence during COVID-19 | Kemi DaSilvalbru [Video]. YouTube. https://www.youtube.com/watch?v=PGdID_ICFII

TED is the author in the reference list entry if the video is posted on YouTube since it is the channel on which the video is posted. For citations, use TED as the author.

Walden University Course Catalog

To include the Walden course catalog in your reference list, use this format:

Walden University. (2020). 2019-2020 Walden University catalog . https://catalog.waldenu.edu/index.php

If you cite from a specific portion of the catalog in your paper, indicate the appropriate section and paragraph number in your text:

...which reflects the commitment to social change expressed in Walden University's mission statement (Walden University, 2020, Vision, Mission, and Goals section, para. 2).

And in the reference list:

Walden University. (2020). Vision, mission, and goals. In 2019-2020 Walden University catalog. https://catalog.waldenu.edu/content.php?catoid=172&navoid=59420&hl=vision&returnto=search

Vartan, S. (2018, January 30). Why vacations matter for your health . CNN. https://www.cnn.com/travel/article/why-vacations-matter/index.html

For webpages on the open web, include the author, date, webpage title, organization/site name, and URL. (There is a slight variation for online versions of print newspapers or magazines. For those sources, follow the models in the previous sections of this page.)

American Federation of Teachers. (n.d.). Community schools . http://www.aft.org/issues/schoolreform/commschools/index.cfm

If there is no specified author, then use the organization’s name as the author. In such a case, there is no need to repeat the organization's name after the title.

In APA 7, active hyperlinks for DOIs and URLs should be used for documents meant for screen reading. Present these hyperlinks in blue and underlined text (the default formatting in Microsoft Word), although plain black text is also acceptable. Be consistent in your formatting choice for DOIs and URLs throughout your reference list.

Related Resources

Knowledge Check: Common Reference List Examples

Didn't find what you need? Email us at [email protected] .

• Previous Page: Reference List
• Next Page: Common Military Reference List Examples
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Roles of references in research papers – a broader assessment

In writing research papers and their evaluation, references or citations play a central role. Knowing the role(s) in detail will help you to cite accurately and responsibly.

Know the definition of referencing here: What is referencing in academic writing? – a different perspective.

Commonly understood roles of references

According to several top search results, when “role of references in research” is searched in Google, referencing have the following important roles:

• Referencing allows you to acknowledge or give credit to the writers and researchers from whom you have borrowed words and ideas, thereby avoiding plagiarism.
• Referencing is a way to provide evidence to support the assertions and claims in your own assignments.
• References allow readers to trace the sources of information you have used.

Subtle but critical roles of referencing

However, these commonly understood roles don’t clearly reflect the deeper impact (as listed below) references make on the research paper itself and science overall.

References (summarised from here ):

• demonstrate the foundation of the study.
• support the novelty and value of the study.
• link one study to others creating a web of knowledge that carries meaning.
• allows researchers to identify work as relevant in general and relevant to them.
• create values that are internal to science (e.g., relevance, credit).
• create values that are external to science (e.g., provide avenues to determine accountability and researchers or funding performance).

A wrong perception of referencing

According to many search results, the most common role of referencing is to acknowledge or give credit to other researchers. This creates a wrong perception among authors that they need to cite references neutrally without constructive scientific evaluation (praise or criticism). No wonder an editorial published in Nature genetics reported that neutral, flavourless or unexamined citations frequently occur in research articles and supporting or contradictory publications are rarely cited. This is an increasing problem for the integrity of scientific communication.

This situation can be improved by acknowledging that referencing is not a neutral act but a political act (without bad politics). In fact, acknowledging the political nature of referencing is a vital rule for responsible referencing. For more on responsible referencing, see here .

But why referencing is a political act?

By assessing many research articles, you select the articles you want to cite and discard others. To pose the research questions and support the claims and novelty of your research work, rather than neutral representations of the references, you accurately represent, inflate, or deflate the contributions of other researchers’ works. See the following examples.

Example 1: Using references to establish the research question.

Many of us have encountered instances where the support of an assertion by the cited reference proves to be ambiguous, non-existent, or even contradictory (often we only notice this when our own work has been mis-cited!). A related practice is the citing of “empty” references (Harzing 2002), also known as “lazy author syndrome” (Gavras 2002), where the citation actually attributes a finding or an opinion to a secondary source such as a review paper, editorial, etc. But how pervasive is citation malpractice and how can it be controlled? SOURCE: Tood, Yeo, Li, and Ladle, Oikos 116, 1599 (2007)

Example 2: Using references to support claim and novelty of the research paper.

We found that the original assertion was “clearly supported” by the citation in 76.1% of the cases; the support was “ambiguous” in 11.1% of the cases; and the citation did “not support” the original statement in 7.2% of the cases. The remaining 5.6% of the cases were classified as “empty”. How do these mis-citation rates compare with other disciplines? A number of biomedical studies have used an approach similar to our own, although they applied the analogous categories “major error” and “minor error” rather than “no support” and “ambiguous”. Combined error rates found by Fenton et al. 2000 (17%) and Lukic et al. 2004 (19%) are comparable to our result of 18.3% for “no support” plus “ambiguous”, though other results for medical journals range from 12.3% (Gosling et al. 2004) to 35.2% (Goldberg et al. 1993). To our knowledge, empty citation data are absent for all the sciences. SOURCE: Tood, Yeo, Li, and Ladle, Oikos 116, 1599 (2007)

Appreciation of the broader roles of referencing and its political nature will enable researchers and scientists to produce research works by making citations accurately.

Does your reference manager support you in demonstrating how accurately you represented other researchers’ contributions in your research work? For the first time, with nXr.iCite you can search quotes/images; make citations based on quotes/images and share all the cited materials with your mentors/reviewers for peer-review. This enables them to validate your research paper with the content of the references side-by-side without leaving the paper. https://nxref.com

Scribbr APA Citation Generator

Accurate APA citations, verified by experts, trusted by millions.

Scribbr for Chrome: Your shortcut to APA citations

Cite any page or article with a single click right from your browser. The extension does the hard work for you by automatically grabbing the title, author(s), publication date, and everything else needed to whip up the perfect APA citation.

Add to Chrome. It's free!

Rely on accurate APA citations, verified by experts.

You don’t want points taken off for incorrect citations. That’s why our APA citation experts have invested countless hours perfecting our algorithms. As a result, we’re proud to be recommended by teachers worldwide.

Enjoy the APA Citation Generator with minimal distraction.

Staying focused is already challenging enough. You don’t need video pop-ups and flickering banner ads slowing you down. At Scribbr, we keep distractions to a minimum while also keeping the APA Citation Generator free for everyone.

Citation Generator features you'll love

Search for your source by title, URL, DOI, ISBN, and more to retrieve the relevant information automatically.

APA 6th & 7th edition

Scribbr's Citation Generator supports both APA 6 and APA 7 (as well as MLA and Harvard ). No matter what edition you're using, we’ve got you covered!

Export to Bib(La)TeX

Easily export in BibTeX format and continue working in your favorite LaTeX editor.

Export to Word

Reference list finished? Export to Word with perfect indentation and spacing set up for you.

Sorting, grouping, and filtering

Organize the reference list the way you want: from A to Z, new to old, or grouped by source type.

Save multiple lists

Stay organized by creating a separate reference list for each of your assignments.

Choose between Times New Roman, Arial, Calibri, and more options to match your style.

Industry-standard technology

The Scribbr Citation Generator is built using the same citation software (CSL) as Mendeley and Zotero, but with an added layer for improved accuracy.

Annotations

Create perfectly formatted annotated bibliographies with just a few clicks.

Explanatory tips help you get the details right to ensure accurate citations.

Citation guides

Getting to grips with citation is simple with the help of our highly rated APA citation guides and videos .

Secure backup

Your work is saved automatically after every change and stored securely in your Scribbr account.

• Introduction
• Parenthetical vs. narrative
• Multiple authors

Missing information

• Sources to include

Tools and resources

• Scroll to top

How to create APA citations

APA Style is widely used by students, researchers, and professionals in the social and behavioral sciences. Scribbr’s free citation generator automatically generates accurate references and in-text citations.

This citation guide outlines the most important citation guidelines from the 7th edition APA Publication Manual (2020).

• Cite a webpage
• Cite a book
• Cite a journal article
• Cite a YouTube video

APA in-text citations

APA in-text citations include the author’s last name, publication date, and, if relevant, a locator such as a page number or timestamp. For example, (Smith, 2021, p. 170) . See it as a shorter version of the entry in the reference list .

You should include in-text citations every time you’re quoting or paraphrasing someone else’s ideas or words. In doing so, you give credit to the original author and avoid plagiarism .

Parenthetical vs. narrative citation

The in-text citation can take two forms: parenthetical and narrative. Both types are generated automatically when citing a source with Scribbr’s APA Citation Generator.

• Parenthetical citation: According to new research … (Smith, 2020) .
• Narrative citation: Smith (2020) notes that …

Multiple authors and corporate authors

The in-text citation changes slightly when a source has multiple authors or an organization as an author. Pay attention to punctuation and the use of the ampersand (&) symbol.

When the author, publication date or locator is unknown, take the steps outlined below.

APA Citation Generator

Generate accurate APA citations in seconds

Get started

APA references

APA references generally include information about the author , publication date , title , and source . Depending on the type of source, you may have to include extra information that helps your reader locate the source.

It is not uncommon for certain information to be unknown or missing, especially with sources found online. In these cases, the reference is slightly adjusted.

Formatting the APA reference page

On the first line of the page, write the section label “References” (in bold and centered). On the second line, start listing your references in alphabetical order .

Apply these formatting guidelines to the APA reference page:

• Double spacing (within and between references)
• Hanging indent of ½ inch
• Legible font (e.g. Times New Roman 12 or Arial 11)
• Page number in the top right header

Which sources to include

On the reference page, you only include sources that you have cited in the text (with an in-text citation ). You should not include references to personal communications that your reader can’t access (e.g. emails, phone conversations or private online material).

In addition to the APA Citation Generator, Scribbr provides many more tools and resources that help millions of students and academics every month.

• Citation Generator : Generate flawless citations in APA, MLA , and Harvard style .
• Citation Checker : Upload your paper and have artificial intelligence check your citations for errors and inconsistencies.
• Free plagiarism checker : Detect plagiarism with unparalleled accuracy with Scribbr’s free plagiarism checker.
• AI Proofreader : Upload and improve unlimited documents and earn higher grades on your assignments. Try it for free!
• Paraphrasing tool: Avoid accidental plagiarism and make your text sound better.
• Grammar checker : Eliminate pesky spelling and grammar mistakes.
• Summarizer: Read more in less time. Distill lengthy and complex texts down to their key points.
• AI detector: Find out if your text was written with ChatGPT or any other AI writing tool. ChatGPT 2 & ChatGPT 3 supported.
• Proofreading services : Have a professional editor (or team of editors) improve your writing so you can submit your paper with pride and confidence. Scribbr offers admission essay editing , paper editing , and academic editing .
• Guides and videos : Explore hundreds of articles, bite-sized videos, time-saving templates, and handy checklists that guide you through the process of research, writing, and citation.

• Key Differences

Know the Differences & Comparisons

Difference Between Citation and Reference

While writing an article, one must quote or refer to the original source of the information, fact or idea, from which it has been taken. It not just to supports your own points, but also to prevent plagiarism and denote that a variety of sources are used to write the piece.

And so, the author provides citations, which are correspondingly mentioned under the head reference, indicating the complete details of the resource. In this article, we are going to talk about the differences between citation and reference.

Content: Citation Vs Reference

Comparison chart.

Definition of Citation

In the citation, the author cites or refer to the source in the text to represent that the information is derived from an external source and to mention that source, in brief. Basically, it is an abbreviated reference, which you can find in the main body of the article or assignment, addressing the source of a quote, image, video, paraphrase, chart, table, etc. Due to this very reason, it is also called as “in-text citation”, which includes a set of parenthesis.

Simply put, it is a short notation, within the intellectual work, which points out to a complete notation, at the end of the page, providing full details of the source i.e. all the authors or publications which have been cited, are to be mentioned in the list of references.

Citation is like a credit to the author, editor or publisher, for their work and to help the readers in consulting the same source when they want more information in this regard. While citing the source of information in the document, you indicate the last name of author and year of publication.

Definition of Reference

Reference can be understood as the list of items which you have read and considered in your piece of work. While providing references, the author actually tells its readers about what kind of source he/she has used in the document.

Along with that it also helps the readers to identify the difference between the author’s words, theories and ideas and those of other authors. Further, it helps the reader to refer to the source for further information in that area, whenever required.

You can find references at the end of the document or article (before bibliography), in alphabetical order, by the first or main author’s last name. One should always use a genuine, reliable and authentic source of information, to ensure support, credence and authority, to the information, ideas and arguments stated.

Reference can be given to books, articles from journals, legal documents, webpage, blogs, official report of government departments and agencies, interview transcripts, conference papers, newspaper articles, films, television, video, etc.

Key Differences Between Citation and Reference

The difference between citation and reference can be drawn clearly on the following grounds:

• Citation can be understood as a formal abbreviated reference, in the main part of your text, which clearly and uniquely identifies the author and publication year, from which the details are derived. On the other hand, a reference list is nothing but a list of resources which you have used particularly for writing your article or assignment and also while performing the research.
• With the help of citation, you inform your readers, about the source, from where the information is extracted. As against, in the case of reference, the reader gets to know about the complete source of information.
• Citations are mainly used to show the source of the material taken. Conversely, references are mainly used to support or critique an argument or point.
• While reference is found as endnote or at the end of the page, the citation is the bracketed piece of information, which informs the reader about the source of material.
• When it comes to information, the reference provides more information than citation. A reference provides all the details of the source, such as author’s name, the title of the book, date of publication and page number, whereas in citation provides a few details only such as year of publication and last name of the author.

Basically, reference and citation are provided to give the user or reader, the source of facts, images, statistics, charts, tables and diagrams, which are a part of the article or assignment. It may also be used when you are talking about a theory, method or model, discovered by or linked to a specific person or writer.

With the help of these two, the author shows acknowledgement to those scholars, whose work or ideas has been used in the article or assignment.

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

DOIs and URLs

This page addresses when to include digital object identifiers (DOIs) and uniform resource locators (URLs) in APA Style references. Also check out the related topic of when to include database information in references .

The DOI or URL is the final component of a reference list entry. Because so much scholarship is available and/or retrieved online, most reference list entries end with either a DOI or a URL.

• A DOI is a unique alphanumeric string that identifies content and provides a persistent link to its location on the internet. DOIs can be found in database records and the reference lists of published works.
• A URL specifies the location of digital information on the internet and can be found in the address bar of your internet browser. URLs in references should link directly to the cited work when possible.

When to include DOIs and URLs

Follow these guidelines for including DOIs and URLs in references:

• Include a DOI for all works that have a DOI, regardless of whether you used the online version or the print version.
• If a print work does not have a DOI, do not include any DOI or URL in the reference.
• If an online work has both a DOI and a URL, include only the DOI.
• For works without DOIs from websites (not including academic research databases), provide a URL in the reference (as long as the URL will work for readers).
• For works without DOIs from most academic research databases , do not include a URL or database information in the reference because these works are widely available. The reference should be the same as the reference for a print version of the work.
• For works from databases that publish original, proprietary material available only in that database (such as the UpToDate database) or for works of limited circulation in databases (such as monographs in the ERIC database), include the name of the database or archive and the URL of the work. If the URL requires a login or is session-specific (meaning it will not resolve for readers), provide the URL of the database or archive home page or login page instead of the URL for the work. See the page on including database information in references for more information.
• If the URL is no longer working or no longer provides readers access to the content you intend to cite, follow the guidance for works with no source .
• Other alphanumeric identifiers such as the International Standard Book Number (ISBN) and the International Standard Serial Number (ISSN) are not included in APA Style references.

DOIs and URLs are covered in the seventh edition APA Style manuals in the Publication Manual Sections 9.34 to 9.36 and the Concise Guide Sections 9.34 to 9.36

Related handouts

• Creating an APA Style Reference List (PDF, 179KB)
• Reference Quick Guide (PDF, 225KB)

Format of DOIs and URLs

Follow these guidelines to format DOIs and URLs:

• Present both DOIs and URLs as hyperlinks (i.e., beginning with “http:” or “https:”).
• Because a hyperlink leads readers directly to the content, it is not necessary to include the words “Retrieved from” or “Accessed from” before a DOI or URL.
• It is acceptable to use either the default display settings for hyperlinks in your word-processing program (e.g., usually blue font, underlined) or plain text that is not underlined.
• Leave links live if the work is to be published or read online.
• Follow the current recommendations of the International DOI Foundation to format DOIs in the reference list, which as of this publication is as follows:

https://doi.org/xxxxx

• The string “https://doi.org/” is a way of presenting a DOI as a link, and “xxxxx” refers to the DOI number.
• The preferred format of the DOI has changed over time. Although older works use previous formats (e.g., “http:/dx.doi.org/” or “doi:” or “DOI:” before the DOI number), in your reference list, standardize DOIs into the current preferred format for all entries. For example, use https://doi.org/10.1037/a0040251 in your reference even though that article, published in 2016, presented the number in an older format.
• Copy and paste the DOI or URL from your web browser directly into your reference list to avoid transcription errors. Do not change the capitalization or punctuation of the DOI or URL. Do not add line breaks manually to the hyperlink; it is acceptable if your word-processing program automatically adds a break or moves the hyperlink to its own line.
• Do not add a period after the DOI or URL because this may interfere with link functionality.

DOI and URL shorteners

When a DOI or URL is long or complex, you may use shortDOIs or shortened URLs if desired.

• Use the shortDOI service provided by the International DOI Foundation to create shortDOIs. A work can have only one DOI and only one shortDOI; the shortDOI service will either produce a new shortDOI for a work that has never had one or retrieve an existing shortDOI.
• Some websites provide their own branded shortened URLs, and independent URL shortening services are available as well. Any shortened URL is acceptable in a reference as long as you check the link to ensure that it takes you to the correct location.

• Election 2024
• Entertainment
• Newsletters
• Photography
• AP Buyline Personal Finance
• AP Buyline Shopping
• Press Releases
• Israel-Hamas War
• Russia-Ukraine War
• Global elections
• Asia Pacific
• Latin America
• Middle East
• Delegate Tracker
• AP & Elections
• 2024 Paris Olympic Games
• Auto Racing
• Movie reviews
• Book reviews
• Financial Markets
• Business Highlights
• Financial wellness
• Artificial Intelligence
• Social Media

Stonehenge’s ‘altar stone’ originally came from Scotland and not Wales, new research shows

The mysterious Altar Stone at Stonehenge might have been transported more than 700km from north east Scotland. Scientists have revealed the fascinating study about the stone circle but the revelation brings as many questions as it does answers.

FILE - The world heritage site of Stonehenge is seen in Wiltshire, England on Dec. 17, 2013. (AP Photo/Alastair Grant, File)

• Copy Link copied

In this photo provided by researchers in August 2024, Stonehenge’s Altar Stone lies underneath two Sarsen stones in Wiltshire, England. (Nick Pearce/Aberystwyth University via AP)

WASHINGTON (AP) — The ancient ritual meaning of Stonehenge is still a mystery, but researchers are one step closer to understanding how the famous stone circle was created.

The unique stone lying flat at the center of the monument was brought to the site in southern England from near the tip of northeast Scotland, researchers reported Wednesday in the journal Nature. It’s not clear whether the 16-foot (5-meter) stone was carried by boat or across land — a journey of more than 460 miles (740 kilometers).

“It’s a surprise that it’s come from so far away,” said University of Exeter archaeologist Susan Greaney, who was not involved in the study.

For more than a hundred years, scientists believed that Stonehenge’s central sandstone slab — long called the “altar stone” — came from much closer Wales. But a study last year by some of the same researchers showed that the stone didn’t match the geology of Wales’ sandstone formations. The actual source of the stone remained unknown until now.

AP AUDIO: Stonehenge’s ‘altar stone’ originally came from Scotland and not Wales, new research shows

AP correspondent Karen Chammas reports on a new study into the origins of the Stone Henge altar stone.

For the study, the team was not permitted to chip away rocks at the site, but instead analyzed minerals in bits of rock that had been collected in previous digs, some dating back to the 1840s. They found a match in the sandstone formations of Orcadian Basin in northeast Scotland, a region that includes parts of the tip of the Scottish peninsula as well as the Orkney Islands.

“That geological ‘fingerprint’ isn’t repeated in any other area of sediment in the U.K.,” said Aberystwyth University geologist Nick Pearce, a study co-author.

Greaney said the difficult logistics of moving the stone such a long distance show a high level of coordination and cultural connection between these two regions of ancient Britain.

Stonehenge was constructed around 5,000 years ago, with stones forming different circles brought to the site at different times. The placement of stones allows for the sun to rise through a stone “window” during summer solstice. The ancient purpose of the altar stone — which lies flat at the heart of Stonehenge, now beneath other rocks — remains a mystery.

“Stonehenge isn’t a settlement site, but a place of ceremony or ritual,” said Heather Sebire, senior curator at English Heritage, who was not involved in the study. She said that past archaeological excavations had not uncovered evidence of feasting or daily living at the site.

Previous research has shown cultural connections — such as similarities in pottery styles — between the area around Stonehenge and Scotland’s Orkney Islands. Other stones at Stonehenge came from western Wales.

While Britain is dotted with other Neolithic stone circles, “the thing that’s unique about Stonehenge is the distance from which the stones have been sourced,” said Aberystwyth University’s Richard Bevins, a study co-author.

The Associated Press Health and Science Department receives support from the Howard Hughes Medical Institute’s Science and Educational Media Group. The AP is solely responsible for all content.

Advertisement

• Previous Issue
• Previous Article
• Next Article

Materials and Methods

Supplemental digital content, hypotension prediction index is equally effective in predicting intraoperative hypotension during noncardiac surgery compared to a mean arterial pressure threshold: a prospective observational study.

Submitted for publication September 26, 2023. Accepted for publication March 22, 2024. Published online first on April 1, 2024.

This article is featured in “This Month in A nesthesiology ,” page A1.

This article is accompanied by an editorial on p. 421.

This article has a related Infographic on p. A16.

Supplemental Digital Content is available for this article. Direct URL citations appear in the printed text and are available in both the HTML and PDF versions of this article. Links to the digital files are provided in the HTML text of this article on the Journal’s Web site ( www.anesthesiology.org ).

• Split-Screen
• Article contents
• Figures & tables
• Supplementary Data
• Peer Review
• Open the PDF for in another window
• Cite Icon Cite
• Get Permissions
• Search Site

Marijn P. Mulder , Mirjam Harmannij-Markusse , Libera Fresiello , Dirk W. Donker , Jan-Willem Potters; Hypotension Prediction Index Is Equally Effective in Predicting Intraoperative Hypotension during Noncardiac Surgery Compared to a Mean Arterial Pressure Threshold: A Prospective Observational Study. Anesthesiology 2024; 141:453–462 doi: https://doi.org/10.1097/ALN.0000000000004990

Download citation file:

• Ris (Zotero)
• Reference Manager

The Hypotension Prediction Index is designed to predict intraoperative hypotension in a timely manner and is based on arterial waveform analysis using machine learning. It has recently been suggested that this algorithm is highly correlated with the mean arterial pressure itself. Therefore, the aim of this study was to compare the index with mean arterial pressure–based prediction methods, and it is hypothesized that their ability to predict hypotension is comparable.

In this observational study, the Hypotension Prediction Index was used in addition to routine intraoperative monitoring during moderate- to high-risk elective noncardiac surgery. The agreement in time between the default Hypotension Prediction Index alarm (greater than 85) and different concurrent mean arterial pressure thresholds was evaluated. Additionally, the predictive performance of the index and different mean arterial pressure–based methods were assessed within 5, 10, and 15 min before hypotension occurred.

A total of 100 patients were included. A mean arterial pressure threshold of 73 mmHg agreed 97% of the time with the default index alarm, whereas a mean arterial pressure threshold of 72 mmHg had the most comparable predictive performance. The areas under the receiver operating characteristic curve of the Hypotension Prediction Index (0.89 [0.88 to 0.89]) and concurrent mean arterial pressure (0.88 [0.88 to 0.89]) were almost identical for predicting hypotension within 5 min, outperforming both linearly extrapolated mean arterial pressure (0.85 [0.84 to 0.85]) and delta mean arterial pressure (0.66 [0.65 to 0.67]). The positive predictive value was 31.9 (31.3 to 32.6)% for the default index alarm and 32.9 (32.2 to 33.6)% for a mean arterial pressure threshold of 72 mmHg.

In clinical practice, the Hypotension Prediction Index alarms are highly similar to those derived from mean arterial pressure, which implies that the machine learning algorithm could be substituted by an alarm based on a mean arterial pressure threshold set at 72 or 73 mmHg. Further research on intraoperative hypotension prediction should therefore include comparison with mean arterial pressure–based alarms and related effects on patient outcome.

The Hypotension Prediction Index is an alarm system approved by the Food and Drug Administration used to predict a mean arterial pressure less than 65 mmHg for at least 1 min in the operating room and critical care environments

It is based on a proprietary algorithm derived using machine learning from components of the arterial waveform processed using pulse contour analysis methods

Preliminary reports have suggested that the Hypotension Prediction Index is highly correlated with mean arterial pressure and that a mean arterial pressure threshold alone might predict imminent hypotension

Using prospectively obtained observational single-center data from 100 patients undergoing elective noncardiac surgery with invasive arterial monitoring, the authors used correlation, receiver operating characteristic curves, and precision-recall curves analyses to characterize the relationship between the index and simultaneous mean arterial pressure

A mean arterial pressure threshold of 73 mmHg correlated 97% of the time with the default Hypotension Prediction Index alarm (greater than 85)

A mean arterial pressure threshold of 72 mmHg had the most comparable predictive performance

Predictive performance metrics were essentially identical for the index and the mean arterial pressure across all analyses, and both surpassed either linearly extrapolated mean arterial pressure or delta mean arterial pressure

The positive predictive value for either the index or a mean arterial pressure threshold of 72 mmHg were both low, suggesting a high rate of false alarms

Intraoperative hypotension is common in surgical patients and is associated with an increased risk for postoperative mortality, acute kidney injury and myocardial injury. 1 , 2   A method to predict hypotension could help anesthesiologists to treat hypotension in a proactive instead of a reactive way and thus ultimately prevent it. The Hypotension Prediction Index has recently been introduced as an innovative method, consisting of a validated machine learning model based on the characteristics of the arterial waveform. 3   It is specifically developed to predict hypotension, defined as a mean arterial pressure (MAP) less than 65 mmHg for at least 1 min, up to 15 min in advance. 3  

Recently, a modeling study hypothesized that the performance of the Hypotension Prediction Index may be overestimated due to the data selection process in the development phase of the machine learning algorithm, which can introduce systematic bias. 4   This elaborate theoretical underpinning further substantiates the notion that MAP may indirectly be over-represented in the index. This is also supported by our recent observational pilot study revealing a high cross-correlation and no time difference between the Hypotension Prediction Index and MAP signals. 5   It is conceivable that with MAP available, the added predictive ability of the Hypotension Prediction Index algorithm is relatively limited, because it largely represents an inverse reflection of the concurrent MAP.

The predictive performance of the Hypotension Prediction Index was compared to several standard clinical hemodynamic variables, including MAP. 6   For the index, an area under the curve (AUC) of the receiver operating characteristic of 0.926 was reported for the prediction of hypotension 5 min in advance. The concurrent MAP was found to be the best predictor among the standard hemodynamic variables tested (AUC of 0.807). 6   However, in a recent erratum to this publication, it was clarified that different data analyses were applied for the Hypotension Prediction Index and other tested variables, and therefore it could not be concluded that the index outperforms concurrent MAP. 6   This erratum was not supported by numeric data or new results, so a formal direct comparison between the Hypotension Prediction Index and MAP is still lacking. Therefore, we set out to compare the Hypotension Prediction Index with MAP-based predictions in a large group of moderate- to high-risk noncardiac surgery patients and hypothesized that respective alarms are equally effective in predicting hypotension.

This single-center prospective, observational cohort study was conducted within the Department of Anesthesiology of the Medisch Spectrum Twente (Enschede, The Netherlands), a tertiary referral hospital. This article is prepared in accordance with the STROBE and STARD reporting guidelines. 7 , 8   Ethical approval for this study was waived by the local ethical committee (#K22-42). All consecutive adult patients aged 18 yr or older and undergoing moderate- to high-risk, noncardiac, elective surgery were included. All patients were scheduled for and received general anesthesia with the need for invasive blood pressure monitoring by means of an arterial line. The inclusion period ranged from March 2022 until July 2023 while aiming for a convenience sample size of 100 patients. There was no follow-up of this cohort. A subanalysis of the first 33 patients from this cohort was published in a research letter before. 5  

In addition to standard care in line with current practice guidelines, the patients were monitored with the HemoSphere advanced monitor including the Acumen Hypotension Prediction Index software (version 2.1, Edwards Lifesciences, Inc., USA). The HemoSphere advanced monitor, including all available additional parameters and secondary screen, was not blinded. The attending anesthesiologists could manage blood pressure according to their preferences; no specific treatment protocol was used. Demographic data, medical history, and details on the surgery performed were obtained from the electronic health records of the patients enrolled. The measured and averaged MAP (in mmHg) and computed Hypotension Prediction Index values (unitless, ranging from 0 to 100) were downloaded from the HemoSphere monitor, with 20-s interval samples.

Data analysis was performed in Python (version 3.9, Python Software Foundation, USA). The Python script, featuring a documented, step-by-step methodology along with example data is available at https://github.com/crph-utwente/HPIvalidation . All data from the start of incision till the end of surgery were used for the analysis; no data were removed. Missing data and artifacts, where a single data point dropped 30 mmHg 3   or greater below its surrounding values, were replaced by a linear interpolation. The analysis included the incidence and severity of hypotension, defined as a MAP less than 65 mmHg for at least 1 min. 3   All other 20-s data points were defined as nonhypotensive. The “area under the threshold” was calculated by multiplying the depth of hypotension (in mmHg) below a MAP of 65 mmHg by the duration of time spent in hypotension (in min). This calculation yields an area value that takes both depth and duration of hypotensive events into account. Additionally, the area under the threshold was normalized by dividing it by the total duration of surgery, resulting in the “time-weighted average” in hypotension.

Statistical Analysis

All statistical analysis were performed in RStudio (version 2023.09.1, RStudio, PBC, USA). The descriptive statistics are presented as means ± SD for normally distributed data, or median [25th, 75th percentiles] for data not normally distributed. The normality of distribution was assessed visually with histograms and Q-Q plots, and categorical data are presented as n (%).

A cross-correlation analysis between the time series of the Hypotension Prediction Index and the MAP was conducted per patient. This is a technique commonly used in the fields of signal processing and statistical time series analysis to quantify the similarity between the two signals as a function of the time shift relative to each other. 9 , 10   Both the index and MAP signals were normalized for this analysis by subtracting their means and dividing by their SDs. Because this method incorporates the temporal relation between data points, it is also used to estimate the time delay between two signals. The cross-correlation at zero time shift is essentially Pearson’s correlation. Because a nonlinear, but monotonic, relation was found between the Hypotension Prediction Index and MAP samples (see fig. 2 ) and the residuals generated during the correlation estimation process were not normally distributed, performing only Pearson’s correlation may formally not be completely sufficient. Therefore, a Spearman’s rank correlation was also calculated to better capture the relationship between the Hypotension Prediction Index values and the concurrent MAP values at zero time shift. The latter two correlation measures were calculated on a per-patient basis and for the entire pooled data set.

Calculation of mean arterial pressure (MAP)–based predictors of hypotension, including concurrent MAP, delta MAP, and linearly extrapolated MAP, illustrated based on a representative time frame of a patient.

Additionally, the timewise relationship between the alarms derived from the Hypotension Prediction Index and different MAP thresholds was evaluated. The Hypotension Prediction Index alarm was defined at the default value greater than 85, whereas different MAP thresholds were defined at concurrent values ranging from 65 to 80 mmHg retrospectively. The proportionate agreement between different MAP thresholds and the index alarm, defined as the percentage of time the thresholds were surpassed or not and the alarms were simultaneously on or off, was calculated. In addition, the Cohen’s kappa coefficient was analyzed, which takes into account the possibility of this agreement occurring by chance.

The performance of predicting hypotension within 5, 10, and 15 min in advance was evaluated for the Hypotension Prediction Index and concurrent MAP. The prediction windows were defined as follows: 1 to 5 min, 1 to 10 min, and 1 to 15 min. The first 1 min was excluded from the prediction windows, because this short prediction time is not deemed clinically relevant when aiming to prevent intraoperative hypotension by starting an intervention. The performance was assessed by means of a receiver operating characteristic curve and the AUC including 95% CI using the bootstrap method with 2,000 repetitions. These were constructed using a generalized estimating equations model that allowed to correct for repeated measures within a single patient. 11   Comparably, precision recall curves were also constructed, in which sensitivity (recall) is plotted against positive predictive value (precision) for each threshold value. The latter curves are more commonly used to assess the performance of tests in an imbalanced data set, because they hold more clinical relevance. For different Hypotension Prediction Index and MAP threshold sensitivity, specificity and positive and negative predictive values were assessed. Additionally, the threshold values at a minimal difference between sensitivity and both specificity and positive predictive values were defined.

For the 5 min prediction window, the performance of ∆MAP and linearly extrapolated MAP was also assessed. 12   ∆MAP was defined as the difference between the concurrent MAP and the MAP 5 min before. The linearly extrapolated MAP served as prediction of the MAP 5 min ahead and was determined on a trendline between concurrent MAP and MAP 5 min before, assuming linearity ( fig. 1 ).

A sensitivity analysis was performed to evaluate the potential effect of different data selection methods, (non)hypotension definitions, and prediction windows.

To achieve a more clinically relevant positive predictive value estimate, a forward analysis was conducted. 13   In this forward analysis, alarms were defined as a Hypotension Prediction Index greater than 85 or a MAP less than 72 mmHg for at least 1 min. From the onset of the alarm, i.e. , the first sample, the presence of hypotension was searched in the upcoming 1- to 5-min, 1- to 10-min, and 1- to 15-min windows. The positive predictive value was defined as the percentage of alarms truly followed by hypotension. Additionally, this forward analysis was repeated while excluding alarms that were followed by a sudden increase in MAP to correct for possible antihypotensive treatments that might conceal the potential predictive performance by preventing hypotension. Sudden increases in MAP were defined as MAP increase of more than 5 mmHg in 20 s or more than 8 mmHg in 2 min, when MAP was initially less than 70 mmHg. 6  

A total of 100 patients were included in the study; no patients were excluded from the analysis. Table 1 shows the baseline characteristics of the study population: 49 of 100 (49%) American Society of Anesthesiologists (ASA) Physical Status II, 42 of 100 (42%) ASA III, and 9 of 100 (9%) ASA IV patients, with hypertension as the most common chronic disease (49 of 100, 49%), followed by diabetes mellitus (24 of 100, 24%). The mean age was 68 ± 11 yr, and about half of the patients (51 of 100, 51%) underwent gastrointestinal surgery. The total number of data points analyzed was 68,583, with 2,061 (3.0%) defined as hypotensive, grouped into 231 hypotensive events. Table 2 shows the hypotension metrics for this study population. Hypotension was present in 62 of 100 (62%) patients, with an area under the threshold of 5 [0, 35] mmHg · min and time-weighted average of 0.03 [0.0, 0.2] mmHg.

Baseline Characteristics of the Study Population

Hypotension Metrics

The maximal cross-correlation between the Hypotension Prediction Index signal and MAP signal was −0.91 ± 0.05 with 0.0 ± 0.0 min time shift. This indicates a strong inverse relation between the two signals and no time delay. For the pooled data set of the index and concurrent MAP data points, the Pearson’s correlation was −0.88. A sigmoidal monotonic relation between Hypotension Prediction Index and concurrent MAP values is visualized in the scatter plot in figure 2 . The Spearman’s rank correlation coefficient between the index and concurrent MAP values per patient was −0.99 ± 0.01, with a −0.97 correlation for the pooled data set.

In figure 2 , the default index alarm and the most corresponding MAP thresholds (72 and 73 mmHg) resulting from the analyses detailed below are also indicated. At least one Hypotension Prediction Index alarm was detected in 89 of 100 (89%) patients, with a concurrent MAP of 71 ± 4 mmHg at the alarm onset (histogram displayed in supplemental digital content, fig. S1, https://links.lww.com/ALN/D530 ). The proportionate agreement between the default index alarm and the different MAP thresholds (65 to 80 mmHg) ranges between 0.79 and 0.97. The specific results of the agreement and Cohen’s kappa analysis can be found in table 3 and more details in supplemental digital content 1, table S1 ( https://links.lww.com/ALN/D530 ). The highest proportionate agreement (0.97) and Cohen’s kappa coefficient (0.85) were found with a MAP threshold of 73 mmHg, indicating that this value has the highest agreement with the default Hypotension Prediction Index alarm. In figure 3 , the agreement between these alarms is displayed in the time frames of two representative hypotensive patients.

Agreement between Default Hypotension Prediction Index (Greater than 85) and Concurrent MAP-based Alarms

Scatter plot of Hypotension Prediction Index and concurrent mean arterial pressure (MAP) values of all 100 patients. Red points indicate presence of hypotension, and blue points indicate its absence within 1 to 5 min. Values for the default index and proposed MAP thresholds of 72 and 73 mmHg ( dashed lines ) were used.

Agreement of the Hypotension Prediction Index default alarm (greater than 85, blue ) and the proposed alarm based on a mean arterial pressure (MAP) threshold (greater than 73 mmHg, red ) during two representative time frames of patients experiencing hypotension, defined as MAP less than 65 mmHg ( dotted line ).

In figure 4A , the receiver operating characteristic curves of the Hypotension Prediction Index, concurrent MAP, ∆MAP, and linearly extrapolated MAP to predict hypotension up to 5 min are shown. The Hypotension Prediction Index and concurrent MAP receiver operating characteristic curves are almost identical, with an AUCs of 0.89 (0.88 to 0.89) and 0.88 (0.88 to 0.89), respectively, yielding a higher AUC than the other two methods: AUC of ∆MAP is 0.66 (0.65 to 0.67), and AUC of linearly extrapolated MAP is 0.85 (0.84 to 0.85). Figure 4B shows the precision recall curves for the four prediction methods within 5 min. Again, Hypotension Prediction Index and concurrent MAP are similar with overlapping CI of the AUC: 0.56 (0.55 to 0.57) and 0.55 (0.54 to 0.56). Linearly extrapolated MAP has a substantially lower precision recall AUC of 0.32 (0.32 to 0.33). The coinciding performance curves for the index and concurrent MAP are also present at the other prediction windows. In figures S2 and S3 (supplemental digital content 1, https://links.lww.com/ALN/D530 ), receiver operating characteristic and precision recall curves for the 10- and 15-min prediction windows can be found.

Performance curves for hypotension prediction with Hypotension Prediction Index ( blue line ), concurrent mean arterial pressure (MAP; red dashed line ), linearly extrapolated MAP ( green line ), and ∆MAP ( orange line ) within a time window of 1 to 5 min. ( A ) Receiver operating characteristic curve. ( B ) Precision recall curve. AUC, area under the curve.

The performance metrics of a selection of the Hypotension Prediction Index alarms with corresponding MAP thresholds and the other MAP-based methods are reported in table 4 . When predicting hypotension within 5 min, the default Hypotension Prediction Index alarm has a sensitivity of 72.4 (71.1 to 73.8)%, specificity of 89.9 (89.7 to 90.1)% and positive predictive value of 31.9 (31.3 to 32.9)%. The MAP threshold set at 72 mmHg was the most comparable to the default Hypotension Prediction Index alarm, gaining a 72.6 (71.3 to 73.9)% sensitivity, 90.3 (90.1 to 90.5)% specificity, and 32.9 (32.2 to 33.6)% positive predictive value. When balancing sensitivity and specificity, the thresholds are 63 for the Hypotension Prediction Index, 75 mmHg for concurrent MAP, −3 mmHg for ∆MAP, and 73 mmHg for linearly extrapolated MAP. Those thresholds are 97 for the Hypotension Prediction Index, 67 mmHg for concurrent MAP, −16 mmHg for ∆MAP, and 57 mmHg for linearly extrapolated MAP when aiming for a minimal difference between sensitivity and positive predictive value. A more complete overview of the performance metrics for other Hypotension Prediction Index and MAP thresholds at different prediction windows can be found in supplemental digital content 1, tables S2 to S4 ( https://links.lww.com/ALN/D530 ).

Performance of Different Hypotension Prediction Index and MAP-based Thresholds in Predicting Hypotension

The sensitivity analysis revealed that absolute performance of both Hypotension Prediction Index and MAP-based prediction methods are affected by data processing, whereas they still perform equally, both being affected in the same way. The performance curves for different data selection and prediction times are displayed in figures S4 and S5 (supplemental digital content 1, https://links.lww.com/ALN/D530 ). At times, we observed an oscillatory behavior in the precision recall curves at low—but not clinically relevant—sensitivity, which is caused by few data points with extremely low Hypotension Prediction Index or MAP values.

The results of the forward analysis are shown in table 5 . The time to hypotension within 15 min from start of a Hypotension Prediction Index alarm was 2.3 [1.0, 7.3] min. The longer the prediction window, the higher the positive predictive value; all the values increase with correction for antihypotensive interventions. The positive predictive values for the Hypotension Prediction Index range between 21.5 and 42.1%; for a MAP alarm of 72 mmHg, they are systematically slightly higher, ranging between 24.3 and 52.0%.

Positive Predictive Values from Forward Analysis for Default Hypotension Prediction Index (Greater than 85) and MAP (Less than 72) Alarm

This observational clinical investigation in 100 surgical patients demonstrates a high correlation between the Hypotension Prediction Index and MAP values in real-world intraoperative practice. Specifically, the MAP thresholds of 72 and 73 mmHg were found to correspond closely to the default Hypotension Prediction Index alarm. When comparing the predictive performance of those methods using AUCs, those for Hypotension Prediction Index and concurrent MAP are virtually identical, while outperforming ∆MAP and linearly extrapolated MAP.

Comparison of Hypotension Prediction Index and MAP

The cross-correlation analysis revealed that the MAP signal is highly similar to the index signal, being largely its mirrored version, as suggested before. 5 , 14–16   When accounting for the nonlinear relation between the two values, an even higher correlation was found (Spearman’s ρ = −0.99). This may well be due to a presumed selection bias in the training of the machine learning model 4   and the fact that MAP is related to the arterial pressure waveform features from which the index is derived. 3   Because no time delay between the two signals was found at maximum correlation, the Hypotension Prediction Index does not seem to be ahead of concurrent MAP in time. The cross-correlation analysis performed on a subgroup published earlier is confirmed by the results found in this larger study population. 5   These results indicate that the Hypotension Prediction Index value, although based on multiple waveform features, does essentially not comprise more or different predictive information than the MAP itself.

The Hypotension Prediction Index was originally not compared to concurrent MAP, but only to ∆MAP, which showed AUCs of 0.50 to 0.62 for predicting hypotension. 3   This low predictive power of ∆MAP is confirmed in the current study, ∆MAP being a parameter that does not take the actual absolute blood pressure into account. Instead, a linear extrapolation of the MAP, as suggested recently, includes both the change and absolute value of MAP 12   and shows a substantially higher performance. Therefore, linearly extrapolated MAP seems a better comparator than ∆MAP when evaluating machine learning models or other innovative methods to predict hypotension, but it still performs substantially less than concurrent MAP.

In our current study, we directly compared the Hypotension Prediction Index and concurrent MAP using identical data and methodology and found that the receiver operating characteristic curves are coinciding. This indicates that not only the overall performance of the two prediction methods is the same, as reflected by the AUCs, but that for every specific alarm threshold the same sensitivity and specificity are achieved. There is no added predictive value of the index over concurrent MAP for any chosen threshold. This is in agreement with the receiver operating characteristic curves in the supplementary material of a recent study on the Hypotension Prediction Index performance in liver transplant surgery. 17   In addition, the precision recall curves of the index and concurrent MAP are also almost identical. The sensitivity analyses revealed that despite the spectrum of studied definitions and the data selection and prediction windows, the index and MAP perform similar. This clinical study highlights the importance of a proper and accurate comparison between the Hypotension Prediction Index and other clinical measures to prevent ambiguous assumptions about its added value.

Strengths and Limitations

During this clinical study, no specific protocol was used for treating hypotension, allowing assessment of the net effect of the presence of the Hypotension Prediction Index. This resulted in real-world data of a large group of patients showing the potential of the Hypotension Prediction Index in the intended clinical context of use and without strict boundaries imposed by a rigid research protocol. We realize that our study represents an observational, nonblinded analysis, that might have led the attending anesthesiologists to act upon Hypotension Prediction Index alarms. The resulting performance measures, especially positive predictive value, may therefore be underestimated because hypotension might have been prevented by timely intervention. We aimed to retrospectively correct for this in the forward analysis. This limits general conclusions about the absolute performance of the index and comparison with blinded studies. However, MAP is never blinded in a routine clinical setting; for a proper comparison of both performances, it is therefore beneficial that the index is also visible to the clinician.

Compared to the original Hypotension Prediction Index article, the AUCs of the Index’s performance found in this study for 5-, 10-, and 15-min prediction are somewhat lower than reported before, when all AUCs were greater than 0.95. 3   In this study, all data points were included to prevent a selection bias, as recently underscored in this context, 4   and consequently reducing the performance of the index. In addition, we used prediction windows instead of a single prediction time point in the future. The current approach may thus result in a more accurate reflection of index performance in everyday clinical practice.

Although our study is confined to a single-center analysis, its adequate representation of real-world clinical data is underlined by a comparison with the recently published European multicenter prospective observational registry: EU HYPROTECT. 18   This registry contains data of 749 surgeries in which the Hypotension Prediction Index was also used in a nonblinded fashion and without a treatment protocol, showing comparable hypotension metrics: 59% incidence with 1 [0, 3] event per patient, 2 [0, 9] min duration, and 0.03 [0.00, 0.20] mmHg time-weighted average 18   (see also supplemental digital content 1, fig. S1, https://links.lww.com/ALN/D530 ).

Clinical Implications

Probably, the most clinically relevant measure is the positive predictive value resulting from the forward analysis. This indicates the likelihood of hypotension occurring upon the onset of an alarm, which would be the trigger to consider a therapeutic intervention. Our results reveal that only around a quarter to half of the Hypotension Prediction Index alarms are actually followed by hypotension. Despite performing a retrospective correction of possible preventive interventions, the positive predictive value still remains rather limited. The MAP alarm of 72 mmHg seems to perform even slightly better than the default Hypotension Prediction Index alarm. The positive predictive values for the index in this study are lower than in previous validation studies using a prediction window of 0 to 20 min. 13 , 19  

Overall, the results of this study imply that the index’s performance in predicting hypotension can equally be achieved by adjusting MAP thresholds to higher set points than usually applied in conventional clinical routine. Notably, the default Hypotension Prediction Index alarm is fixed at 85, whereas theoretically a better balance between sensitivity and specificity could be achieved by setting a much lower alarm value (46 to 63; see table 4 ). This might not be adequate in a clinical setting, potentially inducing alarm fatigue and overtreatment from false positive alarms. Nevertheless, the Hypotension Predicting Index alarm could be set lower to improve sensitivity, for example, at 75 as it is currently proposed for the planned HYPE-2 trail 20   ; or alternatively, concurrent MAP thresholds could be set higher (75 to 79 mmHg; see table 4 ), as supported by our data. Depending on the clinical need, a higher index threshold (90 to 97) or lower MAP (67 to 71 mmHg) could also be chosen to achieve a better precision.

Additionally, an important limitation of the Hypotension Prediction Index is that it can only be used to predict hypotension defined as a MAP of less than 65 mmHg, whereas MAP thresholds can easily be adjusted to predict any desired blood pressure in a timely manner. Therefore, intraoperative alarms based on MAP thresholds seem more suitable than the index when aiming for patient-specific blood pressure targets in relation to the patient’s baseline blood pressure, co-morbidities, and type of surgery to balance the risks of organ ischemia and bleeding. 21   In addition, the predictive value of the MAP thresholds found in this study are limited to our specific population; optimal thresholds could be different for patient groups with different age, sex, and ASA score. 22  

In general, innovative methodologies like the Hypotension Prediction Index are deemed beneficial to prevent intraoperative hypotension, as reflected by a recent systematic review. 23   The data synthesis of several randomized controlled clinical trials 24–28   showed a decrease in the occurrence, duration, and severity of hypotension during noncardiac surgery while using the index, yet these results were confined to limited and low- to very-low-quality evidence. 23   It still needs to be verified whether a MAP-based alarm would yield the same hypotension reduction in clinical practice as the Hypotension Prediction Index in every single patient at every single moment. The current study was not designed to investigate hypotension reduction, treatment decisions, or the use of any other parameters available the HemoSphere platform. However, hypotension was found to be less prevalent than in both the control and intervention groups of the randomized clinical trials as reflected by different hypotension metrics. 23   However, this might also be based on a psychologic effect that arises from the presence of the Hypotension Prediction Index technique, its novelty, and potentially induced increased awareness, leading to more proactive hemodynamic management. The same phenomenon could be present in those randomized controlled trails. Moreover, because most studies used a treatment protocol in addition to the Hypotension Prediction Index, it remains unresolved which of the two would lead to improved outcomes.

Conclusions

Hypotension Prediction Index and concurrent MAP values are highly correlated, their respective alarms and thresholds show strong agreement and interchangeable performance in predicting intraoperative hypotension. A MAP threshold set at 72 or 73 mmHg seems a straightforward clinical alternative to the Hypotension Prediction Index. It is important to compare new hypotension prediction methods against parameters derived from routine hemodynamic monitoring, notably concurrent MAP, to study their added value including patient outcome.

Research Support

Support was provided solely from institutional and/or departmental sources.

Competing Interests

Dr. Mulder, Dr. Donker, and Dr. Fresiello provide research consultancy to Maquet Critical Care AB (Solna, Sweden). Dr. Donker and Dr. Fresiello have a research corporation with Sonion Nederland B.V. (Hoopddorf, The Netherlands), and Dr. Donker provides research consultancy to HBOX Therapies GmbH (Aachen, Germany). None of these authors receive personal fees. The other authors declare no competing interests.

Additional results: Figures and tables, https://links.lww.com/ALN/D530

Visual Abstract

Citing articles via

Most viewed, email alerts, related articles, social media, affiliations.

• ASA Practice Parameters
• Online First
• Author Resource Center
• About the Journal
• Editorial Board
• Rights & Permissions
• Online ISSN 1528-1175
• Print ISSN 0003-3022
• Anesthesiology
• ASA Monitor

• Terms & Conditions Privacy Policy
• Manage Cookie Preferences
• © Copyright 2024 American Society of Anesthesiologists

This Feature Is Available To Subscribers Only

Sign In or Create an Account

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Online First
• Efficacy and safety of sequential therapy with subcutaneous belimumab and one cycle of rituximab in patients with systemic lupus erythematosus: the phase 3, randomised, placebo-controlled BLISS-BELIEVE study
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0001-9299-0053 Cynthia Aranow 1 ,
• Cornelia F Allaart 2 ,
• http://orcid.org/0000-0003-0292-9043 Zahir Amoura 3 ,
• http://orcid.org/0000-0003-3047-500X Ian N Bruce 4 , 5 ,
• Patricia C Cagnoli 6 ,
• Walter W Chatham 7 ,
• Kenneth L Clark 8 ,
• http://orcid.org/0000-0001-6712-1585 Richard Furie 1 ,
• James Groark 9 ,
• http://orcid.org/0000-0001-7506-9166 Murray B Urowitz 10 ,
• http://orcid.org/0000-0001-6438-8663 Ronald van Vollenhoven 11 ,
• Mark Daniels 12 ,
• Norma Lynn Fox 9 ,
• Yun Irene Gregan 13 ,
• Robert B Henderson 14 ,
• André van Maurik 14 ,
• http://orcid.org/0000-0002-8413-131X Josephine C Ocran-Appiah 15 ,
• Mary Oldham 16 ,
• David A Roth 17 ,
• Don Shanahan 18 ,
• http://orcid.org/0000-0002-3532-5409 Paul P Tak 19 ,
• http://orcid.org/0000-0001-9920-2195 Yk Onno Teng 20
• 1 Feinstein Institutes for Medical Research, Northwell Health , Manhasset , New York , USA
• 2 Leids Universitair Medisch Centrum , Leiden , The Netherlands
• 3 Assistance Publique–Hôpitaux de Paris, Groupement Hospitalier Pitié–Salpêtrière, French National Referral Center for Systemic Lupus Erythematosus, Antiphospholipid Antibody Syndrome and Other Autoimmune Disorders, Service de Médecine Interne 2, Institut E3M, Inserm UMRS, Centre d’Immunologie et des Maladies Infectieuses (CIMI-Paris) , Sorbonne Université , Paris , France
• 4 Kellgren Centre for Rheumatology, Manchester University Hospitals NHS Trust , Manchester , UK
• 5 Centre for Musculoskeletal Research, University of Manchester, Manchester Academic Health Science Centre , Manchester , UK
• 6 University of Michigan Medical Center , Ann Arbor , Michigan , USA
• 7 University of Alabama at Birmingham , Birmingham , Alabama , USA
• 8 Clinical Science, GSK , Stevenage , UK
• 9 Clinical Development, GSK , Collegeville , Pennsylvania , USA
• 10 Toronto Western Hospital, University of Toronto, Lupus Clinic , Toronto , Ontario , Canada
• 11 Amsterdam University Medical Center, Amsterdam Rheumatology Center , Amsterdam , The Netherlands
• 12 Global Medical Affairs, GSK , Brentford , UK
• 13 Clinical Science Immunology, GSK , Collegeville , Pennsylvania , USA
• 14 Clinical Biomarker Group, GSK , Stevenage , UK
• 15 Clinical Science, Respiratory and Immunology Clinical Research and Early Programs, GSK , Philadelphia , Pennsylvania , USA
• 16 Biostatistics, GSK , Stevenage , UK
• 17 Research and Development, GSK , Collegeville , Pennsylvania , USA
• 18 Development Biostatistics, GSK, GSK House , Brentford , UK
• 19 Research and Development, GSK , Stevenage , UK
• 20 Department of Nephrology, Leiden University Medical Center , Leiden , The Netherlands
• Correspondence to Dr Cynthia Aranow; caranow{at}northwell.edu

Objectives Disease activity control in patients with systemic lupus erythematosus (SLE) with corticosteroid and immunosuppressant withdrawal is a treatment goal. We evaluated whether this could be attained with sequential subcutaneous belimumab (BEL) and one cycle of rituximab (RTX).

Methods In this phase 3, double-blind BLISS-BELIEVE trial (GSK Study 205646), patients with active SLE initiating subcutaneous BEL 200 mg/week for 52 weeks were randomised to intravenous placebo (BEL/PBO) or intravenous RTX 1000 mg (BEL/RTX) at weeks 4 and 6 while stopping concomitant immunosuppressants/tapering corticosteroids; standard therapy for 104 weeks (BEL/ST; reference arm) was included. Primary endpoint: proportion of patients achieving disease control (SLE Disease Activity Index-2000 (SLEDAI-2K) ≤2; without immunosuppressants; prednisone equivalent ≤5 mg/day) at week 52 with BEL/RTX versus BEL/PBO. Major (alpha-controlled) secondary endpoints: proportion of patients with clinical remission (week 64; clinical SLEDAI-2K=0, without immunosuppressants/corticosteroids); proportion of patients with disease control (week 104). Other assessments: disease control duration, anti-dsDNA antibody, C3/C4 and B cells/B-cell subsets.

Results The modified intention-to-treat population included 263 patients. Overall, 16.7% (12/72) of BEL/PBO and 19.4% (28/144) of BEL/RTX patients achieved disease control (OR (95% CI) 1.27 (0.60 to 2.71); p=0.5342) at week 52. For major secondary endpoints, differences between BEL/RTX and BEL/PBO were not statistically significant. Anti-dsDNA antibodies and most assessed B cells/B-cell subsets were lower with BEL/RTX versus BEL/PBO. Mean disease control duration through 52 weeks was significantly greater with BEL/RTX versus BEL/PBO.

Conclusions BEL/RTX showed no superiority over BEL/PBO for most endpoints analysed; however, it led to significant improvements in disease activity markers compared with BEL/PBO. Further investigation of combination treatment is warranted.

Trial registration number NCT03312907 .

• Autoimmune Diseases
• Biological Therapy
• B-Lymphocytes
• Lupus Erythematosus, Systemic

Data availability statement

Data are available upon reasonable request. GSK is committed to publicly disclosing the results of GSK-sponsored clinical research that evaluates GSK medicines, and as such was involved in the decision to submit the results of this study (GSK Study 205646). Anonymised individual patient data and study documents can be requested for further research from https://www.gsk-studyregister.com/en/ .

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See:  http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/ard-2024-225686

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THIS TOPIC

Achieving low disease activity in the absence of corticosteroids remains an important treatment goal in patients with systemic lupus erythematosus (SLE).

Preliminary studies suggested that sequential therapy with belimumab and rituximab in patients with active SLE may provide clinical benefits with an acceptable safety profile.

WHAT THIS STUDY ADDS

In this robust phase 3 BLISS-BELIEVE study of sequential belimumab and rituximab administration, while the primary and major secondary endpoints were not met, the mean duration of longest disease control was nominally significantly greater in patients treated with belimumab and rituximab sequential therapy compared with belimumab and placebo.

Significant reductions in anti-dsDNA antibody levels, CD19+ B cells and B-cell subsets, were observed with belimumab and rituximab sequential therapy versus belimumab and placebo.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

This is the first randomised study in SLE to prospectively investigate a novel treatment regimen that incorporated a rapid reduction and withdrawal of standard immunosuppressants and thereby it sets the stage for future trials in SLE to aim for the stringent, clinically meaningful endpoint of remission off-therapy.

Introduction

Despite traditional standard therapy (ST), including medications such as corticosteroids, antimalarials and immunosuppressants, a significant proportion of patients with systemic lupus erythematosus (SLE) do not achieve long-term disease control. 1–3 As prolonged exposure to glucocorticoids increases the risk of organ damage accrual, 4 management guidelines recommend tapering corticosteroids to ≤5 mg/day or withdrawing entirely when possible. 3 Furthermore, the treat-to-target principle has been embraced by SLE experts where, besides achieving low disease activity, treatment goals should be remission on-therapy and, even more aspirational, remission off-therapy. 5 6 Notwithstanding these ambitious goals, achieving disease control without corticosteroids remains an unmet treatment goal, but no randomised trial has previously employed these endpoints. 2 These goals could be achieved with disease-modifying therapies targeting the underlying pathogenesis of SLE. 7

B cells play a key role in the pathogenesis of SLE. 8 B-lymphocyte stimulator (BLyS) promotes B-cell activation and differentiation, 9–11 and elevated serum BLyS is associated with higher disease activity, disease relapse and increased numbers of autoantibody-secreting plasma cells. 9 12

Belimumab, a human IgG1λ monoclonal antibody that selectively inhibits soluble BLyS, is approved in combination with ST for treating SLE and lupus nephritis (LN). 13 14 Rituximab, a B-cell-depleting anti-CD20 monoclonal antibody, is used off-label in clinical practice as clinical trials have not demonstrated clinical efficacy. 8 15–18

The scientific justification for sequential therapy with belimumab and rituximab is twofold. Elevated BLyS levels, which occur following B-cell depletion, promote the maturation of autoreactive B cells by allowing them to bypass tolerance checkpoints and enter the immune repertoire. 19–22 Conversely, B-cell reconstitution without high levels of BLyS might result in tolerised B cells without autoreactivity and an enhanced clinical response. This potentially explains the inability of rituximab alone to show superiority over ST in SLE studies. 23 A second rationale for dual therapy is that although rituximab rapidly depletes peripheral B cells, tissue-resident B cells are less affected. 24–26 Thus, since belimumab was shown to increase circulating B-cell levels by either disrupting lymphocyte trafficking and preventing B cells from transmigrating from the blood into tissue or by preventing B cells from being retained at the tissue level, greater B-cell depletion may occur when belimumab is administered before rituximab. 27–34

Three small phase 2 trials, Synbiose (in patients with severe, refractory SLE, of whom the majority had active LN), Belimumab after B cell depletion therapy in patients with SLE (BEAT-LUPUS; in refractory SLE) and The Combination of Antibodies in Lupus Nephritis: Belimumab and Rituximab Assessment of Tolerance and Efficacy (CALIBRATE; in LN), have evaluated the efficacy and safety of rituximab and belimumab sequential therapy in improving disease biomarkers and outcomes, including B-cell depletion. 31 32 35 We have therefore hypothesised that the belimumab/rituximab combination may result in greater B-cell depletion and thus yield better control of disease activity with less need for concomitant immunosuppressants and corticosteroids than belimumab alone. To test this hypothesis, we evaluated the efficacy, safety and tolerability of belimumab with or without a single cycle of rituximab while stopping concomitant immunosuppressants and tapering corticosteroids in adult patients with SLE, using novel and stringent disease control endpoints.

Study design

This phase 3, multicentre, randomised, double-blind, placebo-controlled, 104-week study (GSK Study 205646, NCT03312907 ) was conducted at 71 sites globally between March 2018 and July 2021 ( online supplemental figure S1 ). The study protocol has been published previously. 28

Supplemental material

Eligible patients were ≥18 years of age, fulfilled ≥4 of the 11 American College of Rheumatology classification criteria for SLE, 36 37 and were required to have serum positivity for anti-nuclear antibody (titre ≥1:80) and/or anti-dsDNA antibody (≥30 IU/mL) and an SLE Disease Activity Index-2000 (SLEDAI-2K) score ≥6. Patients with severe LN or severe, active central nervous system lupus were excluded. Full eligibility criteria have been published previously. 28

Randomisation and masking

Patients were centrally randomised in a 1:2:1 ratio to: (1) BEL/PBO arm: subcutaneous (SC) belimumab 200 mg/week for 52 weeks and intravenous placebo at weeks 4 and 6; (2) BEL/RTX arm: belimumab 200 mg/week SC for 52 weeks and rituximab 1000 mg/week intravenous at weeks 4 and 6; or (3) BEL/ST arm: open-label belimumab 200 mg/week SC and ST for 104 weeks. The BEL/ST arm was included to provide an exploratory reference for assessing the relative performance of BEL/RTX versus BEL/PBO. Patients were stratified by their screening SLEDAI-2K score (≤9 or ≥10), immunosuppressant use (yes/no) and corticosteroid dose (prednisone equivalent ≤10 or >10 mg/day).

BEL/PBO and BEL/RTX patients received study treatment until week 52 (primary efficacy endpoint assessment) and then entered a 52-week, treatment-free (no belimumab or rituximab) double-blind observational phase through week 104 to allow assessment of the durability of remission. During the double-blind observational phase, comprehensive clinical assessments were scheduled at week 60, week 64 and every 8 weeks thereafter up to week 104. Data were also collected at any unscheduled visits that occurred. Patients were considered to have failed treatment if they received, at the investigator’s discretion, corticosteroids (>5 mg/day), any immunosuppressants, and/or open-label belimumab ( online supplemental materials ). BEL/ST patients received belimumab and continued their ST throughout the 104 weeks.

Antimalarials, non-steroidal anti-inflammatory drugs (NSAIDs), and/or prednisone-equivalent doses of ≤5 mg/day were allowed within protocol-defined limits in weeks 53 through 104 in all treatment groups.

Discontinuation of immunosuppression and tapering of corticosteroids

All BEL/PBO and BEL/RTX patients discontinued immunosuppressants at or before week 4. Reinitiation of immunosuppressants post week 4 in BEL/PBO and BEL/RTX groups (on the physician discretion) resulted in treatment failure. In the BEL/ST group, an increase of immunosuppressant dose post week 12 resulted in treatment failure. After the initial 12 weeks of study treatment, all treatment groups were required to taper prednisone-equivalent dose to ≤5 mg/day by week 26. Patients were considered treatment failures if their dose exceeded 5 mg/day after week 26 ( online supplemental materials ).

Efficacy endpoints

The primary endpoint was the proportion of patients achieving a state of disease control at week 52, defined as a SLEDAI-2K score ≤2 achieved without immunosuppressants and with a prednisone-equivalent dose of ≤5 mg/day.

Major secondary endpoints were the proportion of patients in clinical remission at week 64, defined as a clinical SLEDAI-2K score of 0 (without a serological activity component) achieved without immunosuppressants and with a prednisone-equivalent dose of 0 mg/day, and the proportion of patients with a state of disease control (defined as per primary endpoint) at week 104. Additional secondary efficacy endpoints included analysis of the primary efficacy endpoint by baseline characteristics (age, race, sex, SLEDAI-2K, use of immunosuppressants, corticosteroid dose, complement 3/4 (C3/C4) and anti-dsDNA antibody levels, BLyS levels, and region). As the BEL/ST arm was open label, the primary and major secondary assessments were conducted by independent blinded assessors (IBA).

Secondary efficacy endpoints that were assessed by principal investigators (PI) included duration of disease control through 52 and 104 weeks in patients achieving disease control at ≥1 time point; time to disease control (defined as per primary endpoint); time to clinical remission (defined as per secondary endpoint); change from baseline in SLEDAI-2K score and the proportion of patients achieving Lupus Low Disease Activity State (LLDAS) 38 39 at weeks 52, 64 and 104.

Time-to-first severe flare was measured through week 104 by a modified SLE Flare Index (SFI), which did not consider severe flares that were triggered only by an increase in SLEDAI-2K score to >12, but included patients classified as a treatment failure by our stringent definition (eg, receiving corticosteroid dose >5 mg/day after week 26, restarting belimumab in year 2 (BEL/PBO or BEL/RTX)), as having a severe flare. Post hoc analyses were also performed to assess disease control at week 52 by baseline proteinuria (≤0.5 g/24 hours vs >0.5 g/24 hours), time to severe flare (modified SFI) without imputing treatment failures as a flare, proportions of patients with proteinuria shift from >0.5 g/24 hours to normal (≤0.5 g/24 hours) and prednisone-equivalent reduction by ≥25% from baseline to ≤7.5 mg/day during weeks 40 through 52.

Biomarker endpoints

Prespecified biomarker endpoints included changes from baseline in IgG, anti-dsDNA antibody and complement C3/C4 levels. Proportions of patients with anti-dsDNA antibody shift from positive at baseline to negative, and with C3/C4 shifts from low at baseline to normal/high were also assessed. B cells and B-cell subsets were assessed by flow cytometric characterisation. Central laboratory testing was used for all biomarker measurements. Biomarker endpoints were assessed statistically at weeks 52, 64 and 104.

Safety endpoints

Safety assessments included adverse events (AEs), serious AEs, AEs of special interest (malignant neoplasms; post-injection/infusion anaphylaxis/hypersensitivity reactions; all infections of special interest (opportunistic infections, herpes zoster, tuberculosis and sepsis); depression/suicide/self-injury) and deaths. AEs are reported on treatment for week 52 (year 1) and on study for week 104 (year 1+2).

Statistical analysis

The target sample size of 280 patients was expected to achieve ≥98% power to detect a treatment difference in the primary endpoint at the 5% two-sided level of significance, assuming response rates of 10% and 35% in the BEL/PBO and BEL/RTX groups (1:2 randomisation), respectively.

All randomised patients who received ≥1 dose of study treatment were included in the intention-to-treat (ITT) population. Efficacy endpoints were assessed using a modified ITT (mITT) population, which comprised the ITT population but excluded 29 patients from the BEL/ST group due to IBAs being potentially unblinded.

Primary and the two major secondary efficacy endpoints were alpha controlled using a prespecified testing order; endpoints could be tested in sequence (two-sided alpha=0.05), provided that statistical significance was achieved in all prior tests. These endpoints were analysed using logistic regression adjusted for baseline SLEDAI-2K score (≤9 or ≥10), immunosuppressant use (use or no use), prednisone-equivalent dose (≤10 mg/day or >10 mg/day) and treatment group. As the primary endpoint was not statistically significant, the p values for the two major secondary endpoints are nominal.

Other endpoints were not adjusted for multiplicity, and all associated p values are nominal.

Patients who met treatment-failure criteria, withdrew or missed assessments (and subsequent data collection was not possible) were regarded as non-responders for primary and secondary efficacy assessments and as experiencing a severe flare for the modified SFI flare endpoint. Other analyses were not alpha controlled; preplanned analyses were performed at weeks 52, 64 and 104.

The post hoc analysis for proteinuria is described in the online supplemental materials .

Biomarker outcomes were performed on observed data for ITT patients who were ongoing after 52 weeks and received both intravenous doses of rituximab or placebo or who remained on open-label belimumab through week 52 (for BEL/ST group) and did not have more than 28 consecutive days from baseline to week 51 without a belimumab dose. Year 1 and 2 analyses used data collected in year 1 before belimumab restart (BEL/PBO and BEL/RTX groups) or discontinuation (BEL/ST group) for patients who completed week 52 on treatment. No imputation was carried out for missing data in these analyses.

Safety endpoints were summarised descriptively.

Patient and public involvement

Patients and/or public were not involved in the design, conduct, reporting or dissemination of this research.

Role of the funding source

This study (GSK Study 205646) was funded by GSK. GSK was involved in designing the study, contributed to the collection, analysis and interpretation of the data, supported the authors in the manuscript development, and funded the medical writing assistance. All authors approved the content of the submitted manuscript and were involved in the decision to submit the manuscript for publication.

Patient disposition and baseline characteristics

Of 396 patients screened, 292 were included in the ITT population and 263 in the mITT population. Overall, 226/292 (77.4%) patients completed the study at week 104, with 215 of these patients having completed the scheduled study treatment up to Week 52. Across all groups, the most common reason for not completing the study was the patient’s decision to withdraw ( figure 1 ).

• Download figure
• Open in new tab
• Download powerpoint

Patient disposition summary. mITT population excludes 29 patients from the BEL/ST group, due to IBAs being potentially unblinded. BEL, belimumab; IBA, independent blinded assessor; ITT, intention-to-treat; mITT, modified intention-to-treat; RTX, rituximab; ST, standard therapy.

Baseline characteristics were similar across treatment groups. The mean (SD) glucocorticoid dose was 10.0 (10.49) mg/day; most patients (80.8%) were taking glucocorticoids, followed by antimalarials (79.8%; table 1 ).

• View inline

Patient baseline characteristics (ITT population, N=292)

Efficacy results

At week 52, 16.7% of BEL/PBO-treated and 19.4% of BEL/RTX-treated patients achieved disease control, which was not statistically significant ( table 2 ). Of note, 25.5% of BEL/ST-treated patients achieved disease control. The observed differences in individual components of disease control between BEL/RTX and BEL/PBO groups are summarised in online supplemental table S1 , and the first reasons for not achieving disease control at Week 52 in online supplemental table S2 .

Summary of primary and major secondary efficacy endpoints, based on IBA assessment (mITT population,* N=263)

Proportions of patients off-treatment achieving clinicalremission at week 64 or disease control at week 104 showed no statistically significant differences between BEL/RTX and BEL/PBO ( table 2 ). The most common first reason for not achieving disease control at week 104 was continuation or restart of belimumab at the investigator’s discretion ( online supplemental table S2 ). In the ITT population, including those for whom it was the first reason for not achieving disease control, 54.8% (n=34/62) of BEL/PBO and 50.0% (n=62/124) of BEL/RTX patients continued or restarted belimumab in year 2.

Few patients achieved disease control or clinical remission that was sustained for at least 24 weeks and maintained through week 104, with no significant difference between BEL/RTX and BEL/PBO ( online supplemental table S3 ).

Prespecified subgroup analyses by baseline characteristics for the primary endpoint, disease control at week 52, showed no significant differences. However, there were numerically higher ORs in favour of BEL/RTX versus BEL/PBO for achieving disease control in patients with low C3/C4 levels and anti-dsDNA antibodies ≥30 IU/mL or SLEDAI-2K score ≥10 at baseline ( figure 2 ). When analysed by baseline proteinuria (post hoc) at week 52, disease control was achieved by 18.0% (n=11/61) of BEL/PBO and 20.8% (n=25/120) of BEL/RTX patients with baseline proteinuria ≤0.5 g/24 hours (OR (95% CI) 1.21 (0.54 to 2.71), p=0.6353), and by 9.1% (n=1/11) of BEL/PBO and 12.5% (n=3/24) of BEL/RTX patients with baseline proteinuria >0.5 g/24 hours (OR (95% CI) 1.74 (0.12 to 25.15), p=0.6845).

Disease control* by baseline characteristics subgroups at week 52, based on IBA assessment (mITT population † ; N=263). Note: OR (95% CI) and p value are from a logistic regression model with covariates: baseline SLEDAI-2K, baseline immunosuppressants, baseline prednisone-equivalent dose and treatment group (however, covariates were excluded from the corresponding subgroup models, for example, baseline SLEDAI-2K was not included in analysis by baseline SLEDAI-2K). *Disease control is defined as a SLEDAI-2K score ≤2 achieved without immunosuppressants and with a prednisone-equivalent dose of ≤5 mg/day. † mITT population excludes 29 patients from the BEL/ST group, due to IBAs being potentially unblinded. BEL, belimumab; BLyS, B-lymphocyte stimulator; C3/4, complement 3/4; IBA, independent blinded assessor; mITT, modified intention-to-treat; PBO, placebo; RTX, rituximab; SLEDAI-2K, SLE Disease Activity Index-2000; SLE, systemic lupus erythematosus; ST, standard therapy.

The mean duration of the longest disease control response (prespecified analysis) was significantly greater in BEL/RTX versus BEL/PBO patients through 52 weeks (adjusted treatment difference (95% CI) 47.0 (8.0 to 86.0) days, p=0.0188). Duration of disease control over 104 weeks, including the observational phase where all immunosuppression was stopped, was not significantly different between treatments ( table 3 ). Reductions in SLEDAI-2K scores through 104 weeks were observed in all treatment groups, with a significantly larger reduction observed with BEL/RTX versus BEL/PBO (adjusted treatment difference (95% CI) −1.6 (−2.4 to –0.7), p=0.0003; online supplemental figure S2A ). The odds of achieving LLDAS were not significantly different for BEL/RTX versus BEL/PBO at week 52 or 104 ( online supplemental figure S2B ).

Duration of disease control,* based on PI assessment (mITT population, † N=263)

Reduction from baseline in SLEDAI-2K ≥4 at week 52 by anti-dsDNA antibody and C3/C4 levels is shown in online supplemental table S4 .

SLE severe flares were assessed using a modified SFI as defined. As such, through week 52, 37.5% of BEL/PBO and 36.1% of BEL/RTX patients had a severe SFI flare. Over 104 weeks, this increased to 75.0% and 64.6%, respectively (HR (95% CI) 0.81 (0.57 to 1.13), p=0.2150) ( online supplemental figure S2C ). The median number of days to the first severe SFI flare was 372 in the BEL/PBO group and 379 in the BEL/RTX group. In post hoc analysis of severe flares that did not impute treatment failure, 20.8% of BEL/PBO-treated patients and 15.3% of BEL/RTX-treated patients experienced severe SFI flares over 104 weeks (HR (95% CI) 0.69 (0.36 to 1.34), p=0.2745). Among these patients, the median number of days to first severe flare was 202.0 and 226.5 in the BEL/PBO and BEL/RTX groups, respectively.

Within the small number of patients with proteinuria >0.5 g/24 hours at baseline (BEL/PBO, n=11; BEL/RTX, n=24; BEL/ST, n=9), at week 52, a numerically greater proportion of BEL/RTX-treated patients versus BEL/PBO-treated patients shifted from high (>0.5 g/24 hours) to normal (≤0.5 g/24 hours), but the difference was not significant (p=0.1318; post hoc analysis; online supplemental figure S3A ).

In a post hoc analysis, among patients with baseline prednisone-equivalent corticosteroid dose of >7.5 mg/day, dose reduction by ≥25% from baseline to ≤7.5 mg/day during week 40 through week 52 was observed in 50.0% (n=18/36) of BEL/PBO and 54.7% (n=41/75) of BEL/RTX patients, with no statistical difference (OR (95% CI) 1.29 (0.57 to 2.90), p=0.5379).

By week 52, a decrease from baseline in IgG was observed in all groups, with the greatest decreases observed with BEL/RTX. On belimumab discontinuation from week 52, IgG levels gradually increased but remained below baseline ( online supplemental figure S3B ).

A significantly greater decrease in anti-dsDNA antibodies from baseline was observed with BEL/RTX versus BEL/PBO at weeks 52 (p=0.0495) and 64 (p=0.0230), but not at week 104 (p=0.2501; online supplemental figure S3C ). Among anti-dsDNA antibody-positive patients at baseline (BEL/PBO, 63.6%; BEL/RTX, 71.4%), a significantly greater proportion of BEL/RTX versus BEL/PBO patients shifted to negative at week 52 only (n=18/74, 24.3% vs n=2/35, 5.7%; p=0.0186).

Increases from baseline in median C3 and C4 levels were observed in all groups, particularly BEL/RTX; levels were generally maintained above baseline during the observation phase ( online supplemental figure S3D, E ). Significant differences between BEL/RTX and BEL/PBO for median change from baseline were observed only for C3 levels at week 64 (p=0.0315). Among patients with low C3 levels (BEL/PBO, 43.6%; BEL/RTX, 44.8%) and those with low C4 levels (BEL/PBO, 18.2%; BEL/RTX, 32.4%) at baseline, there was no difference in the shift to normal/high levels between BEL/RTX and BEL/PBO groups.

At week 52, decreases in B cells and B-cell subsets were observed across all three groups, with significantly greater decreases observed with BEL/RTX versus BEL/PBO (p <0.0001) for all assessed B cells and B-cell subsets ( online supplemental figure S4 ). At week 104, the only statistically significant BEL/RTX versus BEL/PBO treatment difference was for memory CD20+ CD27+ B cells (p=0.0026).

Similar proportions of patients across treatment groups experienced on-treatment AEs (year 1) and on-study AEs (years 1+2; table 4 ). Proportions of patients experiencing belimumab-related AEs on-study were 34.7%–38.2% across all treatment groups and study periods ( table 4 ). Infections and infestations were the most frequently reported serious and non-serious AEs by system organ class, experienced by a higher proportion of the BEL/RTX group than other treatment groups ( table 4 ). In year 1, serious infections and infestations occurred in eight, two and one patients in the BEL/RTX, BEL/PBO and BEL/ST groups, respectively.

Summary of AEs and AESI for year 1 and years 1 + 2 (ITT population; N=292)

The incidence of infections of special interest was low, with one patient reporting sepsis in the BEL/RTX group.

Overall, 9 (12.5%) BEL/PBO-treated patients, 16 (11.1%) BEL/RTX-treated patients, and 5 (6.6%) BEL/ST-treated patients experienced psychiatric disorders of special interest and events of depression/suicide/self-injury ( table 4 ). Suicidal behaviour was reported in one patient in each group. No completed suicides were reported. Three deaths were recorded: one in the BEL/PBO group due to cholangiocarcinoma (fatal SAE started on study, but death occurred after study withdrawal) and two on study in the BEL/RTX group (one due to pneumonia and one attributed to ‘sudden death’).

This study addressed a novel treatment approach of adding rituximab to belimumab in patients with active SLE while investigating a rigorous withdrawal of background immunosuppressants and tapering corticosteroids. As such, BLISS-BELIEVE is the first randomised clinical trial in SLE to investigate disease control and remission off-therapy as stringent study outcomes. Similar proportions of BEL/RTX-treated and BEL/PBO-treated patients achieved disease control; therefore, the study’s primary endpoint was not met. However, the study demonstrated that disease control without immunosuppressants and low-dose corticosteroids can be achieved in 16.7%–19.4% of patients with active SLE.

Belimumab and rituximab have complementary mechanisms of action, providing an immunological rationale for their combined use, 28 as supported by case reports showing improvement with belimumab either preceding rituximab in primary Sjögren’s Syndrome (pSS) or following rituximab in pSS, SLE and LN. 30 33 40–43 While the BEAT-LUPUS trial demonstrated a reduced risk of severe flares among patients receiving belimumab after rituximab compared with placebo after rituximab, this was not observed in the present study of belimumab and subsequent rituximab therapy. The two study designs, sizes and populations differed substantially. For example, BEAT-LUPUS, a small phase 2 trial of 52 patients with refractory SLE, incorporated less stringent corticosteroid tapering and permitted immunosuppression. 32 Furthermore, the risk of severe flares was a secondary endpoint, and a different definition of severe flare was used in BEAT-LUPUS; thus, caution is required in comparing findings across the two trials.

In subgroup analyses, disease control rates at week 52 tended to numerically favour BEL/RTX versus BEL/PBO in patients with low C3/C4 and high anti-dsDNA antibody levels at baseline and in patients with baseline SLEDAI-2K score ≥10. Also, a numerically greater improvement in proteinuria was observed with BEL/RTX versus BEL/PBO in a post hoc analysis; however, these data must be interpreted with caution due to small patient numbers.

In the present study, BEL/RTX led to significant reduction in anti-dsDNA antibody levels at week 52 versus BEL/PBO corroborating the primary outcome results of anti-dsDNA antibody levels from the BEAT-LUPUS trial. 32 Similarly, C3/C4 levels increased from baseline to week 104 in all treatment groups, with a trend for larger increases with BEL/RTX at both week 52 and week 104. With respect to B-cell phenotyping, B-cell subsets (such as naïve B cells) decreased within 16 weeks of belimumab treatment in our study and memory B cells rapidly increased in circulation, consistent with observations made in previous studies. 44–46 More recently, the surge in circulating memory B cells following belimumab initiation has been associated with disrupted lymphocyte trafficking. 34 47 As previously demonstrated, the surge of BLyS levels after rituximab treatment can be targeted by sequentially treating with belimumab and vice versa as the memory B-cell surge following initiation of belimumab can be well targeted by rituximab. It remains, however, to be established whether disrupting lymphocyte trafficking and depriving memory B cells of cell-to-cell interactions can adequately modulate memory B cell-mediated immunological processes driving pathology and/or whether depletion of circulating memory B cells is an absolute requirement to impact the pathological pathways driven by this B-cell subset. We favour the latter, as anti-dsDNA antibodies have been shown to be markedly reduced in all randomised studies of sequential therapy, regardless of the order in which belimumab and rituximab are administered. 31 32 35 Significant decreases in anti-dsDNA antibody levels and increases in complement levels were observed in patients with complete depletion of B cells. 48 Successful depletion of B cells was also shown to predict response to rituximab, whereas repopulation of B cells increased the risk of clinical relapse. 48 Therefore, repopulation of B cells or subsets thereof (such as memory B-cells) could potentially act as an important biomarker for personalised treatment decisions when employing B-cell targeted therapies in active SLE.

Thus, the biomarker investigations as secondary outcomes of this study should prompt further studies to address whether changes seen in serological biomarkers and degree of B-cell depletion can be translated to identify a subgroup of patients with SLE that benefit the most from belimumab/rituximab combination therapy.

Safety findings were consistent with the known safety profiles of belimumab and rituximab, although incidences of AEs and serious infections and infestations, and depression were lower than in the BEAT-LUPUS phase 2 study of rituximab followed by belimumab treatment of patients with SLE. 32 Comparable proportions of patients in all treatment groups experienced ≥1 AE during our study. Although incidence of serious infections and infestations was higher in the BEL/RTX group than in other treatment groups, there was no evidence of new or unexpected safety signals using the combination. Nonetheless, the benefit from sequential belimumab and rituximab treatment in any subgroup of patients will need to be balanced against risk of serious infections and infestations.

In the present study, 25.5% of BEL/ST patients achieved disease control at week 52, which was higher than either BEL/PBO or BEL/RTX patients. One could speculate that a less stringent tapering of immunosuppression in the primary comparator groups BEL/PBO and BEL/RTX could have resulted in better disease control. 49 50 The patients in this study had a median disease duration of 7 years and a baseline mean daily corticosteroid dose of 10 mg/day, perhaps making them better suited to a less stringent tapering regimen, further supporting one of the important learnings of this informative study design.

This study has several limitations. Occurrence of severe SFI flare during the 2-year study period was relatively high, due partly to including the stringent study-specific definition of treatment failure (eg, corticosteroids >5 mg/day rather than 7.5 mg/day, restarting belimumab in year 2 (BEL/PBO or BEL/RTX); see online supplemental materials for full definition). Also, the BEL/ST group was included for descriptive reference only and cannot be directly compared with BEL/RTX; though, in real-world practice, belimumab is administered as an add-on to ST. The study aimed to investigate the hypothesis that combined B-cell targeting therapy would be effective without other ST. As such, the present study was not designed to reflect clinical practice accurately, and the use of less stringent tapering of IS, repetitive treatment cycles of rituximab and a comparison with continued BEL/ST would have been counterproductive to the study’s goal to achieve clinical remission off-therapy. BLISS-BELIEVE used SLEDAI-2K in our stringent efficacy outcomes, which, as a binary assessment, does not measure the improvement in symptoms, nor does it capture incomplete improvement in those symptoms. Furthermore, it is difficult to make direct comparisons between the BLISS-BELIEVE study and previous interventional SLE trials, in which response was determined by the SLE Responder Index 4 or British Isles Lupus Assessment Group-based Composite Lupus Assessment.

Primary and major secondary efficacy endpoints used IBA assessments, while other efficacy endpoints and treatment decisions were based on PI assessments. Some residual bias cannot be ruled out, as PIs were not blinded to the BEL/ST reference group.

The BLISS-BELIEVE study incorporated a unique and stringent efficacy endpoint requiring immunosuppression-free disease remission in patients with active SLE, a concept consistent with treat-to-target criteria. 38 50 Although BEL/RTX treatment was not superior to BEL/PBO and the addition of rituximab to patients’ therapy did not improve disease control using the stringent outcome measures employed by this study, there were significant reductions in circulating B-cells and improvement in serological biomarkers. This warrants further investigation to determine if combination with rituximab provides better disease control for patients. Nonetheless, this large, robust study provides valuable information to clinicians to support their treatment decision-making and informs future studies into combination therapy, perhaps with less rapid immunosuppressant tapering.

Ethics statements

Patient consent for publication.

Consent obtained directly from patient(s).

Ethics approval

This study involves human participants and was approved by "Argentina: 1. Framingham Centro Medico 2. Comite de Etica en Investigacion 3. Comité Independiente de Ética Médica del Noroeste Argentino (CIEM-NOA)""Brazil: 1. CEP do Hospital Universitario Julio Muller / MT 2. Fundação Faculdade Regional de Medicina de são José do Rio Preto 3. Hospital Santa Izabel 4. Comitê de Ética em Pesquisa do Hospital Universitário da Universidade Federal de Juiz de For a""Canada: 1. Advarra Institutional Review Board 2. Hamilton Integrated Research Ethics Board 3. University Health Network, Research Ethics Board""France: 1. Comite de Protection des Personnes Nord-Ouest II - CHU d’Amiens Hôpital Nord ""Germany: 1. Ethik-Kommission des Fachbereichs Medizin der Johann Wolfgang Goethe-Universität 2. Ethik-Kommission bei der Ärztekammer Hamburg 3. Ethik-Kommission der Medizinischen Hochschule Hannover 4. Ethik-Kommission der Med. Fakultaet der Christian-Albrechts-Universitaet zu Kiel""Republic of Korea: 1. IRB of Hanyang University Hospital, College of Medicine 2. IRB of Kyungpook National University Hospital 3. IRB of Chonnam National University Hospital, Clinical trial center 4. IRB of Seoul National University Hospital, Biomedical Research Institute 5. IRB of Seoul Saint Mary’s hospital 6. IRB of Inha University Hospital 7. Ajou University Hospital IRB, 5th Floor Annex building""Mexico: 1. Investigación Biomédica para el Desarrollo de Fárm 2. Hospital Centro de Especialidades Médicas CEM""Netherlands: 1. MEC-U""Russian Federation: 1. Republican Clinical Hospital 2. Local Ethics Committee of The State Educational Institution of the Highest Professional Education 3. University of Roszdrav 4. Institute of Cytology and Genetics 5. BUZ Regional Clinical Hospital 6. I.M. Sechenov First Moscow State Medical University, Local Ethics Committee of Federal State Autonomous Educational Institution of Higher Education 7. Chelyabinsk Regional Clinical Hospital 8. Tver Regional clinical hospital 9. Krasnoyarsk State Medical University 10. Local Ethics Committee at “Clinical Hospital #2, City Hospital #8 11. Kemerovo Regional Clinical Hospital 12. Republican Hospital nom Baranov, Ethics comitee 13. Ulyanovsk Regional Clinical Hospital 14. Local Ethics Committee at LLC Research Medical Complex ""Your Health 15. Local Ethics Committee at State Budgetary Institution of Ryazan Region ‘Regional Clinical Cardiology Dispensary 16. City Rheumatology Center, Hospital #25""Spain1. Comité Etico Hospital U. Germans Trias i Pujol""United States: 1. Advarra Institutional Review Board 2. Biomedical Research Alliance of New York, LLC, Institutional Review Board 3. WCG Institutional Review Board 4. Columbia University Medical Center, Institutional Review Board 5. Ochsner Clinic Foundation Institutional Review Board 6. NYU School of Medicine Institutional Review Board" Participants gave informed consent to participate in the study before taking part.

Acknowledgments

The authors would like to thank the participating patients and their families, clinicians and study investigators. Medical writing support was provided by Olga Conn, PhD, of Fishawack Indicia, UK, part of Avalere Health, and was funded by GSK.

• Malek Mahdavi A ,
• Khabbazi A , et al
• Fanouriakis A ,
• Kostopoulou M ,
• Andersen J , et al
• Ugarte-Gil MF ,
• Leong J , et al
• van Vollenhoven R ,
• Voskuyl A ,
• Bertsias G , et al
• Parra Sánchez AR ,
• Voskuyl AE ,
• van Vollenhoven RF
• Askanase AD ,
• Bomback AS , et al
• Kinnman N ,
• Chatham W , et al
• Cheema GS ,
• Roschke V ,
• Hilbert DM , et al
• Baker KP , et al
• Thompson A ,
• Tang Y , et al
• Duxbury B ,
• Combescure C ,
• Chizzolini C
• Merrill J ,
• Furie R , et al
• Latinis K , et al
• Cambridge G ,
• Leandro MJ , et al
• Miceli-Richard C ,
• Ittah M , et al
• Carter LM ,
• Isenberg DA ,
• Ehrenstein MR
• Lazarus MN ,
• Turner-Stokes T ,
• Chavele K-M , et al
• Vallerskog T ,
• Heimbürger M ,
• Gunnarsson I , et al
• Meijer JM ,
• Bootsma H , et al
• Bootsma H ,
• Yuvaraj S , et al
• Ramwadhdoebe TH ,
• van Baarsen LGM ,
• Boumans MJH , et al
• Dore CJ , et al
• Diamond B , et al
• Ehrenstein MR ,
• Gualtierotti R ,
• Borghi MO ,
• Gerosa M , et al
• Arends EJ ,
• van Dam LS , et al
• Embleton-Thirsk A ,
• Parvaz M , et al
• Espinosa G ,
• Tipton CM , et al
• Atisha‐Fregoso Y ,
• Malkiel S ,
• Harris KM , et al
• Hochberg MC
• Fries JF , et al
• Franklyn K ,
• Navarra SV , et al
• Golder V , et al
• Chevalier K ,
• Belkhir R ,
• Seror R , et al
• De Vita S ,
• Quartuccio L ,
• Salvin S , et al
• Gandolfo S ,
• Huizinga TWJ ,
• Rabelink TJ , et al
• Genovese MC ,
• Satterwhite J , et al
• Merrill JT ,
• Looney RJ , et al
• Thurlings RM ,
• Rossier C , et al
• Dimelow R ,
• Gillespie WR ,
• van Maurik A
• Md Yusof MY ,
• El-Sherbiny YM , et al
• Alunno A , et al
• van Vollenhoven RF ,
• Bertsias G ,
• Doria A , et al

Handling editor Josef S Smolen

Contributors Conception or design: INB, RF, JG, RvV, NLF, RBH, AvM, MO, DAR, PPT and YOT. Acquisition of data: CA, CFA, ZA, PCC, WWC and MBU. Data analysis or interpretation: CA, CFA, ZA, INB, PCC, WWC, KLC, RF, JG, MBU, RvV, MD, MO, NLF, YIG, AvM, JCO-A, DAR, DS, PPT and YOT. JCO-A is guarantor for this publication (KLC, JG, MD, NLF, YIG, DS, PPT: at the time of the author’s contribution to this study).

Funding This study (GSK Study 205646) was funded by GSK.

Competing interests CA has received research support from GSK; consulting fees from GSK, AstraZeneca, BMS, Kezar Life Sciences Inc., Merck Sharp & Dohme and Alumis Inc. CFA has received research support from Janssen, AbbVie, and Eli Lilly. ZA has received research support from GSK, Roche, AstraZeneca, and Amgen; and consulting fees from GSK, AstraZeneca, Amgen, Kezar Life Sciences Inc, and Novartis. INB has received research support from Genzyme, Sanofi, and GSK; consulting fees from AstraZeneca, Eli Lilly, Aurinia Pharmaceuticals, GSK, and ILTOO; and has served as an advisory board member for AstraZeneca and Merck Serono. INB is a National Institute for Health Research (NIHR) Senior Investigator Emeritus and is funded by the NIHR Manchester Biomedical Research Centre (NIHR203308). PCC has received consulting and speaker fees from GSK, Aurinia Pharmaceuticals, and Eli Lilly. WWC has received research support from GSK, UCB, Amgen, Pfizer, and BMS; consulting fees from GSK and Aurinia Pharmaceuticals; honoraria from GSK and Aurinia Pharmaceuticals for disease awareness presentations and curricula on achieving disease control and remission in SLE. RF has received research support and consulting fees from GSK. MBU has received research support from GSK; consulting fees from GSK and UCB; and speaker fees from GSK, Eli Lilly, and AstraZeneca. RvV has received research support for educational programmes and institutional grants from AstraZeneca, Pfizer, and UCB; consulting fees from AbbVie, AstraZeneca, Biogen, Biotest, BMS, Galapagos, Gilead, Janssen, Pfizer, Sanofi, Servier, UCB, and Vielabio; speaker fees from AbbVie, Galapagos, GSK, Janssen, Pfizer, and UCB; and research support from BMS, GSK Eli Lilly, and UCB. RBH, AvM, JCO-A, MO and DAR are employees of GSK and hold stocks and shares in the company. RBH is also an inventor for the following patents: US-20220195062-A1, US-11180569-B2, and US-20180265588-A1. KLC, JG, MD, YIG, DS and PPT were employees of GSK at the time of the study and hold stocks and shares in the company. NLF is a former employee and consultant for GSK and holds stocks in the company. YOT has received unrestricted research grants from GSK, Aurinia Pharmaceuticals, and Vifor Pharma; and consulting fees from Aurinia Pharmaceuticals, Novartis, GSK, Kezar Life Sciences Inc, Vifor Pharma, and Otsuka Pharmaceuticals (paid to Leiden University Medical Center).

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF:

Money blog: Morrisons admits it 'went too far' with self-checkouts - as it changes strategy

Welcome to the Money blog, your place for personal finance and consumer news and tips. Today's posts include Morrisons getting rid of some self-checkouts and a Money Problem on topping up your national insurance. Leave your consumer issue below - remember to include contact details.

Monday 19 August 2024 20:13, UK

• Energy bills to rise 9% this winter - forecast
• Morrisons admits it went too far with self-checkouts
• Kellogg's shrinks size of Corn Flakes

Essential reads

• Money Problem : 'Should I top up my national insurance and could it really get me £6,000 extra?'
• Pay at every supermarket revealed - and perks staff get at each
• Couples on how they split finances when one earns more than other

Tips and advice

• All discounts you get as student or young person
• Save up to half price on top attractions with this trick
• Fines for parents taking kids out of school increasing

Ask a question or make a comment

Morrisons has admitted it "went a bit too far" with self-checkouts.

Chief executive Rami Baitiéh says the supermarket is "reviewing the balance between self-checkouts and manned tills".

Some will be removed.

Mr Baitiéh told The Telegraph : "Morrisons went a bit too far with the self-checkout. This had the advantage of driving some productivity. However, some shoppers dislike it, mainly when they have a full trolley."

The executive also said self checkouts had driven more shoplifting.

What have other supermarkets said about self-checkouts?

In April, the boss of Sainsbury's said customers liked self-checkouts...

That prompted us to ask readers for their thoughts - and we carried out a poll on LinkedIn which suggested the Sainsbury's boss was right...

Asda's chief financial officer Michael Gleeson said last week the technology had reached its limit - and said his firm would be putting more staff on tills.

Northern grocer Booths ditched almost all self-checkouts last year amid customer service concerns.

Over at Marks & Spencer, chairman Archie Norman last year blamed self-checkouts for a rise in "middle-class shoplifting".

But Tesco CEO Ken Murphy is an advocate: "We genuinely believe, at the end of the day, it provides a better customer experience."

The number of drivers visited by bailiffs due to unpaid traffic fines has increased substantially, according to a report.

Four million penalty charge notices (PCNs) were referred to bailiffs in England and Wales in the 2023-24 financial year, it is claimed.

This is up from 2.4 million during the previous 12 months, 1.9 million in 2019-20 and 1.3 million in 2017-18.

Read more ...

Ted Baker is the latest in a string of high-street giants to call in administrators in recent years, with shops set to disappear this week.

But how does it affect you? 

Purchases and returns

You can still buy items online and in store until they close, but you could run into trouble returning them. 

If the retailer stops trading, it may not be able to get your money back to you.

If that is the case, you would have to file a claim with Teneo (Ted Baker's administrator) to join a list of creditors owed money by Ted Baker – and even then there's no guarantee you'd get your money back.

If you have a gift card, you need to use it while you still can.

Credits and debits

You can file a claim with your debit or credit card provider to recover lost funds - but how exactly does that work?

• Credit card:  If you bought any single item costing between £100-£30,000 and paid on a credit card, the card firm is liable if something goes wrong. If any purchase was less than £100, you may still be able to get your money back via chargeback;
• Debit card:  Under chargeback, your bank can try to get your money back from Ted Baker's bank. However, be aware that this is not a legal requirement and it can later be disputed and recalled.

Many retailers boosted wages after living wage/minimum wage changes in spring.

Figures show German discount brands Aldi and Lidl top the list of major UK supermarkets when it comes to staff hourly pay - after Lidl introduced its third pay increase of the year in May to match its closest rival.

Meanwhile, Morrisons is at the bottom of the pack for staff pay outside London, with hourly wages starting at the National Living Wage (£11.44).

How do other companies compare when it comes to pay and benefits? We've taken a look...

Pay: £12.40 an hour outside London and £13.65 inside the M25

Aldi announced in March it was bringing in its second pay rise of the year as part of its aim to be the best-paying UK supermarket.

From 1 June, hourly pay rose from £12 an hour to £12.40 outside the M25 and £13.55 to £13.65 in London. 

Aldi is one of the only supermarkets to give staff paid breaks. It also offers perks such as discounted gym membership and cinema tickets, and financial planning tools. However, there are no cheaper meals, staff discounts or bonus schemes.

Pay:  £12.04 an hour outside London and £13.21 inside the M25

As of 1 July, hourly wages for Asda supermarket staff rose to £12.04 per hour from £11.11, with rates for London staff also going up to £13.21.

As part of the July changes, Asda brought in the option for free later-life care or mortgage advice. The company also offers a pension and a free remote GP service.

Pay:  £12 an hour outside London and £13.15 inside the M25

Co-op boosted its minimum hourly wage for customer team members from £10.90 to £12 nationally as the national living wage rose to £11.44 in April.

For staff inside the M25, rates rose from £12.25 to £13.15.

The perks are better than some. Workers can get 30% off Co-op branded products in its food stores as well as 10% off other brands. Other benefits include a cycle to work scheme, childcare vouchers and discounts on its other services.

Pay:  £11.50 an hour outside London and £12.65 inside the M25

Iceland says it pays £11.50 for staff aged 21 and over - 6p above the minimum wage. Employees in London receive £12.65 per hour.

Staff are also offered a 15% in-store discount, which was raised from 10% in 2022 to help with the cost of living.

The firm says it offers other perks such as a healthcare scheme and Christmas vouchers.

Pay:  £12.40 an hour outside London and £13.65 inside the M25

From June, Lidl matched its rival Aldi by raising its hourly wage to £12.40 for workers outside the M25 and £13.55 for those inside.

Lidl also offers its staff a 10% discount card from the first working day, as well as other perks such as dental insurance and fertility leave. 

Marks and Spencer's hourly rate for store assistants was hiked from £10.90 to £12 for staff outside London and from £12.05 to £13.15 for London workers from April.

The grocer also offers a 20% staff discount after the probation period as well as discretionary bonus schemes and a free virtual GP service.

Pay:  £11.44 an hour outside London and £12.29 inside the M25

Along with many other retailers, Morrisons increased the hourly wage for staff outside the M25 in line with the national living wage of £11.44 in April.

Employees in London receive an 85p supplement.

While it's not the most competitive for hourly pay, Morrisons offers perks including staff discounted meals, a 15% in-store discount and life assurance scheme.

Sainsbury’s

Sainsbury's hourly rate for workers outside London rose to £12 from March, and £13.15 for staff inside the M25.

The company also offers a 10% discount card for staff to use at Sainsbury's, Argos and Habitat, as well as a range of benefits including season ticket loans and long service rewards.

Pay:  £12.02 an hour outside London and £13.15 inside the M25

Since April, Tesco staff have been paid £12.02 an hour nationally - up from £11.02 - while London workers get £13.15 an hour.

The supermarket giant also provides a 10% in-store discount, discounted glasses, health checks and insurance, and free 24/7 access to a virtual GP.

Staff get their pay boosted by 10% on a Sunday if they joined the company before 24 July 2022.

Pay:  £11.55 an hour outside London and £12.89 inside the M25

Waitrose store staff receive £11.55 an hour nationally, while workers inside the M25 get at least £12.89.

Staff can also get access to up to 25% off at Waitrose's partner retailer John Lewis as well as 20% in Waitrose shops. 

JLP (the John Lewis Partnership) gives staff a bonus as an annual share-out of profit determined by the firm's performance. In 2021-22 the bonus was 3% of pay; however, it has not paid the bonus for the past two years.

Dozens of Ted Baker stores will shut for the last time this week amid growing doubts over a future licensing partnership with the retail tycoon Mike Ashley.

Sky News understands that talks between Mr Ashley's Frasers Group and Authentic, Ted Baker's owner, have stalled three months after it appeared that an agreement was imminent.

Administrators are overseeing the closure of its remaining 31 UK shops.

One store source said they had been told that this Tuesday would be the final day of trading.

The housing market experienced a surge in activity following the Bank of England's recent decision to cut interest rates, according to a leading property website.

Estate agents reported a 19% jump in enquiries about properties for sale after 1 August, when compared with the same period last year, research by Rightmove found.

It came after the Bank cut rates for the first time in more than four years from 5.25% to 5%.

The lead negotiator for major train union ASLEF has denied the union sees the new government as a "soft touch" after announcing fresh strikes two days after train drivers were offered a pay deal.

Drivers working for London North Eastern Railway will walk out on weekends from the end of August in a dispute over working agreements.

Lead negotiator Nigel Roebuck said it is a separate issue from the long-running row over pay, which looks likely to be resolved after a much-improved new offer from the government.

Over 40 bottles of fake vodka have been seized from a shop in Scotland after a customer reported "smelling nail varnish".

The 35cl bottles, fraudulently labelled as the popular brand Glen's, were recovered from the shop in Coatbridge, North Lanarkshire.

Officers from the council's environmental health officers and Food Standard Scotland (FSS) sent them for analysis after a customer raised the alarm by saying they smelt nail varnish from one of the bottles.

The bottles were found to be counterfeit.

Britons don't have long left to claim cost of living assistance from the Household Support Fund.

Introduced in October 2021, the scheme provides local councils with funding which can be used to support those struggling most with the rising cost of living.

The vast majority of councils operate their version of the Household Support Fund on a "first come, first serve" basis and will officially end the schemes once the funding has run out in September.

The help provided by councils has ranged from free cash payments, council tax discounts, and vouchers for supermarkets and energy providers.

Who is eligible?

Local authorities were instructed to target the funding at "vulnerable households in most need of support to help with significantly rising living costs" when it was first rolled out.

In particular, councils were guided to make priority considerations for those who: 

• Are eligible but not claiming qualifying benefits;
• Became eligible for benefits after the relevant qualifying dates;
• Are receiving housing benefit only;
• Are normally eligible for benefits but who had a nil award in the qualifying period.

If you do not meet these criteria, you can still contact your local council , with many having broadened their criteria for eligibility.

By Daniel Binns, business reporter

Weapons maker BAE Systems is the big loser on the FTSE 100 this morning, with its shares down almost 3% in early trading.

It comes following reports over the weekend that the German government is planning to scale back aid to Ukraine in its war with Russia – in what would be a blow to the arms industry.

German media said ministers are set to slash support for Kyiv to 6% of current levels by 2027 in their upcoming budget.

However, the government there has rejected the reports and has denied it is "stopping support" to Ukraine.

Whatever the truth, the reports appear to have spooked traders.

Other companies involved in the defence sector, including Rolls-Royce Plc and Chemring Group, are also down more than 2% and 1% respectively on Monday.

It comes amid a slight slump in early trading, with the FTSE 100 down just over 0.2%, although the FTSE 250 is up 0.07%.

Gainers this morning include housebuilders Barratt Developments, up 1.5%, and Redrow Plc, which is up almost 3%.

Barratt said today it intends to push ahead with a planned £2.5bn merger with its rival despite concerns from the competition regulator.

Meanwhile, the price of oil is down amid concerns of weaker demand in China.

Ongoing ceasefire talks in the Israel-Hamas conflict have also raised hopes of cooling tensions in the Middle East, which would help ease supply risks and worries.

A barrel of the benchmark Brent Crude is currently priced at just over $79 (£61).

On the currency markets, this morning £1 buys $1.29 US or €1.17.

Winter energy bills are projected to rise by 9%, according to a closely watched forecast.

The price cap from October to December will go up to £1,714 a year for the average user, Cornwall Insight says.

It would be a £146 rise from the current cap, which is controlled by energy regulator Ofgem and aims to prevent households on variable tariffs being ripped off.

The cap doesn't represent a maximum bill. Instead it creates an average bill by limiting how much you pay per unit of gas and electricity, as well as setting a maximum daily standing charge (which all households must pay to stay connected to the grid).

Ofgem will announce the October cap this Friday.

"This is not the news households want to hear when moving into the colder months," said the principal consultant at Cornwall, Dr Craig Lowrey.

"Following two consecutive falls in the cap, I'm sure many hoped we were on a steady path back to pre-crisis prices. 

"However, the lingering impact of the energy crisis has left us with a market that's still highly volatile and quick to react to any bad news on the supply front.

"Despite this, while we don't expect a return to the extreme prices of recent years, it's unlikely that bills will return to what was once considered normal. Without significant intervention, this may well be the new normal."

Cornwall Insight warned that the highly volatile energy market and unexpected global events, such as the recent escalating tensions in the Russia-Ukraine war, could see prices rise further at the start of the new year.

To avoid this vulnerability, Cornwall Insight said domestic renewable energy production should increase and Britain should wean itself off energy imports.

Kellogg's appears to have shrunk its packets of Corn Flakes. 

Two of its four different pack sizes have reduced in weight by 50g, according to The Sun. 

What used to be 720g boxes are now 670g, while 500g boxes have become 450g. 

The newspaper says the 670g boxes are being sold for £3.20 in Tesco - the same price customers were paying for the larger box back in May. 

The 450g boxes are being sold for £2.19, only slightly less than the previous price of £2.25.

Other supermarkets have similar pricing, although in Morrisons the price has gone down in proportion to the size reduction.

The 250g and 1kg pack sizes remain unchanged. 

Kellogg's has said it is up to shops to choose what they charge, but Tesco said the manufacturer should comment on pricing. 

Sky News has contacted Kellogg's for comment.

A spokesperson is quoted by The Sun: "Kellogg's Corn Flakes are available in four different box sizes to suit different shopper preferences and needs. 

"As the cost of ingredients and production processes increase, it costs us more to make our products than it used to.

"This can impact the recommended retail price. It's the grocer's absolute discretion and decision what price to charge shoppers."

Be the first to get Breaking News

Install the Sky News app for free

• Skip to main content
• Skip to search
• Skip to footer

Products and Services

Configuration engineer

Ben Harting

"I decided to make a career change when I was twenty-four. I chose to pursue Cisco Certifications because I knew it would put me in the best position to start a career in networking."

How it all began

Certifications can improve your ability to do your job

My dad is a systems administrator, so I grew up around computers. Just watching him tinker with them was cool. However, when I went to college, I earned a bachelor’s degree in philosophy. Next, I got a teaching credential, then taught elementary school for a few years. But when I turned 24, I decided I wanted a career change. So, I picked up a book on TCP/IP networking, started studying it, and realized that I really enjoyed it. My dad wasn’t into networking that much, so I didn’t know much about it, and got into it more organically. I considered pursuing Cisco certifications right away since the idea of learning about networking led straight to these credentials. I took a networking class at a local community college, then obtained my Cisco CCNA certification.

I enjoy system administration, I’ve gained communications skills, and I’ve learned tech skills, in Microsoft active directory systems, administration, and networking virtualization. General troubleshooting is a big part of my job, too. Most importantly, I’ve gained the flexibility to work around difficulties and learn new technologies. Just being able to keep up with the constant change in technologies is incredibly valuable.

Certifications

Columbus, OH

Reading Hiking Learning new tech

What does having a Cisco Certification mean to you?

"I’ve gained more knowledge and different skill sets. I’ve opened myself up for more opportunities. And it’s a validation of me and what I’ve learned."

The biggest challenge was where to start, there where a lot of options. A lot of them involved going back to school or spending a lot of money. CCNA was the most cost-effective, and it would put me in the best position to start a career. Cisco is a leader in that. I took a bootcamp course at the local community college that lasted 8 weeks in total.

I was already in my job in IT in tier 1 support, help desk and the CCNA Certification was a catalyst for my first promotion as a second tier support security.

What would you tell your younger self?

Definitely try different things and go all in on whatever interests you the most.

What would you tell a friend?

I’d tell them to go for it. It will open up a lot of opportunities for them. They’ll have all the knowledge they need to go forward in their career.

Ben's journey

Career path.

Current role

Configuration engineers work on systems and network administration.

Previous role

IT Help Desk Support Primary School Educator

Certification path

Most recent certification

Cisco Certified Network Associate (CCNA) certification is the first step toward a career in IT Networking. The CCNA exam covers networking fundamentals, IP services, security fundamentals, automation and programmability.

Read more certification success stories

"The guidance I gained from earning the certification helped— in near real time—to determine what was happening on the job, when I became an SOC operator."

Network security analyst CyberOps Associate

Yasser Auda

"You can do anything. You just need to decide to do it, have the will to do it, and never give up. Be confident in yourself and stop the barriers in your mind."

Network security architect CCNA, CCNP Enterprise, CCNP Security, CyberOps Associate, CCIE Enterprise Infrastructure, Cisco Certified DevNet Associate, Cisco Certified DevNet Professional

Olivia Wolf

"The knowledge that I’ve got from studying for those certifications gave me the confidence that I’ll always be able to get a job if I need to."

Systems engineer CCNA, CCNP Enterprise, DevNet Associate

Share your Cisco Certification Success Story

Has earning a Cisco certification positively changed your life or career, or both? Do you think your Cisco certification story would help encourage other people to earn their Cisco certification? If so, we want to talk to you!