writing a medical literature review

Workshop: Literature Reviews- What you need to know

• How to write a literature review

Useful Titles for All of your Boxes

The Seven Steps to Producing a Literature Review:

1. Identify your question

2. Review discipline style

3. Search the literature

4. Manage your references

5. Critically analyze and evaluate

6. Synthisize

7. Write the review

• University of North Carolina Writing Center "How To" UNC Chapel Hill Writing Center "How To" on writing a literature review.
• Purdue Owl Purdue Owl help on writing a literature review

Feedback and Additional Information

• Library Workshop Evaluation Brief survey for workshop participants. Your feedback is greatly appreciated.
• TTU Library Workshops Additional information about TTU Library Workshops. All workshops are taught by Personal Librarians in Instruction Lab 150 unless otherwise noted.
• Last Updated: Aug 27, 2024 8:06 AM
• URL: https://guides.library.ttu.edu/litreview

Loading metrics

Open Access

Peer-reviewed

Research Article

Perspectives on conducting “sex-normalising” intersex surgeries conducted in infancy: A systematic review

Roles Data curation, Formal analysis, Investigation, Methodology, Project administration, Writing – original draft, Writing – review & editing

* E-mail: [email protected] , [email protected]

Affiliation Department of Public Health and Primary Care, University of Cambridge, Cambridge, United Kingdom

Roles Data curation, Formal analysis, Investigation, Methodology, Writing – review & editing

Affiliation Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia

Roles Writing – review & editing

Affiliation Department of Gender, Rights and Equity, World Health Organization, Geneva, Switzerland

Affiliation Department of Mental Health and Substance Use, World Health Organization, Geneva, Switzerland

Affiliation Department of Sexual and Reproductive Health and Research, UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), World Health Organization, Geneva, Switzerland

Roles Conceptualization, Formal analysis, Supervision, Writing – review & editing

Roles Conceptualization, Data curation, Formal analysis, Supervision, Writing – review & editing

• Luke Muschialli, 
• Connor Luke Allen, 
• Evelyn Boy-Mena, 
• Aiysha Malik, 
• Christina Pallitto, 
• Åsa Nihlén, 
• Lianne Gonsalves

• Published: August 28, 2024
• https://doi.org/10.1371/journal.pgph.0003568
• Reader Comments

Children with intersex variations continue to be subject to elective, irreversible, “sex-normalising” surgical interventions, despite multiple human rights and legislative bodies calling for their prohibition. Our systematic review aims to understand how medical literature reports rationales for “sex-normalising” surgical interventions conducted in childhood, and how they are contextualised within the medical and social controversy surrounding such interventions. PubMed, EMBASE and CINAHL were searched for English language, peer-reviewed articles reporting primary data on elective, genital, “sex-normalising” surgical interventions conducted on individuals <10 years, published 01/07/2006-30/06/2023 (PROSPERO ID: CRD42023460871). Data on outcomes reported, rationale for the conduct and timing of interventions and acknowledgement of controversy were extracted. Narrative synthesis described rationales and controversy. Risk of bias was assessed using Johanna Briggs Institute Tools. 11,042 records were retrieved, with 71 articles included for analysis. One of the most common outcomes collected in included literature were cosmetic outcomes, primarily reported by surgeons or parents. 62.0% of studies reported no rationale for intervention timing, 39.4% reported no rationale for conduct and 52.1% acknowledged no controversy in intervention conduct. Rationales included parental desire for intervention, anatomical/functional/cosmetic reasons, and a perceived goal of aligning with sex assigned by surgical teams or parents. Controversies addressed included concerns about the quality of interventions, the ethics of intervention conduct and gendered and social considerations. “Sex-normalising” interventions are conducted based largely on rationales that were not adequately supported by evidence, a desire from parents and surgeons to match genital cosmesis typically ascribed to male and female bodies, and a parental desire for intervention conduct. Legislating and medical regulatory bodies should advocate for ending the conduct of irreversible, elective, “sex-normalising” interventions conducted without the full, free and informed consent of the person concerned, to promote and protect the highest attainable standard of health for people with intersex variations.

Citation: Muschialli L, Allen CL, Boy-Mena E, Malik A, Pallitto C, Nihlén Å, et al. (2024) Perspectives on conducting “sex-normalising” intersex surgeries conducted in infancy: A systematic review. PLOS Glob Public Health 4(8): e0003568. https://doi.org/10.1371/journal.pgph.0003568

Editor: Sualeha Siddiq Shekhani, Centre of Biomedical Ethics and Culture, PAKISTAN

Received: May 14, 2024; Accepted: July 26, 2024; Published: August 28, 2024

Copyright: © 2024 Muschialli et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: This review was conducted with the financial support of the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), a cosponsored programme executed by the World Health Organization (WHO) to LM, CLA, LG and AN. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Intersex is an umbrella term used to refer to individuals born with physical or biological sex characteristics (including sexual anatomy, reproductive organs and/or chromosomal patterns) that do not fit normative definitions of male or female bodies [ 1 ]. These congenital variations in sex characteristics, also known as differences/disorders of sex development (DSDs), are a large, heterogeneous group of reproductive, urogenital, chromosomal and/or hormonal congenital conditions, with the set of diagnoses lying within these umbrella terms varying across different settings and disciplines [ 2 , 3 ]. Some of these variations may be visible and identifiable at birth, while others may not be recognized until later in childhood, puberty and/or adulthood [ 1 ].

Many congenital variations in sex characteristics pose no threat to physical health, while others may necessitate medical or surgical intervention (i.e., to facilitate excretion or urination). However, standard surgical practice for decades has involved using a variety of elective (i.e., non-urgent) surgical procedures to attempt to ‘normalize’ ‘atypical’ traits in people with congenital variations in sex characteristics, striving for cosmetic, functional and anatomical outcomes that align more with those associated with ‘typical’ male or female bodies [ 4 ], referred to in this manuscript as “sex-normalising” interventions. These procedures frequently take place in infancy or early childhood, due in part to pervasive beliefs that early intervention will facilitate better psychological and/or physiological development, and alleviate anticipated personal and parental stress associated with the congenital variation in sex characteristics [ 5 – 8 ]. Treatment options may also involve hormonal therapy, mechanical procedures (e.g., neovaginal dilations), or surgical, hormonal, and mechanical interventions which take place in adolescence, all of which are outside the scope of this review.

The continued conduct of “sex-normalising” surgical interventions in infancy or early childhood is supported by clinical guidance such as the 2006 Consensus Statement on Management of Intersex Disorders , a statement that attempted to create recommendations for the long-term management, evaluation of, and future research into, congenital variations in sex characteristics [ 9 ]. This is despite a noted lack of evidence regarding the physical, mental, and social outcomes of infants with congenital variations in sex characteristics who have undergone such interventions, with an extant evidence-base primarily consisting of short case reports and cohort data with small sample sizes [ 2 , 4 ].

Evidence on patient preference for infant surgical intervention, despite still being used as a rationale for procedures, is also inconsistent. Although some patients retrospectively express a clear preference for, and satisfaction with, early surgical intervention [ 10 – 12 ], others who have undergone these surgeries share dissatisfaction with long-term outcomes, citing resultant distress and trauma associated with subsequent gender identity, compromised sexual function and pleasure, dissatisfaction with genital appearance and a reflective distress surrounding their compromised autonomy for interventions practiced before they were able to articulate consent for the procedure [ 13 – 23 ].

Human rights experts have recently raised serious concerns about the conduct of elective, irreversible, “sex-normalising” surgical interventions carried out in infancy and childhood. In October 2016, multiple UN human rights monitoring and accountability mechanisms (The international human rights treaty bodies and special procedures behind this joint statement were the UN Committee on the Rights of the Child, UN Committee against Torture, UN Committee on the Rights of People with Disabilities, UN Sub-Committee on Prevention of Torture and other Cruel, Inhuman or Degrading Treatment or Punishment, UN Special Rapporteur on torture and other cruel, inhuman or degrading treatment or punishment, UN Special Rapporteur on the right of everyone to the highest attainable standard of health, UN Special Rapporteur on violence against women, its causes and consequences, Special Representative of the UN Secretary-General on Violence against Children, African Commission on Human and Peoples’ Rights, Council of Europe Commissioner for Human Rights and the Inter-American Commission on Human Rights.) issued a joint statement highlighting the human rights violations associated with ‘medically unnecessary surgeries […] in an attempt to forcibly change [intersex infants’, children’s and adolescents’] appearance to be in line with societal expectations about female and male bodies’ and called on governments to ’prohibit harmful medical practices on intersex children, including unnecessary surgery and treatment without their informed consent’ [ 24 ]. Following this, several UN human rights treaty bodies, including the UN Committee on the Rights of the Child [ 25 ], as well as the World Health Organization (WHO) [ 26 ], have provided further advice, speaking out against medically unnecessary surgical interventions for infants with congenital variations in sex characteristics on the grounds of compromised bodily integrity and rights of the child. Recently, and for the first time, the UN Human Rights Council adopted a resolution calling on countries to enhance efforts to combat discrimination, violence and harmful practices against people with congenital variations in sex characteristics [ 27 ]. Simultaneously, intersex community groups have extensively campaigned against “sex-normalising” interventions on the same grounds, with some suggesting that the existence of such interventions also represent an elimination of intersex communities and an enforcement of strongly normative conceptualizations of gender and sex [ 28 ]. The collective impact of advocacy groups and human rights monitoring bodies has been reflected in a growing wave of national and sub-national legislative change restricting the practice of elective genital surgery conducted on children, focusing on the protection of children’s physical integrity and the prohibition of surgical intervention for infants and children with congenital variations in sex characteristics too young to participate in decision-making [ 29 ].

Both a 2016 Global Disorders of Sex Development Update since 2006 publication and a 2018 European publication, Caring for Individuals with a Difference of Sex Development , emphasise the need for clinicians to consider the ethical implications of conducting “sex-normalising” surgical interventions, and discuss the postponing of genital surgery until an individual is old enough to understand the intervention and participate in decision-making [ 2 , 4 ]. Despite this, there is little evidence to suggest that the prevalence of, or rationales for conducting, “sex-normalising” surgical interventions on individuals too young to consent has changed in recent years [ 30 , 31 ]. Understanding the rationale behind the continued practice of “sex-normalising” surgical interventions and how clinicians acknowledge and contextualise the controversial nature of the interventions they are practicing is necessary for identifying misconceptions in practice, and designing policy and interventions that target common drivers of continued, early surgical procedures.

Our review aims to understand the extent to which the extant medical literature is reporting rationales for elective, “sex-normalising” surgical interventions conducted in infancy and childhood, what these rationales are, and how they are contextualised within the social, medical, and political controversy surrounding “sex-normalising” interventions for people with congenital variations in sex characteristics.

Search and screening

Our review was compliant with PRISMA Reporting Guidelines [ 32 ] (see S1 text ), and is registered with PROSPERO (CRD42023460871).

On 30/06/2023, PubMed, EMBASE and CINAHL were searched for relevant articles published between 01/07/2006-30/06/2023, using search terms derived from previous reviews, recommendations from field experts and terminology from the International Classification of Diseases, Version 11 (ICD-11). The search start date aligns with the year of publication of the 2006 Consensus Statement on Management of Intersex Disorders [ 9 ].

S2 Text reports our search strategy. One search filter captured medical and public health terms that covered intersex as an umbrella term as well as individual congenital variations in sex characteristics that are generally identified in infancy or early childhood (as opposed to during puberty or adulthood). A second search filter captured general surgical terms as well as specific procedures associated with genital surgery for persons with congenital variations in sex characteristics [ 4 ]. English language restrictions were applied due to the risks of losing the complex nuances of surgical rationale in translation. A final search filter attempted to exclude: animal studies; commentaries, letters, and editorials; studies covering gender-affirming care surgeries; studies focusing on adolescence or young people over the age of 10 years.

Title/abstract and full-text screening was carried out by LM and CLA. LG independently screened a random sample of 10% of full texts to ensure consensus between reviewers. Screening was conducted in the systematic review management software, Covidence [ 33 ].

Inclusion and exclusion criteria

Peer-reviewed journal articles reporting primary data on elective, genital, “sex-normalising” surgical interventions conducted before the age of 10 years were included. This age cut-off was selected to align with the WHO’s definition of adolescence (ages 10–19 years) [ 34 ]. Inclusion decisions regarding surgical interventions were guided by the Gardner and Sandberg terminology of ‘urgent’ vs ‘elective’ surgery, with ‘urgent surgeries’ referring to those "performed promptly to avoid life-threatening circumstances or to prevent permanent disability”, while ‘elective surgeries’ include “those that address non-urgent issues” [ 35 ].

The following exclusion criteria were developed:

• Malignancy . Articles that cited malignancy as their rationale for conducting surgeries were excluded. Some individual congenital variations in sex characteristics are associated with increased risk of germ cell tumours, which is sometimes cited as a rationale for surgical intervention, despite the general malignancy risk associated with congenital variations in sex characteristics being debated [ 36 , 37 ]. However, for this review, it was thought that the potential for malignancy as perceived by clinicians and presented to parents/guardians, would, in the eyes of concerned parties, change the suggested surgery from being considered ‘elective’ to ‘urgent’. Whether surgical intervention due to the potential risk of malignancy was recommended for the specific congenital variation in sex characteristics in the 2006 Consensus Statement on Management of Intersex Disorders [ 9 ], which was the last statement to tabulate malignancy risk by individual variation, was recorded, and addressed in the discussion.
• Urinary incontinence . Studies citing preserving urinary continence, or studies citing functional urological goals (e.g., removing urethral obstruction), as their rationale were excluded, as these interventions were deemed to be non-elective [ 35 ]. Studies whose stated urological goals were intended to bring an individual in line with a perceived normative social behaviour (e.g., surgery to allow a boy to urinate standing up rather than sitting down) were included.
• Age. Studies in which it was unclear if all participants were under 10-years-old (i.e., age range was not provided), or studies which reported outcomes for a combination of individuals aged above and below 10-years-old without segregating outcomes, were excluded.
• No patient outcome data . Studies that collected no patient outcomes (e.g., studies focusing on surgical equipment use, histological studies) were excluded.
• Hypospadias, epispadias or cryptorchidism . Studies addressing hypospadias, epispadias or cryptorchidism that did not present these variations as congenital variations in sex characteristics or did not investigate them as part of an established congenital variation in sex characteristics were excluded due to debate among the medical community over their status as congenital variations in sex characteristics.
• Primary focus not “sex-normalising” surgical interventions. Studies that reported patients who had previously received a “sex-normalising” intervention but did not report the outcomes of or rationale for that surgery were excluded.
• Non-intersex populations . Studies that reported interventions that could be classed as “sex-normalising” but were administered to patients both with and without (e.g., transgender and gender diverse patients) congenital variations in sex characteristics without segregating outcomes for individuals with congenital variations in sex characteristics, were excluded.
• Reviews . Relevant systematic and scoping reviews were excluded but studies included in the reviews were screened against our inclusion criteria as part of a secondary search strategy.

Data extraction

Data extraction was conducted in Covidence [ 33 ], using a data-extraction table constructed by LM. LM and LG independently extracted, with CLA extracting a random sample of 25% of studies. In addition to bibliometric data (e.g., author, year of publication), the following data were extracted:

• Surgical outcomes . In line with the recommendations for data collection on long-term surgical outcomes for individuals with congenital variations in sex characteristics in the 2018 publication Caring for Individuals with a Difference of Sex Development [ 2 ] , we recorded if studies reported: complication rates, success rates, cosmetic outcomes, quality of life, sexual functioning, urogenital outcomes, sexuality/gender assignment, re-evaluation, fertility outcomes, social and psychosexual adjustment, mental health, social participation and parental outcomes. The precise and detailed categorisation of long-term outcome reporting used in the 2018 publication was deemed most appropriate for tabulating outcomes, rather than the 2006 Consensus Statement on Management of Intersex Disorders [ 9 ] or the 2016 Global Disorders of Sex Development Update since 2006 [ 4 ].
• Rationale . We recorded whether studies provided a rationale for conducting the surgical intervention, as well as rationale for the timing of the surgery. Rationales were extracted verbatim, alongside any work cited by authors when justifying rationale.
• Acknowledgement of controversy . We extracted verbatim text from studies that acknowledged the social, political, cultural, medical, ethical and/or surgical controversy surrounding the intervention they were conducting, as well as any work cited by authors when discussing this controversy.

Quality assessment was conducted independently by LM and CLA using the Joanna Briggs Institute’s instruments for Quality Assessment [ 38 ], with consensus reached through discussion. As no syntheses of outcomes were conducted, and rationale and controversy reporting were deemed to be independent of manuscript quality, manuscripts were not excluded on the basis of low quality.

Data analysis

Descriptive statistics were used to present the number of studies reporting each of the aforementioned outcomes. Qualitative, verbatim data on rationale and controversy was analysed using a narrative synthesis with an inductive approach. LM, LG and AN independently reviewed data to identify codes which addressed commonly occurring themes. LM, LG, and AN met to discuss and agree upon these high-level themes, after which LM independently coded all articles, discussing and refining emerging subthemes with LG and AN.

Throughout the narrative synthesis, the names of congenital variations in sex characteristics used by study authors will be used to avoid unintentional change in diagnosis through misinterpretation by the authorship team. These terms may not always be the accepted term within the intersex or medical communities.

Justifications and rationales for surgical intervention are presented in the narrative synthesis as presented by study teams, regardless of the accuracy of the assumptions stated. The presentation of such rationales does not represent an endorsement of their validity, and the accuracy of stated rationales is explored in the discussion.

Our search yielded 11,042 records, with 6,963 undergoing title/abstract screening after de-duplication. Following full-text screening of 1,649 records, 70 met all inclusion criteria and underwent data extraction. One further study was identified from searching systematic, scoping and literature reviews identified from the search ( Fig 1 ). S2 Text details the full search results from each database.

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pgph.0003568.g001

Of the 71 studies in the final sample, 73.2% were case studies or case series (n = 52) and 26.8% were cohort studies (n = 19). Countries publishing included studies were India (n = 11; 15.5%), United States of America (n = 7; 9.9%), Brazil (n = 6; 8.5%), Türkiye (n = 5, 7.0%), Japan (n = 4; 5.6%), Algeria (n = 3; 4.2%), United Kingdom of Great Britain and Northern Ireland (UK), Netherlands, Pakistan, Romania, Egypt, Italy (n = 2; 2.8%, respectively), Sweden, Portugal, Republic of Korea, Iran (Islamic Republic of), Canada, Saudi Arabia, Spain, Finland, France, Germany, Indonesia, Argentina, Uganda, Nigeria, China, Poland, North Macedonia, Australia, Iraq, Belgium, Colombia, Kuwait, Syria, Bangladesh, Switzerland and Uzbekistan (n = 1; 1.4%, respectively). Included studies investigated a range of congenital variations in sex characteristics, including 29.6% of studies reporting on congenital adrenal hyperplasia (CAH, n = 21), 15.5% on diphallia and ovotesticular DSD (n = 11, respectively), 8.5% on persistent Mullerian duct syndrome (PMDS, n = 6), 5.6% on mixed gonadal dysgenesis (MGD, n = 4) and 4.2% Mayer-Rokitansky-Küster-Hauser (MRKH) syndrome and partial androgen insensitivity syndrome (PAIS, n = 3, respectively). Table 1 reports study characteristics. Quality of studies was generally low, with limited description of patient demographics, surgical interventions, and outcomes ( S1 Table ).

https://doi.org/10.1371/journal.pgph.0003568.t001

Outcomes measured in the literature

Fig 2 is a representation of the outcomes reported in included articles, categorised in line with the recommended reporting outcomes in the 2018 publication Caring for Individuals with a Difference of Sex Development [ 2 ]. Fig 2 also indicates which articles reported rationales for the conduct or timing of surgery, and those that acknowledged controversy. The most common outcomes collected in included literature were cosmetic outcomes, primarily reported by surgeons or parents, complication rates and urogenital outcomes. There was a notable lack of validated measures used by studies when assessing any surgical outcome, particularly cosmetic, and no studies collected any data on child quality of life or parental outcomes (i.e., parental mental health).

https://doi.org/10.1371/journal.pgph.0003568.g002

Rationales provided for conducting “sex-normalising” procedures and their timings

Four codes were inductively identified to categorise the rationales for “sex-normalising” procedures: Anatomical/Functional/Cosmetic (used in 50.7% of studies for either conduct or timing of intervention; n = 36), Perceived Goal of Aligning with Assigned Sex (39.4%; n = 28), Parental Desire (19.7%; n = 14), Belief of Best Practice (9.9%; n = 7) and Access (1.4%; n = 1). 33.8% of studies provided no rationale for the timing or conduct of “sex-normalising” interventions (n = 24).

Anatomical/functional/cosmetic.

Studies providing anatomical/functional/cosmetic reasons as a rationale for the timing or conduct of procedures investigated CAH (n = 17), diphallia, MGD (n = 3, respectively), PMDS, MRKH syndrome, Herlyn-Werner-Wunderlich syndrome, unspecified DSDs, PAIS (n = 2, respectively), genital ambiguity with high vaginal confluence, true hermaphroditism, genital ambiguity, 5-alpha reductase deficiency, ovotesticular syndrome, ovotesticular DSD, perineal hypospadias in DSD, persistent cloaca and 46, XX DSD (n = 1, respectively).

Rationale for conducting procedure . Cosmetic rationalisations for intervention were primarily based around the goal of achieving a ‘normal’ external genital appearance, corresponding with the sex proposed for children by the surgical team or parents [ 5 – 8 , 39 – 57 ]. This was sometimes done through critically presenting ambiguous genitalia, labelling them as ‘ugly’ [ 44 ] or something to ‘correct’ [ 53 ]. Cosmetic goals were generally unspecific, with authors striving for cosmetic results that were ‘satisfactory’ [ 54 , 57 ], ‘aesthetically pleasing’ [ 49 ], ‘normal’ [ 8 , 50 , 51 ], ‘good’ [ 52 ], ‘natural-looking’ [ 47 ], ‘favourable’ [ 55 ] or ‘typical’ [ 56 ]. Improved cosmesis, determined primarily by parents or surgeons ( Fig 2 ), was perceived by surgical teams to be linked to improved psychosocial outcomes such as anticipated improved parent-child attachment, reduced stigma or psychological distress, or perceived improvements in developments of gender identity [ 6 , 43 ].

Anatomically, surgical interventions were proposed as a way to prevent perceived incorrect virilisation of children and to align genitalia with the perceived ‘typical’ anatomy of the sex proposed by parents or surgeons [ 5 , 7 , 43 , 58 , 59 ]. Studies often articulated this by proposing interventions as a way of resituating anatomical structures to their ‘typical’ position [ 54 , 60 ]. As with cosmetic outcomes, goals were unspecific [ 61 ], with some teams describing the results they were striving for as ‘satisfactory’ [ 8 , 49 , 50 , 62 ]. Preventing downstream complications associated with ambiguous genitalia, such as endometriosis in Herlyn-Werner-Wunderlich Syndrome [ 63 ], was also cited as anatomical rationale for intervention conduct. Anatomical justifications were also used to select and justify the type of procedure being offered, such as one surgical team electing for ‘conservative’ approaches to minimise tissue removal for children who do not yet have definitive gender identity [ 64 ]. There was a notable impression by some authors that anatomical goals were secondary to aforementioned cosmetic rationales [ 50 ].

Many functional rationales were unspecific and subjective, including achieving ‘satisfactory’ [ 57 ], ‘good’ [ 8 , 52 ], ‘proper’ [ 40 ], ‘typical’ [ 56 ], ‘favourable’ [ 55 ] or ‘desirable’ [ 53 ] functional outcomes. More specific functional goals included achieving sexual function, including the prospect of ‘successful’ future intercourse (with the implication that sexual intercourse constituted penis-in-vagina, penetrative sex) [ 42 , 45 , 48 ] and achieving sexual arousal [ 5 , 41 , 44 , 50 ], menstruation [ 5 , 42 , 43 ] and protection of fertility [ 5 , 62 ]. Follow-up times for studies were often not long enough to be able to reliably confirm these functional goals ( Table 1 ). One study suggested a recent shift in prioritisation by global surgical teams towards protecting sexual satisfaction and sensate genital tissues in interventions [ 44 ]. There were also notable instances in which authors recognised uncertainty surrounding the functional impact of interventions, with articles suggesting post-intervention genitalia may be ‘more natural looking, and possibly functioning [emphasis added]’ [ 47 ] and that interventions ‘ may facilitate sexual intercourse [emphasis added]’ [ 45 ]. Similar to anatomic rationales, some authors implied functional outcomes were secondary to the cosmetic results they desired [ 46 , 50 ].

Rationale for timing of procedure . Cosmetic rationalisations for early intervention were often identical to those justifying the conduct of procedures, with no clear distinction articulated on the relative benefits of conducting these interventions in infancy. Some studies argued that the long-term psychological impact of the appearance of ambiguous genitalia is greater if interventions are not conducted early, without citing any recommendations or wider literature justifying this [ 6 , 39 ]. Other studies suggested that better compliance from patients and a lack of recollection in adulthood should justify early conduct of interventions [ 55 ]. Anatomical justifications for early surgery were more detailed, centring around the argument that early conduct is associated with better surgical outcomes. Several studies suggested that increased oestrogen exposure in the first month of life will provide better vascularised tissue and thicker walls of vaginal tissue [ 7 , 55 ], with other studies providing alternate developmental anatomical justifications, such as undeveloped penile circumflex arteries in infants leading to reduced risks of intra-operational bleeding (an uncited justification) [ 44 ]. Early age was also suggested to be the most appropriate time for preventing a range of complications that could have life-long effects [ 61 , 63 , 65 , 66 ]. Studies also suggested general higher success rates of operations conducted in infancy, without expanding on the specificities of this [ 67 ]. “Sex-normalising” interventions in infancy were also rationalised in infants requiring multiple surgeries. For example, one study suggested the benefits of concurrently performing vaginal reconstruction in a child with MRKH syndrome who incidentally had an anorectal malformation, with the justification that scarring of the urinary tract and rectum from anorectal reconstruction in this case may make vaginoplasty more difficult in later life [ 68 ]. Early intervention was also suggested as a way to prevent anticipated incorrect virilisation and puberty associated with retained gonads, the justifications for which were uncited [ 62 , 69 ]. Interestingly, one included study initially promoting early surgical intervention changed their mind following their study, moving to advocating for delaying intervention until puberty, with the anatomical justification that post-pubertal genital skin is more robust than the introital skin of infants [ 7 ]. Functional justifications for early intervention centred around the fact that early surgery will more effectively achieve desired function outcomes [ 61 ], including maintaining reproductive potential [ 63 ] and metabolic development [ 62 ].

Perceived goal of aligning with assigned sex.

The 28 studies reporting a perceived goal of aligning with an assigned sex as a rationale for the timing or conduct of procedures investigated CAH (n = 14), ovotesticular DSD (n = 4), 5-alpha reductase deficiency, MGD (n = 2, respectively), MRKH syndrome, true hermaphroditism, unspecified DSDs, ambiguous genitalia and 46, XX DSD (n = 1, respectively).

Rationale for conducting procedure . Rationalising procedures through pursuing a binary assigned sex for infants manifested in several ways. Anatomically, articles cited an intention to use interventions to create ‘normal’ external genitalia corresponding with the assigned sex of the infant by parents or surgeons [ 7 , 8 , 40 , 42 – 44 , 70 , 71 ] or to remove genitals unconcordant with the sex assigned to infants [ 7 , 65 , 70 – 73 ]. Some articles went further than this and identified interventions as a way to manifest the assigned sex (and subsequently, gender identity) chosen for the infant [ 5 , 74 ], suggesting interventions could ‘ create anatomical […] female gender [emphasis added]’ [ 58 ] or ‘ convert to the male sex [emphasis added]’ [ 75 ]. Facilitating psychosocial adjustment through conforming to a binary sex was also cited as an ethical motivation for conducting interventions [ 43 , 45 , 76 ], based on the assumption that genital appearance influences the development of the sexual psyche [ 39 ]. Teams also suggested that intervention could achieve certain sexual orientations and sexual development associated with the assigned sex [ 62 ], with one study citing hysterectomy and vaginectomy in CAH patients as a way to provide ‘satisfactory male gender identity and heterosexual orientation’ [ 62 ], and another suggesting intervention would facilitate the infant becoming a ‘functional member of the society’ [ 77 ].

Rationale for timing of procedure . The key rationalisation of early intervention timing when enacting the perceived goal of aligning with an assigned sex was the assumption that earlier intervention will more effectively facilitate gender identity development in line with sex assigned to a child by parents or surgeons, and limit psychological distress associated with an ambiguous sex [ 5 , 6 , 41 , 42 , 46 , 62 , 70 , 74 ]. One study extends this to suggest that early intervention will facilitate healthy body image development, a suggestion that did not cite any recommendations or relevant literature [ 39 ]. Some rationales relied on a belief that gender identity formation occurs in early life and can be strongly influenced by environmental surroundings [ 39 , 78 ]. Others cited early intervention as key for preventing psychological distress, without detailing the specific reason or evidence-base as to why early intervention is protective against this [ 8 , 44 ]. One study cites early intervention as an important tool to prevent societal barriers related to assigned sex for people with congenital variations in sex characteristics, specifically highlighting the complicated sports life in female athletes with 5-alpha reductase deficiency [ 79 ].

Parental desire.

The 14 studies reporting parental desire as a rationale for the timing or conduct of procedures investigated CAH (n = 10), ovotesticular DSD (n = 3) and 5-alpha reductase deficiency (n = 1). Both anticipated (n = 11) and vocalised parental desire (n = 4) were used as rationales.

Rationale for conducting procedure . Several articles predicted that external genital appearance that did not match conventional conceptions of male or female bodies would increase parental anxiety, distress and embarrassment, and used mitigating this as a justification for intervention [ 7 ]. One study specifically identifies the anticipated parental embarrassment surrounding a large clitoris in a child with CAH as a rationale for intervention [ 39 ]. Articles also assumed that parental acceptance and parent-child attachment would be improved through surgical intervention, despite often citing no relevant literature or discussions with family to justify this [ 6 ]. Surgical teams also articulated a difficulty in navigating voiced parental distress. Two surgical teams report going against their own recommendation to delay treatment due to intense parental desire for early intervention [ 40 , 78 ]. Parental desire was not only expressed for surgical intervention, but also for the assigned sex of their child. One study reported a strong parental preference to raise their child with ovotesticular DSD as a girl, which was then used as a rationale for conducting reconstructive vaginoplasty [ 80 ].

Rationale for timing of procedure . Some articles anticipated that specifically early reconstruction would alleviate anticipated psychological distress in parents [ 7 , 8 , 41 ], improve parental attachment [ 69 , 70 ] and increase parental comfort [ 62 ]. One study cites parents asking for early orchiopexy for their child with ovotesticular DSD before they began school to ensure wellbeing within the family, school and wider society [ 71 ]. In some cases, authors acknowledged that conducting early intervention was going against current global trends towards delaying intervention, but used strong parental desire in their local context to justify this. For example, one study cited parental anxiety specifically resulting from their sociocultural context as reason to complete the procedure as soon as possible [ 69 ]. Ethical justifications relating to preventing parental harm were also used to justify early intervention, with one study stating the birth of a child with ambiguous genitalia is a ‘tragedy’ for parents and thus ‘prolonging this suffering’ is morally unjustifiable [ 81 ].

Belief of best practice.

The 7 studies reporting belief of best practice as a rationale investigated CAH (n = 5), unspecified DSDs and ambiguous genitalia (n = 1, respectively).

Rationale for conducting procedure . All included studies citing clinical documentation to justify interventions cite the 2006 publication, Consensus Statement on Management of Intersex Disorders [ 9 ], referencing its recommendations to conduct surgical interventions in cases of severe virilisation, with the goal of not only improving cosmesis but also future sexual function [ 6 , 41 , 54 , 67 ].

Rationale for timing of procedure . Studies similarly cited the 2006 publication, Consensus Statement on Management of Intersex Disorders [ 9 ] when justifying early intervention in cases of severe virilisation [ 6 , 41 ], as well as citing clinical practice guidelines from the Endocrine Society [ 40 , 43 , 67 , 82 ] and recommendations from a range of government health authorities [ 45 , 83 ].

One study from the UK investigating CAH reported a logistic issue justifying the timing of infant genital surgery [ 78 ]. Despite teams advocating delayed intervention to parents, an imminent move by the family to their country of origin led the clinical team to elect to conduct the intervention in their specialist unit rather than risk the intervention being conducted in non-specialist facilities at a later date.

No rationale.

44 studies provided no rationale for timing of surgery, 28 no rationale for the conduct of surgery, and 24 studies no rationale for either. These studies providing no rationale for either timing or conduct investigated diphallia (n = 8), PMDS, ovotesticular DSD (n = 4, respectively), CAH, transverse testicular ectopia(n = 3, respectively), gonadal dysgenesis (n = 2), 5-alpha reductase deficiency, 45,X/46, XY DSD, Antley-Bixler syndrome, PAIS, severe hypospadias and microphallus, sex chromosome mosaicism and Herlyn-Weber-Wunderlich syndrome (n = 1, respectively)

Acknowledgement of controversy

47.9% of studies (n = 34) acknowledged controversy surrounding the interventions they were reporting. Three codes were developed to describe studies’ acknowledgment of the controversy surrounding “sex-normalising” surgical interventions: controversy surrounding the quality and standards of care (n = 29), controversy surrounding the cultural, gender and social well-being implications of interventions (n = 18) and controversy surrounding the ethics of the procedure (n = 15).

Standards of care and quality.

There was extensive debate in included studies addressing concerns about the quality of interventions, specifically highlighting a lack of information about long-term outcomes of the surgery [ 42 , 43 , 55 , 60 , 64 , 84 ], harms caused by the surgery [ 6 , 13 , 40 , 43 , 56 , 84 ], lack of evidence on the number of operations required [ 7 , 39 , 43 ] and contention about intervention timing [ 7 , 8 , 39 , 40 , 42 , 43 , 53 , 55 , 62 , 64 , 68 , 71 , 74 , 76 , 81 ]. Studies navigated these controversies in different ways. Some authors introduced the ethical debate surrounding interventions, and went on to dismiss the validity of these concerns and continue with the intervention regardless [ 60 , 62 , 68 , 85 ]. Several studies also explored concerns about intervention quality, but continued with various levels of accommodations for these controversies, including leaving surgical decision-making up to expertise of the surgical team [ 7 , 55 ], changing surgical procedure [ 5 , 40 , 42 , 43 , 53 , 64 , 81 , 86 ], individualising care [ 87 ] or providing psychological support to parents and children [ 71 , 75 ].

Culture/gender/social well-being.

Studies that addressed controversies surrounding the cultural, gender and social considerations of interventions introduced this debate through discussing the age of gender identity and sexual development, largely suggesting that by 2–3 years of age, children may be able to correctly label themselves of a certain gender, thus justifying intervention before this age [ 39 , 78 ]. Some studies recognised that controversies surrounding sex determination were increasing [ 7 ], particularly with concerns around higher prevalence of gender dysphoria among individuals with congenital variations in sex characteristics due to assignment of sex not matching future gender identity [ 42 , 64 , 69 ]. Often these controversies were dismissed through authors citing previous experience or small-scale qualitative evidence citing low levels of gender-related regret in adults having undergone infant genital surgery [ 64 , 78 ]. The rights of the child and perceived rights of parents also pervaded in these debates, with authors debating the gender preferences of families and the future gender identity of individuals with congenital variations in sex characteristics [ 46 , 71 , 73 , 75 ]. In addition to the aforementioned ways of navigating these controversies, such as offering patient-centred, multidisciplinary, individualised care for patients [ 7 , 46 , 64 , 71 ], studies also cited the possibility of a sex reversal later in life to rationalise controversial decisions to undergo procedures [ 71 , 74 , 79 ] or recommended psychological support for children who develop gender dysphoria from incorrect sex assignment from surgeons or parents post-intervention [ 69 , 75 ]. Authors also justified intervention in the face of these controversies by suggesting conservative surgery could balance future risks of gender dysphoria while protecting children from growing up in non-accepting environments [ 64 , 71 ].

A range of ethical controversies were acknowledged by included studies, primarily concerning the ability to provide informed consent for an irreversible intervention in infancy, and the interventions’ implied impact on cosmesis and gender identity [ 40 , 42 , 53 , 64 , 69 , 78 ]. Some studies further extend this debate to discuss the relative rights of the individual and the perceived rights of parents/guardians to request such intervention [ 46 , 55 , 74 , 81 ]. There is a recognition in some articles about a changing trend in the ethical approach to “sex-normalising” interventions, suggesting a global trend towards prioritising rights of the individual and postponing interventions [ 46 , 71 ]. These ethical controversies were navigated in different ways. Often ethical concerns were considered but not explored or integrated into decision-making [ 86 , 88 ]. The aforementioned surgical, functional, and anatomical rationales are also used by authors as considerations that overwhelm the ethical concerns of interventions, despite many concurrently noting that evidence on the relative efficacies of early and late surgery are lacking [ 40 , 53 , 55 , 74 ]. Notably, one team changed their recommendations to promote later intervention after their study, citing ethical concerns arising when considering high complication rates and low patient satisfaction with early intervention [ 42 ]. Placement of the global ethical debate within the context teams were working was also used to rationalise why teams found it appropriate to continue to conduct interventions, primarily suggesting that perceived rights of parents overwhelm the rights of the child in certain settings [ 46 , 69 , 74 , 75 ]. Other studies introduced ethical controversies surrounding interventions but rationalised their conduct through providing individualised care to patients from a multidisciplinary team and providing psychological counselling to those involved [ 69 , 71 ]. Some studies adapted their interventions based off of ethical concerns about irreversibility [ 64 ], with authors of one diphallia study promoting early surgery using a technique that did not remove penile tissue as a way to (in their view) address ethical concerns surrounding irreversibility [ 86 ]. One study adapted their study to focus on vaginoplasty rather than clitoroplasty in their report on a child with CAH to incorporate ethical concerns, due to vaginoplasty being ‘less debated’ in their setting [ 89 ].

This manuscript reports the findings of the first systematic review investigating the rationales, outcomes and controversies surrounding elective, “sex-normalising” interventions for infants with congenital variations in sex characteristics. Included articles were largely case reports, matching previous observations about the extant literature [ 2 , 4 ]. Among a geographically diverse set of articles, we noted outcome reporting that did not align with recommended reporting guidance. Rationales reported by surgical teams both concerning the conduct of interventions and their timings were heterogeneous across included studies, but broadly included anatomical/cosmetic/functional rationales, a perceived desire of aligning with a sex assigned by surgeons and parents, parental desire, and a belief of best practice. Studies acknowledged controversies surrounding the quality of interventions, cultural, gender and social considerations, and the ethics surrounding surgeries, while demonstrating different ways of integrating these controversies into their rationale for the continued justification of surgical intervention. 62.0% of studies reported no specific rationale for intervention timing, 39.4% reported no rationale for conduct and 52.1% acknowledged no controversy in the interventions they were conducting, implying that in some settings these surgical interventions remain accepted, common practice.

Cosmesis emerged as an important rationale for the conduct of interventions, as well as being one of the most commonly reported outcome measures collected in included articles. There was notable subjectivity in the reporting of cosmesis, with subjective, descriptive terminology used by surgeons and parents to determine successful cosmetic outcomes, combined with a lack of validated measures used for outcome appraisal. There is no guarantee that infant genital surgery will improve long-term cosmetic outcomes for the individual concerned; indeed, some adults with congenital variations in sex characteristics who have received surgical intervention in infancy report dissatisfaction and distress associated with their post-surgical genital appearance [ 90 ], concerns which will sometimes require multiple repeat surgeries to address [ 91 ]. Decisions about genital cosmesis should not be made without the active participation and consent of the individual to whom the genitalia belong, and thus elective, irreversible genital surgical interventions for infants and children with congenital variations in sex characteristics should not take place before the individual can decide for themselves whether they would like to undergo such procedures [ 92 ].

19.7% of studies reported parental distress as rationale for either the conduct or timing of interventions, primarily citing an anticipated or voiced parental desire for improved genital cosmesis or for genitals that align more with those associated with ‘typical’ male or female bodies. Medical teams also suggested surgery could relieve psychological distress in parents, improve parental attachment and increase parental comfort. Importantly, a belief that infant surgery was not best practice by surgical teams was occasionally overwhelmed by intense parental desire for intervention in infancy or early childhood, indicating the respect given to parental desire in many settings. Medical teams in some settings struggled with balancing clearly voiced parental desire for early intervention against medical recommendations and human rights imperatives to delay decision-making surrounding intervention until children with congenital variations in sex characteristics are able to participate in decision-making processes. This underscores the need for multidisciplinary teams supporting infants and children with congenital variations in sex characteristics and their parents. Medical teams have a critical role to play in guiding and managing ethical decision-making with parents in clinical settings and sociocultural environments where the rights of the child and obligations of the parents to make decisions which benefit the child do not necessarily match parental desire [ 93 ]. Known drivers of parental distress should be better and consistently integrated into parent and family counselling and psychosocial support, and delivered by qualified professionals like psychologists and social workers [ 4 ]. This counselling may include, for example, a focus on destigmatising congenital variations in sex characteristics, providing clear supportive pathways for communicating with their child on their variation as they grow, being candid on the risks of early surgery for a child, and connecting parents to the intersex community and parents of children with congenital variations in sex characteristics.

39.4% of studies reported a perceived goal of aligning with an assigned sex from parents or surgeons as a rationale for intervention conduct or timing, with arguments around promoting a specific gender also pervading into cosmetic, functional, and anatomical rationales. These rationales largely rely on the assumption that early life experience, including cosmetic, social, and functional features, predicts future gender identity; a theory cited by several studies when navigating sex assignment controversies [ 39 , 78 ]. This is reminiscent of mid-20th century arguments about gender identity development, popularized by John Money (the work of whom is cited by some included articles [ 42 , 81 ] which suggests gender identity is socially constructed in childhood and thus can be re-assigned in infancy, with external factors such as social environment and anatomical features made to match this chosen assignment [ 94 ]. Although this theory is now debunked [ 95 , 96 ], it still pervades in rationales for intervention conduct and a belief that earlier intervention is more efficacious at promoting gender identity development aligned with an assigned sex from parents or medical teams. There is no evidence to suggest surgical intervention for infants with congenital variations in sex characteristics is associated with ‘better’ gender identity development. In fact, several studies note an increased prevalence of gender dysphoria in individuals having undergone “sex-normalising” surgical intervention in infancy [ 2 , 97 , 98 ]. Irreversible, elective genital surgical interventions with the rationale of promoting the development of a specific gender identity are unethical without the full, free, and informed consent and decision-making of the person concerned. Rather, counselling and support for children and parents should be provided throughout infancy, childhood and adolescence, until an individual is old enough to decide for themselves whether they wish to undergo such interventions, and, if they do desire intervention, are able to understand and provide informed consent for any desired interventions.

Forty-six studies were excluded from analysis after full-text screening for citing malignancy as a rationale for conducting interventions, as we determined mitigation of a perceived malignancy risk to be sufficient to consider an intervention to be non-elective. The importance of mitigating the risk of childhood malignancy should not be understated. However, it is important to ensure that the choice to undertake prophylactic genital surgeries (versus a more conservative course of biopsy and monitoring) before a child can provide full, free and informed consent and participate in decision-making processes is done with caution, and that any risk of malignancy is justified and evidence-based. Twenty-six of these forty-six studies cited malignancy as a rationale for surgery for a congenital variation in sex characteristics for which surgical intervention was not recommended as a malignancy mitigation strategy in the 2006 publication, Consensus Statement on Management of Intersex Disorders [ 9 ], which was the last guidance to segregate malignancy risk by individual congenital variation in sex characteristics. This echoes a lack of clarity on the malignant potential of different congenital variations in sex characteristics, which is noted in the extant literature [ 36 , 37 , 99 ]. The expertise of this team did not allow for a critical appraisal of the validity of oncological concerns justifying interventions. Future research should focus on fully elucidating causal pathways and epidemiological connections between individual congenital variations in sex characteristics and malignancy to ensure irreversible genital surgeries are not being conducted when biopsy and monitoring may be sufficient malignancy mitigation strategies.

The social and cultural complexity of congenital variations in sex characteristics across different geographic settings and demographic groups was cited as a reason for conducting interventions and permeated throughout discussions about intervention controversy. This was of particular importance when considering parental desire and concerns about raising a child in a setting that could potentially be stigmatising. Attitudes towards people with congenital variations in sex characteristics differ largely across settings, with cultural considerations about gender preference, identity and stigma towards children with congenital variations in sex characteristics driving strong parental and surgeon desire for intervention in certain settings [ 69 , 100 ]. Concerns about raising a child with congenital variations in sex characteristics should be acknowledged and integrated into discussions with parents (for example, by providing resources, including through peer support from intersex persons and their parents, on how to communicate with other family and community members). Parental or surgeon concerns, however, should not be used to justify the conduct of elective, irreversible genital surgeries before an individual can provide full, free and informed consent.

International human rights treaty bodies have recommended the prohibition of irreversible, elective, genital surgeries conducted on children with congenital variations in sex characteristics before they can participate in decision-making and give full, free and informed consent [ 25 ]. Despite this, we identified that interventions continue to be conducted, based largely around heterogeneous anatomical and functional goals that were not adequately supported by recommendations and the extant medical literature, a desire from parents and surgeons to match genital cosmesis with that typically ascribed to male and female bodies, and a parental desire for intervention conduct. In-keeping with the recommendations of human rights bodies, we recommend that human rights, including the rights of the child, should be protected and prioritised by health systems, and, as such, desire for intervention from parents or surgical teams is not sufficient to justify conducting irreversible, elective, genital interventions before a child can decide for themselves if they want or do not want such procedures, and if they do, provide informed consent to any intervention.

There are notable limitations to this review. Our findings are not entirely representative of the diversity of persons with congenital variations in sex characteristics, given the notable discrepancy in the categorisation of these variations across different disciplines and countries. Our conservative approach to study inclusion, such as excluding studies citing generic urological functional goals and all studies citing malignancy regardless of recommendations from oncological literature, may have led to the exclusion of relevant studies. However, a strength of this review is that this conservative approach ensured that all included interventions were clearly elective. There is also likely to be a publication bias towards congenital variations in sex characteristics that aren’t commonly occurring, given the predominance of case reports emerging from our inclusion criteria (i.e., the large number of diphallia cases included, when the variation itself is reported to be limited to one in every five million live births [ 101 ]). A decision to exclude articles in which it was not certain if all individuals were under the age of 10, to ensure our findings represented only interventions conducted in infancy and childhood, led to the exclusion of cohort and larger scale studies where these data were not disaggregated by age, potentially biasing the study towards primarily considering lower-quality case reports. Available clinical data lacks adequate sample sizes, independence and relevant control groups [ 28 ], undermining the generalisability of the outcomes reported in this review. Although some guidance recommend upscaling longitudinal (including retrospective) outcome assessment studies to address this lack of data [ 2 , 9 ], it is important that any long-term outcome data is collected from individuals who provided full, free and informed consent to undergo elective, irreversible, genital interventions and to being a participant in any research. Outcome measurements and research priorities should focus on understanding the perspectives, health and wellbeing of the individuals affected and how health services can better meet their current and future needs.

International human rights monitoring and accountability mechanisms call for the deferral of decision-making around elective, irreversible, “sex-normalising” genital surgeries for infants and children with congenital variations in sex characteristics until individuals can participate in decision-making and provide free, full and informed consent to any desired interventions. This review has identified a heterogenous literature reporting the surgical outcomes, rationales, and controversies of surgical interventions, with notably inconsistent study designs, methods, and outcome reporting. Assessment of surgical rationale and outcomes has revealed that medical teams continue to conduct these interventions in the face of controversy due to desires to mitigate parental distress surrounding the perceived difficulties of raising a child with a congenital variation in sex characteristics, as well as a variety of under-researched or outdated beliefs that doing so mitigates suboptimal anatomical, cosmetic, physiological, and psychological outcomes for a child, or that surgical intervention is best practice.

Irreversible, elective, “sex-normalising” genital interventions on infants or children with congenital variations of sex characteristics are unethical without the full, free, and informed consent and active participation in decision-making of the individual concerned. Rationales of achieving a cosmetic outcome perceived as satisfactory by individuals to whom the genitalia do not belong, matching parental desire or alleviating parental distress, or promoting a specific gender identity do not justify continuation of these procedures.

National legislating and medical regulatory bodies, in collaboration with relevant international and regional organisations, should enhance efforts to end the conduct of irreversible, elective, “sex-normalising” interventions conducted without the full, free, and informed consent of the person concerned, in an effort to promote and protect the right to the highest attainable standard of physical and mental health for people with congenital variations in sex characteristics.

Supporting information

S1 text. prisma checklist..

https://doi.org/10.1371/journal.pgph.0003568.s001

S2 Text. Search strategy.

https://doi.org/10.1371/journal.pgph.0003568.s002

S1 Table. Quality assessment table.

https://doi.org/10.1371/journal.pgph.0003568.s003

S2 Table. Articles excluded at full text with exclusion reason.

https://doi.org/10.1371/journal.pgph.0003568.s004

S3 Table. Study characteristics with information on data extraction.

https://doi.org/10.1371/journal.pgph.0003568.s005

Acknowledgments

The authors wish to thank Morgan Carpenter and Mauro Cabral Grinspan for their help to define the review’s search strategy, and Anna Coates, Jenny Cresswell, Eva Krpelanova, Andreas Reis, for their inputs on the review’s results and interpretation.

Disclaimer: The named authors alone are responsible for the views expressed in this publication and do not necessarily represent the decisions or the policies of the UNDP-UNFPA-UNICEF-WHO-World Bank Special Programme of Research , Development and Research Training in Human Reproduction (HRP) or the World Health Organization (WHO) .

• 1. World Health Organisation. Gender, Equity and Human Rights; FAQ on Health and Sexual Diversity. An Introduction to Key Concepts [Internet]. 2016. Available from: https://iris.who.int/bitstream/handle/10665/255340/WHO-FWC-GER-16.2-eng.pdf?sequence=1 .
• View Article
• PubMed/NCBI
• Google Scholar
• 24. Office of the High Commissioner for Human Rights, African Commission on Human and Peoples’ Rights, Council of Europe, Office of the Commissioner for Human Rights, Inter-American Commission on Human Rights, Special Rapporteur on torture and other cruel, inhuman or degrading treatment or punishment, et al. Intersex Awareness Day–Wednesday 26 October. End violence and harmful medical practices on intersex children and adults, UN and regional experts urge. [Internet]. Office of the High Commissioner for Human Rights; 2016 [cited 2024 May 1]. Available from: https://www.ohchr.org/en/2016/10/intersex-awareness-day-wednesday-26-october?LangID=E&NewsID=20739 .
• 25. Office of the High Commissioner for Human Rights (OHCHR). OHCHR Technical Note on the Human RIghts of Intersex People: Human Rights Standards and Good Practice [Internet]. 2023 [cited 2024 Mar 22]. Available from: https://www.ohchr.org/en/documents/tools-and-resources/ohchr-technical-note-human-rights-intersex-people-human-rights .
• 26. World Health Organization. Sexual health, human rights and the law [Internet]. Geneva: World Health Organization; 2015 [cited 2023 Aug 2]. Available from: https://apps.who.int/iris/handle/10665/175556 .
• 27. United Nations Human Rights Council. Combating discrimination, violence and harmful practices against intersex persons [Internet]. United Nations General Assembly; 2024 [cited 2024 Apr 23]. Report No.: A/HRC/55/L.9. Available from: https://documents.un.org/doc/undoc/ltd/g24/048/06/pdf/g2404806.pdf?token=XlcT9Mfyw1sK9JJJoM&fe=true .
• 29. ILGA World Database. Legal Frameworks—Restrictions on Interventions on Intersex Minors [Internet]. 2024 [cited 2024 Apr 6]. Available from: https://database.ilga.org/interventions-intersex-minors .
• 33. Veritas Health Innovation. Covidence systematic review software [Internet]. Melbourne, Australia; Available from: www.covidence.org .
• 34. World Health Organisation (WHO). Adolescent Health [Internet]. World Health Organisation Health Topics. 2023 [cited 2023 Aug 31]. Available from: https://www.who.int/health-topics/adolescent-health#tab=tab_1 .
• 37. Amnesty International. First, Do No Harm: Ensuring the Rights of Children with Variations of Sex Characteristics in Denmark and Germany [Internet]. UK; 2017. Available from: https://www.amnesty.org/en/documents/eur01/6086/2017/en/ .
• 92. Zilli L, Khosla R. The notion of consent in the UN Treaty Bodies’ general comments and jurisprudence. United Nations University International Institute for Global Health; 2023.
• 93. Council of Europe. Convention for the Protection of Human Rights and Dignity of the Human Being with regard to the Application of Biology and Medicine: Convention on Human Rights and Biomedicine * [Internet]. Oviedo; 1997. Report No.: No. 164. Available from: https://rm.coe.int/168007cf98 .
• 96. Colapinto J. As Nature Made Him: The Boy Who Was Raised as a Girl. Harper Collins; 2013.
• 101. Murphy JP, Gatti JM. Chapter 122—Abnormalities of the Urethra, Penis, and Scrotum. In: Coran AG, editor. Pediatric Surgery (Seventh Edition) [Internet]. Philadelphia: Mosby; 2012 [cited 2024 Apr 18]. p. 1555–1563. Available from: https://www.sciencedirect.com/science/article/pii/B9780323072557001227 .

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• CAREER FEATURE
• 04 December 2020
• Correction 09 December 2020

How to write a superb literature review

Andy Tay is a freelance writer based in Singapore.

You can also search for this author in PubMed   Google Scholar

Credit: Getty

Literature reviews are important resources for scientists. They provide historical context for a field while offering opinions on its future trajectory. Creating them can provide inspiration for one’s own research, as well as some practice in writing. But few scientists are trained in how to write a review — or in what constitutes an excellent one. Even picking the appropriate software to use can be an involved decision (see ‘Tools and techniques’). So Nature asked editors and working scientists with well-cited reviews for their tips.

WENTING ZHAO: Be focused and avoid jargon

Assistant professor of chemical and biomedical engineering, Nanyang Technological University, Singapore.

When I was a research student, review writing improved my understanding of the history of my field. I also learnt about unmet challenges in the field that triggered ideas.

For example, while writing my first review 1 as a PhD student, I was frustrated by how poorly we understood how cells actively sense, interact with and adapt to nanoparticles used in drug delivery. This experience motivated me to study how the surface properties of nanoparticles can be modified to enhance biological sensing. When I transitioned to my postdoctoral research, this question led me to discover the role of cell-membrane curvature, which led to publications and my current research focus. I wouldn’t have started in this area without writing that review.

Collection: Careers toolkit

A common problem for students writing their first reviews is being overly ambitious. When I wrote mine, I imagined producing a comprehensive summary of every single type of nanomaterial used in biological applications. It ended up becoming a colossal piece of work, with too many papers discussed and without a clear way to categorize them. We published the work in the end, but decided to limit the discussion strictly to nanoparticles for biological sensing, rather than covering how different nanomaterials are used in biology.

My advice to students is to accept that a review is unlike a textbook: it should offer a more focused discussion, and it’s OK to skip some topics so that you do not distract your readers. Students should also consider editorial deadlines, especially for invited reviews: make sure that the review’s scope is not so extensive that it delays the writing.

A good review should also avoid jargon and explain the basic concepts for someone who is new to the field. Although I trained as an engineer, I’m interested in biology, and my research is about developing nanomaterials to manipulate proteins at the cell membrane and how this can affect ageing and cancer. As an ‘outsider’, the reviews that I find most useful for these biological topics are those that speak to me in accessible scientific language.

Bozhi Tian likes to get a variety of perspectives into a review. Credit: Aleksander Prominski

BOZHI TIAN: Have a process and develop your style

Associate professor of chemistry, University of Chicago, Illinois.

In my lab, we start by asking: what is the purpose of this review? My reasons for writing one can include the chance to contribute insights to the scientific community and identify opportunities for my research. I also see review writing as a way to train early-career researchers in soft skills such as project management and leadership. This is especially true for lead authors, because they will learn to work with their co-authors to integrate the various sections into a piece with smooth transitions and no overlaps.

After we have identified the need and purpose of a review article, I will form a team from the researchers in my lab. I try to include students with different areas of expertise, because it is useful to get a variety of perspectives. For example, in the review ‘An atlas of nano-enabled neural interfaces’ 2 , we had authors with backgrounds in biophysics, neuroengineering, neurobiology and materials sciences focusing on different sections of the review.

After this, I will discuss an outline with my team. We go through multiple iterations to make sure that we have scanned the literature sufficiently and do not repeat discussions that have appeared in other reviews. It is also important that the outline is not decided by me alone: students often have fresh ideas that they can bring to the table. Once this is done, we proceed with the writing.

I often remind my students to imagine themselves as ‘artists of science’ and encourage them to develop how they write and present information. Adding more words isn’t always the best way: for example, I enjoy using tables to summarize research progress and suggest future research trajectories. I’ve also considered including short videos in our review papers to highlight key aspects of the work. I think this can increase readership and accessibility because these videos can be easily shared on social-media platforms.

ANKITA ANIRBAN: Timeliness and figures make a huge difference

Editor, Nature Reviews Physics .

One of my roles as a journal editor is to evaluate proposals for reviews. The best proposals are timely and clearly explain why readers should pay attention to the proposed topic.

It is not enough for a review to be a summary of the latest growth in the literature: the most interesting reviews instead provide a discussion about disagreements in the field.

Careers Collection: Publishing

Scientists often centre the story of their primary research papers around their figures — but when it comes to reviews, figures often take a secondary role. In my opinion, review figures are more important than most people think. One of my favourite review-style articles 3 presents a plot bringing together data from multiple research papers (many of which directly contradict each other). This is then used to identify broad trends and suggest underlying mechanisms that could explain all of the different conclusions.

An important role of a review article is to introduce researchers to a field. For this, schematic figures can be useful to illustrate the science being discussed, in much the same way as the first slide of a talk should. That is why, at Nature Reviews, we have in-house illustrators to assist authors. However, simplicity is key, and even without support from professional illustrators, researchers can still make use of many free drawing tools to enhance the value of their review figures.

Yoojin Choi recommends that researchers be open to critiques when writing reviews. Credit: Yoojin Choi

YOOJIN CHOI: Stay updated and be open to suggestions

Research assistant professor, Korea Advanced Institute of Science and Technology, Daejeon.

I started writing the review ‘Biosynthesis of inorganic nanomaterials using microbial cells and bacteriophages’ 4 as a PhD student in 2018. It took me one year to write the first draft because I was working on the review alongside my PhD research and mostly on my own, with support from my adviser. It took a further year to complete the processes of peer review, revision and publication. During this time, many new papers and even competing reviews were published. To provide the most up-to-date and original review, I had to stay abreast of the literature. In my case, I made use of Google Scholar, which I set to send me daily updates of relevant literature based on key words.

Through my review-writing process, I also learnt to be more open to critiques to enhance the value and increase the readership of my work. Initially, my review was focused only on using microbial cells such as bacteria to produce nanomaterials, which was the subject of my PhD research. Bacteria such as these are known as biofactories: that is, organisms that produce biological material which can be modified to produce useful materials, such as magnetic nanoparticles for drug-delivery purposes.

Synchronized editing: the future of collaborative writing

However, when the first peer-review report came back, all three reviewers suggested expanding the review to cover another type of biofactory: bacteriophages. These are essentially viruses that infect bacteria, and they can also produce nanomaterials.

The feedback eventually led me to include a discussion of the differences between the various biofactories (bacteriophages, bacteria, fungi and microalgae) and their advantages and disadvantages. This turned out to be a great addition because it made the review more comprehensive.

Writing the review also led me to an idea about using nanomaterial-modified microorganisms to produce chemicals, which I’m still researching now.

PAULA MARTIN-GONZALEZ: Make good use of technology

PhD student, University of Cambridge, UK.

Just before the coronavirus lockdown, my PhD adviser and I decided to write a literature review discussing the integration of medical imaging with genomics to improve ovarian cancer management.

As I was researching the review, I noticed a trend in which some papers were consistently being cited by many other papers in the field. It was clear to me that those papers must be important, but as a new member of the field of integrated cancer biology, it was difficult to immediately find and read all of these ‘seminal papers’.

That was when I decided to code a small application to make my literature research more efficient. Using my code, users can enter a query, such as ‘ovarian cancer, computer tomography, radiomics’, and the application searches for all relevant literature archived in databases such as PubMed that feature these key words.

The code then identifies the relevant papers and creates a citation graph of all the references cited in the results of the search. The software highlights papers that have many citation relationships with other papers in the search, and could therefore be called seminal papers.

My code has substantially improved how I organize papers and has informed me of key publications and discoveries in my research field: something that would have taken more time and experience in the field otherwise. After I shared my code on GitHub, I received feedback that it can be daunting for researchers who are not used to coding. Consequently, I am hoping to build a more user-friendly interface in a form of a web page, akin to PubMed or Google Scholar, where users can simply input their queries to generate citation graphs.

Tools and techniques

Most reference managers on the market offer similar capabilities when it comes to providing a Microsoft Word plug-in and producing different citation styles. But depending on your working preferences, some might be more suitable than others.

Reference managers

Here is a comparison of the more popular collaborative writing tools, but there are other options, including Fidus Writer, Manuscript.io, Authorea and Stencila.

Collaborative writing tools

*Markdown and LaTex are code-based formatting languages favoured by physicists, mathematicians and computer scientists who code on a regular basis, and less popular in other disciplines such as biology and chemistry.

doi: https://doi.org/10.1038/d41586-020-03422-x

Interviews have been edited for length and clarity.

Updates & Corrections

Correction 09 December 2020 : An earlier version of the tables in this article included some incorrect details about the programs Zotero, Endnote and Manubot. These have now been corrected.

Hsing, I.-M., Xu, Y. & Zhao, W. Electroanalysis 19 , 755–768 (2007).

Article   Google Scholar  

Ledesma, H. A. et al. Nature Nanotechnol. 14 , 645–657 (2019).

Article   PubMed   Google Scholar  

Brahlek, M., Koirala, N., Bansal, N. & Oh, S. Solid State Commun. 215–216 , 54–62 (2015).

Choi, Y. & Lee, S. Y. Nature Rev. Chem . https://doi.org/10.1038/s41570-020-00221-w (2020).

Download references

Related Articles

• Research management

Tales of a migratory marine biologist

Career Feature 28 AUG 24

Nail your tech-industry interviews with these six techniques

Career Column 28 AUG 24

How to harness AI’s potential in research — responsibly and ethically

Career Feature 23 AUG 24

Binning out-of-date chemicals? Somebody think about the carbon!

Correspondence 27 AUG 24

No more hunting for replication studies: crowdsourced database makes them easy to find

Nature Index 27 AUG 24

Partners in drug discovery: how to collaborate with non-governmental organizations

Exclusive: the papers that most heavily cite retracted studies

News 28 AUG 24

Chain retraction: how to stop bad science propagating through the literature

Comment 28 AUG 24

Global Faculty Recruitment of School of Life Sciences, Tsinghua University

The School of Life Sciences at Tsinghua University invites applications for tenure-track or tenured faculty positions at all ranks (Assistant/Ass...

Beijing, China

Tsinghua University (The School of Life Sciences)

Tenure-Track/Tenured Faculty Positions

Tenure-Track/Tenured Faculty Positions in the fields of energy and resources.

Suzhou, Jiangsu, China

School of Sustainable Energy and Resources at Nanjing University

ATLAS - Joint PhD Program from BioNTech and TRON with a focus on translational medicine

5 PhD positions for ATLAS, the joint PhD Program from BioNTech and TRON with a focus on translational medicine.

Mainz, Rheinland-Pfalz (DE)

Translational Oncology (TRON) Mainz

Alzheimer's Disease (AD) Researcher/Associate Researcher

Xiaoliang Sunney XIE’s Group is recruiting researchers specializing in Alzheimer's disease (AD).

Changping Laboratory

Supervisory Bioinformatics Specialist CTG Program Head

The National Library of Medicine (NLM) is a global leader in biomedical informatics and computational health data science and the world’s largest b...

Bethesda, Maryland (US)

National Library of Medicine, National Center for Biotechnology Information

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Health (Nursing, Medicine, Allied Health)

• Find Articles/Databases
• Reference Resources
• Evidence Summaries & Clinical Guidelines
• Drug Information
• Health Data & Statistics
• Patient/Consumer Facing Materials
• Images and Streaming Video
• Grey Literature
• Mobile Apps & "Point of Care" Tools
• Tests & Measures This link opens in a new window
• Citing Sources
• Selecting Databases
• Framing Research Questions
• Crafting a Search
• Narrowing / Filtering a Search
• Expanding a Search
• Cited Reference Searching
• Saving Searches
• Term Glossary
• Critical Appraisal Resources
• What are Literature Reviews?
• Conducting & Reporting Systematic Reviews
• Finding Systematic Reviews
• Tutorials & Tools for Literature Reviews
• Finding Full Text

What are Systematic Reviews? (3 minutes, 24 second YouTube Video)

Systematic Literature Reviews: Steps & Resources

These steps for conducting a systematic literature review are listed below . 

Also see subpages for more information about:

• The different types of literature reviews, including systematic reviews and other evidence synthesis methods
• Tools & Tutorials

Literature Review & Systematic Review Steps

• Develop a Focused Question
• Scope the Literature  (Initial Search)
• Refine & Expand the Search
• Limit the Results
• Download Citations
• Abstract & Analyze
• Create Flow Diagram
• Synthesize & Report Results

1. Develop a Focused   Question 

Consider the PICO Format: Population/Problem, Intervention, Comparison, Outcome

Focus on defining the Population or Problem and Intervention (don't narrow by Comparison or Outcome just yet!)

"What are the effects of the Pilates method for patients with low back pain?"

Tools & Additional Resources:

• PICO Question Help
• Stillwell, Susan B., DNP, RN, CNE; Fineout-Overholt, Ellen, PhD, RN, FNAP, FAAN; Melnyk, Bernadette Mazurek, PhD, RN, CPNP/PMHNP, FNAP, FAAN; Williamson, Kathleen M., PhD, RN Evidence-Based Practice, Step by Step: Asking the Clinical Question, AJN The American Journal of Nursing : March 2010 - Volume 110 - Issue 3 - p 58-61 doi: 10.1097/01.NAJ.0000368959.11129.79

2. Scope the Literature

A "scoping search" investigates the breadth and/or depth of the initial question or may identify a gap in the literature. 

Eligible studies may be located by searching in:

• Background sources (books, point-of-care tools)
• Article databases
• Trial registries
• Grey literature
• Cited references
• Reference lists

When searching, if possible, translate terms to controlled vocabulary of the database. Use text word searching when necessary.

Use Boolean operators to connect search terms:

• Combine separate concepts with AND  (resulting in a narrower search)
• Connecting synonyms with OR  (resulting in an expanded search)

Search:  pilates AND ("low back pain"  OR  backache )

Video Tutorials - Translating PICO Questions into Search Queries

• Translate Your PICO Into a Search in PubMed (YouTube, Carrie Price, 5:11) 
• Translate Your PICO Into a Search in CINAHL (YouTube, Carrie Price, 4:56)

3. Refine & Expand Your Search

Expand your search strategy with synonymous search terms harvested from:

• database thesauri
• reference lists
• relevant studies

Example: 

(pilates OR exercise movement techniques) AND ("low back pain" OR backache* OR sciatica OR lumbago OR spondylosis)

As you develop a final, reproducible strategy for each database, save your strategies in a:

• a personal database account (e.g., MyNCBI for PubMed)
• Log in with your NYU credentials
• Open and "Make a Copy" to create your own tracker for your literature search strategies

4. Limit Your Results

Use database filters to limit your results based on your defined inclusion/exclusion criteria.  In addition to relying on the databases' categorical filters, you may also need to manually screen results.  

• Limit to Article type, e.g.,:  "randomized controlled trial" OR multicenter study
• Limit by publication years, age groups, language, etc.

NOTE: Many databases allow you to filter to "Full Text Only".  This filter is  not recommended . It excludes articles if their full text is not available in that particular database (CINAHL, PubMed, etc), but if the article is relevant, it is important that you are able to read its title and abstract, regardless of 'full text' status. The full text is likely to be accessible through another source (a different database, or Interlibrary Loan).  

• Filters in PubMed
• CINAHL Advanced Searching Tutorial

5. Download Citations

Selected citations and/or entire sets of search results can be downloaded from the database into a citation management tool. If you are conducting a systematic review that will require reporting according to PRISMA standards, a citation manager can help you keep track of the number of articles that came from each database, as well as the number of duplicate records.

In Zotero, you can create a Collection for the combined results set, and sub-collections for the results from each database you search.  You can then use Zotero's 'Duplicate Items" function to find and merge duplicate records.

• Citation Managers - General Guide

6. Abstract and Analyze

• Migrate citations to data collection/extraction tool
• Screen Title/Abstracts for inclusion/exclusion
• Screen and appraise full text for relevance, methods, 
• Resolve disagreements by consensus

Covidence is a web-based tool that enables you to work with a team to screen titles/abstracts and full text for inclusion in your review, as well as extract data from the included studies.

• Covidence Support
• Critical Appraisal Tools
• Data Extraction Tools

7. Create Flow Diagram

The PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) flow diagram is a visual representation of the flow of records through different phases of a systematic review.  It depicts the number of records identified, included and excluded.  It is best used in conjunction with the PRISMA checklist .

Example from: Stotz, S. A., McNealy, K., Begay, R. L., DeSanto, K., Manson, S. M., & Moore, K. R. (2021). Multi-level diabetes prevention and treatment interventions for Native people in the USA and Canada: A scoping review. Current Diabetes Reports, 2 (11), 46. https://doi.org/10.1007/s11892-021-01414-3

• PRISMA Flow Diagram Generator (ShinyApp.io, Haddaway et al. )
• PRISMA Diagram Templates  (Word and PDF)
• Make a copy of the file to fill out the template
• Image can be downloaded as PDF, PNG, JPG, or SVG
• Covidence generates a PRISMA diagram that is automatically updated as records move through the review phases

8. Synthesize & Report Results

There are a number of reporting guideline available to guide the synthesis and reporting of results in systematic literature reviews.

It is common to organize findings in a matrix, also known as a Table of Evidence (ToE).

• Reporting Guidelines for Systematic Reviews
• Download a sample template of a health sciences review matrix  (GoogleSheets)

Steps modified from: 

Cook, D. A., & West, C. P. (2012). Conducting systematic reviews in medical education: a stepwise approach.   Medical Education , 46 (10), 943–952.

• << Previous: Critical Appraisal Resources
• Next: What are Literature Reviews? >>
• Last Updated: Aug 14, 2024 1:19 PM
• URL: https://guides.nyu.edu/health

• Reserve a study room
• Library Account
• Undergraduate Students
• Graduate Students
• Faculty & Staff

How to Conduct a Literature Review (Health Sciences and Beyond)

What is a literature review, traditional (narrative) literature review, integrative literature review, systematic reviews, meta-analysis, scoping review.

• Developing a Research Question
• Selection Criteria
• Database Search
• Documenting Your Search
• Organize Key Findings
• Reference Management

Ask Us! Health Sciences Library

The health sciences library.

Call toll-free:  (844) 352-7399 E-mail:   Ask Us More contact information

Related Guides

• Systematic Reviews by Roy Brown Last Updated Oct 17, 2023 884 views this year
• Write a Literature Review by John Glover Last Updated Jul 26, 2024 4534 views this year

A literature review provides an overview of what's been written about a specific topic. There are many different types of literature reviews. They vary in terms of comprehensiveness, types of study included, and purpose. 

The other pages in this guide will cover some basic steps to consider when conducting a traditional health sciences literature review. See below for a quick look at some of the more popular types of literature reviews.

For additional information on a variety of review methods, the following article provides an excellent overview.

Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009 Jun;26(2):91-108. doi: 10.1111/j.1471-1842.2009.00848.x. Review. PubMed PMID: 19490148.

• Next: Developing a Research Question >>
• Last Updated: Mar 15, 2024 12:22 PM
• URL: https://guides.library.vcu.edu/health-sciences-lit-review

• - Google Chrome

Intended for healthcare professionals

• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Performing a...

Performing a literature review

• Related content
• Peer review
• Gulraj S Matharu , academic foundation doctor ,
• Christopher D Buckley , Arthritis Research UK professor of rheumatology
• 1 Institute of Biomedical Research, College of Medical and Dental Sciences, School of Immunity and Infection, University of Birmingham, UK

A necessary skill for any doctor

What causes disease, which drug is best, does this patient need surgery, and what is the prognosis? Although experience helps in answering these questions, ultimately they are best answered by evidence based medicine. But how do you assess the evidence? As a medical student, and throughout your career as a doctor, critical appraisal of published literature is an important skill to develop and refine. At medical school you will repeatedly appraise published literature and write literature reviews. These activities are commonly part of a special study module, research project for an intercalated degree, or another type of essay based assignment.

Formulating a question

Literature reviews are most commonly performed to help answer a particular question. While you are at medical school, there will usually be some choice regarding the area you are going to review.

Once you have identified a subject area for review, the next step is to formulate a specific research question. This is arguably the most important step because a clear question needs to be defined from the outset, which you aim to answer by doing the review. The clearer the question, the more likely it is that the answer will be clear too. It is important to have discussions with your supervisor when formulating a research question as his or her input will be invaluable. The research question must be objective and concise because it is easier to search through the evidence with a clear question. The question also needs to be feasible. What is the point in having a question for which no published evidence exists? Your supervisor’s input will ensure you are not trying to answer an unrealistic question. Finally, is the research question clinically important? There are many research questions that may be answered, but not all of them will …

Log in using your username and password

BMA Member Log In

If you have a subscription to The BMJ, log in:

• Need to activate
• Log in via institution
• Log in via OpenAthens

Log in through your institution

Subscribe from £184 *.

Subscribe and get access to all BMJ articles, and much more.

* For online subscription

Access this article for 1 day for: £50 / $60/ €56 ( excludes VAT )

You can download a PDF version for your personal record.

Buy this article

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 29, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

JAY SIWEK, M.D., MARGARET L. GOURLAY, M.D., DAVID C. SLAWSON, M.D., AND ALLEN F. SHAUGHNESSY, PHARM.D.

Am Fam Physician. 2002;65(2):251-258

Traditional clinical review articles, also known as updates, differ from systematic reviews and meta-analyses. Updates selectively review the medical literature while discussing a topic broadly. Nonquantitative systematic reviews comprehensively examine the medical literature, seeking to identify and synthesize all relevant information to formulate the best approach to diagnosis or treatment. Meta-analyses (quantitative systematic reviews) seek to answer a focused clinical question, using rigorous statistical analysis of pooled research studies. This article presents guidelines for writing an evidence-based clinical review article for American Family Physician . First, the topic should be of common interest and relevance to family practice. Include a table of the continuing medical education objectives of the review. State how the literature search was done and include several sources of evidence-based reviews, such as the Cochrane Collaboration, BMJ's Clinical Evidence , or the InfoRetriever Web site. Where possible, use evidence based on clinical outcomes relating to morbidity, mortality, or quality of life, and studies of primary care populations. In articles submitted to American Family Physician , rate the level of evidence for key recommendations according to the following scale: level A (randomized controlled trial [RCT], meta-analysis); level B (other evidence); level C (consensus/expert opinion). Finally, provide a table of key summary points.

American Family Physician is particularly interested in receiving clinical review articles that follow an evidence-based format. Clinical review articles, also known as updates, differ from systematic reviews and meta-analyses in important ways. 1 Updates selectively review the medical literature while discussing a topic broadly. An example of such a topic is, “The diagnosis and treatment of myocardial ischemia.” Systematic reviews comprehensively examine the medical literature, seeking to identify and synthesize all relevant information to formulate the best approach to diagnosis or treatment. Examples are many of the systematic reviews of the Cochrane Collaboration or BMJ's Clinical Evidence compendium. Meta-analyses are a special type of systematic review. They use quantitative methods to analyze the literature and seek to answer a focused clinical question, using rigorous statistical analysis of pooled research studies. An example is, “Do beta blockers reduce mortality following myocardial infarction?”

The best clinical review articles base the discussion on existing systematic reviews and meta-analyses, and incorporate all relevant research findings about the management of a given disorder. Such evidence-based updates provide readers with powerful summaries and sound clinical guidance.

In this article, we present guidelines for writing an evidence-based clinical review article, especially one designed for continuing medical education (CME) and incorporating CME objectives into its format. This article may be read as a companion piece to a previous article and accompanying editorial about reading and evaluating clinical review articles. 1 , 2 Some articles may not be appropriate for an evidence-based format because of the nature of the topic, the slant of the article, a lack of sufficient supporting evidence, or other factors. We encourage authors to review the literature and, wherever possible, rate key points of evidence. This process will help emphasize the summary points of the article and strengthen its teaching value.

Topic Selection

Choose a common clinical problem and avoid topics that are rarities or unusual manifestations of disease or that have curiosity value only. Whenever possible, choose common problems for which there is new information about diagnosis or treatment. Emphasize new information that, if valid, should prompt a change in clinical practice, such as the recent evidence that spironolactone therapy improves survival in patients who have severe congestive heart failure. 3 Similarly, new evidence showing that a standard treatment is no longer helpful, but may be harmful, would also be important to report. For example, patching most traumatic corneal abrasions may actually cause more symptoms and delay healing compared with no patching. 4

Searching the Literature

When searching the literature on your topic, please consult several sources of evidence-based reviews ( Table 1 ) . Look for pertinent guidelines on the diagnosis, treatment, or prevention of the disorder being discussed. Incorporate all high-quality recommendations that are relevant to the topic. When reviewing the first draft, look for all key recommendations about diagnosis and, especially, treatment. Try to ensure that all recommendations are based on the highest level of evidence available. If you are not sure about the source or strength of the recommendation, return to the literature, seeking out the basis for the recommendation.

In particular, try to find the answer in an authoritative compendium of evidence-based reviews, or at least try to find a meta-analysis or well-designed randomized controlled trial (RCT) to support it. If none appears to be available, try to cite an authoritative consensus statement or clinical guideline, such as a National Institutes of Health Consensus Development Conference statement or a clinical guideline published by a major medical organization. If no strong evidence exists to support the conventional approach to managing a given clinical situation, point this out in the text, especially for key recommendations. Keep in mind that much of traditional medical practice has not yet undergone rigorous scientific study, and high-quality evidence may not exist to support conventional knowledge or practice.

Patient-Oriented vs. Disease-Oriented Evidence

With regard to types of evidence, Shaughnessy and Slawson 5 – 7 developed the concept of Patient-Oriented Evidence that Matters (POEM), in distinction to Disease-Oriented Evidence (DOE). POEM deals with outcomes of importance to patients, such as changes in morbidity, mortality, or quality of life. DOE deals with surrogate end points, such as changes in laboratory values or other measures of response. Although the results of DOE sometimes parallel the results of POEM, they do not always correspond ( Table 2 ) . 2 When possible, use POEM-type evidence rather than DOE. When DOE is the only guidance available, indicate that key clinical recommendations lack the support of outcomes evidence. Here is an example of how the latter situation might appear in the text: “Although prostate-specific antigen (PSA) testing identifies prostate cancer at an early stage, it has not yet been proved that PSA screening improves patient survival.” (Note: PSA testing is an example of DOE, a surrogate marker for the true outcomes of importance—improved survival, decreased morbidity, and improved quality of life.)

Evaluating the Literature

Evaluate the strength and validity of the literature that supports the discussion (see the following section, Levels of Evidence). Look for meta-analyses, high-quality, randomized clinical trials with important outcomes (POEM), or well-designed, nonrandomized clinical trials, clinical cohort studies, or case-controlled studies with consistent findings. In some cases, high-quality, historical, uncontrolled studies are appropriate (e.g., the evidence supporting the efficacy of Papanicolaou smear screening). Avoid anecdotal reports or repeating the hearsay of conventional wisdom, which may not stand up to the scrutiny of scientific study (e.g., prescribing prolonged bed rest for low back pain).

Look for studies that describe patient populations that are likely to be seen in primary care rather than subspecialty referral populations. Shaughnessy and Slawson's guide for writers of clinical review articles includes a section on information and validity traps to avoid. 2

Levels of Evidence

Readers need to know the strength of the evidence supporting the key clinical recommendations on diagnosis and treatment. Many different rating systems of varying complexity and clinical relevance are described in the medical literature. Recently, the third U.S. Preventive Services Task Force (USPSTF) emphasized the importance of rating not only the study type (RCT, cohort study, case-control study, etc.), but also the study quality as measured by internal validity and the quality of the entire body of evidence on a topic. 8

While it is important to appreciate these evolving concepts, we find that a simplified grading system is more useful in AFP . We have adopted the following convention, using an ABC rating scale. Criteria for high-quality studies are discussed in several sources. 8 , 9 See the AFP Web site ( www.aafp.org/afp/authors ) for additional information about levels of evidence and see the accompanying editorial in this issue discussing the potential pitfalls and limitations of any rating system.

Level A (randomized controlled trial/meta-analysis): High-quality randomized controlled trial (RCT) that considers all important outcomes. High-quality meta-analysis (quantitative systematic review) using comprehensive search strategies.

Level B (other evidence): A well-designed, nonrandomized clinical trial. A nonquantitative systematic review with appropriate search strategies and well-substantiated conclusions. Includes lower quality RCTs, clinical cohort studies, and case-controlled studies with non-biased selection of study participants and consistent findings. Other evidence, such as high-quality, historical, uncontrolled studies, or well-designed epidemiologic studies with compelling findings, is also included.

Level C (consensus/expert opinion): Consensus viewpoint or expert opinion.

Each rating is applied to a single reference in the article, not to the entire body of evidence that exists on a topic. Each label should include the letter rating (A, B, C), followed by the specific type of study for that reference. For example, following a level B rating, include one of these descriptors: (1) nonrandomized clinical trial; (2) nonquantitative systematic review; (3) lower quality RCT; (4) clinical cohort study; (5) case-controlled study; (6) historical uncontrolled study; (7) epidemiologic study.

Here are some examples of the way evidence ratings should appear in the text:

“To improve morbidity and mortality, most patients in congestive heart failure should be treated with an angiotensin-converting enzyme inhibitor. [Evidence level A, RCT]”

“The USPSTF recommends that clinicians routinely screen asymptomatic pregnant women 25 years and younger for chlamydial infection. [Evidence level B, non-randomized clinical trial]”

“The American Diabetes Association recommends screening for diabetes every three years in all patients at high risk of the disease, including all adults 45 years and older. [Evidence level C, expert opinion]”

When scientifically strong evidence does not exist to support a given clinical recommendation, you can point this out in the following way:

“Physical therapy is traditionally prescribed for the treatment of adhesive capsulitis (frozen shoulder), although there are no randomized outcomes studies of this approach.”

Format of the Review

Introduction.

The introduction should define the topic and purpose of the review and describe its relevance to family practice. The traditional way of doing this is to discuss the epidemiology of the condition, stating how many people have it at one point in time (prevalence) or what percentage of the population is expected to develop it over a given period of time (incidence). A more engaging way of doing this is to indicate how often a typical family physician is likely to encounter this problem during a week, month, year, or career. Emphasize the key CME objectives of the review and summarize them in a separate table entitled “CME Objectives.”

The methods section should briefly indicate how the literature search was conducted and what major sources of evidence were used. Ideally, indicate what predetermined criteria were used to include or exclude studies (e.g., studies had to be independently rated as being high quality by an established evaluation process, such as the Cochrane Collaboration). Be comprehensive in trying to identify all major relevant research. Critically evaluate the quality of research reviewed. Avoid selective referencing of only information that supports your conclusions. If there is controversy on a topic, address the full scope of the controversy.

The discussion can then follow the typical format of a clinical review article. It should touch on one or more of the following subtopics: etiology, pathophysiology, clinical presentation (signs and symptoms), diagnostic evaluation (history, physical examination, laboratory evaluation, and diagnostic imaging), differential diagnosis, treatment (goals, medical/surgical therapy, laboratory testing, patient education, and follow-up), prognosis, prevention, and future directions.

The review will be comprehensive and balanced if it acknowledges controversies, unresolved questions, recent developments, other viewpoints, and any apparent conflicts of interest or instances of bias that might affect the strength of the evidence presented. Emphasize an evidence-supported approach or, where little evidence exists, a consensus viewpoint. In the absence of a consensus viewpoint, you may describe generally accepted practices or discuss one or more reasoned approaches, but acknowledge that solid support for these recommendations is lacking.

In some cases, cost-effectiveness analyses may be important in deciding how to implement health care services, especially preventive services. 10 When relevant, mention high-quality cost-effectiveness analyses to help clarify the costs and health benefits associated with alternative interventions to achieve a given health outcome. Highlight key points about diagnosis and treatment in the discussion and include a summary table of the key take-home points. These points are not necessarily the same as the key recommendations, whose level of evidence is rated, although some of them will be.

Use tables, figures, and illustrations to highlight key points, and present a step-wise, algorithmic approach to diagnosis or treatment when possible.

Rate the evidence for key statements, especially treatment recommendations. We expect that most articles will have at most two to four key statements; some will have none. Rate only those statements that have corresponding references and base the rating on the quality and level of evidence presented in the supporting citations. Use primary sources (original research, RCTs, meta-analyses, and systematic reviews) as the basis for determining the level of evidence. In other words, the supporting citation should be a primary research source of the information, not a secondary source (such as a nonsystematic review article or a textbook) that simply cites the original source. Systematic reviews that analyze multiple RCTs are good sources for determining ratings of evidence.

The references should include the most current and important sources of support for key statements (i.e., studies referred to, new information, controversial material, specific quantitative data, and information that would not usually be found in most general reference textbooks). Generally, these references will be key evidence-based recommendations, meta-analyses, or landmark articles. Although some journals publish exhaustive lists of reference citations, AFP prefers to include a succinct list of key references. (We will make more extensive reference lists available on our Web site or provide links to your personal reference list.)

You may use the following checklist to ensure the completeness of your evidence-based review article; use the source list of reviews to identify important sources of evidence-based medicine materials.

Checklist for an Evidence-Based Clinical Review Article

The topic is common in family practice, especially topics in which there is new, important information about diagnosis or treatment.

The introduction defines the topic and the purpose of the review, and describes its relevance to family practice.

A table of CME objectives for the review is included.

The review states how you did your literature search and indicates what sources you checked to ensure a comprehensive assessment of relevant studies (e.g., MEDLINE, the Cochrane Collaboration Database, the Center for Research Support, TRIP Database).

Several sources of evidence-based reviews on the topic are evaluated ( Table 1 ) .

Where possible, POEM (dealing with changes in morbidity, mortality, or quality of life) rather than DOE (dealing with mechanistic explanations or surrogate end points, such as changes in laboratory tests) is used to support key clinical recommendations ( Table 2 ) .

Studies of patients likely to be representative of those in primary care practices, rather than subspecialty referral centers, are emphasized.

Studies that are not only statistically significant but also clinically significant are emphasized; e.g., interventions with meaningful changes in absolute risk reduction and low numbers needed to treat. (See http://www.cebm.net/index.aspx?o=1116 .) 11

The level of evidence for key clinical recommendations is labeled using the following rating scale: level A (RCT/meta-analysis), level B (other evidence), and level C (consensus/expert opinion).

Acknowledge controversies, recent developments, other viewpoints, and any apparent conflicts of interest or instances of bias that might affect the strength of the evidence presented.

Highlight key points about diagnosis and treatment in the discussion and include a summary table of key take-home points.

Use tables, figures, and illustrations to highlight key points and present a step-wise, algorithmic approach to diagnosis or treatment when possible.

Emphasize evidence-based guidelines and primary research studies, rather than other review articles, unless they are systematic reviews.

The essential elements of this checklist are summarized in Table 3 .

Siwek J. Reading and evaluating clinical review articles. Am Fam Physician. 1997;55:2064-2069.

Shaughnessy AF, Slawson DC. Getting the most from review articles: a guide for readers and writers. Am Fam Physician. 1997;55:2155-60.

Pitt B, Zannad F, Remme WJ, Cody R, Castaigne A, Perez A, et al. The effect of spironolactone on morbidity and mortality in patients with severe heart failure. N Engl J Med. 1999;341:709-17.

Flynn CA, D'Amico F, Smith G. Should we patch corneal abrasions? A meta-analysis. J Fam Pract. 1998;47:264-70.

Slawson DC, Shaughnessy AF, Bennett JH. Becoming a medical information master: feeling good about not knowing everything. J Fam Pract. 1994;38:505-13.

Shaughnessy AF, Slawson DC, Bennett JH. Becoming an information master: a guidebook to the medical information jungle. J Fam Pract. 1994;39:489-99.

Slawson DC, Shaughnessy AF. Becoming an information master: using POEMs to change practice with confidence. Patient-oriented evidence that matters. J Fam Pract. 2000;49:63-7.

Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, et al. Methods Work Group, Third U.S. Preventive Services Task Force. Current methods of the U.S. Preventive Services Task Force. A review of the process. Am J Prev Med. 2001;20(3 suppl):21-35.

CATbank topics: levels of evidence and grades of recommendations. Retrieved November 2001, from: http://www.cebm.net/ .

Saha S, Hoerger TJ, Pignone MP, Teutsch SM, Helfand M, Mandelblatt JS. for the Cost Work Group of the Third U.S. Preventive Services Task Force. The art and science of incorporating cost effectiveness into evidence-based recommendations for clinical preventive services. Am J Prev Med. 2001;20(3 suppl):36-43.

Evidence-based medicine glossary. Retrieved November 2001, from: http://www.cebm.net/index.aspx?o=1116 .

Continue Reading

More in afp, more in pubmed.

Copyright © 2002 by the American Academy of Family Physicians.

This content is owned by the AAFP. A person viewing it online may make one printout of the material and may use that printout only for his or her personal, non-commercial reference. This material may not otherwise be downloaded, copied, printed, stored, transmitted or reproduced in any medium, whether now known or later invented, except as authorized in writing by the AAFP.  See permissions  for copyright questions and/or permission requests.

Copyright © 2024 American Academy of Family Physicians. All Rights Reserved.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

How to write a good scientific review article

Affiliation.

• 1 The FEBS Journal Editorial Office, Cambridge, UK.
• PMID: 35792782
• DOI: 10.1111/febs.16565

Literature reviews are valuable resources for the scientific community. With research accelerating at an unprecedented speed in recent years and more and more original papers being published, review articles have become increasingly important as a means to keep up to date with developments in a particular area of research. A good review article provides readers with an in-depth understanding of a field and highlights key gaps and challenges to address with future research. Writing a review article also helps to expand the writer's knowledge of their specialist area and to develop their analytical and communication skills, amongst other benefits. Thus, the importance of building review-writing into a scientific career cannot be overstated. In this instalment of The FEBS Journal's Words of Advice series, I provide detailed guidance on planning and writing an informative and engaging literature review.

© 2022 Federation of European Biochemical Societies.

PubMed Disclaimer

Similar articles

• Rules to be adopted for publishing a scientific paper. Picardi N. Picardi N. Ann Ital Chir. 2016;87:1-3. Ann Ital Chir. 2016. PMID: 28474609
• How to write an original article. Mateu Arrom L, Huguet J, Errando C, Breda A, Palou J. Mateu Arrom L, et al. Actas Urol Esp (Engl Ed). 2018 Nov;42(9):545-550. doi: 10.1016/j.acuro.2018.02.011. Epub 2018 May 18. Actas Urol Esp (Engl Ed). 2018. PMID: 29779648 Review. English, Spanish.
• [Writing a scientific review, advice and recommendations]. Turale S. Turale S. Soins. 2013 Dec;(781):39-43. Soins. 2013. PMID: 24558688 French.
• How to write a research paper. Alexandrov AV. Alexandrov AV. Cerebrovasc Dis. 2004;18(2):135-8. doi: 10.1159/000079266. Epub 2004 Jun 23. Cerebrovasc Dis. 2004. PMID: 15218279 Review.
• How to write a review article. Williamson RC. Williamson RC. Hosp Med. 2001 Dec;62(12):780-2. doi: 10.12968/hosp.2001.62.12.2389. Hosp Med. 2001. PMID: 11810740 Review.
• A scoping review of the methodological approaches used in retrospective chart reviews to validate adverse event rates in administrative data. Connolly A, Kirwan M, Matthews A. Connolly A, et al. Int J Qual Health Care. 2024 May 10;36(2):mzae037. doi: 10.1093/intqhc/mzae037. Int J Qual Health Care. 2024. PMID: 38662407 Free PMC article. Review.
• Ado-tratuzumab emtansine beyond breast cancer: therapeutic role of targeting other HER2-positive cancers. Zheng Y, Zou J, Sun C, Peng F, Peng C. Zheng Y, et al. Front Mol Biosci. 2023 May 11;10:1165781. doi: 10.3389/fmolb.2023.1165781. eCollection 2023. Front Mol Biosci. 2023. PMID: 37251081 Free PMC article. Review.
• Connecting authors with readers: what makes a good review for the Korean Journal of Women Health Nursing. Kim HK. Kim HK. Korean J Women Health Nurs. 2023 Mar;29(1):1-4. doi: 10.4069/kjwhn.2023.02.23. Epub 2023 Mar 31. Korean J Women Health Nurs. 2023. PMID: 37037445 Free PMC article. No abstract available.
• Ketcham C, Crawford J. The impact of review articles. Lab Invest. 2007;87:1174-85. https://doi.org/10.1038/labinvest.3700688
• Muka T, Glisic M, Milic J, Verhoog S, Bohlius J, Bramer W, et al. A 24-step guide on how to design, conduct, and successfully publish a systematic review and meta-analysis in medical research. Eur J Epidemiol. 2020;35:49-60. https://doi.org/10.1007/s10654-019-00576-5
• Tawfik GM, Dila KAS, Mohamed MYF, Tam DNH, Kien ND, Ahmed AM, et al. A step by step guide for conducting a systematic review and meta-analysis with simulation data. Trop Med Health. 2019;47:46. https://doi.org/10.1186/s41182-019-0165-6
• Zimba O, Gasparyan AY. Scientific authorship: a primer for researchers. Reumatologia. 2020;58(6):345-9. https://doi.org/10.5114/reum.2020.101999
• Gasparyan AY, Yessirkepov M, Voronov AA, Maksaev AA, Kitas GD. Article-level metrics. J Korean Med Sci. 2021;36(11):e74.

Publication types

• Search in MeSH

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Writing an effective literature review

Part II: Citation technique

• The Writer’s Craft
• Open access
• Published: 02 March 2018
• Volume 7 , pages 133–135, ( 2018 )

Cite this article

You have full access to this open access article

• Lorelei Lingard 1  

17k Accesses

8 Citations

232 Altmetric

Explore all metrics

Avoid common mistakes on your manuscript.

In the Writer’s Craft section we offer simple tips to improve your writing in one of three areas: Energy, Clarity and Persuasiveness. Each entry focuses on a key writing feature or strategy, illustrates how it commonly goes wrong, teaches the grammatical underpinnings necessary to understand it and offers suggestions to wield it effectively. We encourage readers to share comments on or suggestions for this section on Twitter, using the hashtag: #how’syourwriting?

This Writer’s Craft instalment is the second in a two-part series that offers strategies for effectively presenting the literature review section of a research manuscript. This piece argues that citation is not just a technical practice but also a rhetorical one, and offers writers an expanded vocabulary for using citation to maximal effect.

Many writers think of citation as the formal system we use to avoid plagiarism and acknowledge others’ work. But citation is a much more nuanced practice than this. Not only does citation allow us to represent the source of knowledge, but it also allows us to position ourselves in relation to that knowledge, and to place that knowledge in relation to other knowledge . In short, citation is how we artfully tell the story of what the field knows, how it came to that knowledge, and where we stand in relation to it as we write the literature review section to frame our own work. Seen this way, citation is a sophisticated task, requiring in-depth knowledge of the literature in a domain.

Citation is more than just referencing; it is how we represent the social construction of knowledge in a field. A citation strategy is any indication in the text about the source and nature of knowledge. Consider the following passage, in which all citation strategies are italicized:

Despite years of effort to teach and enforce positive professional norms and standards, many reports of challenges to medical professionalism continue to appear, both in the medical and education literature and, often in reaction, in the lay press . 1,2,3,4,5 Examples of professional lapses dot the health care landscape: regulations are thwarted, records are falsified, patients are ignored, colleagues are berated. 2,4,6 The medical profession has articulated its sense of what professionalism is in a number of important position statements . 7,8 These statements tend to be built upon abstracted principles and values, such as the taxonomy presented in the American Board of Internal Medicine’s (ABIM’s) Project Professionalism : altruism, accountability, excellence, duty, honour, integrity, and respect for others. 7 (From Ginsburg et al., The anatomy of a professional lapse [ 1 ])

In this passage, citation as referencing (in the form of Vancouver format superscript numbers) is used to acknowledge the source of knowledge. There are more than just references in this passage, however. Citation strategies also include statements that characterize the density of that knowledge (‘many reports’), its temporal patterns (‘continue to appear’), its diverse origins (‘both in the medical and education literature’), its social nature (‘often in reaction’), and its social import (‘important position statements’). Citation does more than just acknowledge the source of something you’ve read. It is how you represent the social nature of knowledge as coming from somewhere, being debated and developed, and having impact on the world [ 2 ]. If we remove all these citation strategies, the passage sounds at best like common sense or, at worst, like unsubstantiated personal opinion:

Despite years of effort to teach and enforce positive professional norms and standards, challenges to medical professionalism continue. Examples of professional lapses dot the health care landscape: regulations are thwarted, records are falsified, patients are ignored, colleagues are berated. Professionalism is a set of principles and values: altruism, accountability, excellence, duty, honour, integrity, and respect for others.

But perhaps you’ve been told that your literature review should be ‘objective’—that you should simply present what is known without taking a stance on it. This is largely untrue, for two reasons. The first involves the distinction between summary and critical summary. A summary is a neutral description of material, but a good literature review contains very little pure summary because, as we review, we must also judge the quality, source and reliability of the knowledge claims we are presenting [ 3 ]. To do this, we engage in critical summary, not only summarizing existing knowledge but offering a stance on it.

The second reason is that, even when we’re aiming for simple summary, a completely neutral presentation of knowledge claims is very difficult to achieve. We take a stance in ways we hardly even notice. Consider how the verb in each of these statements adds a flavour of stance to what is otherwise a summary of a knowledge claim in the field:

Anderson describes how the assessment is overly time-consuming for use in the Emergency Department. Anderson discovered that the assessment was overly time-consuming for use in the Emergency Department. Anderson claims that the assessment is overly time-consuming for use in the Emergency Department.

The first verb, ‘describes’, is neutral: it is not possible to ascertain the writer’s stance on the knowledge Anderson has contributed to the field. The second verb, ‘discovered’, expresses an affiliation or positive stance in the writer, while the third verb, ‘claims’, distances the writer from Anderson’s work. Even these brief summary sentences contain a flavour of critical summary. This is not a flaw; in fact, it is an important method of portraying existing knowledge as a conversation in which the writer is positioning herself and her work. But it should be done consciously and strategically. Tab.  1 offers examples to help writers think about how the verbs in their literature review position them in relation to existing knowledge in the field. Meaning is subject to context and these examples should only be taken as a guide: e. g., ‘suggests’ can be used to signal neutrality or distancing.

Most of us have favourite verbs that we default to almost unconsciously when we are writing—reports, argues, describes, studies, explains, asserts—but these verbs are not interchangeable. They each inscribe a slightly different stance towards the knowledge—not only the writer’s stance, but also the stance of the researcher who created the knowledge. It is critical to get the original stance right in your critical summary. Nothing irritates me more than seeing my stance mispresented in someone else’s literature review. For example, if I wrote a paragraph offering tentative reflections on a new idea, I don’t want to see that summarized in someone’s literature review as ‘Lingard argues’, when more accurate would be ‘Lingard suggests’ or ‘Lingard explored’.

Writers need to extend their library of citation verbs to allow themselves to accurately and persuasively position knowledge claims published by authors in their field. You can find many online resources to help extend your vocabulary: Tab.  2 , adapted from one such online source [ 4 ], provides some suggestions. Tables like these should be thought of as tools, not rules—keep in mind that words have flexible meanings depending on context and purpose. This is why one word, such as suggest or conclude , can appear in more than one list.

Knowledge is a social construction and it accumulates as researchers debate, extend and refine one another’s contributions. To avoid your literature review reading like a laundry list of disconnected ‘facts’, reporting verbs are an important resource. Tab.  3 offers a selection of verbs organized to reflect different relationships among authors in the field of knowledge being reviewed.

Finally, although we have focused on citation verbs in this article, adverbs (e. g., similarly, consequently) and prepositional phrases (e. g., by contrast, in addition) are also important for expressing similar, contrasting or responding relations among knowledge claims and their authors in the field being reviewed.

In summary, an effective literature review not only summarizes existing knowledge, it also critically presents that knowledge to depict an evolving conversation and understanding in a particular domain of study. As writers we need to know when we are summarizing and when we are critically summarizing—summary alone makes for a literature review that reads like a laundry list of undigested ‘facts-in-the-world’. Finally, writers need to attend to the subtle power of citation verbs to position themselves and the authors they are citing in relation to the knowledge being reviewed. Broadening our catalogue of ‘go-to’ verbs is an important step in enlivening and strengthening our writing.

Ginsburg S, Regehr G, Stern D, Lingard L. The anatomy of the professional lapse: Bridging the gap between traditional frameworks and students’ perceptions. Acad Med. 2002;77:516–22.

Article   Google Scholar  

Giltrow J, Gooding R, Burgoyne D, Academic Writing SM. An Introduction. 3rd ed. ition. Peterborough, Ontario: Broadview Press; 2014.

Google Scholar  

Lingard L. Mapping the gap. Perspect Med Educ. 2017;6. https://doi.org/10.1007/s40037-017-0401-x .

Hampton M. Writing about others’ work: verbs for citations. Department of Curriculum and Quality Enhancement. http://www.port.ac.uk/media/contacts-and-departments/student-support-services/ask/downloads/Verbs-for-citation.pdf Accessed 7 September 2017.

Download references

Acknowledgements

Thanks to Mark Goldszmidt for his feedback on an early version of this manuscript.

Author information

Authors and affiliations.

Schulich School of Medicine & Dentistry, Health Sciences Addition, Western University, London, Ontario, Canada

Lorelei Lingard

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Lorelei Lingard .

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Lingard, L. Writing an effective literature review. Perspect Med Educ 7 , 133–135 (2018). https://doi.org/10.1007/s40037-018-0407-z

Download citation

Published : 02 March 2018

Issue Date : April 2018

DOI : https://doi.org/10.1007/s40037-018-0407-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research

Writing in the Health Sciences: Research and Lit Reviews

• Research and Lit Reviews
• Tables and Figures
• Citation Management
• Further Reference

What Is a Literature Review?

In simple terms, a literature review investigates the available information on a certain topic. It may be only a knowledge survey with an intentional focus. However, it is often a well-organized examination of the existing research which evaluates each resource in a systematic way. Often a lit review will involve a series of inclusion/exclusion criteria or an assessment rubric which examines the research in-depth. Below are some interesting sources to consider.

The Writing Center's Literature Reviews - UNC-Chapel Hill's writing center explains some of the key criteria involved in doing a literature review.

Literature Review vs. Systematic Review - This recent article details the difference between a literature review and a systematic review. Though the two share similar attributes, key differences are identified here.

Literature Review Steps

1. Identify a research question. For example: "Does the use of warfarin in elderly patients recovering from myocardial infarction help prevent stroke?"

2. Consider which databases might provide information for your topic. Often PubMed or CINAHL will cover a wide spectrum of biomedical issues. However, other databases and grey literature sources may specialize in certain disciplines. Embase is generally comprehensive but also specializes in pharmacological interventions.

3. Select the major subjects or ideas from your question.  Focus in on the particular concepts involved in your research. Then brainstorm synonyms and related terminology for these topics.

4. Look for the  preferred indexing terms for each concept in your question. This is especially important with databases such as PubMed, CINAHL, or Scopus where headings within the MeSH database or under the Emtree umbrella are present.  For example, the above question's keywords such as " warfarin " or "myocardial infarction" can involve related terminology or subject headings such as "anti-coagulants" or "cardiovascular disease."

5. Build your search using boolean operators. Combine the synonyms in your database using boolean operators such as AND or OR. Sometimes it is necessary to research parts of a question rather than the whole. So you might link searches for things like the preventive effects of anti-coagulants with stroke or embolism, then AND these results with the therapy for patients with cardiovascular disease.

6. Filter and save your search results from the first database (do this for all databases). This may be a short list because of your topic's limitations, but it should be no longer than 15 articles for an initial search. Make sure your list is saved or archived and presents you with what's needed to access the full text.

7. Use the same process with the next databases on your list. But pay attention to how certain major headings may alter the terminology. "Stroke" may have a suggested term of "embolism" or even "cerebrovascular incident" depending on the database.

8. Read through the material for inclusion/exclusion . Based on your project's criteria and objective, consider which studies or reviews deserve to be included and which should be discarded. Make sure the information you have permits you to go forward. 

9. Write the literature review. Begin by summarizing why your research is important and explain why your approach will help fill gaps in current knowledge. Then incorporate how the information you've selected will help you to do this. You do not need to write about all of the included research you've chosen, only the most pe rtinent.

10. Select the most relevant literature for inclusion in the body of your report. Choose the articles and data sets that are most particularly relevant to your experimental approach. Consider how you might arrange these sources in the body of your draft. 

Library Books

Call #: WZ 345 G192h 2011

ISBN #: 9780763771867

This book details a practical, step-by-step method for conducting a literature review in the health sciences. Aiming to  synthesize the information while also analyzing it, the Matrix Indexing System enables users to establish a  structured process for tracking, organizing and integrating the knowledge within a collection.

Key Research Databases

PubMed -  The premier medical database for review articles in medicine, nursing, healthcare, other related biomedical disciplines. PubMed contains over 20 million citations and can be navigated through multiple database capabilities and searching strategies.

CINAHL Ultimate - Offers comprehensive coverage of health science literature. CINAHL is particularly useful for those researching the allied disciplines of nursing, medicine, and pharmaceutical sciences.

Scopus - Database with over 12 million abstracts and citations which include peer-reviewed titles from international and Open Access journals. Also includes interactive bibliometrics and researcher profiling.

Embase - Elsevier's fully interoperable database of both Medline and Emtree-indexed articles. Embase also specializes in pharmacologic interventions.

Cochrane - Selected evidence-based medicine resources from the Cochrane Collaboration that includes peer-reviewed systematic reviews and randomized controlled trials. Access this database through OVID with TTUHSC Libraries.

DARE - Literally the Datatase of Abstracts of Reviews of Effectiveness, this collection of systematic reviews and other evidence-based research contains critical assessments from a wide variety of medical journals.

TRIP - This TRIP database is structured according to the level of evidence for its EBM content. It allows users to quickly and easily locate high-quality, accredited medical literature for clinical and research purposes.

Web of Science - Contains bibliographic articles and data from a wide variety of publications in the life sciences and other fields. Also, see this link for conducting a lit review exclusively within Web of Science.

• << Previous: Welcome
• Next: Drafting >>
• Last Updated: Sep 29, 2023 10:07 AM
• URL: https://ttuhsc.libguides.com/Writing_HealthSciences

• University of Detroit Mercy
• Health Professions

Health Operations Management

• Writing a Literature Review
• Find Articles (Databases)
• Evidence-based Practice
• eBooks & Articles
• General Writing Support
• Creating & Printing Posters
• Research Project Web Resources
• Statistics: Health / Medical
• Searching Tips
• Streaming Video
• Database & Library Help
• Medical Apps & Mobile Sites
• Faculty Publications

Literature Review Overview

What is a Literature Review? Why Are They Important?

A literature review is important because it presents the "state of the science" or accumulated knowledge on a specific topic. It summarizes, analyzes, and compares the available research, reporting study strengths and weaknesses, results, gaps in the research, conclusions, and authors’ interpretations.

Tips and techniques for conducting a literature review are described more fully in the subsequent boxes:

• Literature review steps
• Strategies for organizing the information for your review
• Literature reviews sections
• In-depth resources to assist in writing a literature review
• Templates to start your review
• Literature review examples

Literature Review Steps

Graphic used with permission: Torres, E. Librarian, Hawai'i Pacific University

1. Choose a topic and define your research question

• Try to choose a topic of interest. You will be working with this subject for several weeks to months.
• Ideas for topics can be found by scanning medical news sources (e.g MedPage Today), journals / magazines, work experiences, interesting patient cases, or family or personal health issues.
• Do a bit of background reading on topic ideas to familiarize yourself with terminology and issues. Note the words and terms that are used.
• Develop a focused research question using PICO(T) or other framework (FINER, SPICE, etc - there are many options) to help guide you.
• Run a few sample database searches to make sure your research question is not too broad or too narrow.
• If possible, discuss your topic with your professor. 

2. Determine the scope of your review

The scope of your review will be determined by your professor during your program. Check your assignment requirements for parameters for the Literature Review.

• How many studies will you need to include?
• How many years should it cover? (usually 5-7 depending on the professor)
• For the nurses, are you required to limit to nursing literature?

3. Develop a search plan

• Determine which databases to search. This will depend on your topic. If you are not sure, check your program specific library website (Physician Asst / Nursing / Health Services Admin) for recommendations.
• Create an initial search string using the main concepts from your research (PICO, etc) question. Include synonyms and related words connected by Boolean operators
• Contact your librarian for assistance, if needed.

4. Conduct searches and find relevant literature

• Keep notes as you search - tracking keywords and search strings used in each database in order to avoid wasting time duplicating a search that has already been tried
• Read abstracts and write down new terms to search as you find them
• Check MeSH or other subject headings listed in relevant articles for additional search terms
• Scan author provided keywords if available
• Check the references of relevant articles looking for other useful articles (ancestry searching)
• Check articles that have cited your relevant article for more useful articles (descendancy searching). Both PubMed and CINAHL offer Cited By links
• Revise the search to broaden or narrow your topic focus as you peruse the available literature
• Conducting a literature search is a repetitive process. Searches can be revised and re-run multiple times during the process.
• Track the citations for your relevant articles in a software citation manager such as RefWorks, Zotero, or Mendeley

5. Review the literature

• Read the full articles. Do not rely solely on the abstracts. Authors frequently cannot include all results within the confines of an abstract. Exclude articles that do not address your research question.
• While reading, note research findings relevant to your project and summarize. Are the findings conflicting? There are matrices available than can help with organization. See the Organizing Information box below.
• Critique / evaluate the quality of the articles, and record your findings in your matrix or summary table. Tools are available to prompt you what to look for. (See Resources for Appraising a Research Study box on the HSA, Nursing , and PA guides )
• You may need to revise your search and re-run it based on your findings.

6. Organize and synthesize

• Compile the findings and analysis from each resource into a single narrative.
• Using an outline can be helpful. Start broad, addressing the overall findings and then narrow, discussing each resource and how it relates to your question and to the other resources.
• Cite as you write to keep sources organized.
• Write in structured paragraphs using topic sentences and transition words to draw connections, comparisons, and contrasts.
• Don't present one study after another, but rather relate one study's findings to another. Speak to how the studies are connected and how they relate to your work.

Organizing Information

Options to assist in organizing sources and information :

1. Synthesis Matrix

• helps provide overview of the literature
• information from individual sources is entered into a grid to enable writers to discern patterns and themes
• article summary, analysis, or results
• thoughts, reflections, or issues
• each reference gets its own row
• mind maps, concept maps, flowcharts
• at top of page record PICO or research question
• record major concepts / themes from literature
• list concepts that branch out from major concepts underneath - keep going downward hierarchically, until most specific ideas are recorded
• enclose concepts in circles and connect the concept with lines - add brief explanation as needed

3. Summary Table

• information is recorded in a grid to help with recall and sorting information when writing
• allows comparing and contrasting individual studies easily
• purpose of study
• methodology (study population, data collection tool)

Efron, S. E., & Ravid, R. (2019). Writing the literature review : A practical guide . Guilford Press.

Literature Review Sections

• Lit reviews can be part of a larger paper / research study or they can be the focus of the paper
• Lit reviews focus on research studies to provide evidence
• New topics may not have much that has been published

* The sections included may depend on the purpose of the literature review (standalone paper or section within a research paper)

Standalone Literature Review (aka Narrative Review):

• presents your topic or PICO question
• includes the why of the literature review and your goals for the review.
• provides background for your the topic and previews the key points
• Narrative Reviews: tmay not have an explanation of methods.
• include where the search was conducted (which databases) what subject terms or keywords were used, and any limits or filters that were applied and why - this will help others re-create the search
• describe how studies were analyzed for inclusion or exclusion
• review the purpose and answer the research question
• thematically - using recurring themes in the literature
• chronologically - present the development of the topic over time
• methodological - compare and contrast findings based on various methodologies used to research the topic (e.g. qualitative vs quantitative, etc.)
• theoretical - organized content based on various theories
• provide an overview of the main points of each source then synthesize the findings into a coherent summary of the whole
• present common themes among the studies
• compare and contrast the various study results
• interpret the results and address the implications of the findings
• do the results support the original hypothesis or conflict with it
• provide your own analysis and interpretation (eg. discuss the significance of findings; evaluate the strengths and weaknesses of the studies, noting any problems)
• discuss common and unusual patterns and offer explanations
•  stay away from opinions, personal biases and unsupported recommendations
• summarize the key findings and relate them back to your PICO/research question
• note gaps in the research and suggest areas for further research
• this section should not contain "new" information that had not been previously discussed in one of the sections above
• provide a list of all the studies and other sources used in proper APA 7

Literature Review as Part of a Research Study Manuscript:

• Compares the study with other research and includes how a study fills a gap in the research.
• Focus on the body of the review which includes the synthesized Findings and Discussion

Literature Reviews vs Systematic Reviews

Systematic Reviews are NOT the same as a Literature Review:

Literature Reviews:

• Literature reviews may or may not follow strict systematic methods to find, select, and analyze articles, but rather they selectively and broadly review the literature on a topic
• Research included in a Literature Review can be "cherry-picked" and therefore, can be very subjective

Systematic Reviews:

• Systemic reviews are designed to provide a comprehensive summary of the evidence for a focused research question
• rigorous and strictly structured, using standardized reporting guidelines (e.g. PRISMA, see link below)
• uses exhaustive, systematic searches of all relevant databases
• best practice dictates search strategies are peer reviewed
• uses predetermined study inclusion and exclusion criteria in order to minimize bias
• aims to capture and synthesize all literature (including unpublished research - grey literature) that meet the predefined criteria on a focused topic resulting in high quality evidence

Literature Review Examples

• Breastfeeding initiation and support: A literature review of what women value and the impact of early discharge (2017). Women and Birth : Journal of the Australian College of Midwives
• Community-based participatory research to promote healthy diet and nutrition and prevent and control obesity among African-Americans: A literature review (2017). Journal of Racial and Ethnic Health Disparities

• Vitamin D deficiency in individuals with a spinal cord injury: A literature review (2017). Spinal Cord

Resources for Writing a Literature Review

These sources have been used in developing this guide.

Resources Used on This Page

Aveyard, H. (2010). Doing a literature review in health and social care : A practical guide . McGraw-Hill Education.

Purdue Online Writing Lab. (n.d.). Writing a literature review . Purdue University. https://owl.purdue.edu/owl/research_and_citation/conducting_research/writing_a_literature_review.html

Torres, E. (2021, October 21). Nursing - graduate studies research guide: Literature review. Hawai'i Pacific University Libraries. Retrieved January 27, 2022, from https://hpu.libguides.com/c.php?g=543891&p=3727230

• << Previous: General Writing Support
• Next: Creating & Printing Posters >>
• Last Updated: Aug 29, 2024 1:07 PM
• URL: https://udmercy.libguides.com/hsa

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 19, Issue 1
• Reviewing the literature
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Joanna Smith 1 ,
• Helen Noble 2
• 1 School of Healthcare, University of Leeds , Leeds , UK
• 2 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• Correspondence to Dr Joanna Smith , School of Healthcare, University of Leeds, Leeds LS2 9JT, UK; j.e.smith1{at}leeds.ac.uk

https://doi.org/10.1136/eb-2015-102252

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review.

Are there different approaches to undertaking a literature review?

What stages are required to undertake a literature review.

The rationale for the review should be established; consider why the review is important and relevant to patient care/safety or service delivery. For example, Noble et al 's 4 review sought to understand and make recommendations for practice and research in relation to dialysis refusal and withdrawal in patients with end-stage renal disease, an area of care previously poorly described. If appropriate, highlight relevant policies and theoretical perspectives that might guide the review. Once the key issues related to the topic, including the challenges encountered in clinical practice, have been identified formulate a clear question, and/or develop an aim and specific objectives. The type of review undertaken is influenced by the purpose of the review and resources available. However, the stages or methods used to undertake a review are similar across approaches and include:

Formulating clear inclusion and exclusion criteria, for example, patient groups, ages, conditions/treatments, sources of evidence/research designs;

Justifying data bases and years searched, and whether strategies including hand searching of journals, conference proceedings and research not indexed in data bases (grey literature) will be undertaken;

Developing search terms, the PICU (P: patient, problem or population; I: intervention; C: comparison; O: outcome) framework is a useful guide when developing search terms;

Developing search skills (eg, understanding Boolean Operators, in particular the use of AND/OR) and knowledge of how data bases index topics (eg, MeSH headings). Working with a librarian experienced in undertaking health searches is invaluable when developing a search.

Once studies are selected, the quality of the research/evidence requires evaluation. Using a quality appraisal tool, such as the Critical Appraisal Skills Programme (CASP) tools, 5 results in a structured approach to assessing the rigour of studies being reviewed. 3 Approaches to data synthesis for quantitative studies may include a meta-analysis (statistical analysis of data from multiple studies of similar designs that have addressed the same question), or findings can be reported descriptively. 6 Methods applicable for synthesising qualitative studies include meta-ethnography (themes and concepts from different studies are explored and brought together using approaches similar to qualitative data analysis methods), narrative summary, thematic analysis and content analysis. 7 Table 1 outlines the stages undertaken for a published review that summarised research about parents’ experiences of living with a child with a long-term condition. 8

• View inline

An example of rapid evidence assessment review

In summary, the type of literature review depends on the review purpose. For the novice reviewer undertaking a review can be a daunting and complex process; by following the stages outlined and being systematic a robust review is achievable. The importance of literature reviews should not be underestimated—they help summarise and make sense of an increasingly vast body of research promoting best evidence-based practice.

• ↵ Centre for Reviews and Dissemination . Guidance for undertaking reviews in health care . 3rd edn . York : CRD, York University , 2009 .
• ↵ Canadian Best Practices Portal. http://cbpp-pcpe.phac-aspc.gc.ca/interventions/selected-systematic-review-sites / ( accessed 7.8.2015 ).
• Bridges J , et al
• ↵ Critical Appraisal Skills Programme (CASP). http://www.casp-uk.net / ( accessed 7.8.2015 ).
• Dixon-Woods M ,
• Shaw R , et al
• Agarwal S ,
• Jones D , et al
• Cheater F ,

Twitter Follow Joanna Smith at @josmith175

Competing interests None declared.

Read the full text or download the PDF:

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health (m-health) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers