research paper on peer review

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Peer Review? | Types & Examples

What Is Peer Review? | Types & Examples

Published on December 17, 2021 by Tegan George . Revised on June 22, 2023.

Peer review, sometimes referred to as refereeing , is the process of evaluating submissions to an academic journal. Using strict criteria, a panel of reviewers in the same subject area decides whether to accept each submission for publication.

Peer-reviewed articles are considered a highly credible source due to the stringent process they go through before publication.

There are various types of peer review. The main difference between them is to what extent the authors, reviewers, and editors know each other’s identities. The most common types are:

• Single-blind review
• Double-blind review
• Triple-blind review

Collaborative review

Open review.

Relatedly, peer assessment is a process where your peers provide you with feedback on something you’ve written, based on a set of criteria or benchmarks from an instructor. They then give constructive feedback, compliments, or guidance to help you improve your draft.

Table of contents

What is the purpose of peer review, types of peer review, the peer review process, providing feedback to your peers, peer review example, advantages of peer review, criticisms of peer review, other interesting articles, frequently asked questions about peer reviews.

Many academic fields use peer review, largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the manuscript. For this reason, academic journals are among the most credible sources you can refer to.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Prevent plagiarism. Run a free check.

Depending on the journal, there are several types of peer review.

Single-blind peer review

The most common type of peer review is single-blind (or single anonymized) review . Here, the names of the reviewers are not known by the author.

While this gives the reviewers the ability to give feedback without the possibility of interference from the author, there has been substantial criticism of this method in the last few years. Many argue that single-blind reviewing can lead to poaching or intellectual theft or that anonymized comments cause reviewers to be too harsh.

Double-blind peer review

In double-blind (or double anonymized) review , both the author and the reviewers are anonymous.

Arguments for double-blind review highlight that this mitigates any risk of prejudice on the side of the reviewer, while protecting the nature of the process. In theory, it also leads to manuscripts being published on merit rather than on the reputation of the author.

Triple-blind peer review

While triple-blind (or triple anonymized) review —where the identities of the author, reviewers, and editors are all anonymized—does exist, it is difficult to carry out in practice.

Proponents of adopting triple-blind review for journal submissions argue that it minimizes potential conflicts of interest and biases. However, ensuring anonymity is logistically challenging, and current editing software is not always able to fully anonymize everyone involved in the process.

In collaborative review , authors and reviewers interact with each other directly throughout the process. However, the identity of the reviewer is not known to the author. This gives all parties the opportunity to resolve any inconsistencies or contradictions in real time, and provides them a rich forum for discussion. It can mitigate the need for multiple rounds of editing and minimize back-and-forth.

Collaborative review can be time- and resource-intensive for the journal, however. For these collaborations to occur, there has to be a set system in place, often a technological platform, with staff monitoring and fixing any bugs or glitches.

Lastly, in open review , all parties know each other’s identities throughout the process. Often, open review can also include feedback from a larger audience, such as an online forum, or reviewer feedback included as part of the final published product.

While many argue that greater transparency prevents plagiarism or unnecessary harshness, there is also concern about the quality of future scholarship if reviewers feel they have to censor their comments.

In general, the peer review process includes the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to the author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits and resubmit it to the editor for publication.

In an effort to be transparent, many journals are now disclosing who reviewed each article in the published product. There are also increasing opportunities for collaboration and feedback, with some journals allowing open communication between reviewers and authors.

It can seem daunting at first to conduct a peer review or peer assessment. If you’re not sure where to start, there are several best practices you can use.

Summarize the argument in your own words

Summarizing the main argument helps the author see how their argument is interpreted by readers, and gives you a jumping-off point for providing feedback. If you’re having trouble doing this, it’s a sign that the argument needs to be clearer, more concise, or worded differently.

If the author sees that you’ve interpreted their argument differently than they intended, they have an opportunity to address any misunderstandings when they get the manuscript back.

Separate your feedback into major and minor issues

It can be challenging to keep feedback organized. One strategy is to start out with any major issues and then flow into the more minor points. It’s often helpful to keep your feedback in a numbered list, so the author has concrete points to refer back to.

Major issues typically consist of any problems with the style, flow, or key points of the manuscript. Minor issues include spelling errors, citation errors, or other smaller, easy-to-apply feedback.

Tip: Try not to focus too much on the minor issues. If the manuscript has a lot of typos, consider making a note that the author should address spelling and grammar issues, rather than going through and fixing each one.

The best feedback you can provide is anything that helps them strengthen their argument or resolve major stylistic issues.

Give the type of feedback that you would like to receive

No one likes being criticized, and it can be difficult to give honest feedback without sounding overly harsh or critical. One strategy you can use here is the “compliment sandwich,” where you “sandwich” your constructive criticism between two compliments.

Be sure you are giving concrete, actionable feedback that will help the author submit a successful final draft. While you shouldn’t tell them exactly what they should do, your feedback should help them resolve any issues they may have overlooked.

As a rule of thumb, your feedback should be:

• Easy to understand
• Constructive

Here's why students love Scribbr's proofreading services

Discover proofreading & editing

Below is a brief annotated research example. You can view examples of peer feedback by hovering over the highlighted sections.

Influence of phone use on sleep

Studies show that teens from the US are getting less sleep than they were a decade ago (Johnson, 2019) . On average, teens only slept for 6 hours a night in 2021, compared to 8 hours a night in 2011. Johnson mentions several potential causes, such as increased anxiety, changed diets, and increased phone use.

The current study focuses on the effect phone use before bedtime has on the number of hours of sleep teens are getting.

For this study, a sample of 300 teens was recruited using social media, such as Facebook, Instagram, and Snapchat. The first week, all teens were allowed to use their phone the way they normally would, in order to obtain a baseline.

The sample was then divided into 3 groups:

• Group 1 was not allowed to use their phone before bedtime.
• Group 2 used their phone for 1 hour before bedtime.
• Group 3 used their phone for 3 hours before bedtime.

All participants were asked to go to sleep around 10 p.m. to control for variation in bedtime . In the morning, their Fitbit showed the number of hours they’d slept. They kept track of these numbers themselves for 1 week.

Two independent t tests were used in order to compare Group 1 and Group 2, and Group 1 and Group 3. The first t test showed no significant difference ( p > .05) between the number of hours for Group 1 ( M = 7.8, SD = 0.6) and Group 2 ( M = 7.0, SD = 0.8). The second t test showed a significant difference ( p < .01) between the average difference for Group 1 ( M = 7.8, SD = 0.6) and Group 3 ( M = 6.1, SD = 1.5).

This shows that teens sleep fewer hours a night if they use their phone for over an hour before bedtime, compared to teens who use their phone for 0 to 1 hours.

Peer review is an established and hallowed process in academia, dating back hundreds of years. It provides various fields of study with metrics, expectations, and guidance to ensure published work is consistent with predetermined standards.

• Protects the quality of published research

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. Any content that raises red flags for reviewers can be closely examined in the review stage, preventing plagiarized or duplicated research from being published.

• Gives you access to feedback from experts in your field

Peer review represents an excellent opportunity to get feedback from renowned experts in your field and to improve your writing through their feedback and guidance. Experts with knowledge about your subject matter can give you feedback on both style and content, and they may also suggest avenues for further research that you hadn’t yet considered.

• Helps you identify any weaknesses in your argument

Peer review acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process. This way, you’ll end up with a more robust, more cohesive article.

While peer review is a widely accepted metric for credibility, it’s not without its drawbacks.

• Reviewer bias

The more transparent double-blind system is not yet very common, which can lead to bias in reviewing. A common criticism is that an excellent paper by a new researcher may be declined, while an objectively lower-quality submission by an established researcher would be accepted.

• Delays in publication

The thoroughness of the peer review process can lead to significant delays in publishing time. Research that was current at the time of submission may not be as current by the time it’s published. There is also high risk of publication bias , where journals are more likely to publish studies with positive findings than studies with negative findings.

• Risk of human error

By its very nature, peer review carries a risk of human error. In particular, falsification often cannot be detected, given that reviewers would have to replicate entire experiments to ensure the validity of results.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Normal distribution
• Measures of central tendency
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Thematic analysis
• Discourse analysis
• Cohort study
• Ethnography

Research bias

• Implicit bias
• Cognitive bias
• Conformity bias
• Hawthorne effect
• Availability heuristic
• Attrition bias
• Social desirability bias

Peer review is a process of evaluating submissions to an academic journal. Utilizing rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication. For this reason, academic journals are often considered among the most credible sources you can use in a research project– provided that the journal itself is trustworthy and well-regarded.

In general, the peer review process follows the following steps: 

• Reject the manuscript and send it back to author, or 
• Send it onward to the selected peer reviewer(s) 
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made. 
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field. It acts as a first defense, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure. 

A credible source should pass the CRAAP test  and follow these guidelines:

• The information should be up to date and current.
• The author and publication should be a trusted authority on the subject you are researching.
• The sources the author cited should be easy to find, clear, and unbiased.
• For a web source, the URL and layout should signify that it is trustworthy.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

George, T. (2023, June 22). What Is Peer Review? | Types & Examples. Scribbr. Retrieved September 3, 2024, from https://www.scribbr.com/methodology/peer-review/

Is this article helpful?

Tegan George

Other students also liked, what are credible sources & how to spot them | examples, ethical considerations in research | types & examples, applying the craap test & evaluating sources, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 12 November 2021

Demystifying the process of scholarly peer-review: an autoethnographic investigation of feedback literacy of two award-winning peer reviewers

• Sin Wang Chong   ORCID: orcid.org/0000-0002-4519-0544 1 &
• Shannon Mason 2  

Humanities and Social Sciences Communications volume  8 , Article number:  266 ( 2021 ) Cite this article

3259 Accesses

8 Citations

18 Altmetric

Metrics details

• Language and linguistics

A Correction to this article was published on 26 November 2021

This article has been updated

Peer reviewers serve a vital role in assessing the value of published scholarship and improving the quality of submitted manuscripts. To provide more appropriate and systematic support to peer reviewers, especially those new to the role, this study documents the feedback practices and experiences of two award-winning peer reviewers in the field of education. Adopting a conceptual framework of feedback literacy and an autoethnographic-ecological lens, findings shed light on how the two authors design opportunities for feedback uptake, navigate responsibilities, reflect on their feedback experiences, and understand journal standards. Informed by ecological systems theory, the reflective narratives reveal how they unravel the five layers of contextual influences on their feedback practices as peer reviewers (micro, meso, exo, macro, chrono). Implications related to peer reviewer support are discussed and future research directions are proposed.

Similar content being viewed by others

The transformative power of values-enacted scholarship

What matters in the cultivation of student feedback literacy: exploring university efl teachers’ perceptions and practices, why english exploring chinese early career returnee academics’ motivations for writing and publishing in english, introduction.

The peer-review process is the longstanding method by which research quality is assured. On the one hand, it aims to assess the quality of a manuscript, with the desired outcome being (in theory if not always in practice) that only research that has been conducted according to methodological and ethical principles be published in reputable journals and other dissemination outlets (Starck, 2017 ). On the other hand, it is seen as an opportunity to improve the quality of manuscripts, as peers identify errors and areas of weakness, and offer suggestions for improvement (Kelly et al., 2014 ). Whether or not peer review is actually successful in these areas is open to considerable debate, but in any case it is the “critical juncture where scientific work is accepted for publication or rejected” (Heesen and Bright, 2020 , p. 2). In contemporary academia, where higher education systems across the world are contending with decreasing levels of public funding, there is increasing pressure on researchers to be ‘productive’, which is largely measured by the number of papers published, and of funding grants awarded (Kandiko, 2010 ), both of which involve peer review.

Researchers are generally invited to review manuscripts once they have established themselves in their disciplinary field through publication of their own research. This means that for early career researchers (ECRs), their first exposure to the peer-review process is generally as an author. These early experiences influence the ways ECRs themselves conduct peer review. However, negative experiences can have a profound and lasting impact on researchers’ professional identity. This appears to be particularly true when feedback is perceived to be unfair, with feedback tone largely shaping author experience (Horn, 2016 ). In most fields, reviewers remain anonymous to ensure freedom to give honest and critical feedback, although there are concerns that a lack of accountability can result in ‘bad’ and ‘rude’ reviews (Mavrogenis et al., 2020 ). Such reviews can negatively impact all researchers, but disproportionately impact underrepresented researchers (Silbiger and Stubler, 2019 ). Regardless of career phase, no one is served well by unprofessional reviews, which contribute to the ongoing problem of bullying and toxicity prevalent in academia, with serious implications on the health and well-being of researchers (Keashly and Neuman, 2010 ).

Because of its position as the central process through which research is vetted and refined, peer review should play a similarly central role in researcher training, although it rarely features. In surveying almost 3000 researchers, Warne ( 2016 ) found that support for reviewers was mostly received “in the form of journal guidelines or informally as advice from supervisors or colleagues” (p. 41), with very few engaging in formal training. Among more than 1600 reviewers of 41 nursing journals, only one third received any form of support (Freda et al., 2009 ), with participants across both of these studies calling for further training. In light of the lack of widespread formal training, most researchers learn ‘on the job’, and little is known about how researchers develop their knowledge and skills in providing effective assessment feedback to their peers. In this study, we undertake such an investigation, by drawing on our first-hand experiences. Through a collaborative and reflective process, we look to identify the forms and forces of our feedback literacy development, and seek to answer specifically the following research questions:

What are the exhibited features of peer reviewer feedback literacy?

What are the forces at work that affect the development of feedback literacy?

Literature review

Conceptualisation of feedback literacy.

The notion of feedback literacy originates from the research base of new literacy studies, which examines ‘literacies’ from a sociocultural perspective (Gee, 1999 ; Street, 1997 ). In the educational context, one of the most notable types of literacy is assessment literacy (Stiggins, 1999 ). Traditionally, assessment literacy is perceived as one of the indispensable qualities of a successful educator, which refers to the skills and knowledge for teachers “to deal with the new world of assessment” (Fulcher, 2012 , p. 115). Following this line of teacher-oriented assessment literacy, recent attempts have been made to develop more subject-specific assessment literacy constructs (e.g., Levi and Inbar-Lourie, 2019 ). Given the rise of student-centred approaches and formative assessment in higher education, researchers began to make the case for students to be ‘assessment literate’; comprising of such knowledge and skills as understanding of assessment standards, the relationship between assessment and learning, peer assessment, and self-assessment skills (Price et al., 2012 ). Feedback literacy, as argued by Winstone and Carless ( 2019 ), is essentially a subset of assessment literacy because “part of learning through assessment is using feedback to calibrate evaluative judgement” (p. 24). The notion of feedback literacy was first extensively discussed by Sutton ( 2012 ) and more recently by Carless and Boud ( 2018 ). Focusing on students’ feedback literacy, Sutton ( 2012 ) conceptualised feedback literacy as a three-dimensional construct—an epistemological dimension (what do I know about feedback?), an ontological dimension (How capable am I to understand feedback?), and a practical dimension (How can I engage with feedback?). In close alignment with Sutton’s construct, the seminal conceptual paper by Carless and Boud ( 2018 ) further illustrated the four distinctive abilities of feedback literate students: the abilities to (1) understand the formative role of feedback, (2) make informed and accurate evaluative judgement against standards, (3) manage emotions especially in the face of critical and harsh feedback, and (4) take action based on feedback. Since the publication of Carless and Boud ( 2018 ), student and teacher feedback literacy has been in the limelight of assessment research in higher education (e.g., Chong 2021b ; Carless and Winstone 2020 ). These conceptual contributions expand the notion of feedback literacy to consider not only the manifestations of various forms of effective student engagement with feedback but also the confluence of contexts and individual differences of students in developing students’ feedback literacy by drawing upon various theoretical perspectives (e.g., ecological systems theory; sociomaterial perspective) and disciplines (e.g., business and human resource management). Others address practicalities of feedback literacy; for example, how teachers and students can work in synergy to develop feedback literacy (Carless and Winstone, 2020 ) and ways to maximise student engagement with feedback at a curricular level (Malecka et al., 2020). In addition to conceptualisation, advancement of the notion of feedback literacy is evident in the recent proliferation of primary studies. The majority of these studies are conducted in the field of higher education, focusing mostly on student feedback literacy in classrooms (e.g., Molloy et al., 2019 ; Winstone et al., 2019 ) and in the workplace (Noble et al., 2020 ), with a handful focused on teacher feedback literacy (e.g., Xu and Carless 2016 ). Some studies focusing on student feedback literacy adopt a qualitative case study research design to delve into individual students’ experience of engaging with various forms of feedback. For example, Han and Xu ( 2019 ) analysed the profiles of feedback literacy of two Chinese undergraduate students. Findings uncovered students’ resistance to engagement with feedback, which relates to the misalignment between the cognitive, social, and affective components of individual students’ feedback literacy profiles. Others reported interventions designed to facilitate students’ uptake of feedback, focusing on their effectiveness and students’ perceptions. Specifically, affordances and constraints of educational technology such as electronic feedback portfolio (Chong, 2019 ; Winstone et al., 2019 ) are investigated. Of particular interest is a recent study by Noble et al. ( 2020 ), which looked into student feedback literacy in the workplace by probing into the perceptions of a group of Australian healthcare students towards a feedback literacy training programme conducted prior to their placement. There is, however, a dearth of primary research in other areas where elicitation, process, and enactment of feedback are vital; for instance, academics’ feedback literacy. In the ‘publish or perish’ culture of higher education, academics, especially ECRs, face immense pressure to publish in top-tiered journals in their fields and face the daunting peer-review process, while juggling other teaching and administrative responsibilities (Hollywood et al., 2019 ; Tynan and Garbett 2007 ). Taking up the role of authors and reviewers, researchers have to possess the capacity and disposition to engage meaningfully with feedback provided by peer reviewers and to provide constructive comments to authors. Similar to students, researchers have to learn how to manage their emotions in the face of critical feedback, to understand the formative values of feedback, and to make informed judgements about the quality of feedback (Gravett et al., 2019 ). At the same time, feedback literacy of academics also resembles that of teachers. When considering the kind of feedback given to authors, academics who serve as peer reviewers have to (1) design opportunities for feedback uptake, (2) maintain a professional and supportive relationship with authors, and (3) take into account the practical dimension of giving feedback (e.g., how to strike a balance between quality of feedback and time constraints due to multiple commitments) (Carless and Winstone 2020 ). To address the above, one of the aims of the present study is to expand the application of feedback literacy as a useful analytical lens to areas outside the classroom, that is, scholarly peer-review activities in academia, by presenting, analysing, and synthesising the personal experiences of the authors as successful peer reviewers for academic journals.

Conceptual framework

We adopt a feedback literacy of peer reviewers framework (Chong 2021a ) as an analytical lens to analyse, systemise, and synthesise our own experiences and practices as scholarly peer reviewers (Fig. 1 ). This two-tier framework includes a dimension on the manifestation of feedback literacy, which categorises five features of feedback literacy of peer reviewers, informed by student and teacher feedback literacy frameworks by Carless and Boud ( 2018 ) and Carless and Winstone ( 2020 ). When engaging in scholarly peer review, reviewers are expected to be able to provide constructive and formative feedback, which authors can act on in their revisions ( engineer feedback uptake ). Besides, peer reviewers who are usually full-time researchers or academics lead hectic professional lives; thus, when writing reviewers’ reports, it is important for them to consider practically and realistically the time they can invest and how their various degrees of commitment may have an impact on the feedback they provide ( navigate responsibilities ). Furthermore, peer reviewers should consider the emotional and relational influences their feedback exert on the authors. It is crucial for feedback to be not only informative but also supportive and professional (Chong, 2018 ) ( maintain relationships ). Equally important, it is imperative for peer reviewers to critically reflect on their own experience in the scholarly peer-review process, including their experience of receiving and giving feedback to academic peers, as well as the ways authors and editors respond to their feedback ( reflect on feedback experienc e). Lastly, acting as gatekeepers of journals to assess the quality of manuscripts, peer reviewers have to demonstrate an accurate understanding of the journals’ aims, remit, guidelines and standards, and reflect those in their written assessments of submitted manuscripts ( understand standards ). Situated in the context of scholarly peer review, this collaborative autoethnographic study conceptualises feedback literacy not only as a set of abilities but also orientations (London and Smither, 2002 ; Steelman and Wolfeld, 2016 ), which refers to academics’ tendency, beliefs, and habits in relation to engaging with feedback (London and Smither, 2002 ). According to Cheung ( 2000 ), orientations are influenced by a plethora of factors, namely experiences, cultures, and politics. It is important to understand feedback literacy as orientations because it takes into account that feedback is a convoluted process and is influenced by a plethora of contextual and personal factors. Informed by ecological systems theory (Bronfenbrenner, 1986 ; Neal and Neal, 2013 ) and synthesising existing feedback literacy models (Carless and Boud, 2018 ; Carless and Winstone, 2020 ; Chong, 2021a , 2021b ), we consider feedback literacy as a malleable, situated, and emergent construct, which is influenced by the interplay of various networked layers of ecological systems (Neal and Neal, 2013 ) (Fig. 1 ). Also important is that conceptualising feedback literacy as orientations avoids dichotomisation (feedback literate vs. feedback illiterate), emphasises the developmental nature of feedback literacy, and better captures the multifaceted manifestations of feedback engagement.

The outer ring of the figure shows the components of feedback literacy while the inner ring concerns the layers of contexts (ecosystems) which influence the manifestation of feedback literacy of peer reviewers.

Echoing recent conceptual papers on feedback literacy which emphasises the indispensable role of contexts (Chong 2021b ; Boud and Dawson, 2021 ; Gravett et al., 2019 ), our conceptual framework includes an underlying dimension of networked ecological systems (micro, meso, exo, macro, and chrono), which portrays the contextual forces shaping our feedback orientations. Informed by the networked ecological system theory of Neal and Neal ( 2013 ), we postulate that there are five systems of contextual influence, which affect the feedback experience and development of feedback literacy of peer reviewers. The five ecological systems refer to ‘settings’, which is defined by Bronfenbrenner ( 1986 ) as “place[s] where people can readily engage in social interactions” (p. 22). Even though Bronfenbrenner’s ( 1986 ) somewhat dated definition of ‘place’ is limited to ‘physical space’, we believe that ‘places’ should be more broadly defined in the 21st century to encompass physical and virtual, recent and dated, closed and distanced locations where people engage; as for ‘interactions’, from a sociocultural perspective, we understand that ‘interactions’ can include not only social, but also cognitive and emotional exchanges (Vygotsky, 1978 ). Microsystem refers to a setting where people, including the focal individual, interact. Mesosystem , on the other hand, means the interactions between people from different settings and the influence they exert on the focal individual. An exosystem , similar to a microsystem, is understood as a single setting but this setting excludes the focal individual but it is likely that participants in this setting would interact with the focal individual. The remaining two systems, macrosystem and chronosystem, refer not only to ‘settings’ but ‘forces that shape the patterns of social interactions that define settings’ (Neal and Neal, 2013 , p. 729). Macrosystem is “the set of social patterns that govern the formation and dissolution of… interactions… and thus the relationship among ecological systems” (ibid). Some examples of macrosystems given by Neal and Neal ( 2013 ) include political and cultural systems. Finally, chronosystem is “the observation that patterns of social interactions between individuals change over time, and that such changes impact on the focal individual” (ibid, p. 729). Figure 2 illustrates this networked ecological systems theory using a hypothetical example of an early career researcher who is involved in scholarly peer review for Journal A; at the same time, they are completing a PhD and are working as a faculty member at a university.

This is a hypothetical example of an early career researcher who is involved in scholarly peer review for Journal A.

From the reviewed literature on the construct of feedback literacy, the investigation of feedback literacy as a personal, situated, and unfolding process is best done through an autoethnographic lens, which underscores critical self-reflection. Autoethnography refers to “an approach to research and writing that seeks to describe and systematically analyse (graphy) personal experience (auto) in order to understand cultural experience (ethno)” (Ellis et al., 2011 , p. 273). Autoethnography stems from research in the field of anthropology and is later introduced to the fields of education by Ellis and Bochner ( 1996 ). In higher education research, autoethnographic studies are conducted to illuminate on topics related to identity and teaching practices (e.g., Abedi Asante and Abubakari, 2020 ; Hains-Wesson and Young 2016 ; Kumar, 2020 ). In this article, a collaborative approach to autoethnography is adopted. Based on Chang et al. ( 2013 ), Lapadat ( 2017 ) defines collaborative autoethnography (CAE) as follows:

… an autobiographic qualitative research method that combines the autobiographic study of self with ethnographic analysis of the sociocultural milieu within which the researchers are situated, and in which the collaborating researchers interact dialogically to analyse and interpret the collection of autobiographic data. (p. 598)

CAE is not only a product but a worldview and process (Wall, 2006 ). CAE is a discrete view about the world and research, which straddles between paradigmatic boundaries of scientific and literary studies. Similar to traditional scientific research, CAE advocates systematicity in the research process and consideration is given to such crucial research issues as reliability, validity, generalisability, and ethics (Lapadat, 2017 ). In closer alignment with studies on humanities and literature, the goal of CAE is not to uncover irrefutable universal truths and generate theories; instead, researchers of CAE are interested in co-constructing and analysing their own personal narratives or ‘stories’ to enrich and/or challenge mainstream beliefs and ideas, embracing diverse rather than canonical ways of behaviour, experience, and thinking (Ellis et al., 2011 ). Regarding the role of researchers, CAE researchers openly acknowledge the influence (and also vulnerability) of researchers throughout the research process and interpret this juxtaposition of identities between researchers and participants of research as conducive to offering an insider’s perspective to illustrate sociocultural phenomena (Sughrua, 2019 ). For our CAE on the scholarly peer-review experiences of two ECRs, the purpose is to reconstruct, analyse, and publicise our lived experience as peer reviewers and how multiple forces (i.e., ecological systems) interact to shape our identity, experience, and feedback practice. As a research process, CAE is a collaborative and dynamic reflective journey towards self-discovery, resulting in narratives, which connect with and add to the existing literature base in a personalised manner (Ellis et al., 2011 ). The collaborators should go beyond personal reflection to engage in dialogues to identify similarities and differences in experiences to throw new light on sociocultural phenomena (Merga et al., 2018 ). The iterative process of self- and collective reflections takes place when CAE researchers write about their own “remembered moments perceived to have significantly impacted the trajectory of a person’s life” and read each other’s stories (Ellis et al., 2011 , p. 275). These ‘moments’ or vignettes are usually written retrospectively, selectively, and systematically to shed light on facets of personal experience (Hughes et al., 2012 ). In addition to personal stories, some autoethnographies and CAEs utilise multiple data sources (e.g., reflective essays, diaries, photographs, interviews with co-researchers) and various ways of expressions (e.g., metaphors) to achieve some sort of triangulation and to present evidence in a ‘systematic’ yet evocative manner (Kumar, 2020 ). One could easily notice that overarching methodological principles are discussed in lieu of a set of rigid and linear steps because the process of reconstructing experience through storytelling can be messy and emergent, and certain degree of flexibility is necessary. However, autoethnographic studies, like other primary studies, address core research issues including reliability (reader’s judgement of the credibility of the narrator), validity (reader’s judgement that the narratives are believable), and generalisability (resemblance between the reader’s experience and the narrative, or enlightenment of the reader regarding unfamiliar cultural practices) (Ellis et al., 2011 ). Ethical issues also need to be considered. For example, authors are expected to be honest in reporting their experiences; to protect the privacy of the people who ‘participated’ in our stories, pseudonyms need to be used (Wilkinson, 2019 ). For the current study, we follow the suggested CAE process outlined by Chang et al. ( 2013 ), which includes four stages: deciding on topic and method , collecting materials , making meaning , and writing . When deciding on the topic, we decided to focus on our experience as scholarly peer reviewers because doing peer review and having our work reviewed are an indispensable part of our academic lives. The next is to collect relevant autoethnographic materials. In this study, we follow Kumar ( 2020 ) to focus on multiple data sources: (1) reflective essays which were written separately through ‘recalling’, which is referred to by Chang et al. ( 2013 ) as ‘a free-spirited way of bringing out memories about critical events, people, place, behaviours, talks, thoughts, perspectives, opinions, and emotions pertaining to the research topic’ (p. 113), and (2) discussion meetings. In our reflective essays, we included written records of reflection and excerpts of feedback in our peer-review reports. Following material collection is meaning making. CAE, as opposed to autoethnography, emphasises the importance of engaging in dialogues with collaborators and through this process we identify similarities and differences in our experiences (Sughrua, 2019 ). To do so, we exchanged our reflective essays; we read each other’s reflections and added questions or comments on the margins. Then, we met online twice to share our experiences and exchange views regarding the two reflective essays we wrote. Both meetings lasted for approximately 90 min, were audio-recorded and transcribed. After each meeting, we coded our stories and experiences with reference to the two dimensions of the ecological framework of feedback literacy (Fig. 1 ). With regards to coding our data, we followed the model of Miles and Huberman ( 1994 ), which comprises four stages: data reduction (abstracting data), data display (visualising data in tabular form), conclusion-drawing, and verification. The coding and writing processes were done collaboratively on Google Docs and care was taken to address the aforesaid ethical (e.g., honesty, privacy) and methodological issues (e.g., validity, reliability, generalisability). As a CAE study, the participants are the researchers themselves, that is, the two authors of this paper. We acknowledge that research data are collected from human subjects (from the two authors), such data are collected in accordance with the standards and guidelines of the School Research Ethics Committee at the School of Social Sciences, Education and Social Work, Queen’s University Belfast (Ref: 005_2021). Despite our different experiences in our unique training and employment contexts, we share some common characteristics, both being ECRs (<5 years post-PhD), working in the field of education, active in the scholarly publication process as both authors and peer reviewers. Importantly for this study, we were both recipients of Reviewer of the Year Award 2019 awarded jointly by the journal, Higher Education Research & Development and the publisher , Taylor & Francis. This award in recognition of the quality of our reviewing efforts, as determined by the editorial board of a prestigious higher education journal, provided a strong impetus for this study, providing an opportunity to reflect on our own experiences and practices. The extent of our peer-review activities during our early career leading up to the time of data collection is summarised in Table 1 .

Findings and discussion

Analysis of the four individual essays (E1 and E2 for each participant) and transcripts of the two subsequent discussions (D1 and D2) resulted in the identification of multiple descriptive codes and in turn a number of overarching themes (Supplementary Appendix 1). Our reporting of these themes is guided by our conceptual framework, where we first focus on the five manifestations of feedback literacy to highlight the experiences that contribute to our growth as effective and confident peer reviewers. Then, we report on the five ecological systems to unravel how each contextual layer develops our feedback literacy as peer reviewers. (Note that the discussion of the chronosystem has been necessarily incorporated into each of the four others dimensions: microsystem , mesosystem , exosystem , and macrosystem in order to demonstrate temporal changes). In particular, similarities and differences will be underscored, and connections with manifested feedback beliefs and behaviours will be made. We include quotes from both Author 1 (A1) and Author 2 (A2), in order to illustrate our findings, and to show the richness and depth of the data collected (Corden and Sainsbury, 2006 ). Transcribed quotes may be lightly edited while retaining meaning, for example through the removal of fillers and repetitions, which is generally accepted practice to ensure readability ( ibid ).

Manifestations of feedback literacy

Engineering feedback uptake.

The two authors have a strong sense of the purpose of peer review as promoting not only research quality, but the growth of researchers. One way that we engineer author uptake is to ensure that feedback is ‘clear’ (A2,E1), ‘explicit’ (A2,E1), ‘specific’ (A1,E1), and importantly ‘actionable… to ensure that authors can act on this feedback so that their manuscripts can be improved and ultimately accepted for publication’ (A1,E1). In less than favourable author outcomes, we ensure that there is reference to the role of the feedback in promoting the development of the manuscript, which A1 refers to as ‘promotion of a growth mindset’ (A1,E1). For example, after requesting a second round of major revisions, A2 ‘acknowledged the frustration that the author might have felt on getting further revisions by noting how much improvement was made to the paper, but also making clear the justification for sending it off for more work’ (A2,E1). We both note that we tend to write longer reviews when a rejection is the recommended outcome, as our ultimate goal is to aid in the development of a manuscript.

Rejections doesn’t mean a paper is beyond repair. It can still be fixed and improved; a rejection simply means that the fix may be too extensive even for multiple review cycles. It is crucial to let the authors whose manuscripts are rejected know that they can still act on the feedback to improve their work; they should not give up on their own work. I think this message is especially important to first-time authors or early career researchers. (A1,E1)

In promoting a growth mindset and in providing actionable feedback, we hope to ‘show the authors that I’m not targeting them, but their work’ (A1,D1). We particularly draw on our own experiences as ECRs, with first-hand understanding that ‘everyone takes it personally when they get rejected. Yeah. Moreover, it is hard to separate (yourself from the paper)’ (A2,D1).

Navigating responsibilities

As with most academics, the two authors have multiple pressures on their time, and there ‘isn’t much formal recognition or reward’ (A1,E1) and ‘little extrinsic incentive for me to review’ (A2,E1). Nevertheless we both view our roles as peer reviewers as ‘an important part of the process’ (A2,E1), ‘a modest way for me to give back to the academic community’ (A1,E1). Through peer review we have built a sense of ‘identity as an academic’ (A1,D1), through ‘being a member of the academic community’ (A2,D1). While A1 commits to ‘review as many papers as possible’ (A1,E1) and A2 will usually accept offers to review, there are still limits on our time and therefore we consider the topic and methods employed when deciding whether or not to accept an invitation, as well as the journal itself, as we feel we can review more efficiently for journals with which we are more familiar. A1 and A2 have different processes for conducting their review that are most efficient for their own situations. For A1, the process begins with reading the whole manuscript in one go, adding notes to the pdf document along the way, which he then reviews, and makes a tentative decision, including ‘a few reasons why I have come to this decision’ (A1,E1). After waiting at least one day, he reviews all of the notes and begins writing the report, which is divided into the sections of the paper. He notes it ‘usually takes me 30–45 min to write a report. I then proofread this report and submit it to the system. So it usually takes me no more than three hours to complete a review’ (A1,E1). For A2, the process for reviewing and structuring the report is quite different, with a need to ‘just find small but regular opportunities to work on the review’ (A2,E1). As was the case during her Ph.D, which involved juggling research and raising two babies, ‘I’ve trained myself to be able to do things in bits’ (A2,D1). So A2 also begins by reading the paper once through, although generally without making initial comments. The next phase involves going through the paper at various points in time whenever possible, and at the same time building up the report, making the report structurally slightly different to that of A1.

What my reviews look like are bullet points, basically. And they’re not really in a particular order. They generally… follow the flow (of the paper). But I mean, I might think of something, looking at the methods and realise, hey, you haven’t defined this concept in the literature review so I’ll just add you haven’t done this. And so I will usually preface (the review)… Here’s a list of suggestions. Some of them are minor, some of them are serious, but they’re in no particular order. (A1,D1)

As such, both reviewers engage in personalised strategies to make more effective use of their time. Both A1 and A2 give explicit but not exhaustive examples of an area of concern, and they also pose questions for the author to consider, in both cases placing the onus back on the author to take action. As A1 notes, ‘I’m not going to do a summary of that reference for you. I’m just going to include that there. If you’d like you can check it out’ (A1,D1). For A2, a lack of adequate reporting of the methods employed in a study makes it difficult to proceed, and in such cases will not invest further time, sending it back to the editor, because ‘I can’t even comment on the findings… I can’t go on. I’m not gonna waste my time’ (A2,D1). In cases where the authors may be ‘on the fence’ about a particular review, they will use the confidential comments to the editor to help work through difficult cases as ‘they are obviously very experienced reviewers’ (A1,D1). Delegating tasks to the expertise of the editorial teams when appropriate also ensures time is used more prudently.

Maintaining relationships

Except in a few cases where A2 has reviewed for journals with a single-blind model, the vast majority of the reviews that we have completed have been double-blind. This means that we are unaware of the identity of the author/s, and we are unknown to them. However, ‘even with blind-reviews I tend to think of it as a conversation with a person’ (A2,E1). A1 talks about the need to have respect for the author and their expertise and effort ‘regardless of the quality of the submission (which can be in some cases subjective)’ (A1,E1). A2 writes similarly about the ‘privilege’ and ‘responsibility’ of being able to review manuscripts that authors ‘have put so much time and energy into possibly over an extended period’ (A2,E1). In this way it is possible to develop a sort of relationship with an author even without knowing their identity. In trying to articulate the nature of that relationship (which we struggle to do so definitively), we note that it is more than just a reviewer, and A2 reflected on a recent review, which went through a number of rounds of resubmission where ‘it felt like we were developing a relationship, more like a mentor than a reviewer’ (A2,E1).

I consider this role as a peer reviewer more than giving helpful and actionable feedback; I would like to be a supporter and critical friend to the authors, even though in most cases I don’t even know who they are or what career stage they are at (A1,E1).

In any case, as A1 notes, ‘we don’t even need to know who that person is because we know that people like encouragement’ (A1,D1), and we are very conscious of the emotional impact that feedback can have on authors, and the inherent power imbalance in the relationship. For this reason, A1 is ‘cautious about the way I write so that I don’t accidentally make the authors the target of my feedback’. As A2 notes ‘I don’t want authors feeling depressed after reading a review’ (A2,E1). While we note that we try to deliver our feedback with ‘respect’ (A1,E1; A1,E2; A2,D1) ‘empathy’ (A1,E1), and ‘kindness’ (A2,D1), we both noted that we do not ‘sugar coat’ our feedback and A1 describes himself as ‘harsh’ and ‘critical’ (A1,E1) while A2 describes herself as ‘pretty direct’ (A2,E1). In our discussion, we tried to delve into this seeming contradiction:… the encouragement, hopefully is to the researcher, but the directness it should be, I hope, is related directly to whatever it is, the methods or the reporting or the scope of the literature review. It’s something specific about the manuscript itself. And I know myself, being an ECR and being reviewed, that it’s hard to separate yourself from your work… And I want to make it really explicit. If it’s critical, it’s not about the person. It’s about the work, you know, the weakness of the work, but not the person. (A2,D1)

A1 explains that at times his initial report may be highly critical, and at times he will ‘sit back and rethink… With empathy, I will write feedback, which is more constructive’ (A1,E1). However, he adds that ‘I will never try to overrate a piece or sugar-coat my comments just to sound “friendly”’ (A1,E1), with the ultimate goal being to uphold academic rigour. Thus, honesty is seen as the best strategy to maintain a strong, professional relationship with reviewers. Another strategy employed by A2 is showing explicit commitment to the review process. One way this is communicated is by prefacing a review with a summary of the paper, not only ‘to confirm with the author that I am interpreting the findings in the way that they intended, but also importantly to show that I have engaged with the paper’ (A2,E1). Further, if the recommendation is for a further round of review, she will state directly to the authors ‘that I would be happy to review a revised manuscript’ (A2,E1).

Reflecting on feedback experience

As ECRs we have engaged in the scholarly publishing process initially as authors, subsequently as reviewers, and most recently as Associate Editors. Insights gained in each of these roles have influenced our feedback practices, and have interacted to ‘develop a more holistic understanding of the whole review process’ (A1,E1).

We reflect on our experiences as authors beginning in our doctoral candidatures, with reviews that ranged from ‘the most helpful to the most cynical’ (A1,E1). A2 reflected on two particular experiences both of which resulted in rejection, one being ‘snarky’ and ‘unprofessional’ with ‘no substance’, the other providing ‘strong encouragement … the focus was clearly on the paper and not me personally’ (A2,E1). It was this experience that showed the divergence between the tone and content of review despite the same outcome, and as result A2 committed to being ‘ the amazing one’. A1 also drew from a negative experience noting that ‘I remember the least useful feedback as much as I do with the most constructive one’ (A1,E1). This was particularly the case when a reviewer made apparently politically-motivated judgements that A1 ‘felt very uncomfortable with’ and flagged with the editor (A1,E1). Through these experiences both authors wrote in their essays about the need to focus on the work and not on the individual, with an understanding that a review ‘can have a really serious impact’ (A2,D1) on an author.

It is important to note that neither authors have been involved in any formal or informal training on how to conduct peer review, although A1 expresses appreciation of the regular practice of one journal for which he reviews, where ‘the editor would write an email to the reviewers giving feedback on the feedback we have given’ (A1,E1). For A2, an important source of learning is in comparing her reviews with that of others who have reviewed the same manuscript, the norm for some journals being to send all reports to all reviewers along with the final decision.

I’m always interested to see how [my] review compares with others. Have I given the same recommendation? Have I identified the same areas of weakness? Have I formatted my review in the same way? How does the tone of delivery differ? I generally find that I give a similar if not the same response to other reviews, and I’m happy to see that I often pick up the same issues with methodology. (A2,E1)

For A2 there is comfort in seeing reviews that are similar to others, although we both draw on experiences where our recommendation diverged from others, with a source of assurance being the ultimate decision of the editor.

So it’s like, I don’t think it can be published and that [other] reviewer thinks it’s excellent. So usually, what the editor would do in this instance is invite the third one. Right, yeah. But then this editor told me… that they decided to go with my decision to reject because they find that my comments are more convincing. (A1,D1)

A2 also was surprised to read another report of the same manuscript she reviewed, that raised similar concerns and gave the same recommendation for major revisions, but noted the ‘wording is soooo snarky. What need?’ (A2,E1). In one case that A1 detailed in our first discussion, significant but improbable changes made to the methodology section of a resubmitted paper caused him to question the honesty of the reporting, making him ‘uncomfortable’ and as a result reported his concerns to the editor. In this case the review took some time to craft, trying to balance the ‘fine line between catering for the emotion [of the author], right, and upholding the academic standards’ (A1,D1). While he conceded initially his report was ‘kind of too harsh… later I think I rephrased it a little bit, I kind of softened (it)’.

While the role of Associate Editor is very new to A2 and thus was yet unable to comment, for A1 the ‘opportunity to read various kinds of comments given by reviewers’ (A1,E1) is viewed favourably. This includes not only how reviewers structure their feedback, but also how they use the confidential comments to the editors to express their thoughts more openly, providing important insights into the process that are largely hidden.

Understanding standards

While our reviewing practices are informed more broadly ‘according to more general academic standards of the study itself, and the clarity and fullness of the reporting’ (A2,E1), we look in the first instance to advice and guidelines from journals to develop an understanding of journal-specific standards, although A2 notes that a lack of review guidelines for one of the earliest journals she reviewed led her to ‘searching Google for standard criteria’ (A2,E1). However, our development in this area seems to come from developing a familiarity with a journal, particularly through engagement with the journal as an author.

In addition to reading the scope and instructions for authors to obtain such basic information as readership, length of submissions, citation style, the best way for me to understand the requirements and preferences of the journals is my own experience as an author. I review for journals which I have published in and for those which I have not. I always find it easier to make a judgement about whether the manuscripts I review meet the journal’s standards if I have published there before. (A1,E1)

Indeed, it seems that journal familiarity is connected closely to our confidence in reviewing, and while both authors ‘review for journals which I have published in and for those which I have not’ (A1,E1), A2 states that she is reluctant to ‘readily accept an offer to review for a journal that I’m not familiar with’, and A1 takes extra time to ‘do more preparatory work before I begin reading the manuscript and writing the review’ when reviewing for an unfamiliar journal.

Ecological systems

Microsystem.

Three microsystems exert influence on A1’s and A2’s development of feedback literacy: university, journal community, and Twitter.

In regards to the university, we are full-time academics in research-intensive universities in the UK and Japan where expectations for academics include publishing research in high-impact journals ‘which is vital to promotion’ (A1,E2). It is especially true in A2’s context where the national higher education agenda is to increase world rankings of universities. Thus, ‘there is little value placed on peer review, as it is not directly related to the broader agenda’ (A2,E2). When considering his recent relocation to the UK together with the current pandemic, A1 navigated his responsibilities within the university context and decided to allocate more time to his university-related responsibilities, especially providing learning and pastoral support to his students, who are mostly international students. Besides, A2 observed that there is a dearth of institution-wide support on conducting peer review although ‘there are a lot of training opportunities related to how to write academic papers in English, how to present at international conferences, how to write grant applications’, etc. (A2,E2). As a result, she ‘struggled for a couple of years’ because of the lack of institutional support for her development as a peer reviewer’ (A2,D2); but this helplessness also motivated her to seek her own ways to learn how to give feedback, such as ‘seeing through glimpses of other reviews, how others approach it, in terms of length, structure, tone, foci etc.’ (A2,E2). A1 shares the same view that no training is available at his institution to support his development as a peer reviewer. However, his postgraduate supervision experiences enabled him to reflect on how his feedback can benefit researchers. In our second online discussion, A1 shared that he held individual advising sessions with some postgraduate students, which made him realise that it is important for feedback to serve the function to inspire rather than to ‘give them right answers’ (A1,D2).

Because of the lack of formal training provided by universities, both authors searched for other professional communities to help us develop our expertise in giving feedback as peer reviewers, with journal communities being the next microsystem. We found that international journals provide valuable opportunities for us to understand more about the whole peer-review process, in particular the role of feedback. For A1, the training which he received from the editor-in-chief when he took up the associate editorship of a language education journal two years ago was particularly useful. A1 benefited greatly from meetings with the editor who walked him through every stage in the review process and provided ‘hands-on experience on how to handle delicate scenarios’ (A1,E2). Since then, A1 has had plenty of opportunities to oversee various stages of peer review and read a large number of reviewers’ reports which helped him gain ‘a holistic understanding of the peer-review process’ (A1,E2) and gradually made him become more cognizant of how he wants to give feedback. Although there was no explicit instruction on the technical aspect of giving feedback, A1 found that being an associate editor has developed his ‘consciousness’ and ‘awareness’ of giving feedback as a peer reviewer (A1,D2). Further, he felt that his editorial experiences provided him the awareness to constantly refine and improve his ways of giving feedback, especially ways to make his feedback ‘more structured, evidence-based, and objective’ (A1,E2). Despite not reflecting from the perspective of an editor, A2 recalled her experience as an author who received in-depth and constructive feedback from a reviewer, which really impacted the way she viewed the whole review process. She understood from this experience that even though the paper under review may not be particularly strong, peer reviewers should always aim to provide formative feedback which helps the authors to improve their work. These positive experiences of the two authors are impactful on the ways they give feedback as peer reviewers. In addition, close engagement with a specific journal has helped A2 to develop a sense of belonging, making it ‘much more than a journal, but also a way to become part of an academic community’ (A2,E2). With such a sense of belonging, it is more likely for her to be ‘pulled towards that journal than others’ when she can only review a limited number of manuscripts (A2,D2).

Another professional community in which we are both involved is Twitter. We regard Twitter as a platform for self-learning, reflection, and inspiration. We perceive Twitter as a space where we get to learn from others’ peer-review experiences and disciplinary practices. For example, A1 found the tweets on peer-review informative ‘because they are written by different stakeholders in the process—the authors, editors, reviewers’ and offer ‘different perspectives and sometimes different versions of the same story’ (A1,E2). A2 recalled a tweet she came across about the ‘infamous Reviewer 2’ and how she learned to not make the same mistakes (A2,D2). Reading other people’s experiences helps us reconsider our own feedback practices and, more broadly, the whole peer-review system because we ‘get a glimpse of the do’s and don’ts for peer reviewers’ (A1,E2).

Further to our three common microsystems, A2 also draws on a unique microsystem, that of her former profession as a teacher, which shapes her feedback practices in three ways. First, in her four years of teacher training, a lot of emphasis was placed on assessment and feedback such as ‘error correction’; this understanding related to giving feedback to students and was solidified through ‘learning on the job’ (A2,D2). Second, A2 acknowledges that as a teacher, she has a passion to ‘guide others in their knowledge and skill development… and continue this in our review practices’ (A2,E2). Finally, her teaching experience prepared her to consider the authors’ emotional responses in her peer-review feedback practices, constantly ‘thinking there’s a person there who’s going to be shattered getting a rejection’ (A2,D2).

Mesosystem considers the confluence of our interactions in various microsystems. Particularly, we experienced a lack of support from our institutions, which pushed us to seek alternative paths to acquire the art of giving feedback. This has made us realise the importance of self-learning in developing feedback literacy as peer reviewers, especially in how to develop constructive and actionable feedback. Both authors self-learn how to give feedback by reading others’ feedback. A1 felt ‘fortunate to be involved in journal editing and Twitter’ because he gets ‘a glimpse of how other peer reviewers give feedback to authors’ (A1,E2). A2, on the other hand, learned through her correspondences with a journal editor who made her stop ‘looking for every word’ and move away from ‘over proofreading and over editing’ (A2,D2).

Focusing on the chronosystem, it is noticed that both authors adjusted how they give feedback over time because of the aggregated influence of their microsystems. What stands out is that they have become more strategic in giving feedback. One way this is achieved is through focusing their comments on the arguments of the manuscripts instead of burning the midnight oil with error-correcting.

Exosystem concerns the environment where the focal individuals do not have direct interactions with the people in it but have access to information about. In his case, A1’s understanding of advising techniques promoted by a self-access language learning centre is conducive to the cultivation of his feedback literacy. Although A1 is not a part of the language advising team, he has a working relationship with the director. A1 was especially impressed by the learner-centeredness of an advising process:

The primary duty of the language advisor is not to be confused with that of a language teacher. Language teachers may teach a lecture on a linguistic feature or correct errors on an essay, but language advisors focus on designing activities and engaging students in dialogues to help them reflect on their own learning needs… The advisors may also suggest useful resources to the students which cater to their needs. In short, language advisors work in partnership with the students to help them improve their language while language teachers are often perceived as more authoritative figures (A1, E2).

His understanding of advising has affected how A1 provides feedback as a peer reviewer in a number of ways. First, A1 places much more emphasis on humanising his feedback, for example, by considering ‘ways to work in partnership with the authors and making this “partnership mindset” explicit to the authors through writing’ (A1,E2). One way to operationalise this ‘partnership mindset’ in peer review is to ‘ask a lot of questions’ and provide ‘multiple suggestions’ for the authors to choose from (A1,E2). Furthermore, his knowledge of the difference between feedback as giving advice and feedback as instruction has led him to include feedback, which points authors to additional resources. Below is a feedback point A1 gave in one of his reviews:

The description of the data analysis process was very brief. While we are not aiming at validity and reliability in qualitative studies, it is important for qualitative researchers to describe in detail how the data collected were analysed (e.g. iterative coding, inductive/deductive coding, thematic analysis) in order to ascertain that the findings were credible and trustworthy. See Johnny Saldaña’s ‘The Coding Manual for Qualitative Researchers’.

Another exosystem that we have knowledge about is formal peer-review training courses provided by publishers. These online courses are usually run asynchronously. Even though we did not enrol in these courses, our interest in peer review has led us to skim the content of these courses. Both of us questioned the value of formal peer-review training in developing feedback literacy of peer reviewers. For example, A2 felt that opportunities to review are more important because they ‘put you in that position where you have responsibility and have to think critically about how you are going to respond’ (A2,D2). To A1, formal peer-review training mostly focuses on developing peer reviewers’ ‘understanding of the whole mechanism’ but not providing ‘training on how to give feedback… For example, do you always ask a question without giving the answers you know? What is a good suggestion?’ (A1,D2).

Macrosystem

The two authors have diverse sociocultural experiences because of their family backgrounds and work contexts. When reflecting on their sociocultural experiences, A1 focused on his upbringing in Hong Kong where both of his parents are school teachers and his professional experience as a language teacher in secondary and tertiary education in Hong Kong while A2 discussed her experience of working in academia in Japan as an anglophone.

Observing his parents’ interactions with their students in schools, A1 was immersed in an Asian educational discourse characterised by ‘mutual respect and all sorts of formality’ (A1,E2). After he finished university, A1 became a school teacher and then a university lecturer (equivalent to a teaching fellow in the UK), getting immersed continuously in the etiquette of educational discourse in Hong Kong. Because of this, A1 knows that being professional means to be ‘formal and objective’ and there is a constant expectation to ‘treat people with respect’ (A1,E2). At the same time, his parents are unlike typical Asian parents; they are ‘more open-minded’, which made him more willing to listen and ‘consider different perspectives’ (A1,D2). Additionally, social hierarchy also impacted his approach to giving feedback as a peer reviewer. A1 started his career as a school teacher and then a university lecturer in Hong Kong with no formal research training. After obtaining his BA and MA, it is not until recently that A1 obtained his PhD by Prior Publication. Perhaps because of his background as a frontline teacher, A1 did not regard himself as ‘a formally trained researcher’ and perceived himself as not ‘elite enough to give feedback to other researchers’ (A1,E2). Both his childhood and his self-perceived identity have led to the formation of two feedback strategies: asking questions and providing a structured report mimicking the sections in the manuscript. A1 frequently asks questions in his reports ‘in a bid to offset some of the responsibilities to the authors’ (A1,E2). A1 struggles to decide whether to address authors using second- or third-person pronouns. A1 consistently uses third-person pronouns in his feedback because he wants to sound ‘very formal’ (A1,D2). However, A1 shared that he has recently started using second-person pronouns to make his feedback more interactive.

A2, on the other hand, pondered upon her sociocultural experiences as a school teacher in Australia, her position as an anglophone in a Japanese university, and her status as first-generation high school graduate. Reflecting on her career as a school teacher, A2 shared that her students had high expectations on her feedback:

So if you give feedback that seems unfair, you know … they’ll turn around and say, ‘What are you talking about’? They’re going to react back if your feedback is not clear. I think a lot of them [the students] appreciate the honesty. (A2,D2)

A2 acknowledges that her identity as a native English speaker has given her the advantage to publish extensively in international journals because of her high level of English proficiency and her access to ‘data from the US and from Australia which are more marketable’ (A2,D2). At the same time, as a native English speaker, she has empathy for her Japanese colleagues who struggle to write proficiently in English and some who even ‘pay thousands of dollars to have their work translated’ (A2,D2). Therefore, when giving feedback as a peer reviewer, she tries not to make a judgement on an author’s English proficiency and will not reject a paper based on the standard of English alone. Finally, as a first-generation scholar without any previous connections to academia, she struggles with belonging and self-confidence. As a result she notes that it usually takes her a long time to complete a review because she would like to be sure what she is saying is ‘right or constructive and is not on the wrong track’ (A2,D2).

Implications and future directions

In investigating the manifestations of the authors’ feedback literacy development, and the ecological systems in which this development occurs, this study unpacks the various sources of influence behind our feedback behaviours as two relatively new but highly commended peer reviewers. The findings show that our feedback literacy development is highly personalised and contextualised, and the sources of influence are diverse and interconnected, albeit largely informal. Our peer-review practices are influenced by our experiences within academia, but influences are much broader and begin much earlier. Peer-review skills were enhanced through direct experience not only in peer review but also in other activities related to the peer-review process, and as such more hands-on, on-site feedback training for peer reviewers may be more appropriate than knowledge-based training. The authors gain valuable insights from seeing the reviews of others, and as this is often not possible until scholars take on more senior roles within journals, co-reviewing is a potential way for ECRs to gain experience (McDowell et al., 2019 ). We draw practical and moral support from various communities, particularly online to promote “intellectual candour”, which refers to honest expressions of vulnerability for learning and trust building (Molloy and Bearman, 2019 , p. 32); in response to this finding we have developed an online community of practice, specifically as a space for discussing issues related to peer review (a Twitter account called “Scholarly Peers”). Importantly, our review practices are a product not only of how we review, but why we review, and as such training should not focus solely on the mechanics of review, but extend to its role within academia, and its impact not only on the quality of scholarship, but on the growth of researchers.

The significance of this study is its insider perspective, and the multifaceted framework that allows the capturing of the complexity of factors that influence individual feedback literacy development of two recognised peer reviewers. It must be stressed that the findings of this study are highly idiosyncratic, focusing on the experiences of only two peer reviewers and the educational research discipline. While the research design is such that it is not an attempt to describe a ‘typical’ or ‘expected’ experience, the scope of the study is a limitation, and future research could be expanded to studies of larger cohorts in order to identify broader trends. In this study, we have not included the reviewer reports themselves, and these reports provide a potentially rich source of data, which will be a focus in our continued investigation in this area. Further research could also investigate the role that peer-review training courses play in the feedback literacy development and practices of new and experienced peer reviewers. Since journal peer review is a communication process, it is equally important to investigate authors’ perspectives and experiences, especially pertaining to how authors interpret reviewers’ feedback based on the ways that it is written.

Data availability

Because of the sensitive nature of the data these are not made available.

Change history

26 november 2021.

A Correction to this paper has been published: https://doi.org/10.1057/s41599-021-00996-3

Abedi Asante L, Abubakari Z (2020) Pursuing PhD by publication in geography: a collaborative autoethnography of two African doctoral researchers. J Geogr High Educ 45(1):87–107. https://doi.org/10.1080/03098265.2020.1803817

Article   Google Scholar  

Boud D, Dawson P (2021). What feedback literate teachers do: An empirically-derived competency framework. Assess Eval High Educ. Advanced online publication. https://doi.org/10.1080/02602938.2021.1910928

Bronfenbrenner U (1986) Ecology of the family as a context for human development. Res Perspect Dev Psychol 22:723–742. https://doi.org/10.1037/0012-1649.22.6.723

Carless D, Boud D (2018) The development of student feedback literacy: enabling uptake of feedback. Assess Eval High Educ 43(8):1315–1325. https://doi.org/10.1080/02602938.2018.1463354

Carless D, Winstone N (2020) Teacher feedback literacy and its interplay with student feedback literacy. Teach High Educ, 1–14. https://doi.org/10.1080/13562517.2020.1782372

Chang H, Ngunjiri FW, Hernandez KC (2013) Collaborative autoethnography. Left Coast Press

Cheung D (2000) Measuring teachers’ meta-orientations to curriculum: application of hierarchical confirmatory factor analysis. The J Exp Educ 68(2):149–165. https://doi.org/10.1080/00220970009598500

Chong SW (2021a) Improving peer-review by developing peer reviewers’ feedback literacy. Learn Publ 34(3):461–467. https://doi.org/10.1002/leap.1378

Chong SW (2021b) Reconsidering student feedback literacy from an ecological perspective. Assess Eval High Educ 46(1):92–104. https://doi.org/10.1080/02602938.2020.1730765

Chong SW (2019) College students’ perception of e-feedback: a grounded theory perspective. Assess Eval High Educ 44(7):1090–1105. https://doi.org/10.1080/02602938.2019.1572067

Chong SW (2018) Interpersonal aspect of written feedback: a community college students’ perspective. Res Post-Compul Educ 23(4):499–519. https://doi.org/10.1080/13596748.2018.1526906

Corden A, Sainsbury R (2006) Using verbatim quotations in reporting qualitative social research: the views of research users. University of York Social Policy Research Unit

Ellis C, Adams TE, Bochner AP (2011) Autoethnography: An Overview. Historical Soc Res, 12:273–290

Ellis C, Bochner A (1996) Composing ethnography: Alternative forms of qualitative writing. Sage

Freda MC, Kearney MH, Baggs JG, Broome ME, Dougherty M (2009) Peer reviewer training and editor support: results from an international survey of nursing peer reviewers. J Profession Nurs 25(2):101–108. https://doi.org/10.1016/j.profnurs.2008.08.007

Fulcher G (2012) Assessment literacy for the language classroom. Lang Assess Quart 9(2):113–132. https://doi.org/10.1080/15434303.2011.642041

Gee JP (1999) Reading and the new literacy studies: reframing the national academy of sciences report on reading. J Liter Res 3(3):355–374. https://doi.org/10.1080/10862969909548052

Gravett K, Kinchin IM, Winstone NE, Balloo K, Heron M, Hosein A, Lygo-Baker S, Medland E (2019) The development of academics’ feedback literacy: experiences of learning from critical feedback via scholarly peer review. Assess Eval High Educ 45(5):651–665. https://doi.org/10.1080/02602938.2019.1686749

Hains-Wesson R, Young K (2016) A collaborative autoethnography study to inform the teaching of reflective practice in STEM. High Educ Res Dev 36(2):297–310. https://doi.org/10.1080/07294360.2016.1196653

Han Y, Xu Y (2019) Student feedback literacy and engagement with feedback: a case study of Chinese undergraduate students. Teach High Educ, https://doi.org/10.1080/13562517.2019.1648410

Heesen R, Bright LK (2020) Is Peer Review a Good Idea? Br J Philos Sci, https://doi.org/10.1093/bjps/axz029

Hollywood A, McCarthy D, Spencely C, Winstone N (2019) ‘Overwhelmed at first’: the experience of career development in early career academics. J Furth High Educ 44(7):998–1012. https://doi.org/10.1080/0309877X.2019.1636213

Horn SA (2016) The social and psychological costs of peer review: stress and coping with manuscript rejection. J Manage Inquiry 25(1):11–26. https://doi.org/10.1177/1056492615586597

Hughes S, Pennington JL, Makris S (2012) Translating Autoethnography Across the AERA Standards: Toward Understanding Autoethnographic Scholarship as Empirical Research. Educ Researcher, 41(6):209–219

Kandiko CB(2010) Neoliberalism in higher education: a comparative approach. Int J Art Sci 3(14):153–175. http://www.openaccesslibrary.org/images/BGS220_Camille_B._Kandiko.pdf

Keashly L, Neuman JH (2010) Faculty experiences with bullying in higher education-causes, consequences, and management. Adm Theory Prax 32(1):48–70. https://doi.org/10.2753/ATP1084-1806320103

Kelly J, Sadegieh T, Adeli K (2014) Peer review in scientific publications: benefits, critiques, & a survival guide. J Int Fed Clin Chem Labor Med 25(3):227–243. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4975196/

Google Scholar  

Kumar KL (2020) Understanding and expressing academic identity through systematic autoethnography. High Educ Res Dev, https://doi.org/10.1080/07294360.2020.1799950

Lapadat JC (2017) Ethics in autoethnography and collaborative autoethnography. Qual Inquiry 23(8):589–603. https://doi.org/10.1177/1077800417704462

Levi T, Inbar-Lourie O (2019) Assessment literacy or language assessment literacy: learning from the teachers. Lang Assess Quarter 17(2):168–182. https://doi.org/10.1080/15434303.2019.1692347

London MS, Smither JW (2002) Feedback orientation, feedback culture, and the longitudinal performance management process. Hum Res Manage Rev 12(1):81–100. https://doi.org/10.1016/S1053-4822(01)00043-2

Malecka B, Boud D, Carless D (2020) Eliciting, processing and enacting feedback: mechanisms for embedding student feedback literacy within the curriculum. Teach High Educ, 1–15. https://doi.org/10.1080/13562517.2020.1754784

Mavrogenis AF, Quaile A, Scarlat MM (2020) The good, the bad and the rude peer-review. Int Orthopaed 44(3):413–415. https://doi.org/10.1007/s00264-020-04504-1

McDowell GS, Knutsen JD, Graham JM, Oelker SK, Lijek RS (2019) Co-reviewing and ghostwriting by early-career researchers in the peer review of manuscripts. ELife 8:e48425. https://doi.org/10.7554/eLife.48425

Article   CAS   PubMed   PubMed Central   Google Scholar  

Merga MK, Mason S, Morris JE (2018) Early career experiences of navigating journal article publication: lessons learned using an autoethnographic approach. Learn Publ 31(4):381–389. https://doi.org/10.1002/leap.1192

Miles MB, Huberman AM (1994) Qualitative data analysis: An expanded sourcebook (2nd edn.). Sage

Molloy E, Bearman M (2019) Embracing the tension between vulnerability and credibility: ‘Intellectual candour’ in health professions education. Med Educ 53(1):32–41. https://doi.org/10.1111/medu.13649

Article   PubMed   Google Scholar  

Molloy E, Boud D, Henderson M (2019) Developing a learning-centred framework for feedback literacy. Assess Eval High Educ 45(4):527–540. https://doi.org/10.1080/02602938.2019.1667955

Neal JW, Neal ZP (2013) Nested or networked? Future directions for ecological systems theory. Soc Dev 22(4):722–737. https://doi.org/10.1111/sode.12018

Noble C, Billett S, Armit L, Collier L, Hilder J, Sly C, Molloy E (2020) “It’s yours to take”: generating learner feedback literacy in the workplace. Adv Health Sci Educ Theory Pract 25(1):55–74. https://doi.org/10.1007/s10459-019-09905-5

Price M, Rust C, O’Donovan B, Handley K, Bryant R (2012) Assessment literacy: the foundation for improving student learning. Oxford Centre for Staff and Learning Development

Silbiger NJ, Stubler AD (2019) Unprofessional peer reviews disproportionately harm underrepresented groups in STEM. PeerJ 7:e8247. https://doi.org/10.7717/peerj.8247

Article   PubMed   PubMed Central   Google Scholar  

Starck JM (2017) Scientific peer review: guidelines for informative peer review. Springer Spektrum

Steelman LA, Wolfeld L (2016) The manager as coach: the role of feedback orientation. J Busi Psychol 33(1):41–53. https://doi.org/10.1007/s10869-016-9473-6

Stiggins RJ (1999) Evaluating classroom assessment training in teacher education programs. Educ Meas: Issue Pract 18(1):23–27. https://doi.org/10.1111/j.1745-3992.1999.tb00004.x

Street B (1997) The implications of the ‘new literacy studies’ for literacy Education. Engl Educ 31(3):45–59. https://doi.org/10.1111/j.1754-8845.1997.tb00133.x

Sughrua WM (2019) A nomenclature for critical autoethnography in the arena of disciplinary atomization. Cult Stud Crit Methodol 19(6):429–465. https://doi.org/10.1177/1532708619863459

Sutton P (2012) Conceptualizing feedback literacy: knowing, being, and acting. Innov Educ Teach Int 49(1):31–40. https://doi.org/10.1080/14703297.2012.647781

Article   MathSciNet   Google Scholar  

Tynan BR, Garbett DL (2007) Negotiating the university research culture: collaborative voices of new academics. High Educ Res Dev 26(4):411–424. https://doi.org/10.1080/07294360701658617

Vygotsky LS (1978) Mind in society: The development of higher psychological processes. Harvard University Press

Wall S (2006) An autoethnography on learning about autoethnography. Int J Qual Methods 5(2):146–160. https://doi.org/10.1177/160940690600500205

Article   ADS   MathSciNet   Google Scholar  

Warne V (2016) Rewarding reviewers-sense or sensibility? A Wiley study explained. Learn Publ 29:41–40. https://doi.org/10.1002/leap.1002

Wilkinson S (2019) The story of Samantha: the teaching performances and inauthenticities of an early career human geography lecturer. High Educ Res Dev 38(2):398–410. https://doi.org/10.1080/07294360.2018.1517731

Winstone N, Carless D (2019) Designing effective feedback processes in higher education: a learning-focused approach. Routledge

Winstone NE, Mathlin G, Nash RA (2019) Building feedback literacy: students’ perceptions of the developing engagement with feedback toolkit. Front Educ 4:1–11. https://doi.org/10.3389/feduc.2019.00039

Xu Y, Carless D (2016) ‘Only true friends could be cruelly honest’: cognitive scaffolding and social-affective support in teacher feedback literacy. Assess Eval High Educ 42(7):1082–1094. https://doi.org/10.1080/02602938.2016.1226759

Download references

Author information

Authors and affiliations.

Queen’s University Belfast, Belfast, UK

Sin Wang Chong

Nagasaki University, Nagasaki, Japan

Shannon Mason

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Sin Wang Chong .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

We acknowledge that research data are collected from human subjects (from the two authors), such data are collected in accordance with the standards and guidelines of the School Research Ethics Committee at the School of Social Sciences, Education and Social Work, Queen’s University Belfast (Ref: 005_2021).

Informed consent

Since the participants are the two authors, there is no informed consent form.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplemental material file #1, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Chong, S.W., Mason, S. Demystifying the process of scholarly peer-review: an autoethnographic investigation of feedback literacy of two award-winning peer reviewers. Humanit Soc Sci Commun 8 , 266 (2021). https://doi.org/10.1057/s41599-021-00951-2

Download citation

Received : 02 August 2021

Accepted : 12 October 2021

Published : 12 November 2021

DOI : https://doi.org/10.1057/s41599-021-00951-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Peer review

A key convention in the publication of research is the peer review process, in which the quality and potential contribution of each manuscript is evaluated by one's peers in the scientific community.

Like other scientific journals, APA journals utilize a peer review process to guide manuscript selection and publication decisions.

Toward the goal of impartiality, the majority of APA journals follow a masked review policy, in which authors' and reviewers' identities are concealed from each other. Reviewer identities are never shared unless the reviewer requests to sign their review.

APA journal reviewers are qualified individuals selected by the action editor (typically, the journal editor or associate editor) to review a manuscript on the basis of their expertise in particular content areas of their field.

The role of a peer reviewer is to highlight unique, original manuscripts that fit within the scope of the journal.

To aid the editor's objectivity, two to three peer reviewers are selected to evaluate a manuscript.

These reviewers should be able to provide fair reviews, free from conflicts of interest, as well as submit the reviews on time.

In addition to technical expertise, criteria for selection of reviewers may include familiarity with a particular controversy or attention to a balance of perspectives (APA, 2010, p. 226).

Whereas the journal editor holds final responsibility for a manuscript, the action editor usually weighs reviewers' inputs heavily.

Authors can expect their manuscripts to be reviewed fairly, in a skilled, conscientious manner. The comments received should be constructive, respectful and specific.

Reviewers must present a clear decision recommendation regarding publication, considering the quality of the manuscript, its scientific contribution, and its appropriateness for the particular journal; support the recommendation with a detailed, comprehensive analysis of the quality and coherence of the study's conceptual basis, methods, results, and interpretations; and offer specific, constructive suggestions to authors.

Journal editors may request that reviewers evaluate manuscripts based on specific criteria, which may vary across journals or for non-empirical article types, such as commentaries or reviews.

The action editor scans the paper to gain an independent view of the work. This "quick read" provides a foundation for the more thorough reading that follows — it by no means determines the final decision, but does parallel how authors can expect many reviewers (and readers) to approach their papers.

First, the editor scans the paper from beginning to end for obvious flaws in the research substance and writing style. If problems show on the surface, a deeper reading is likely to uncover other matters needing attention.

After this initial examination of your manuscript, the action editors, as well as any peer reviewers, will follow these general guidelines:

Read the abstract

Major problems in the abstract often reflect internal flaws.

The major goal in reading the abstract is to understand the research question:

• Is it clearly defined, relevant, and supported by the methodology?
• What is the sense of the research question, methodology, findings, and interpretations?

APA publication policy emphasizes conclusion-oriented abstracts: What did the research find, and what do the findings mean?

Examine the full manuscript

If it is more than 35 typed, double-spaced pages (including references, tables, and figures), this could pose a problem for some journals.

• How long are the Introduction and the Discussion sections relative to other sections of the paper?
• Does the paper adhere to journal-specific guidelines?

These guidelines can be found on the Manuscript Submission tab of each journal's webpage.

Scan the paper's headings

• Are they well organized?
• Does a clear structure emerge?

If not, the author has not achieved coherence.

Scan the references

• Are they in APA Style?

If not, the author is not using APA publication format.

Scan the tables and figures

• Do they portray the information clearly?
• Can they stand alone without captions?
• Are they well constructed and in APA Style?

A "no" to any of these questions suggests problems in the author's presentation of findings.

• If the text contains a large number of statistics, could they be more appropriately put into tables or figures?

The editor drafting the decision letter should be synthesizing the input from multiple reviewers into a cohesive list of improvements that should be made to the manuscript. Any comments from the reviewers will be appended to the official decision letter.

These categories constitute the editorial actions that may be taken on a manuscript.

The flaws that lead to this decision generally center on substantive or methodological issues. A manuscript is usually rejected because it is outside the area of coverage of the journal; it contains serious flaws of design, methodology, analysis, or interpretation; or it is judged to make only a limited novel contribution to the field.

Revise and resubmit

In most cases, manuscripts may have publication potential but are not yet ready for final publication. The study as presented may not merit acceptance as is but may warrant consideration after substantive revision (e.g., reorganizing the conceptual structure, conducting additional experiments, or modifying analyses).

The action editor will give the author an invitation to revise and resubmit for another round of reviews (usually with the same reviewers). An action editor cannot guarantee acceptance of a revised manuscript, but authors who respond flexibly and attend closely to suggested revisions enhance their chances for an acceptance.

Authors must include a detailed cover letter outlining their responses to the revisions. Authors may receive this decision multiple times prior to acceptance.

In very few cases, a manuscript may be accepted for publication on first reading, with only minor revisions required. More typically, acceptances follow the successful revision of a manuscript previously rejected with invitation to revise and resubmit.

Once a manuscript is accepted and appropriate paperwork has been obtained, it enters the production phase of publication. At this point, no further changes can be made by the author other than those suggested by the copyeditor.

• Guidelines for Effective Manuscript Evaluation (from Psychotherapy )
• Peer review ethics: Six things every author should know (from Division Dialogue , March 2018)
• Current Peer Review Trends and Standards

If your manuscript is rejected, and if you believe a pertinent point was overlooked or misunderstood by the reviewers, you may appeal the editorial decision by contacting the editor responsible for the journal.

The editor might then decide to send the appeal to the (associate) editor who handled the initial submission.

If you appeal to the editor and are not satisfied with the editor's response, the next step in the APA editorial appeal procedure is to contact the APA chief editorial advisor .

If a satisfactory resolution is still not achieved, and you still believe that the process was unfair, you may appeal to the Publications and Communications (P&C) Board.

An initial review by the journals publisher and P&C Board chair and chair-elect will determine if the appeal will go before the full board for final decision.

Cases in which an appeal might not go before the full board are those in which an author submitted a manuscript against a journal’s policy (e.g., if a rejected submission was revised and submitted as a new submission without invitation to do so).

• APA publishing resources
• Reviewer Resource Center
• Editor resource center
• Why publish with APA Journals™
• Equity, diversity, and inclusion in APA Journals™

APA Publishing Insider

APA Publishing Insider is a free monthly newsletter with tips on APA Style, open science initiatives, active calls for papers, research summaries, and more.

Visit the APA Style website for style and grammar guidelines, free instructional aids, reference examples, the APA Style blog, APA Style products, and more.

Contact Journals

• Science Notes Posts
• Contact Science Notes
• Todd Helmenstine Biography
• Anne Helmenstine Biography
• Free Printable Periodic Tables (PDF and PNG)
• Periodic Table Wallpapers
• Interactive Periodic Table
• Periodic Table Posters
• Science Experiments for Kids
• How to Grow Crystals
• Chemistry Projects
• Fire and Flames Projects
• Holiday Science
• Chemistry Problems With Answers
• Physics Problems
• Unit Conversion Example Problems
• Chemistry Worksheets
• Biology Worksheets
• Periodic Table Worksheets
• Physical Science Worksheets
• Science Lab Worksheets
• My Amazon Books

Understanding Peer Review in Science

Peer review is an essential element of the scientific publishing process that helps ensure that research articles are evaluated, critiqued, and improved before release into the academic community. Take a look at the significance of peer review in scientific publications, the typical steps of the process, and and how to approach peer review if you are asked to assess a manuscript.

What Is Peer Review?

Peer review is the evaluation of work by peers, who are people with comparable experience and competency. Peers assess each others’ work in educational settings, in professional settings, and in the publishing world. The goal of peer review is improving quality, defining and maintaining standards, and helping people learn from one another.

In the context of scientific publication, peer review helps editors determine which submissions merit publication and improves the quality of manuscripts prior to their final release.

Types of Peer Review for Manuscripts

There are three main types of peer review:

• Single-blind review: The reviewers know the identities of the authors, but the authors do not know the identities of the reviewers.
• Double-blind review: Both the authors and reviewers remain anonymous to each other.
• Open peer review: The identities of both the authors and reviewers are disclosed, promoting transparency and collaboration.

There are advantages and disadvantages of each method. Anonymous reviews reduce bias but reduce collaboration, while open reviews are more transparent, but increase bias.

Key Elements of Peer Review

Proper selection of a peer group improves the outcome of the process:

• Expertise : Reviewers should possess adequate knowledge and experience in the relevant field to provide constructive feedback.
• Objectivity : Reviewers assess the manuscript impartially and without personal bias.
• Confidentiality : The peer review process maintains confidentiality to protect intellectual property and encourage honest feedback.
• Timeliness : Reviewers provide feedback within a reasonable timeframe to ensure timely publication.

Steps of the Peer Review Process

The typical peer review process for scientific publications involves the following steps:

• Submission : Authors submit their manuscript to a journal that aligns with their research topic.
• Editorial assessment : The journal editor examines the manuscript and determines whether or not it is suitable for publication. If it is not, the manuscript is rejected.
• Peer review : If it is suitable, the editor sends the article to peer reviewers who are experts in the relevant field.
• Reviewer feedback : Reviewers provide feedback, critique, and suggestions for improvement.
• Revision and resubmission : Authors address the feedback and make necessary revisions before resubmitting the manuscript.
• Final decision : The editor makes a final decision on whether to accept or reject the manuscript based on the revised version and reviewer comments.
• Publication : If accepted, the manuscript undergoes copyediting and formatting before being published in the journal.

Pros and Cons

While the goal of peer review is improving the quality of published research, the process isn’t without its drawbacks.

• Quality assurance : Peer review helps ensure the quality and reliability of published research.
• Error detection : The process identifies errors and flaws that the authors may have overlooked.
• Credibility : The scientific community generally considers peer-reviewed articles to be more credible.
• Professional development : Reviewers can learn from the work of others and enhance their own knowledge and understanding.
• Time-consuming : The peer review process can be lengthy, delaying the publication of potentially valuable research.
• Bias : Personal biases of reviews impact their evaluation of the manuscript.
• Inconsistency : Different reviewers may provide conflicting feedback, making it challenging for authors to address all concerns.
• Limited effectiveness : Peer review does not always detect significant errors or misconduct.
• Poaching : Some reviewers take an idea from a submission and gain publication before the authors of the original research.

Steps for Conducting Peer Review of an Article

Generally, an editor provides guidance when you are asked to provide peer review of a manuscript. Here are typical steps of the process.

• Accept the right assignment: Accept invitations to review articles that align with your area of expertise to ensure you can provide well-informed feedback.
• Manage your time: Allocate sufficient time to thoroughly read and evaluate the manuscript, while adhering to the journal’s deadline for providing feedback.
• Read the manuscript multiple times: First, read the manuscript for an overall understanding of the research. Then, read it more closely to assess the details, methodology, results, and conclusions.
• Evaluate the structure and organization: Check if the manuscript follows the journal’s guidelines and is structured logically, with clear headings, subheadings, and a coherent flow of information.
• Assess the quality of the research: Evaluate the research question, study design, methodology, data collection, analysis, and interpretation. Consider whether the methods are appropriate, the results are valid, and the conclusions are supported by the data.
• Examine the originality and relevance: Determine if the research offers new insights, builds on existing knowledge, and is relevant to the field.
• Check for clarity and consistency: Review the manuscript for clarity of writing, consistent terminology, and proper formatting of figures, tables, and references.
• Identify ethical issues: Look for potential ethical concerns, such as plagiarism, data fabrication, or conflicts of interest.
• Provide constructive feedback: Offer specific, actionable, and objective suggestions for improvement, highlighting both the strengths and weaknesses of the manuscript. Don’t be mean.
• Organize your review: Structure your review with an overview of your evaluation, followed by detailed comments and suggestions organized by section (e.g., introduction, methods, results, discussion, and conclusion).
• Be professional and respectful: Maintain a respectful tone in your feedback, avoiding personal criticism or derogatory language.
• Proofread your review: Before submitting your review, proofread it for typos, grammar, and clarity.
• Couzin-Frankel J (September 2013). “Biomedical publishing. Secretive and subjective, peer review proves resistant to study”. Science . 341 (6152): 1331. doi: 10.1126/science.341.6152.1331
• Lee, Carole J.; Sugimoto, Cassidy R.; Zhang, Guo; Cronin, Blaise (2013). “Bias in peer review”. Journal of the American Society for Information Science and Technology. 64 (1): 2–17. doi: 10.1002/asi.22784
• Slavov, Nikolai (2015). “Making the most of peer review”. eLife . 4: e12708. doi: 10.7554/eLife.12708
• Spier, Ray (2002). “The history of the peer-review process”. Trends in Biotechnology . 20 (8): 357–8. doi: 10.1016/S0167-7799(02)01985-6
• Squazzoni, Flaminio; Brezis, Elise; Marušić, Ana (2017). “Scientometrics of peer review”. Scientometrics . 113 (1): 501–502. doi: 10.1007/s11192-017-2518-4

Related Posts

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Methodology
• What Is Peer Review? | Types & Examples

What Is Peer Review? | Types & Examples

Published on 6 May 2022 by Tegan George . Revised on 2 September 2022.

Peer review, sometimes referred to as refereeing , is the process of evaluating submissions to an academic journal. Using strict criteria, a panel of reviewers in the same subject area decides whether to accept each submission for publication.

Peer-reviewed articles are considered a highly credible source due to the stringent process they go through before publication.

There are various types of peer review. The main difference between them is to what extent the authors, reviewers, and editors know each other’s identities. The most common types are:

• Single-blind review
• Double-blind review
• Triple-blind review

Collaborative review

Open review.

Relatedly, peer assessment is a process where your peers provide you with feedback on something you’ve written, based on a set of criteria or benchmarks from an instructor. They then give constructive feedback, compliments, or guidance to help you improve your draft.

Table of contents

What is the purpose of peer review, types of peer review, the peer review process, providing feedback to your peers, peer review example, advantages of peer review, criticisms of peer review, frequently asked questions about peer review.

Many academic fields use peer review, largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the manuscript. For this reason, academic journals are among the most credible sources you can refer to.

However, peer review is also common in non-academic settings. The United Nations, the European Union, and many individual nations use peer review to evaluate grant applications. It is also widely used in medical and health-related fields as a teaching or quality-of-care measure.

Peer assessment is often used in the classroom as a pedagogical tool. Both receiving feedback and providing it are thought to enhance the learning process, helping students think critically and collaboratively.

Prevent plagiarism, run a free check.

Depending on the journal, there are several types of peer review.

Single-blind peer review

The most common type of peer review is single-blind (or single anonymised) review . Here, the names of the reviewers are not known by the author.

While this gives the reviewers the ability to give feedback without the possibility of interference from the author, there has been substantial criticism of this method in the last few years. Many argue that single-blind reviewing can lead to poaching or intellectual theft or that anonymised comments cause reviewers to be too harsh.

Double-blind peer review

In double-blind (or double anonymised) review , both the author and the reviewers are anonymous.

Arguments for double-blind review highlight that this mitigates any risk of prejudice on the side of the reviewer, while protecting the nature of the process. In theory, it also leads to manuscripts being published on merit rather than on the reputation of the author.

Triple-blind peer review

While triple-blind (or triple anonymised) review – where the identities of the author, reviewers, and editors are all anonymised – does exist, it is difficult to carry out in practice.

Proponents of adopting triple-blind review for journal submissions argue that it minimises potential conflicts of interest and biases. However, ensuring anonymity is logistically challenging, and current editing software is not always able to fully anonymise everyone involved in the process.

In collaborative review , authors and reviewers interact with each other directly throughout the process. However, the identity of the reviewer is not known to the author. This gives all parties the opportunity to resolve any inconsistencies or contradictions in real time, and provides them a rich forum for discussion. It can mitigate the need for multiple rounds of editing and minimise back-and-forth.

Collaborative review can be time- and resource-intensive for the journal, however. For these collaborations to occur, there has to be a set system in place, often a technological platform, with staff monitoring and fixing any bugs or glitches.

Lastly, in open review , all parties know each other’s identities throughout the process. Often, open review can also include feedback from a larger audience, such as an online forum, or reviewer feedback included as part of the final published product.

While many argue that greater transparency prevents plagiarism or unnecessary harshness, there is also concern about the quality of future scholarship if reviewers feel they have to censor their comments.

In general, the peer review process includes the following steps:

• First, the author submits the manuscript to the editor.
• Reject the manuscript and send it back to the author, or
• Send it onward to the selected peer reviewer(s)
• Next, the peer review process occurs. The reviewer provides feedback, addressing any major or minor issues with the manuscript, and gives their advice regarding what edits should be made.
• Lastly, the edited manuscript is sent back to the author. They input the edits and resubmit it to the editor for publication.

In an effort to be transparent, many journals are now disclosing who reviewed each article in the published product. There are also increasing opportunities for collaboration and feedback, with some journals allowing open communication between reviewers and authors.

It can seem daunting at first to conduct a peer review or peer assessment. If you’re not sure where to start, there are several best practices you can use.

Summarise the argument in your own words

Summarising the main argument helps the author see how their argument is interpreted by readers, and gives you a jumping-off point for providing feedback. If you’re having trouble doing this, it’s a sign that the argument needs to be clearer, more concise, or worded differently.

If the author sees that you’ve interpreted their argument differently than they intended, they have an opportunity to address any misunderstandings when they get the manuscript back.

Separate your feedback into major and minor issues

It can be challenging to keep feedback organised. One strategy is to start out with any major issues and then flow into the more minor points. It’s often helpful to keep your feedback in a numbered list, so the author has concrete points to refer back to.

Major issues typically consist of any problems with the style, flow, or key points of the manuscript. Minor issues include spelling errors, citation errors, or other smaller, easy-to-apply feedback.

The best feedback you can provide is anything that helps them strengthen their argument or resolve major stylistic issues.

Give the type of feedback that you would like to receive

No one likes being criticised, and it can be difficult to give honest feedback without sounding overly harsh or critical. One strategy you can use here is the ‘compliment sandwich’, where you ‘sandwich’ your constructive criticism between two compliments.

Be sure you are giving concrete, actionable feedback that will help the author submit a successful final draft. While you shouldn’t tell them exactly what they should do, your feedback should help them resolve any issues they may have overlooked.

As a rule of thumb, your feedback should be:

• Easy to understand
• Constructive

Below is a brief annotated research example. You can view examples of peer feedback by hovering over the highlighted sections.

Influence of phone use on sleep

Studies show that teens from the US are getting less sleep than they were a decade ago (Johnson, 2019) . On average, teens only slept for 6 hours a night in 2021, compared to 8 hours a night in 2011. Johnson mentions several potential causes, such as increased anxiety, changed diets, and increased phone use.

The current study focuses on the effect phone use before bedtime has on the number of hours of sleep teens are getting.

For this study, a sample of 300 teens was recruited using social media, such as Facebook, Instagram, and Snapchat. The first week, all teens were allowed to use their phone the way they normally would, in order to obtain a baseline.

The sample was then divided into 3 groups:

• Group 1 was not allowed to use their phone before bedtime.
• Group 2 used their phone for 1 hour before bedtime.
• Group 3 used their phone for 3 hours before bedtime.

All participants were asked to go to sleep around 10 p.m. to control for variation in bedtime . In the morning, their Fitbit showed the number of hours they’d slept. They kept track of these numbers themselves for 1 week.

Two independent t tests were used in order to compare Group 1 and Group 2, and Group 1 and Group 3. The first t test showed no significant difference ( p > .05) between the number of hours for Group 1 ( M = 7.8, SD = 0.6) and Group 2 ( M = 7.0, SD = 0.8). The second t test showed a significant difference ( p < .01) between the average difference for Group 1 ( M = 7.8, SD = 0.6) and Group 3 ( M = 6.1, SD = 1.5).

This shows that teens sleep fewer hours a night if they use their phone for over an hour before bedtime, compared to teens who use their phone for 0 to 1 hours.

Peer review is an established and hallowed process in academia, dating back hundreds of years. It provides various fields of study with metrics, expectations, and guidance to ensure published work is consistent with predetermined standards.

• Protects the quality of published research

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. Any content that raises red flags for reviewers can be closely examined in the review stage, preventing plagiarised or duplicated research from being published.

• Gives you access to feedback from experts in your field

Peer review represents an excellent opportunity to get feedback from renowned experts in your field and to improve your writing through their feedback and guidance. Experts with knowledge about your subject matter can give you feedback on both style and content, and they may also suggest avenues for further research that you hadn’t yet considered.

• Helps you identify any weaknesses in your argument

Peer review acts as a first defence, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process. This way, you’ll end up with a more robust, more cohesive article.

While peer review is a widely accepted metric for credibility, it’s not without its drawbacks.

• Reviewer bias

The more transparent double-blind system is not yet very common, which can lead to bias in reviewing. A common criticism is that an excellent paper by a new researcher may be declined, while an objectively lower-quality submission by an established researcher would be accepted.

• Delays in publication

The thoroughness of the peer review process can lead to significant delays in publishing time. Research that was current at the time of submission may not be as current by the time it’s published.

• Risk of human error

By its very nature, peer review carries a risk of human error. In particular, falsification often cannot be detected, given that reviewers would have to replicate entire experiments to ensure the validity of results.

Peer review is a process of evaluating submissions to an academic journal. Utilising rigorous criteria, a panel of reviewers in the same subject area decide whether to accept each submission for publication.

For this reason, academic journals are often considered among the most credible sources you can use in a research project – provided that the journal itself is trustworthy and well regarded.

Peer review can stop obviously problematic, falsified, or otherwise untrustworthy research from being published. It also represents an excellent opportunity to get feedback from renowned experts in your field.

It acts as a first defence, helping you ensure your argument is clear and that there are no gaps, vague terms, or unanswered questions for readers who weren’t involved in the research process.

Peer-reviewed articles are considered a highly credible source due to this stringent process they go through before publication.

In general, the peer review process follows the following steps:

• Reject the manuscript and send it back to author, or
• Lastly, the edited manuscript is sent back to the author. They input the edits, and resubmit it to the editor for publication.

Many academic fields use peer review , largely to determine whether a manuscript is suitable for publication. Peer review enhances the credibility of the published manuscript.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

George, T. (2022, September 02). What Is Peer Review? | Types & Examples. Scribbr. Retrieved 3 September 2024, from https://www.scribbr.co.uk/research-methods/peer-reviews/

Is this article helpful?

Tegan George

Other students also liked, what is a double-blind study | introduction & examples, a quick guide to experimental design | 5 steps & examples, data cleaning | a guide with examples & steps.

Page Content

Overview of the review report format, the first read-through, first read considerations, spotting potential major flaws, concluding the first reading, rejection after the first reading, before starting the second read-through, doing the second read-through, the second read-through: section by section guidance, how to structure your report, on presentation and style, criticisms & confidential comments to editors, the recommendation, when recommending rejection, additional resources, step by step guide to reviewing a manuscript.

When you receive an invitation to peer review, you should be sent a copy of the paper's abstract to help you decide whether you wish to do the review. Try to respond to invitations promptly - it will prevent delays. It is also important at this stage to declare any potential Conflict of Interest.

The structure of the review report varies between journals. Some follow an informal structure, while others have a more formal approach.

" Number your comments!!! " (Jonathon Halbesleben, former Editor of Journal of Occupational and Organizational Psychology)

Informal Structure

Many journals don't provide criteria for reviews beyond asking for your 'analysis of merits'. In this case, you may wish to familiarize yourself with examples of other reviews done for the journal, which the editor should be able to provide or, as you gain experience, rely on your own evolving style.

Formal Structure

Other journals require a more formal approach. Sometimes they will ask you to address specific questions in your review via a questionnaire. Or they might want you to rate the manuscript on various attributes using a scorecard. Often you can't see these until you log in to submit your review. So when you agree to the work, it's worth checking for any journal-specific guidelines and requirements. If there are formal guidelines, let them direct the structure of your review.

In Both Cases

Whether specifically required by the reporting format or not, you should expect to compile comments to authors and possibly confidential ones to editors only.

Following the invitation to review, when you'll have received the article abstract, you should already understand the aims, key data and conclusions of the manuscript. If you don't, make a note now that you need to feedback on how to improve those sections.

The first read-through is a skim-read. It will help you form an initial impression of the paper and get a sense of whether your eventual recommendation will be to accept or reject the paper.

Keep a pen and paper handy when skim-reading.

Try to bear in mind the following questions - they'll help you form your overall impression:

• What is the main question addressed by the research? Is it relevant and interesting?
• How original is the topic? What does it add to the subject area compared with other published material?
• Is the paper well written? Is the text clear and easy to read?
• Are the conclusions consistent with the evidence and arguments presented? Do they address the main question posed?
• If the author is disagreeing significantly with the current academic consensus, do they have a substantial case? If not, what would be required to make their case credible?
• If the paper includes tables or figures, what do they add to the paper? Do they aid understanding or are they superfluous?

While you should read the whole paper, making the right choice of what to read first can save time by flagging major problems early on.

Editors say, " Specific recommendations for remedying flaws are VERY welcome ."

Examples of possibly major flaws include:

• Drawing a conclusion that is contradicted by the author's own statistical or qualitative evidence
• The use of a discredited method
• Ignoring a process that is known to have a strong influence on the area under study

If experimental design features prominently in the paper, first check that the methodology is sound - if not, this is likely to be a major flaw.

You might examine:

• The sampling in analytical papers
• The sufficient use of control experiments
• The precision of process data
• The regularity of sampling in time-dependent studies
• The validity of questions, the use of a detailed methodology and the data analysis being done systematically (in qualitative research)
• That qualitative research extends beyond the author's opinions, with sufficient descriptive elements and appropriate quotes from interviews or focus groups

Major Flaws in Information

If methodology is less of an issue, it's often a good idea to look at the data tables, figures or images first. Especially in science research, it's all about the information gathered. If there are critical flaws in this, it's very likely the manuscript will need to be rejected. Such issues include:

• Insufficient data
• Unclear data tables
• Contradictory data that either are not self-consistent or disagree with the conclusions
• Confirmatory data that adds little, if anything, to current understanding - unless strong arguments for such repetition are made

If you find a major problem, note your reasoning and clear supporting evidence (including citations).

After the initial read and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review - the first summarizing the research question addressed and the second the contribution of the work. If the journal has a prescribed reporting format, this draft will still help you compose your thoughts.

The First Paragraph

This should state the main question addressed by the research and summarize the goals, approaches, and conclusions of the paper. It should:

• Help the editor properly contextualize the research and add weight to your judgement
• Show the author what key messages are conveyed to the reader, so they can be sure they are achieving what they set out to do
• Focus on successful aspects of the paper so the author gets a sense of what they've done well

The Second Paragraph

This should provide a conceptual overview of the contribution of the research. So consider:

• Is the paper's premise interesting and important?
• Are the methods used appropriate?
• Do the data support the conclusions?

After drafting these two paragraphs, you should be in a position to decide whether this manuscript is seriously flawed and should be rejected (see the next section). Or whether it is publishable in principle and merits a detailed, careful read through.

Even if you are coming to the opinion that an article has serious flaws, make sure you read the whole paper. This is very important because you may find some really positive aspects that can be communicated to the author. This could help them with future submissions.

A full read-through will also make sure that any initial concerns are indeed correct and fair. After all, you need the context of the whole paper before deciding to reject. If you still intend to recommend rejection, see the section "When recommending rejection."

Once the paper has passed your first read and you've decided the article is publishable in principle, one purpose of the second, detailed read-through is to help prepare the manuscript for publication. You may still decide to recommend rejection following a second reading.

" Offer clear suggestions for how the authors can address the concerns raised. In other words, if you're going to raise a problem, provide a solution ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Preparation

To save time and simplify the review:

• Don't rely solely upon inserting comments on the manuscript document - make separate notes
• Try to group similar concerns or praise together
• If using a review program to note directly onto the manuscript, still try grouping the concerns and praise in separate notes - it helps later
• Note line numbers of text upon which your notes are based - this helps you find items again and also aids those reading your review

Now that you have completed your preparations, you're ready to spend an hour or so reading carefully through the manuscript.

As you're reading through the manuscript for a second time, you'll need to keep in mind the argument's construction, the clarity of the language and content.

With regard to the argument’s construction, you should identify:

• Any places where the meaning is unclear or ambiguous
• Any factual errors
• Any invalid arguments

You may also wish to consider:

• Does the title properly reflect the subject of the paper?
• Does the abstract provide an accessible summary of the paper?
• Do the keywords accurately reflect the content?
• Is the paper an appropriate length?
• Are the key messages short, accurate and clear?

Not every submission is well written. Part of your role is to make sure that the text’s meaning is clear.

Editors say, " If a manuscript has many English language and editing issues, please do not try and fix it. If it is too bad, note that in your review and it should be up to the authors to have the manuscript edited ."

If the article is difficult to understand, you should have rejected it already. However, if the language is poor but you understand the core message, see if you can suggest improvements to fix the problem:

• Are there certain aspects that could be communicated better, such as parts of the discussion?
• Should the authors consider resubmitting to the same journal after language improvements?
• Would you consider looking at the paper again once these issues are dealt with?

On Grammar and Punctuation

Your primary role is judging the research content. Don't spend time polishing grammar or spelling. Editors will make sure that the text is at a high standard before publication. However, if you spot grammatical errors that affect clarity of meaning, then it's important to highlight these. Expect to suggest such amendments - it's rare for a manuscript to pass review with no corrections.

A 2010 study of nursing journals found that 79% of recommendations by reviewers were influenced by grammar and writing style (Shattel, et al., 2010).

1. The Introduction

A well-written introduction:

• Sets out the argument
• Summarizes recent research related to the topic
• Highlights gaps in current understanding or conflicts in current knowledge
• Establishes the originality of the research aims by demonstrating the need for investigations in the topic area
• Gives a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript

Originality and Topicality

Originality and topicality can only be established in the light of recent authoritative research. For example, it's impossible to argue that there is a conflict in current understanding by referencing articles that are 10 years old.

Authors may make the case that a topic hasn't been investigated in several years and that new research is required. This point is only valid if researchers can point to recent developments in data gathering techniques or to research in indirectly related fields that suggest the topic needs revisiting. Clearly, authors can only do this by referencing recent literature. Obviously, where older research is seminal or where aspects of the methodology rely upon it, then it is perfectly appropriate for authors to cite some older papers.

Editors say, "Is the report providing new information; is it novel or just confirmatory of well-known outcomes ?"

It's common for the introduction to end by stating the research aims. By this point you should already have a good impression of them - if the explicit aims come as a surprise, then the introduction needs improvement.

2. Materials and Methods

Academic research should be replicable, repeatable and robust - and follow best practice.

Replicable Research

This makes sufficient use of:

• Control experiments
• Repeated analyses
• Repeated experiments

These are used to make sure observed trends are not due to chance and that the same experiment could be repeated by other researchers - and result in the same outcome. Statistical analyses will not be sound if methods are not replicable. Where research is not replicable, the paper should be recommended for rejection.

Repeatable Methods

These give enough detail so that other researchers are able to carry out the same research. For example, equipment used or sampling methods should all be described in detail so that others could follow the same steps. Where methods are not detailed enough, it's usual to ask for the methods section to be revised.

Robust Research

This has enough data points to make sure the data are reliable. If there are insufficient data, it might be appropriate to recommend revision. You should also consider whether there is any in-built bias not nullified by the control experiments.

Best Practice

During these checks you should keep in mind best practice:

• Standard guidelines were followed (e.g. the CONSORT Statement for reporting randomized trials)
• The health and safety of all participants in the study was not compromised
• Ethical standards were maintained

If the research fails to reach relevant best practice standards, it's usual to recommend rejection. What's more, you don't then need to read any further.

3. Results and Discussion

This section should tell a coherent story - What happened? What was discovered or confirmed?

Certain patterns of good reporting need to be followed by the author:

• They should start by describing in simple terms what the data show
• They should make reference to statistical analyses, such as significance or goodness of fit
• Once described, they should evaluate the trends observed and explain the significance of the results to wider understanding. This can only be done by referencing published research
• The outcome should be a critical analysis of the data collected

Discussion should always, at some point, gather all the information together into a single whole. Authors should describe and discuss the overall story formed. If there are gaps or inconsistencies in the story, they should address these and suggest ways future research might confirm the findings or take the research forward.

4. Conclusions

This section is usually no more than a few paragraphs and may be presented as part of the results and discussion, or in a separate section. The conclusions should reflect upon the aims - whether they were achieved or not - and, just like the aims, should not be surprising. If the conclusions are not evidence-based, it's appropriate to ask for them to be re-written.

5. Information Gathered: Images, Graphs and Data Tables

If you find yourself looking at a piece of information from which you cannot discern a story, then you should ask for improvements in presentation. This could be an issue with titles, labels, statistical notation or image quality.

Where information is clear, you should check that:

• The results seem plausible, in case there is an error in data gathering
• The trends you can see support the paper's discussion and conclusions
• There are sufficient data. For example, in studies carried out over time are there sufficient data points to support the trends described by the author?

You should also check whether images have been edited or manipulated to emphasize the story they tell. This may be appropriate but only if authors report on how the image has been edited (e.g. by highlighting certain parts of an image). Where you feel that an image has been edited or manipulated without explanation, you should highlight this in a confidential comment to the editor in your report.

6. List of References

You will need to check referencing for accuracy, adequacy and balance.

Where a cited article is central to the author's argument, you should check the accuracy and format of the reference - and bear in mind different subject areas may use citations differently. Otherwise, it's the editor’s role to exhaustively check the reference section for accuracy and format.

You should consider if the referencing is adequate:

• Are important parts of the argument poorly supported?
• Are there published studies that show similar or dissimilar trends that should be discussed?
• If a manuscript only uses half the citations typical in its field, this may be an indicator that referencing should be improved - but don't be guided solely by quantity
• References should be relevant, recent and readily retrievable

Check for a well-balanced list of references that is:

• Helpful to the reader
• Fair to competing authors
• Not over-reliant on self-citation
• Gives due recognition to the initial discoveries and related work that led to the work under assessment

You should be able to evaluate whether the article meets the criteria for balanced referencing without looking up every reference.

7. Plagiarism

By now you will have a deep understanding of the paper's content - and you may have some concerns about plagiarism.

Identified Concern

If you find - or already knew of - a very similar paper, this may be because the author overlooked it in their own literature search. Or it may be because it is very recent or published in a journal slightly outside their usual field.

You may feel you can advise the author how to emphasize the novel aspects of their own study, so as to better differentiate it from similar research. If so, you may ask the author to discuss their aims and results, or modify their conclusions, in light of the similar article. Of course, the research similarities may be so great that they render the work unoriginal and you have no choice but to recommend rejection.

"It's very helpful when a reviewer can point out recent similar publications on the same topic by other groups, or that the authors have already published some data elsewhere ." (Editor feedback)

Suspected Concern

If you suspect plagiarism, including self-plagiarism, but cannot recall or locate exactly what is being plagiarized, notify the editor of your suspicion and ask for guidance.

Most editors have access to software that can check for plagiarism.

Editors are not out to police every paper, but when plagiarism is discovered during peer review it can be properly addressed ahead of publication. If plagiarism is discovered only after publication, the consequences are worse for both authors and readers, because a retraction may be necessary.

For detailed guidelines see COPE's Ethical guidelines for reviewers and Wiley's Best Practice Guidelines on Publishing Ethics .

8. Search Engine Optimization (SEO)

After the detailed read-through, you will be in a position to advise whether the title, abstract and key words are optimized for search purposes. In order to be effective, good SEO terms will reflect the aims of the research.

A clear title and abstract will improve the paper's search engine rankings and will influence whether the user finds and then decides to navigate to the main article. The title should contain the relevant SEO terms early on. This has a major effect on the impact of a paper, since it helps it appear in search results. A poor abstract can then lose the reader's interest and undo the benefit of an effective title - whilst the paper's abstract may appear in search results, the potential reader may go no further.

So ask yourself, while the abstract may have seemed adequate during earlier checks, does it:

• Do justice to the manuscript in this context?
• Highlight important findings sufficiently?
• Present the most interesting data?

Editors say, " Does the Abstract highlight the important findings of the study ?"

If there is a formal report format, remember to follow it. This will often comprise a range of questions followed by comment sections. Try to answer all the questions. They are there because the editor felt that they are important. If you're following an informal report format you could structure your report in three sections: summary, major issues, minor issues.

• Give positive feedback first. Authors are more likely to read your review if you do so. But don't overdo it if you will be recommending rejection
• Briefly summarize what the paper is about and what the findings are
• Try to put the findings of the paper into the context of the existing literature and current knowledge
• Indicate the significance of the work and if it is novel or mainly confirmatory
• Indicate the work's strengths, its quality and completeness
• State any major flaws or weaknesses and note any special considerations. For example, if previously held theories are being overlooked

Major Issues

• Are there any major flaws? State what they are and what the severity of their impact is on the paper
• Has similar work already been published without the authors acknowledging this?
• Are the authors presenting findings that challenge current thinking? Is the evidence they present strong enough to prove their case? Have they cited all the relevant work that would contradict their thinking and addressed it appropriately?
• If major revisions are required, try to indicate clearly what they are
• Are there any major presentational problems? Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work?
• Are there any ethical issues? If you are unsure it may be better to disclose these in the confidential comments section

Minor Issues

• Are there places where meaning is ambiguous? How can this be corrected?
• Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased?
• Are there any factual, numerical or unit errors? If so, what are they?
• Are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not

Your review should ultimately help the author improve their article. So be polite, honest and clear. You should also try to be objective and constructive, not subjective and destructive.

You should also:

• Write clearly and so you can be understood by people whose first language is not English
• Avoid complex or unusual words, especially ones that would even confuse native speakers
• Number your points and refer to page and line numbers in the manuscript when making specific comments
• If you have been asked to only comment on specific parts or aspects of the manuscript, you should indicate clearly which these are
• Treat the author's work the way you would like your own to be treated

Most journals give reviewers the option to provide some confidential comments to editors. Often this is where editors will want reviewers to state their recommendation - see the next section - but otherwise this area is best reserved for communicating malpractice such as suspected plagiarism, fraud, unattributed work, unethical procedures, duplicate publication, bias or other conflicts of interest.

However, this doesn't give reviewers permission to 'backstab' the author. Authors can't see this feedback and are unable to give their side of the story unless the editor asks them to. So in the spirit of fairness, write comments to editors as though authors might read them too.

Reviewers should check the preferences of individual journals as to where they want review decisions to be stated. In particular, bear in mind that some journals will not want the recommendation included in any comments to authors, as this can cause editors difficulty later - see Section 11 for more advice about working with editors.

You will normally be asked to indicate your recommendation (e.g. accept, reject, revise and resubmit, etc.) from a fixed-choice list and then to enter your comments into a separate text box.

Recommending Acceptance

If you're recommending acceptance, give details outlining why, and if there are any areas that could be improved. Don't just give a short, cursory remark such as 'great, accept'. See Improving the Manuscript

Recommending Revision

Where improvements are needed, a recommendation for major or minor revision is typical. You may also choose to state whether you opt in or out of the post-revision review too. If recommending revision, state specific changes you feel need to be made. The author can then reply to each point in turn.

Some journals offer the option to recommend rejection with the possibility of resubmission – this is most relevant where substantial, major revision is necessary.

What can reviewers do to help? " Be clear in their comments to the author (or editor) which points are absolutely critical if the paper is given an opportunity for revisio n." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Recommending Rejection

If recommending rejection or major revision, state this clearly in your review (and see the next section, 'When recommending rejection').

Where manuscripts have serious flaws you should not spend any time polishing the review you've drafted or give detailed advice on presentation.

Editors say, " If a reviewer suggests a rejection, but her/his comments are not detailed or helpful, it does not help the editor in making a decision ."

In your recommendations for the author, you should:

• Give constructive feedback describing ways that they could improve the research
• Keep the focus on the research and not the author. This is an extremely important part of your job as a reviewer
• Avoid making critical confidential comments to the editor while being polite and encouraging to the author - the latter may not understand why their manuscript has been rejected. Also, they won't get feedback on how to improve their research and it could trigger an appeal

Remember to give constructive criticism even if recommending rejection. This helps developing researchers improve their work and explains to the editor why you felt the manuscript should not be published.

" When the comments seem really positive, but the recommendation is rejection…it puts the editor in a tough position of having to reject a paper when the comments make it sound like a great paper ." (Jonathon Halbesleben, Editor of Journal of Occupational and Organizational Psychology)

Visit our Wiley Author Learning and Training Channel for expert advice on peer review.

Watch the video, Ethical considerations of Peer Review

You are using an outdated browser . Please upgrade your browser today !

What Is Peer Review and Why Is It Important?

It’s one of the major cornerstones of the academic process and critical to maintaining rigorous quality standards for research papers. Whichever side of the peer review process you’re on, we want to help you understand the steps involved.

This post is part of a series that provides practical information and resources for authors and editors.

Peer review – the evaluation of academic research by other experts in the same field – has been used by the scientific community as a method of ensuring novelty and quality of research for more than 300 years. It is a testament to the power of peer review that a scientific hypothesis or statement presented to the world is largely ignored by the scholarly community unless it is first published in a peer-reviewed journal.

It is also safe to say that peer review is a critical element of the scholarly publication process and one of the major cornerstones of the academic process. It acts as a filter, ensuring that research is properly verified before being published. And it arguably improves the quality of the research, as the rigorous review by like-minded experts helps to refine or emphasise key points and correct inadvertent errors.

Ideally, this process encourages authors to meet the accepted standards of their discipline and in turn reduces the dissemination of irrelevant findings, unwarranted claims, unacceptable interpretations, and personal views.

If you are a researcher, you will come across peer review many times in your career. But not every part of the process might be clear to you yet. So, let’s have a look together!

Types of Peer Review

Peer review comes in many different forms. With single-blind peer review , the names of the reviewers are hidden from the authors, while double-blind peer review , both reviewers and authors remain anonymous. Then, there is open peer review , a term which offers more than one interpretation nowadays.

Open peer review can simply mean that reviewer and author identities are revealed to each other. It can also mean that a journal makes the reviewers’ reports and author replies of published papers publicly available (anonymized or not). The “open” in open peer review can even be a call for participation, where fellow researchers are invited to proactively comment on a freely accessible pre-print article. The latter two options are not yet widely used, but the Open Science movement, which strives for more transparency in scientific publishing, has been giving them a strong push over the last years.

If you are unsure about what kind of peer review a specific journal conducts, check out its instructions for authors and/or their editorial policy on the journal’s home page.

Why Should I Even Review?

To answer that question, many reviewers would probably reply that it simply is their “academic duty” – a natural part of academia, an important mechanism to monitor the quality of published research in their field. This is of course why the peer-review system was developed in the first place – by academia rather than the publishers – but there are also benefits.

Are you looking for the right place to publish your paper? Find out here whether a De Gruyter journal might be the right fit.

Besides a general interest in the field, reviewing also helps researchers keep up-to-date with the latest developments. They get to know about new research before everyone else does. It might help with their own research and/or stimulate new ideas. On top of that, reviewing builds relationships with prestigious journals and journal editors.

Clearly, reviewing is also crucial for the development of a scientific career, especially in the early stages. Relatively new services like Publons and ORCID Reviewer Recognition can support reviewers in getting credit for their efforts and making their contributions more visible to the wider community.

The Fundamentals of Reviewing

You have received an invitation to review? Before agreeing to do so, there are three pertinent questions you should ask yourself:

• Does the article you are being asked to review match your expertise?
• Do you have time to review the paper?
• Are there any potential conflicts of interest (e.g. of financial or personal nature)?

If you feel like you cannot handle the review for whatever reason, it is okay to decline. If you can think of a colleague who would be well suited for the topic, even better – suggest them to the journal’s editorial office.

But let’s assume that you have accepted the request. Here are some general things to keep in mind:

Please be aware that reviewer reports provide advice for editors to assist them in reaching a decision on a submitted paper. The final decision concerning a manuscript does not lie with you, but ultimately with the editor. It’s your expert guidance that is being sought.

Reviewing also needs to be conducted confidentially . The article you have been asked to review, including supplementary material, must never be disclosed to a third party. In the traditional single- or double-blind peer review process, your own anonymity will also be strictly preserved. Therefore, you should not communicate directly with the authors.

When writing a review, it is important to keep the journal’s guidelines in mind and to work along the building blocks of a manuscript (typically: abstract, introduction, methods, results, discussion, conclusion, references, tables, figures).

After initial receipt of the manuscript, you will be asked to supply your feedback within a specified period (usually 2-4 weeks). If at some point you notice that you are running out of time, get in touch with the editorial office as soon as you can and ask whether an extension is possible.

Some More Advice from a Journal Editor

• Be critical and constructive. An editor will find it easier to overturn very critical, unconstructive comments than to overturn favourable comments.
• Justify and specify all criticisms. Make specific references to the text of the paper (use line numbers!) or to published literature. Vague criticisms are unhelpful.
• Don’t repeat information from the paper , for example, the title and authors names, as this information already appears elsewhere in the review form.
• Check the aims and scope. This will help ensure that your comments are in accordance with journal policy and can be found on its home page.
• Give a clear recommendation . Do not put “I will leave the decision to the editor” in your reply, unless you are genuinely unsure of your recommendation.
• Number your comments. This makes it easy for authors to easily refer to them.
• Be careful not to identify yourself. Check, for example, the file name of your report if you submit it as a Word file.

Sticking to these rules will make the author’s life and that of the editors much easier!

Explore new perspectives on peer review in this collection of blog posts published during Peer Review Week 2021

[Title image by AndreyPopov/iStock/Getty Images Plus

David Sleeman

David Sleeman worked as a Senior Journals Manager in the field of Physical Sciences at De Gruyter.

You might also be interested in

Academia & Publishing

Taking Libraries into the Future, Part 4: How IFLA Harnesses Social Media

10 summer reads for the intellectually curious, how to maximize your message through social media: a global masterclass from library professionals, visit our shop.

De Gruyter publishes over 1,300 new book titles each year and more than 750 journals in the humanities, social sciences, medicine, mathematics, engineering, computer sciences, natural sciences, and law.

Pin It on Pinterest

Peer review process

Introduction to peer review, what is peer review.

Peer review is the system used to assess the quality of a manuscript before it is published. Independent researchers in the relevant research area assess submitted manuscripts for originality, validity and significance to help editors determine whether a manuscript should be published in their journal.

How does it work?

When a manuscript is submitted to a journal, it is assessed to see if it meets the criteria for submission. If it does, the editorial team will select potential peer reviewers within the field of research to peer-review the manuscript and make recommendations.

There are four main types of peer review used by BMC:

Single-blind: the reviewers know the names of the authors, but the authors do not know who reviewed their manuscript unless the reviewer chooses to sign their report.

Double-blind: the reviewers do not know the names of the authors, and the authors do not know who reviewed their manuscript.

Open peer: authors know who the reviewers are, and the reviewers know who the authors are. If the manuscript is accepted, the named reviewer reports are published alongside the article and the authors’ response to the reviewer.

Transparent peer: the reviewers know the names of the authors, but the authors do not know who reviewed their manuscript unless the reviewer chooses to sign their report. If the manuscript is accepted, the anonymous reviewer reports are published alongside the article and the authors’ response to the reviewer.

Different journals use different types of peer review. You can find out which peer-review system is used by a particular journal in the journal’s ‘About’ page.

Why do peer review?

Peer review is an integral part of scientific publishing that confirms the validity of the manuscript. Peer reviewers are experts who volunteer their time to help improve the manuscripts they review. By undergoing peer review, manuscripts should become:

More robust - peer reviewers may point out gaps in a paper that require more explanation or additional experiments.

Easier to read - if parts of your paper are difficult to understand, reviewers can suggest changes.

More useful - peer reviewers also consider the importance of your paper to others in your field.

For more information and advice on how to get published, please see our blog series here .

How peer review works

The peer review process can be single-blind, double-blind, open or transparent.

You can find out which peer review system is used by a particular journal in the journal's 'About' page.

N. B. This diagram is a representation of the peer review process, and should not be taken as the definitive approach used by every journal.

Peer Reviewed Literature

What is peer review, terminology, peer review what does that mean, what types of articles are peer-reviewed, what information is not peer-reviewed, what about google scholar.

• How do I find peer-reviewed articles?
• Scholarly vs. Popular Sources

Research Librarian

For more help on this topic, please contact our Research Help Desk: [email protected] or 781-768-7303. Stay up-to-date on our current hours . Note: all hours are EST.

This Guide was created by Carolyn Swidrak (retired).

Research findings are communicated in many ways.  One of the most important ways is through publication in scholarly, peer-reviewed journals.

Research published in scholarly journals is held to a high standard.  It must make a credible and significant contribution to the discipline.  To ensure a very high level of quality, articles that are submitted to scholarly journals undergo a process called peer-review.

Once an article has been submitted for publication, it is reviewed by other independent, academic experts (at least two) in the same field as the authors.  These are the peers.  The peers evaluate the research and decide if it is good enough and important enough to publish.  Usually there is a back-and-forth exchange between the reviewers and the authors, including requests for revisions, before an article is published. 

Peer review is a rigorous process but the intensity varies by journal.  Some journals are very prestigious and receive many submissions for publication.  They publish only the very best, most highly regarded research. 

The terms scholarly, academic, peer-reviewed and refereed are sometimes used interchangeably, although there are slight differences.

Scholarly and academic may refer to peer-reviewed articles, but not all scholarly and academic journals are peer-reviewed (although most are.)  For example, the Harvard Business Review is an academic journal but it is editorially reviewed, not peer-reviewed.

Peer-reviewed and refereed are identical terms.

From  Peer Review in 3 Minutes  [Video], by the North Carolina State University Library, 2014, YouTube (https://youtu.be/rOCQZ7QnoN0).

Peer reviewed articles can include:

• Original research (empirical studies)
• Review articles
• Systematic reviews
• Meta-analyses

There is much excellent, credible information in existence that is NOT peer-reviewed.  Peer-review is simply ONE MEASURE of quality. 

Much of this information is referred to as "gray literature."

Government Agencies

Government websites such as the Centers for Disease Control (CDC) publish high level, trustworthy information.  However, most of it is not peer-reviewed.  (Some of their publications are peer-reviewed, however. The journal Emerging Infectious Diseases, published by the CDC is one example.)

Conference Proceedings

Papers from conference proceedings are not usually peer-reviewed.  They may go on to become published articles in a peer-reviewed journal. 

Dissertations

Dissertations are written by doctoral candidates, and while they are academic they are not peer-reviewed.

Many students like Google Scholar because it is easy to use.  While the results from Google Scholar are generally academic they are not necessarily peer-reviewed.  Typically, you will find:

• Peer reviewed journal articles (although they are not identified as peer-reviewed)
• Unpublished scholarly articles (not peer-reviewed)
• Masters theses, doctoral dissertations and other degree publications (not peer-reviewed)
• Book citations and links to some books (not necessarily peer-reviewed)
• Next: How do I find peer-reviewed articles? >>
• Last Updated: Aug 14, 2024 10:25 AM
• URL: https://libguides.regiscollege.edu/peer_review

How to Write and Publish a Research Paper for a Peer-Reviewed Journal

• Open access
• Published: 30 April 2020
• Volume 36 , pages 909–913, ( 2021 )

Cite this article

You have full access to this open access article

• Clara Busse   ORCID: orcid.org/0000-0002-0178-1000 1 &
• Ella August   ORCID: orcid.org/0000-0001-5151-1036 1 , 2  

282k Accesses

17 Citations

708 Altmetric

Explore all metrics

Communicating research findings is an essential step in the research process. Often, peer-reviewed journals are the forum for such communication, yet many researchers are never taught how to write a publishable scientific paper. In this article, we explain the basic structure of a scientific paper and describe the information that should be included in each section. We also identify common pitfalls for each section and recommend strategies to avoid them. Further, we give advice about target journal selection and authorship. In the online resource 1 , we provide an example of a high-quality scientific paper, with annotations identifying the elements we describe in this article.

Similar content being viewed by others

How to Choose the Right Journal

The Point Is…to Publish?

Writing and publishing a scientific paper

Explore related subjects.

• Artificial Intelligence

Avoid common mistakes on your manuscript.

Introduction

Writing a scientific paper is an important component of the research process, yet researchers often receive little formal training in scientific writing. This is especially true in low-resource settings. In this article, we explain why choosing a target journal is important, give advice about authorship, provide a basic structure for writing each section of a scientific paper, and describe common pitfalls and recommendations for each section. In the online resource 1 , we also include an annotated journal article that identifies the key elements and writing approaches that we detail here. Before you begin your research, make sure you have ethical clearance from all relevant ethical review boards.

Select a Target Journal Early in the Writing Process

We recommend that you select a “target journal” early in the writing process; a “target journal” is the journal to which you plan to submit your paper. Each journal has a set of core readers and you should tailor your writing to this readership. For example, if you plan to submit a manuscript about vaping during pregnancy to a pregnancy-focused journal, you will need to explain what vaping is because readers of this journal may not have a background in this topic. However, if you were to submit that same article to a tobacco journal, you would not need to provide as much background information about vaping.

Information about a journal’s core readership can be found on its website, usually in a section called “About this journal” or something similar. For example, the Journal of Cancer Education presents such information on the “Aims and Scope” page of its website, which can be found here: https://www.springer.com/journal/13187/aims-and-scope .

Peer reviewer guidelines from your target journal are an additional resource that can help you tailor your writing to the journal and provide additional advice about crafting an effective article [ 1 ]. These are not always available, but it is worth a quick web search to find out.

Identify Author Roles Early in the Process

Early in the writing process, identify authors, determine the order of authors, and discuss the responsibilities of each author. Standard author responsibilities have been identified by The International Committee of Medical Journal Editors (ICMJE) [ 2 ]. To set clear expectations about each team member’s responsibilities and prevent errors in communication, we also suggest outlining more detailed roles, such as who will draft each section of the manuscript, write the abstract, submit the paper electronically, serve as corresponding author, and write the cover letter. It is best to formalize this agreement in writing after discussing it, circulating the document to the author team for approval. We suggest creating a title page on which all authors are listed in the agreed-upon order. It may be necessary to adjust authorship roles and order during the development of the paper. If a new author order is agreed upon, be sure to update the title page in the manuscript draft.

In the case where multiple papers will result from a single study, authors should discuss who will author each paper. Additionally, authors should agree on a deadline for each paper and the lead author should take responsibility for producing an initial draft by this deadline.

Structure of the Introduction Section

The introduction section should be approximately three to five paragraphs in length. Look at examples from your target journal to decide the appropriate length. This section should include the elements shown in Fig.  1 . Begin with a general context, narrowing to the specific focus of the paper. Include five main elements: why your research is important, what is already known about the topic, the “gap” or what is not yet known about the topic, why it is important to learn the new information that your research adds, and the specific research aim(s) that your paper addresses. Your research aim should address the gap you identified. Be sure to add enough background information to enable readers to understand your study. Table 1 provides common introduction section pitfalls and recommendations for addressing them.

The main elements of the introduction section of an original research article. Often, the elements overlap

Methods Section

The purpose of the methods section is twofold: to explain how the study was done in enough detail to enable its replication and to provide enough contextual detail to enable readers to understand and interpret the results. In general, the essential elements of a methods section are the following: a description of the setting and participants, the study design and timing, the recruitment and sampling, the data collection process, the dataset, the dependent and independent variables, the covariates, the analytic approach for each research objective, and the ethical approval. The hallmark of an exemplary methods section is the justification of why each method was used. Table 2 provides common methods section pitfalls and recommendations for addressing them.

Results Section

The focus of the results section should be associations, or lack thereof, rather than statistical tests. Two considerations should guide your writing here. First, the results should present answers to each part of the research aim. Second, return to the methods section to ensure that the analysis and variables for each result have been explained.

Begin the results section by describing the number of participants in the final sample and details such as the number who were approached to participate, the proportion who were eligible and who enrolled, and the number of participants who dropped out. The next part of the results should describe the participant characteristics. After that, you may organize your results by the aim or by putting the most exciting results first. Do not forget to report your non-significant associations. These are still findings.

Tables and figures capture the reader’s attention and efficiently communicate your main findings [ 3 ]. Each table and figure should have a clear message and should complement, rather than repeat, the text. Tables and figures should communicate all salient details necessary for a reader to understand the findings without consulting the text. Include information on comparisons and tests, as well as information about the sample and timing of the study in the title, legend, or in a footnote. Note that figures are often more visually interesting than tables, so if it is feasible to make a figure, make a figure. To avoid confusing the reader, either avoid abbreviations in tables and figures, or define them in a footnote. Note that there should not be citations in the results section and you should not interpret results here. Table 3 provides common results section pitfalls and recommendations for addressing them.

Discussion Section

Opposite the introduction section, the discussion should take the form of a right-side-up triangle beginning with interpretation of your results and moving to general implications (Fig.  2 ). This section typically begins with a restatement of the main findings, which can usually be accomplished with a few carefully-crafted sentences.

Major elements of the discussion section of an original research article. Often, the elements overlap

Next, interpret the meaning or explain the significance of your results, lifting the reader’s gaze from the study’s specific findings to more general applications. Then, compare these study findings with other research. Are these findings in agreement or disagreement with those from other studies? Does this study impart additional nuance to well-accepted theories? Situate your findings within the broader context of scientific literature, then explain the pathways or mechanisms that might give rise to, or explain, the results.

Journals vary in their approach to strengths and limitations sections: some are embedded paragraphs within the discussion section, while some mandate separate section headings. Keep in mind that every study has strengths and limitations. Candidly reporting yours helps readers to correctly interpret your research findings.

The next element of the discussion is a summary of the potential impacts and applications of the research. Should these results be used to optimally design an intervention? Does the work have implications for clinical protocols or public policy? These considerations will help the reader to further grasp the possible impacts of the presented work.

Finally, the discussion should conclude with specific suggestions for future work. Here, you have an opportunity to illuminate specific gaps in the literature that compel further study. Avoid the phrase “future research is necessary” because the recommendation is too general to be helpful to readers. Instead, provide substantive and specific recommendations for future studies. Table 4 provides common discussion section pitfalls and recommendations for addressing them.

Follow the Journal’s Author Guidelines

After you select a target journal, identify the journal’s author guidelines to guide the formatting of your manuscript and references. Author guidelines will often (but not always) include instructions for titles, cover letters, and other components of a manuscript submission. Read the guidelines carefully. If you do not follow the guidelines, your article will be sent back to you.

Finally, do not submit your paper to more than one journal at a time. Even if this is not explicitly stated in the author guidelines of your target journal, it is considered inappropriate and unprofessional.

Your title should invite readers to continue reading beyond the first page [ 4 , 5 ]. It should be informative and interesting. Consider describing the independent and dependent variables, the population and setting, the study design, the timing, and even the main result in your title. Because the focus of the paper can change as you write and revise, we recommend you wait until you have finished writing your paper before composing the title.

Be sure that the title is useful for potential readers searching for your topic. The keywords you select should complement those in your title to maximize the likelihood that a researcher will find your paper through a database search. Avoid using abbreviations in your title unless they are very well known, such as SNP, because it is more likely that someone will use a complete word rather than an abbreviation as a search term to help readers find your paper.

After you have written a complete draft, use the checklist (Fig. 3 ) below to guide your revisions and editing. Additional resources are available on writing the abstract and citing references [ 5 ]. When you feel that your work is ready, ask a trusted colleague or two to read the work and provide informal feedback. The box below provides a checklist that summarizes the key points offered in this article.

Checklist for manuscript quality

Data Availability

Michalek AM (2014) Down the rabbit hole…advice to reviewers. J Cancer Educ 29:4–5

Article   Google Scholar  

International Committee of Medical Journal Editors. Defining the role of authors and contributors: who is an author? http://www.icmje.org/recommendations/browse/roles-and-responsibilities/defining-the-role-of-authosrs-and-contributors.html . Accessed 15 January, 2020

Vetto JT (2014) Short and sweet: a short course on concise medical writing. J Cancer Educ 29(1):194–195

Brett M, Kording K (2017) Ten simple rules for structuring papers. PLoS ComputBiol. https://doi.org/10.1371/journal.pcbi.1005619

Lang TA (2017) Writing a better research article. J Public Health Emerg. https://doi.org/10.21037/jphe.2017.11.06

Download references

Acknowledgments

Ella August is grateful to the Sustainable Sciences Institute for mentoring her in training researchers on writing and publishing their research.

Code Availability

Not applicable.

Author information

Authors and affiliations.

Department of Maternal and Child Health, University of North Carolina Gillings School of Global Public Health, 135 Dauer Dr, 27599, Chapel Hill, NC, USA

Clara Busse & Ella August

Department of Epidemiology, University of Michigan School of Public Health, 1415 Washington Heights, Ann Arbor, MI, 48109-2029, USA

Ella August

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Ella August .

Ethics declarations

Conflicts of interests.

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

(PDF 362 kb)

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Busse, C., August, E. How to Write and Publish a Research Paper for a Peer-Reviewed Journal. J Canc Educ 36 , 909–913 (2021). https://doi.org/10.1007/s13187-020-01751-z

Download citation

Published : 30 April 2020

Issue Date : October 2021

DOI : https://doi.org/10.1007/s13187-020-01751-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Manuscripts
• Scientific writing
• Find a journal
• Publish with us
• Track your research

Research Guides

Understanding peer review, peer review is...., is your journal article peer-reviewed, limit your search results to peer-reviewed articles, is everything in a peer-reviewed journal peer-reviewed, peer review process.

Get research help from anywhere! Select the "Ask! Chat with a librarian" icon to start a chat session. We can help you:

• Find sources about a topic
• Locate a specific book or article
• Use library services, resources, and tools effectively  

September 3-December 6

• Monday-Thursday: 10am-10pm
• Friday: 10am-5pm
• Saturday & Sunday: 12pm-6pm
• SERVICE CLOSURES: Monday, October 14th
• More information about the Ask a Librarian chat service
• the process by which scholars critically appraise each other's work to ensure a high level of scholarship in a journal and to improve the quality and readability of a manuscript. 
• applied to both primary articles (i.e. articles which present findings from original research) and review articles that summarize primary research. 

Note:    'Peer reviewed' and 'refereed' are synonyms .

How do you know if an article is from a peer-reviewed journal? Some databases allow you to limit your search to peer-reviewed journals. For other databases, you need to look up the title of the journal in Ulrich's Directory. 

1. Go to Ulrich's.

2. Type the JOURNAL TITLE ( not the article title ) into the search box, and click the green search button. Look for the journal title in your search results.

3. In the search results, look for a referee jersey icon to indicate that a journal is refereed. Refereed means the same as peer reviewed.

4. The Journal of Infectious Diseases is peer reviewed.  

REMEMBER:   It's the journal that's peer reviewed/refereed, so you are looking for the journal title in your search results, NOT the article title.

Many library databases include  a peer-reviewed filter which you can select to limits your search results to articles published in scholarly journals that use a peer-review editorial process  Find examples below. Questions?   Ask for assistance at your library.     

LibrarySearch

• Submit your search in  UofT LibrarySearch  and select "Peer-reviewed Articles" from the filter options on the left side of the screen. 

Submit your search in  Proquest  and select "Peer-reviewed Articles" from the filter options on the left side of the screen.

Not necessarily.   It is also possible that some contents of a peer reviewed journal will not have been peer reviewed.

For example, editorials may not be peer reviewed but research articles generally will be. This varies from journal to journal; look for "about the journal" or "instructions to authors"  or "journal homepage" for guidance on this.   See the following examples:

• see the statement on peer review at Science ("...only some of the papers are reviewed in depth....")
• see the statement on peer review at Nature ("...the following types of contribution...are peer reviewed...other contributed articles are not usually peer reviewed...."). Note especially the 2nd and 3rd paragraphs.

When the manuscript of an article is submitted to a peer-reviewed journal, independent experts are asked to read and comment on the manuscript. If approved by the reviewers, the manuscript is accepted for publication as an article in the journal.   

Journals have different peer review standards and procedures but there are a few main types:

• Most peer review is double-blind, which means that neither the reviewers nor the authors know each other's identities.
• Single-blind i s a variation where the reviewer knows who the author is, but the author does not know the reviewer.
• Open review refers to a process in which the reviewer's comments and author's replies are openly discussed before formal publication.

Every journal describes their specific peer review process in the author guidelines section of their website.  

(Image source)

• Last Updated: Sep 5, 2024 10:48 AM
• URL: https://guides.library.utoronto.ca/peer-review

Library links

• UTSC Library home
• U of T Libraries home
• Catalogue Search
• Renew items and pay fines
• All U of T Libraries' hours
• Engineering
• UT Mississauga Library
• UT Scarborough Library
• Information Commons
• All libraries

University of Toronto Scarborough Library 1265 Military Trail, Toronto, ON M1C 1A4 Canada Email help 416-287-7500 Map About web accessibility . Tell us about a web accessibility problem . About online privacy and data collection .

© University of Toronto . All rights reserved.

Connect with us

Unfortunately we don't fully support your browser. If you have the option to, please upgrade to a newer version or use Mozilla Firefox , Microsoft Edge , Google Chrome , or Safari 14 or newer. If you are unable to, and need support, please send us your feedback .

We'd appreciate your feedback. Tell us what you think! opens in new tab/window

What is peer review?

Reviewers play a pivotal role in scholarly publishing. The peer review system exists to validate academic work, helps to improve the quality of published research, and increases networking possibilities within research communities. Despite criticisms, peer review is still the only widely accepted method for research validation and has continued successfully with relatively minor changes for some 350 years.

Elsevier relies on the peer review process to uphold the quality and validity of individual articles and the journals that publish them.

Peer review has been a formal part of scientific communication since the first scientific journals appeared more than 300 years ago. The Philosophical Transactions opens in new tab/window of the Royal Society is thought to be the first journal to formalize the peer review process opens in new tab/window under the editorship of Henry Oldenburg (1618- 1677).

Despite many criticisms about the integrity of peer review, the majority of the research community still believes peer review is the best form of scientific evaluation. This opinion was endorsed by the outcome of a survey Elsevier and Sense About Science conducted in 2009 opens in new tab/window and has since been further confirmed by other publisher and scholarly organization surveys. Furthermore, a  2015 survey by the Publishing Research Consortium opens in new tab/window , saw 82% of researchers agreeing that “without peer review there is no control in scientific communication.”

To learn more about peer review, visit Elsevier’s free e-learning platform  Researcher Academy opens in new tab/window and see our resources below.

The peer review process

Types of peer review.

Peer review comes in different flavours. Each model has its own advantages and disadvantages, and often one type of review will be preferred by a subject community. Before submitting or reviewing a paper, you must therefore check which type is employed by the journal so you are aware of the respective rules. In case of questions regarding the peer review model employed by the journal for which you have been invited to review, consult the journal’s homepage or contact the editorial office directly.  

Single anonymized review

In this type of review, the names of the reviewers are hidden from the author. This is the traditional method of reviewing and is the most common type by far. Points to consider regarding single anonymized review include:

Reviewer anonymity allows for impartial decisions, as the reviewers will not be influenced by potential criticism from the authors.

Authors may be concerned that reviewers in their field could delay publication, giving the reviewers a chance to publish first.

Reviewers may use their anonymity as justification for being unnecessarily critical or harsh when commenting on the authors’ work.

Double anonymized review

Both the reviewer and the author are anonymous in this model. Some advantages of this model are listed below.

Author anonymity limits reviewer bias, such as on author's gender, country of origin, academic status, or previous publication history.

Articles written by prestigious or renowned authors are considered based on the content of their papers, rather than their reputation.

But bear in mind that despite the above, reviewers can often identify the author through their writing style, subject matter, or self-citation – it is exceedingly difficult to guarantee total author anonymity. More information for authors can be found in our  double-anonymized peer review guidelines .

Triple anonymized review

With triple anonymized review, reviewers are anonymous to the author, and the author's identity is unknown to both the reviewers and the editor. Articles are anonymized at the submission stage and are handled in a way to minimize any potential bias towards the authors. However, it should be noted that: 

The complexities involved with anonymizing articles/authors to this level are considerable.

As with double anonymized review, there is still a possibility for the editor and/or reviewers to correctly identify the author(s) from their writing style, subject matter, citation patterns, or other methodologies.

Open review

Open peer review is an umbrella term for many different models aiming at greater transparency during and after the peer review process. The most common definition of open review is when both the reviewer and author are known to each other during the peer review process. Other types of open peer review consist of:

Publication of reviewers’ names on the article page 

Publication of peer review reports alongside the article, either signed or anonymous 

Publication of peer review reports (signed or anonymous) with authors’ and editors’ responses alongside the article 

Publication of the paper after pre-checks and opening a discussion forum to the community who can then comment (named or anonymous) on the article 

Many believe this is the best way to prevent malicious comments, stop plagiarism, prevent reviewers from following their own agenda, and encourage open, honest reviewing. Others see open review as a less honest process, in which politeness or fear of retribution may cause a reviewer to withhold or tone down criticism. For three years, five Elsevier journals experimented with publication of peer review reports (signed or anonymous) as articles alongside the accepted paper on ScienceDirect ( example opens in new tab/window ).

Read more about the experiment

More transparent peer review

Transparency is the key to trust in peer review and as such there is an increasing call towards more  transparency around the peer review process . In an effort to promote transparency in the peer review process, many Elsevier journals therefore publish the name of the handling editor of the published paper on ScienceDirect. Some journals also provide details about the number of reviewers who reviewed the article before acceptance. Furthermore, in order to provide updates and feedback to reviewers, most Elsevier journals inform reviewers about the editor’s decision and their peers’ recommendations. 

Article transfer service: sharing reviewer comments

Elsevier authors may be invited to  transfer  their article submission from one journal to another for free if their initial submission was not successful. 

As a referee, your review report (including all comments to the author and editor) will be transferred to the destination journal, along with the manuscript. The main benefit is that reviewers are not asked to review the same manuscript several times for different journals. 

Tools & resources

Elsevier researcher academy modules.

The certified peer reviewer course opens in new tab/window

Transparency in peer review opens in new tab/window

Reviewers’ Update articles

Peer review using today’s technology

Lifting the lid on publishing peer review reports: an interview with Bahar Mehmani and Flaminio Squazzoni

How face-to-face peer review can benefit authors and journals alike

Innovation in peer review: introducing “volunpeers”

Results masked review: peer review without publication bias

Interesting reads

"Is Peer Review in Crisis?" Perspectives in Publishing No 2, August 2004, by Adrian Mulligan opens in new tab/window

“The history of the peer-review process” Trends in Biotechnology, 2002, by Ray Spier opens in new tab/window

Publishing Research Consortium Peer review survey . 2015. Elsevier; 2015 

Start Your Research

• Generating Topic Ideas
• Using AI for Topic Ideas
• Getting More Specific
• Learn About Your Topic
• Finding In-Depth Sources
• Finding Data Sources

Peer-Review Articles

Recognizing peer-review articles, confirming peer-review (when in doubt).

• Primary vs. Secondary Sources
• Think About How a Source Fits Your Project
• Learn How to Efficiently Evaluate Sources
• Save Your Work
• In OneSearch
• On Your Computer

How Does the Review Process Work?

Reading Academic Articles

Reading academic articles is challenging.

First pass: 

• Read the abstract, introduction, and discussion/conclusion
• Is this article relevant and useful for your project?

Second pass 

• Read the methods and results section
• Even if you don't understand everything in these sections, can you understand how the author(s) arrived at their conclusion?

Many people use "academic" or "scholarly" sources to mean the same thing as peer-reviewed sources.

Peer-reviewed articles are papers that are reviewed by  a panel of experts prior to acceptance and publication

• reviewers often evaluate the  validity ,  significance , and  originality  of a work

As a result, the quality of peer-review articles is generally viewed as high.

• Google the journal title. Look at the journal homepage or Wikipedia for details.
• Look for a "peer reviewed" symbol in a library database. For example, in OneSearch you can see if an article is published in a peer reviewed journal by the symbol. 

• << Previous: Finding Data Sources
• Next: Peer-Review Articles >>
• Last Updated: Sep 4, 2024 12:07 PM
• URL: https://libguides.cvtc.edu/research

Chippewa Valley Technical College Library

620 West Clairemont Avenue, Eau Claire, WI 715-833-6285 | [email protected] Admin Login

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Saudi J Anaesth
• v.13(Suppl 1); 2019 Apr

The peer review process

Dmitry tumin.

1 Department of Anesthesiology and Pain Medicine, Nationwide Children's Hospital, Columbus, Ohio, USA

2 Department of Pediatrics, Nationwide Children's Hospital and The Ohio State University, Columbus, Ohio, USA

Joseph Drew Tobias

3 Department of Anesthesiology and Pain Medicine, The Ohio State University College of Medicine, Columbus, Ohio, USA

The peer review process provides a foundation for the credibility of scientific findings in medicine. The following article discusses the history of peer review in scientific and medical journals, the process for the selection of peer reviewers, and how journal editors arrive at a decision on submitted manuscripts. To aid authors who are invited to revise their manuscripts for further consideration, we outline steps for considering reviewer comments and provide suggestions for organizing the author's response to reviewers. We also examine ethical issues in peer review and provide recommendations for authors interested in becoming peer reviewers themselves.

Introduction

The review of research articles by peer experts prior to their publication is considered a mainstay of publishing in the medical literature.[ 1 , 2 ] This peer review process serves at least two purposes. For journal editors, peer review is an important tool for evaluating manuscripts submitted for publication. Reviewers assess the novelty and importance of the study, the validity of the methods, including the statistical analysis, the quality of the writing, the presentation of the data, and the connections drawn between the study findings and the existing literature. For authors, peer review is an important source of feedback on scientific writing and study design, and may aid in professionalization of junior researchers still learning the conventions of their field. Nevertheless, peer review can be frustrating, intimidating, or mysterious. This can deter authors from publishing their work or lead them to seek publication in less credible venues that use less rigorous peer review or do not subject manuscripts to peer review at all. In this article, we trace the origins of the scientific peer review system, explain its contemporary workings, and present authors with a brief guide on shepherding their manuscripts through peer review in medical journals.

The History of Scientific Peer Review

The introduction of peer review has been popularly attributed to the Royal Society of Edinburg, which compiled a collection of articles that had undergone peer review in 1731.[ 2 , 3 ] However, this initial process did not meet the criteria of peer review in its modern form, and well into the twentieth century, external and blinded peer review was still far from a requisite for scientific publication. Albert Einstein protested to the editor of an American journal in 1936 that his article was sent out for review, whereas this was not the practice of the German journals to which he had previously contributed.[ 4 ] Nevertheless, by the 1960s, the scientific value of peer review was becoming widely accepted, and in recent years, publication in a peer-reviewed journal has become a standard metric of scientific productivity (for the researchers) and validity (for the study).[ 5 , 6 ] In fact, publication in peer-reviewed quality journals is used to evaluate the quality of research during the academic promotion process. Today, peer review continues to evolve with the introduction of open review (reviewer comments posted publicly with the final article), postpublication review (reviews solicited from readers in an open forum after article publication), and journal review networks (where reviews are transferred from one journal to another when an article is rejected).[ 7 , 8 , 9 ] The constant at the center of this change remains the individual reviewer, who is asked to contribute their expertise to evaluating a manuscript that may or may not ever be shared with a wider scientific audience.

Reviewer Selection

The opacity of the peer review process is due, in part, to the anonymity of the reviewers and authors' lack of familiarity with how reviewers are selected. Typically, reviewers are selected by an editor of the journal, although depending on the size and organization of the journal, this may be the Editor-in-Chief, an Associate Editor, a Managing Editor, or an Editorial Assistant. Some journals permit authors to suggest their own reviewers, although the extent to which editors use these suggestions is variable. Authors may also be asked specifically or allowed to oppose reviewers, if they feel that certain scholars cannot grant their manuscript an unbiased hearing. Again, it is at the editors' discretion whether these requests are heeded. It has been suggested that these “opposed” reviewers may even be deliberately selected to ensure critical evaluation of a controversial manuscript. Alternatively, for very specific and narrow subject areas, there may be a limited number of appropriately qualified reviewers.

In general, reviewers may be of any academic rank and from a wide range of medical disciplines. A reviewer may be selected for their expertise in the topic of the study, but also for their general methodological expertise, or because they have been a reliable reviewer for the journal in the past. Qualified reviewers may not be invited if they cannot be reached by the editorial team, if they tend to submit late or uninformative reviews, or if they are too closely connected with the manuscript authors (e.g., colleagues at the same institution) and therefore may not provide an unbiased review. The reviewers initially selected by the editors may decline the invitation to review, mandating that the editors seek other reviewers. Unfortunately, this process of waiting for a response from the initial invitation to review (aside from the time taken to review) is one of the more common causes resulting in a delay in getting a response from the journal when a manuscript is submitted. The invited reviewer may pass the review on to a junior faculty member to allow them to participate and experience the academic peer review process. This may be performed with the permission of the editor, and noted after the review is submitted to the editor when the invited reviewer identifies that another person has participated in the process.

The initially received reviews may conflict with one another, leading the editors to cast a wider net for experts who will agree to review a submission. Because many factors may delay the completion of the review process, editors may proactively invite more reviews than they require and decide on the manuscript after a minimum number of reviews have been completed. The use of email and the internet has greatly facilitated communication for the review process, which used to be accomplished via telephone and postal mail. In most instances, an initial email is sent to the reviewer inquiring regarding their availability and interest. They are then asked to agree to review, at which time, a secondary email with a link to the journal site, the manuscript, and the review forms is sent.

How Reviewers Assess a Manuscript

From the reviewer's perspective, participation in the review process begins with an invitation from the journal editors to consider reviewing a submitted manuscript. If they accept, the reviewers will be able to access the submitted manuscript files, and sometimes the authors' cover letter, and other article metadata (e.g., the authors' list of preferred reviewers, figures, tables, etc.). Some journals ask reviewers to complete a structured questionnaire regarding the manuscript, rating its attributes on a numeric scale, or answering specific questions about each article section. All journals permit the submission of free-response evaluations. It is these evaluations that typically carry the greatest weight in the editors' final decision. The free-text reviewer reports also give the authors specific instructions about revising their manuscript and responding to the concerns that are raised. Reviewers may also submit confidential free-response comments to the editors (not seen by the authors) and indicate to the editors if they would be willing to review a revised version of the manuscript. In the end, the reviewer is asked to indicate their final recommendation to accept the manuscript without changes, accept after minor revisions, reconsider after major revisions, or reject. Some journals may offer additional variations on these recommendations, such as “reject but allow resubmission,” discussed below.

Regardless of the requested format for reviews, reviewers will typically evaluate several key aspects of submitted manuscripts. For original research studies, these will include the importance of the research question, the rigor of the methods, the completeness, accuracy, and novelty of the study and its results, and the validity of conclusions drawn from the data. The presentation of the manuscript, including the writing style, structure, grammar, and syntax also determine how the manuscript is received by the reviewers. Although the study design and results may be valid, these findings may be lost if the presentation is not precise or if there are grammar and spelling errors.

Reviewers also consider whether the study adds to existing knowledge in the field, whether it was ethically conducted, and whether it may be subject to any conflicts of interest. The editor and the reviewers also evaluate the study content and decide whether it is valuable and relevant to the readers of the journal. Although the study may be valid and well performed, it may be decided that the subject matter fits more appropriately in a journal of a different specialty. Along those lines, there may be overlap in the interests and fit of journals in different specialties, so that common topics in anesthesiology research may be of interest to journals from surgical specialties, pain medicine, or healthcare quality and patient safety, depending on the article content.

Some reviewers may submit their comments in paragraph form, building a narrative of the study's strengths and weaknesses section by section, whereas others may submit a short summary of the study followed by a list of criticisms or suggested corrections. Less commonly, reviewers may annotate the original manuscript with specific changes and questions or using the track-changes function of the word-processing software. Although the reviewers may recommend a specific editorial decision (e.g., recommend accepting an article with revisions, recommend rejection) in their comments to authors, this is generally discouraged by most journals and does not override the final decision reached by the editorial team. The ultimate decision generally resides with the section editor or the editor-in-chief, once they have seen and evaluated the comments of the reviewers. Depending on the format of the journal, the manuscript may be reviewed by one to five individuals. When there are specific statistical questions or advanced methods used, a separate review of the analytic methods may be required. For high-profile journals with high Impact Factors, a recommendation to accept may be required from all reviewers to receive a favorable editorial decision. At times, if there is a split decision, an additional reviewer or member of the editorial board may be asked to evaluate the manuscript to break the tie.

Almost all journals practice blinded review, where the reviewers' identities are not revealed to the authors. Double-blind review, where authors' identities are concealed from reviewers, although previously uncommon in medical journals, has been increasingly used. The editors communicate their decision and reviewers' evaluations to the authors in a decision letter (e-mail), informing of manuscript acceptance or rejection.

Reviews and the Editorial Decision

The comments submitted by external reviewers are collected by the editorial team and considered when determining the overall decision on the submitted article. The reviews may be read directly by the Editor-in-Chief, or by one or more Associate or Section Editors. The first editor reading the reviews might provide a recommendation that is then considered by the more senior editor; or the editors may convene to discuss the reviews and reach a decision as a group. In some journals, editors may write their own summary of the reviewers' criticism (sometimes adding their own) or may point out the critiques they consider most important to their decision. In other journals, editors weigh the number of positive and negative reviews or may reject an article unless all reviewers endorse its acceptance or revision.

Based on the external reviews and their own reading of the manuscript, the editors will reach one of several options regarding the manuscript. Unconditional acceptance of an article on its first submission to a journal (without any requested revisions) is very rare. Sometimes, articles will be conditionally accepted or accepted with minor revisions, meaning that the editors wish the authors to make changes to their manuscript based on the reviewers' comments but will not send the revised manuscript for a further round of external review. Rather, if the comments are generally minor, the editor will ensure that the comments are appropriately addressed in the authors' revision. The more common decision is “major revision,” where editors are willing to consider a revised version of the article but will subject it to further external review, by the original reviewers, a new set of reviewers, or a combination of both. Some journals also use a “reject and resubmit” decision, indicating lower enthusiasm for a resubmitted version of the article but still permitting resubmission, perhaps in an alternative format (e.g., brief report or letter to the editor, vs. full article) or with extensive revisions. For this latter decision, a full review will be accomplished as the revised manuscript is handled in much the same way as a new submission.

If the editors feel an article is a poor fit for their journal or falls too far below its standards, they may reject submissions outright without sending the manuscript for external review. This “desk reject” should not be confused with articles being “unsubmitted” by a managing editor or editorial assistant. The latter can happen due to style or formatting issues with the initial submission, which the author is asked to correct before the manuscript proceeds to review. Having a manuscript “unsubmitted” does not preclude resubmission of a corrected manuscript and is unlikely to affect reviewer assessment and, eventually, editorial decision.

Revising the Manuscript

When the initial editorial decision is positive, but not an unconditional acceptance, authors may elect to revise their manuscript and resubmit it to the same journal with a point-by-point response to the reviewers (discussed in the next section). The primary aim of the authors for this revision should be to address the criticisms and concerns raised during the initial review. Yet, this may be easier said than done when faced with conflicting recommendations, hostile reviews, or simply a large number of suggestions to be accommodated within a strict manuscript word limit. To streamline the process of responding to reviews, we offer the following roadmap as a suggestion.

Address the “fatal flaws”

Reviewers or editors may point out critical weaknesses of the study that prevent it from drawing the intended conclusions or even any conclusions at all. For example, an inaccuracy in the data, a bias in patient recruitment, a limitation of sample size, or a lack of follow-up may be so severe that the manuscript cannot provide credible evidence on the treatment or exposure it is meant to study. In particular, a lack of appropriate ethical approval would disqualify a study from publication, no matter how methodologically rigorous it may have been. In systematic reviews and analyses of existing databases, prior publication of a near-identical paper by a different group may also fundamentally preclude a paper from acceptance. On the rare occasions when the paper's central conclusions are found invalid and cannot be corrected through new analysis or a different framing of the authors' argument, reconceiving the study may be a better approach than attempting to revise and resubmit. At other times, some of these issues may be approached and the editor and reviewers satisfied by adding text to the discussion outlining the limitations of the current study. This may allow authors to acknowledge the concerns expressed by the reviewers and yet not redo their study from the beginning.

Amend the data analysis

More commonly, reviewers ask for changes to the data analysis without implying that these requests invalidate the entire study. We recommend making these changes before any further edits to the manuscript, because the intent is often to see if the paper's original findings are robust. In the best case scenario, any additional analysis will only confirm and strengthen the central conclusions. However, additional analyses sometimes reveal contradicting findings, which the authors should frankly address in the revised manuscript, by pointing out the contradiction and speculating about why different analyses of their data may have reached different conclusions. Especially when the study design was prospectively registered, the authors should explain in the manuscript which analyses were planned a priori and which were added post hoc . In these studies, authors should also avoid changing the pre-specified primary outcome, which would have been used for any a priori power or sample size calculation.

Decline infeasible or inappropriate suggestions

Some requests may not be feasible, for example, when requested data were not collected for a prospective study, or when collecting the data would mean starting chart review from scratch for a retrospective study. At other times, it may not be feasible to comply with the reviewers' requests if they disagree with the study type, the study cohort, or make other requests that would require a new or different study to address. Reviewers could also request changes to the statistical analysis that are not appropriate for the data at hand or for the study aims. In these cases, authors have the choice of rebutting the reviewers' comments while making no change in their manuscript, but an argumentative revision that leans too heavily on this option may be received poorly on re-review, resulting in rejection of the manuscript. In our experience, authors may be successful in responding to the reviews while rebutting one or two of the reviewers' suggestions, but a legitimate argument must be made for the rebuttal, and the reasons clearly stated.

Explain the study rationale and methods

Having completed the revision of the data analysis, authors should check that their methods section includes a complete and correct explanation of how the data were collected and explains how the analysis was performed. It may be appropriate to end the introduction by stating the hypothesis of the study. In the methods section, reviewers will often ask about the ethical committee approval of the study, the site(s) where the study was conducted, patient inclusion and exclusion criteria, the consent process, the procedures involved and the protocol for anesthetic management, and the specific data points that were collected during the study. For prospective clinical studies, authors should also indicate whether the study was submitted to a trial registry (such as ClinicalTrials.gov), and whether this was done before or after study enrollment had started. Clearly stated ethical approval and trial registration information must be provided for all submissions. Explanations may be sought if the editors and reviewers believe that the study did not meet standards for ethical approval, patient consent, or trial registration. Other requests related to methods may ask to clarify how the primary and secondary aims outlined in the introduction were addressed in the analysis, and how the sample size was determined, whether based on a statistical power analysis or logistical considerations (e.g., how many patients could be recruited with available resources). When a statistical power analysis is performed, reviewers may ask for more detail about the assumptions of this analysis and any supporting data from pilot studies or previous publications.

Check the conclusions and limitations

Having revised the introduction, methods, and results, the authors should revise the discussion to make any changes to the conclusions required by new or different study findings. We recommend that authors start the discussion with a review of what the study found, and then discuss how the study findings relate to similar work that has been previously published. An excessively long discussion does not ensure that a study will be published and, in fact, may detract from the quality of the manuscript. For a scientific study (retrospective or prospective), the discussion should not read like a comprehensive review of the literature. Typically, the discussion of study limitations will be expanded in the revised manuscript to include additional study weaknesses pointed out by reviewers, acknowledge suggested changes that could not be made to the study methods, and mention other suggestions for future studies that would build on the current results or answer questions left unanswered by the current study. Reviewers may ask that the conclusions be more specific in addressing the primary aim or hypothesis of the study (stated in the introduction), but they may also encourage authors to go further afield in their discussion, connecting their findings to results from previous publications and describing how their findings support or challenge current clinical practice.

Writing the Response to Reviewers

As seen above, manuscript revision can require more writing and (re)analysis than even the initial submission. Therefore, the aim of the revision memo (response to reviewers) is to summarize for the editors and reviewers how each change addresses the concerns raised on the initial review. This document is handled differently by different journals; some require it to be uploaded as a separate file, others require that the revision memo be entered in a text-box during the online submission process, and still others require that the response to review be included in the cover letter for the resubmitted manuscript. Therefore, authors should pay close attention to the decision letter and its instructions as to how they should submit their response to reviewers and how they should refer to manuscript edits in the revision memo (e.g., by page number, by line number, or copying sections of the revised manuscript into the memo).

Typically, the reviewers' comments should be copied and entered in the response memo so that each comment is numbered and the response clearly listed after it, in a different font style or color. It is equally important to determine how the journal would like the changes tracked in the revised manuscript. Some journals will ask that the authors use the track-changes mode in the word processing software, whereas others may ask for changes to be highlighted or be added in a different color font. Deleted manuscript text may need to be shown in strike-through font or simply removed from the revised submission, depending on the journal. Journals may ask for two copies of the revised manuscript: one showing the changes and one in a clean format that is ready for copyediting.

A typical revision memo will include a short paragraph acknowledging the editorial decision and reviewer comments and briefly summarizing key changes made to the manuscript. This would be followed by a numbered list of comments from the editors and reviewers (as received in the decision letter), with the authors' response to each one. Although not all reviewers and editors submit their comments as a numbered list, the authors may want to break up long sections or paragraphs of the reviews into shorter, numbered comments, to separately describe how each one was addressed in the revision. The authors' responses need not be excessively ingratiating but should respect the reviewers' effort in evaluating the manuscript, and concisely explain what was changed or why a change was not or could not be made. Different reviewers may have conflicting recommendations for revision. This may be as simple as one asking for a more concise definition of a method while another asking for a more detailed explanation. With conflicting reviews, the authors may consider taking the recommendation that is endorsed in the editor's comments (if this is provided), the one that is best aligned with the study aims, or the one that best matches the methods and writing style used in other contemporary papers in the field; and explaining this rationale when responding to the reviewers.

What to Do with a Rejected Manuscript

Based on reviewer reports or their own judgment, editors may reject a manuscript with no option to resubmit. It is essential to read the decision letter closely as some journals will state that they cannot publish a manuscript in its current form but offer to consider a new submission of a substantially revised manuscript (“reject and resubmit,” as mentioned above, in contrast to “revise and resubmit”). When the manuscript is rejected with no option of resubmission, authors may appeal this decision, but this option is rarely exercised and may not change the editors' decision. Appeals are also generally only successful when made by experienced and recognized scholars in the field.

Unless the study is discovered to be so flawed as to preclude publication in any venue, authors will usually consider submitting it to another journal after the initial rejection. Taking a single rejection and tabling a manuscript without further submission is rarely a good option. It is possible that multiple rejections will precede an eventual acceptance for valuable work. Given the amount of time taken to devise, implement, and up a study, we encourage authors to consider resubmission to a new journal, if the study is well conceived and addresses an important problem or question. In this case, the criticisms in the initial review are not binding, but still worth the authors' consideration. Particularly, authors should address any major flaws in the study's approach and conclusions (distinct from reviewers' preferences for additional data analysis unrelated to the primary aims), and correct any factual, spelling, or grammatical errors prior to resubmission. Adding recommended secondary analyses could sometimes strengthen the next submission, although just as often, the reviewers at the next journal may find these additional analyses superfluous, and will have their own set of analyses to recommend.

Becoming a Reviewer

Like any complex skill, navigating the peer review process is best learned through repetition. Becoming a peer reviewer for scientific journals is an important way to hone this skill, as well as providing a service to the scientific community, and adding to one's academic credentials as an expert whose opinion is sought by journal editors. The most common entry point to becoming a reviewer is through scientific publication; the authors of published articles can be contacted by another journal to provide a review on a related submission. One's expertise in a specific area may be noted by the editor who performs a topic search of key words when looking for reviewers. Alternatively, editors and associate editors may call on colleagues who they know are recognized experts in a particular field. Academic mentorship is also important, as mentors may ask junior colleagues and faculty to help them with reviewing article submissions, or may pass their name along to journal editors to be considered for inclusion in the reviewer pool. Once one has successfully reviewed for a journal, they are frequently called upon to review other submissions, especially if their review was returned in a timely manner. Many journals will give a specific timeframe within which the review is to be completed, while others will not. In most cases, a response within 2–4 weeks is considered acceptable. Some journals have now started editorial fellowships that aim to provide an immersive experience in the peer review and publishing process for early-career scientists. Lastly, researchers wishing to become peer reviewers may contact journal editors themselves, or register reviewer accounts in journal online submission systems. Although the general structure of peer review reports is described above, more specific guidance on performing peer review is available in other publications.[ 10 , 11 ]

Peer Review Ethics

Authors, reviewers, and editors have a shared responsibility for the ethical conduct of peer review. This is necessary to sustain the professional and public trust in peer review, as a system of evaluation that is accurate, constructive, and free from bias. Recently reported ethics violations have included authors misrepresenting the identity of suggested reviewers, reviewers plagiarizing a manuscript sent to them for review or recommending its rejection and then conducting a similar study, and editors inappropriately pressuring authors to cite articles published in their journal.[ 12 , 13 , 14 ] Some journals and publishers have also been criticized for circumventing the peer review process for submitted manuscripts.[ 15 ] For reviewers, it is most important that they be unbiased and not have any hidden agendas or personal vendettas to settle. For authors, ethical conduct in peer review includes disclosing the study's ethics committee approval, trial registration, and consent process; disclosing any related or overlapping prior publications; disclosing any actual or potential conflicts of interest; and submitting the manuscript only to one journal. These requirements are typically stated in the journal's guidelines for authors, and may need to be acknowledged in the cover letter accompanying the manuscript. In responding to reviews, authors should also carefully consider whether their revisions still fall within the scope of the ethics committee approval for the study and the informed consent that was obtained, and whether the revised manuscript remains faithful to the aims and study design of any pre-registered trial protocol.

Scientific research is not complete until it is published, but not all research can or should be published. It falls to peer-review to determine the difference. By engaging with the process of peer review, authors can improve the quality of their work as well as gain confidence that it is published in a reputable medium. Furthermore, the fact that a study has been peer reviewed will increase its stature and potential for recognition. However, the peer review process does not assure this. Although responding to reviews can be challenging, we hope that the suggestions sketched out in this article will help authors plan their approach to manuscript revision and resubmission. We also encourage authors to participate in this process as reviewers, so that the labor of peer review is properly shared among the community of scientists.

Financial support and sponsorship

Conflicts of interest.

There are no conflicts of interest.