Overview: For a scientific community to grow its knowledge, individual researchers need to enter a conversation with the rest of this scientific community about their newest research results, the validity of those results, and how those results fit into the existing knowledge. The peer review of manuscripts by journals, the publication of accepted articles, and then the discussions of the articles by others (citations in articles by other researchers) is one way such dialog occurs.
 |
| Photo by Steve Johnson on Unsplash |
Post Contributor(s): Marian Kennedy and Dr. Sapna Sarupria, Associate Professor in the University of Minnesota Department of Chemistry. Professor Sarupria studies materials using computational methods (molecular modeling, simulations, and statistical mechanics) and won one of the coveted NSF CAREER awards in 2017. Her research involves developing sampling techniques in molecular simulations and applying them in understanding long-standing problems in condensed matter.
*****
Purpose of a journals and journal articles
Disseminating new knowledge to your community of practice – other researchers and practitioners who will build on your contribution to the field – is a vital part of research, and journal articles are a major vehicle of such dissemination. A journal’s importance in its field can be gauged by its “impact factor,” which is based on the average number of citations of each article it has published in the last two years. Journals are established for technical areas and operate under the oversight of an editorial board. The people on that board and the journal’s editors have the responsibility of making sure that articles published align with the focus of the journal and will be of interest to its readership.
What is the peer review process?
The review process for publication in a journal acts as a proving ground where reviewers probe an author’s breakthroughs and the merit of the work. If the reviewers are satisfied, then the journal editors allocate space within the journal for the study, Peer review is the process by which other scholars in a field to which a journal article aims to contribute review its technical quality and scholarly rigor, including the appropriate application of theories or frameworks and appropriate interpretation based on the current knowledge of the field.
What is the “blinding”?
Reviews may be single blind or double blind. During single-blind peer review, the reviewers can see who the authors are, but the authors do not know who the specific reviewers were. This is the most common type of review method in our fields (chemical engineering, materials science). In double-blind peer review, neither the authors nor reviewers know who the other is. The authors therefore must remove identifying information such as names, self-citations, and acknowledgements from the manuscript and put their name only on a separable cover page. Many social science journals use the double-blind system, and engineering education articles are also frequently double blind.
What is the difference between “traditional” and “open-access” journals?
Journals’ funding mechanisms are diverging. Open-access or “pay-to-publish” journals charge the authors to publish their work. Those charges allow the journal to give the article to anyone who wants to read it. This is quite different from “traditional” journals where researchers (or their institutions) pay for access to the articles either through a site license or average number of licenses.
What happens when you submit a manuscript to a journal for publication?
When a paper is submitted to a journal, the editor of that journal typically scans the article to confirm that it matches the scope or theme of the journal. If the editor deems it does not, this is called a desk rejection. If the submitted manuscript does fit the journal focus, the editor sends requests to reviewers (typically 2-4) who the editor thinks can confirm that the manuscript, is of interest to the field (contributes new knowledge) and is of high technical quality. These reviewers, who remain unknown to the author, are given a window of time (often 2-4 weeks) to read the manuscript, provide a critique, and assign their final recommendation as to how the journal should respond. The editor reads all the reviewers’ remarks and recommendation, and then determines how the journal will respond: accept, accept with minor revisions, accept with major revisions, revise and resubmit [R&R], or decline. Most papers are declined or given an R&R.
Authors given the opportunity to revise will typically have significant time (3-6 months) to make the requested revisions. This timeframe can be shorter in some instances such as the editor wanting to include the article in a special issue. The authors are asked to respond thoughtfully to the reviewers’ comments, explaining what if any changes they made based on the comments, and listing locations in the manuscript where edits were made or quoting directly from the new version. Revisions might be as simple as altering the text to clarify a point or require significantly more work such as the inclusion of more samples or a change in the analysis methodology. Reviewers may also point out errors of style, grammar, punctuation, etc. When the revisions are significant, often the revised manuscript is sent back to the reviewers for critique. This process can continue until the reviewers are satisfied with the manuscript. Accepted papers, either after the R&R or without one, normally go through an editorial review with a staff member. If not, authors may elect to employ a freelance copy editor familiar with the journal’s preferred style. For some journals, those final editorial steps can take some time, up to a year. During this time, the author(s) can list the article on their CV or résumé as “accepted for publication.”
How do reviewers approach the review process?
We can only answer this based on our own experience. When we get a request to review a paper, the editor normally sends an email with the title and the paper’s abstract. We scan the paper to make sure that we have the expertise in that area, have an interest in the study, and the bandwidth to do an appropriate review. If we do, we accept the review assignment. We note our reasons for doing so in the next paragraph. After accepting a request to review, we typically print out the paper and designate time on our calendars to focus on it. When the time arrives, we first read the abstract and conclusions, then evaluate the figures, then read the entire paper from title to references. We then take notes about questions that arose throughout the process and draft the comments and recommendation based on those notes.
Elisabeth Pain published an article entitled “How to review a paper” in Science that provides thoughtful insights on the process by assembling comments from a range of researchers. We really appreciated the article highlighting that a good review requires:
- expertise in the field (can you offer an intelligent insight?)
- knowledge of related research methods
- a critical mind
- the ability to provide constructive feedback
- sensitivity to the feelings of authors
Unfortunately, there are plenty of reviews submitted to editors (and eventually sent on to the authors) where the reviewer does not evaluate the articles in the current context of the field (recent published findings). In addition, some review comments sometimes are not clearly crafted, such that the authors and journal editors are confused during the revision phase. The most difficult part of a good review may be the last component – “sensitivity to the feelings of the authors.”
When responding to reviewer comments and revising manuscripts, it is important to respond professionally to a reviewer who hasn’t succeed in all the aspects outlined above. This person is a peer in your field who just spent time reading your article without compensation. Give them the benefit of the doubt, assume they are trying their best and word your response to reflect that assumption.
Reviewers have been on both sides of the publication process (submitting and reviewing) and they ultimately want the authors to publish valuable work and are hoping that their critiques help the authors to increase the quality of the final paper. This is true even if the reviewer recommends rejection. At a minimum they expect you to produce another study, having learned from the process of preparing and submitting this one. But in general they expect you to do what you always should: regroup, revise, etc., and then rejoin the scientific discussion with their community by submitting to another journal more suited to your paper.
The value of being a reviewer and when should you start
Ultimately all members of a scientific community should work together to increase the knowledge of their field and therefore participate in reviewing manuscripts for journals. This activity is unpaid, but it is an accepted part of being a member of the field. Those who have completed some peer reviews:
· Will have learned how to professionally critique the work of their peers.
· Will be more thoughtful as they continue to develop their writing by understanding how editors, reviewers, and readers look at manuscripts.
When to start participating as a reviewer
You will be asked to review articles sooner that you probably anticipate. Dr. Sarupria made a group of students laugh uncomfortably when she commented, “If you have published, then you might get asked!” Editors are always in need of reviewers and often look at the list of authors recently published in their journals as potential reviewers. While being asked is a signal that editors think you have something significant to contribute to their process, you should decline if it is not an appropriate use of your time relative to your other responsibilities. When initially starting as early career scientists, we found it difficult to turn down a review opportunity. Dr. Kennedy remembers feeling she might never be asked again if she said no. But most editors probably appreciate a quick and honest response and will ask again if they know you will not accept an assignment and then complete it badly or later than promised.
For graduate students, we would suggest discussing the benefits and downsides of accepting an editor’s request with your mentoring team. If you do decide that you want to review, you might take the suggestion of the report by the APA student council in 2007 (“A graduate students’ guide to involvement in the peer review process”) of pairing with your research advisor to learn how to write a good review. The Journal of Engineering Education, in line with its mission and understanding of the importance and value of this procedure in expanding useful knowledge in the discipline, has begun to require that novice and master reviewers team together on an article reviews.
Takeaways for graduate students
For students pursuing PhDs who want to enter the professoriate, it is vital to have some papers in print by the time you graduate. Try to get an average of least one publication for each year you are a graduate student (such that four years results in four publications). Talk with your advisor about finding the “right” journals to submit to, working on preparing the manuscripts and on the submission process.
Today, the research productivity of academics is often, but not always, measured by the prestige of the journals in which they publish, the number of publications in press, and the impact of those publications. So, if you are considering an academic career, you should learn to identify the most influential journals in your field, the citation index of a journal, etc. However, not all research fields have journals with high impact factors. For example, some research fields, like hydrogen embrittlement, are small enough that they do not have journals with large impact factors. In contrast, journals for research fields like cardiovascular biomaterials will inherently have higher impact factors because research in that area tends to be picked up by a broader audience outside of that specific field area. This isn’t to say that one field is “better” than another. It’s just an artifact of how impact factors get calculated. Therefore, it’s important to know what the impact factors for journal in your particular field are and not compare across disciplines too much (e.g., an impact factor of 2.5 might be very high for one field but terribly low for another).
Discussions about the publishing landscape are ongoing, and the scientific ecosystem is not perfect. Indeed, the scientific community is struggling to build and maintain a diverse cohort of researchers. For now our community is lacking the data to identify if there are biases in the peer review process. Journal publishers are working to collect data. Authors will soon be asked to submit information on their race and gender to journals going forward so that the scientific community can identify whether there are biases in editing or review that sway which findings get published. This article describes this effort, which we applaud.
Acknowledgements: This blog post was edited by Kate Epstein of EpsteinWords. She specializes in editing and coaching for academics, and she can be reached at kate at epsteinwords.com. Dr. Sarupria acknowledges funding from National Science Foundation under grant number 2224643. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.