Skip to main content

Table 5 Explanations and Examples of quality domains and subdomains

From: Tools used to assess the quality of peer review reports: a methodological systematic review

N Domains Subdomains Explanations and Examples
1 Relevance of the study   Explanation: Items inquiring if the reviewer has discussed in the peer review report the importance of the research question and usefulness of the study.
Example: ‘Did the reviewer give appropriate attention to the importance of the question?’ [28]
2 Originality of the study   Explanation: Items inquiring if the reviewer has commented in the peer review report on the originality of the manuscript.
Example: ‘Did the reviewer discuss the originality of the paper?’ [23, 27]
3 Interpretation of the study results   Explanation: Items inquiring if the reviewer has commented in the peer review report on how authors interpreted and discussed the results of the study.
Example: ‘The reviewer commented accurately and productively on the quality of the author’s interpretation of the data, including acknowledgment of the data’s limitations.’ [26]
4 Strengths and weaknesses of the study General Explanation: Items inquiring if the reviewer has identified and commented in the peer review report on the general strong and weak points of the study.
Example: ‘How well it identified the study’s strengths and weaknesses?’ [35]
Methods Explanation: Items inquiring if the reviewer has identified and commented in the peer review report on the strong and weak points specifically related to study’s methods
Example: ‘Did the reviewer clearly identify strengths and weaknesses in the study’s methods?’ [28]
Statistical methods Explanation: Items inquiring if the reviewer has identified and commented in the peer review report on the strong and weak points specifically related to study’s statistical methods
Example: ‘Confidence intervals/p-values/overall fit’ [36]
5 Presentation and organization of the manuscript   Explanation: Items inquiring if the reviewer has made comments in the peer review report on the data presentation such as tables and figures and on the organization of the manuscript such as writing communication.
Example: ‘Are there any constructive suggestions on improvement of a. writing; b. data presentation and c. interpretation’ [54]
6 Structure of reviewer’s comments   Explanation: Items inquiring if the reviewer has made in the peer review report organized and structured comments.
Example: ‘Concise well-organized comments to the editor’ [50]
7 Characteristics of reviewer’s comments Clarity Explanation: Items inquiring if the reviewer has provided in the peer review report clear and easily to read comments.
Example: ‘How clear was this review? The review was easily read and interpreted by the editor and authors.’ [38]
Constructiveness Explanation: Items inquiring if the reviewer has provided in the peer review report helpful, relevant and realistic comments.
Example: ‘Were the reviewer’s comments constructive?’ [23, 27]
Detail/Thoroughness Explanation: Items inquiring if the reviewer has provided in the peer review report detailed and thorough comments supplying appropriate evidence.
Example: ‘Detail of commentary’ [33]
Fairness Explanation: Items inquiring if the reviewer has provided in the peer review report balanced and objective comments.
Example: ‘Balanced/fair’ [24, 36]
Knowledgeability Explanation: Items inquiring if the reviewer has showed in the peer review report to know and understand correctly the content of the manuscript.
Example: ‘Knowledge of the manuscript’s content area.’ [28]
Tone Explanation: Items inquiring if the reviewer has used a courteous tone in the peer review report.
Example: ‘Overall tone of the reviewers was also assessed as harsh or courteous.’ [34]
8 Timeliness of the review report   Explanation: Items inquiring if the reviewer has completed the peer review report on time.
Example: ‘Punctuality of the review’ [49]
9 Usefulness of the review report Decision making Explanation: Items inquiring if the reviewer has provided a peer review report useful to make a decision about the acceptance, revision or rejection of a manuscript
Example: ‘The reviewer provided the editor with the proper context and perspective to make a decision about acceptance or revision of the manuscript.’ [26]
Manuscript improvement Explanation: Items inquiring if the reviewer has provided useful suggestions in the peer review report to improve the manuscript.
Example: ‘This aspect is solely interested in how well the review aids the authors for improving their work and/or writing. Whether the review makes a good judgment regarding acceptance of the submission plays no role here whatsoever.’ [53]