Table 5 Explanations and Examples of quality domains and subdomains
From: Tools used to assess the quality of peer review reports: a methodological systematic review
N | Domains | Subdomains | Explanations and Examples |
|---|---|---|---|
1 | Relevance of the study | Explanation: Items inquiring if the reviewer has discussed in the peer review report the importance of the research question and usefulness of the study. Example: ‘Did the reviewer give appropriate attention to the importance of the question?’ [28] | |
2 | Originality of the study | Explanation: Items inquiring if the reviewer has commented in the peer review report on the originality of the manuscript. Example: ‘Did the reviewer discuss the originality of the paper?’ [23, 27] | |
3 | Interpretation of the study results | Explanation: Items inquiring if the reviewer has commented in the peer review report on how authors interpreted and discussed the results of the study. Example: ‘The reviewer commented accurately and productively on the quality of the author’s interpretation of the data, including acknowledgment of the data’s limitations.’ [26] | |
4 | Strengths and weaknesses of the study | General | Explanation: Items inquiring if the reviewer has identified and commented in the peer review report on the general strong and weak points of the study. Example: ‘How well it identified the study’s strengths and weaknesses?’ [35] |
Methods | Explanation: Items inquiring if the reviewer has identified and commented in the peer review report on the strong and weak points specifically related to study’s methods Example: ‘Did the reviewer clearly identify strengths and weaknesses in the study’s methods?’ [28] | ||
Statistical methods | Explanation: Items inquiring if the reviewer has identified and commented in the peer review report on the strong and weak points specifically related to study’s statistical methods Example: ‘Confidence intervals/p-values/overall fit’ [36] | ||
5 | Presentation and organization of the manuscript | Explanation: Items inquiring if the reviewer has made comments in the peer review report on the data presentation such as tables and figures and on the organization of the manuscript such as writing communication. Example: ‘Are there any constructive suggestions on improvement of a. writing; b. data presentation and c. interpretation’ [54] | |
6 | Structure of reviewer’s comments | Explanation: Items inquiring if the reviewer has made in the peer review report organized and structured comments. Example: ‘Concise well-organized comments to the editor’ [50] | |
7 | Characteristics of reviewer’s comments | Clarity | Explanation: Items inquiring if the reviewer has provided in the peer review report clear and easily to read comments. Example: ‘How clear was this review? The review was easily read and interpreted by the editor and authors.’ [38] |
Constructiveness | Explanation: Items inquiring if the reviewer has provided in the peer review report helpful, relevant and realistic comments. Example: ‘Were the reviewer’s comments constructive?’ [23, 27] | ||
Detail/Thoroughness | Explanation: Items inquiring if the reviewer has provided in the peer review report detailed and thorough comments supplying appropriate evidence. Example: ‘Detail of commentary’ [33] | ||
Fairness | Explanation: Items inquiring if the reviewer has provided in the peer review report balanced and objective comments. | ||
Knowledgeability | Explanation: Items inquiring if the reviewer has showed in the peer review report to know and understand correctly the content of the manuscript. Example: ‘Knowledge of the manuscript’s content area.’ [28] | ||
Tone | Explanation: Items inquiring if the reviewer has used a courteous tone in the peer review report. Example: ‘Overall tone of the reviewers was also assessed as harsh or courteous.’ [34] | ||
8 | Timeliness of the review report | Explanation: Items inquiring if the reviewer has completed the peer review report on time. Example: ‘Punctuality of the review’ [49] | |
9 | Usefulness of the review report | Decision making | Explanation: Items inquiring if the reviewer has provided a peer review report useful to make a decision about the acceptance, revision or rejection of a manuscript Example: ‘The reviewer provided the editor with the proper context and perspective to make a decision about acceptance or revision of the manuscript.’ [26] |
Manuscript improvement | Explanation: Items inquiring if the reviewer has provided useful suggestions in the peer review report to improve the manuscript. Example: ‘This aspect is solely interested in how well the review aids the authors for improving their work and/or writing. Whether the review makes a good judgment regarding acceptance of the submission plays no role here whatsoever.’ [53] |