Skip to main content

Table 2 Examples of definition of scoring system instructions

From: Tools used to assess the quality of peer review reports: a methodological systematic review

Scoring system instructions
Defined Partially defined Not defined
5 (Exceptional) = The rare outstanding critique that is comprehensive, objective, and insightful. Evaluates purpose of the study, study design, scientific validity, and conclusions by numbering questions and constructive suggestions to be addressed by the author. Includes comments to the editor about whether this is something new and important and useful to our readers.
4 (Very good) = Excellent review indicating that the paper was carefully evaluated. Helpful comments to the author and editor with well-documented reasons for decision.
3 (Good) = Useful type of very satisfactory review. Analysis not as well organized, documented, or as complete as above but is reasonable, with adequate comments for the authors.
2 (Below average) = Very brief, superficial evaluation. Reasons for the decision not explained and comments to authors not helpful.
1 (Unacceptable) = Such a poor review that consideration should be given to not sending further papers to this reviewer. Reasons could include evidence of bias, unfair, faulty reasoning, or evaluation (totally disagrees with the opinion of other reviewers and editor) and comments to author either absent, inappropriate, or inadequate to explain how the paper was rated.
(Landkroon 2006) [42]
1 (Poor) = Does not follow reviewer guideline structure or preferred formatting in providing comments; unfavourable timeliness.
2 (Acceptable) = Comments are somewhat helpful; review meets timeline.
3 (Reliable) = Thorough and helpful comments; timely submission.
4 (Excellent) = Very strong and detailed comments; review was submitted early or on time; comments enhance the manuscript’s merit and relevance in the field.
(Rajesh 2013) [32]
1 = poor;
2 = fair;
3 = good;
4 = excellent
(Friedam1995) [22]