Table 2 Quality assessment tools – comparison of key quality constructs

Max.= 2 points out of total score of 5

1. Was the study described as randomized? (1 point for yes) Give an additional point if the method to generate the sequence of randomization was described and it was appropriate. Deduct a point if it was inappropriate.

Max. = 10 points out of total score of 100

2. Have the patients been randomly allocated to treatment groups? (1 point for yes)

If yes: i) Is the method of randomization explicitly detailed? (1.5 points)

ii) Is it valid? i.e. Are there any threats to internal validity re: designation of subjects to groups? (2.5 points)

iii) Is patient consent sought prior to randomization? (2.5 points)

iv) Is it secure and 'blind' to the assessors? (2.5 points)

Max. = 2 points out of total score of 13

23. Were the study subjects randomised to intervention groups? Studies which state that subjects were randomised should be answered yes except where method of randomisation would not ensure random allocation. For example alternate allocation would score no... (1 point for yes)

24. Was the randomised intervention assignment concealed from both patients and health care staff until recruitment was complete and irrevocable? (1 point)

Max. = 2 points out of total score of 5

2. Was the study described as double blind? (1 point)

Give an additional point if the method of double blinding was described and it was appropriate. Deduct a point if it was inappropriate.

Max. = 5 points out of total score of 100

9. Is the assessment blind?

a) If yes, who is blinded:

i) patients? (2 points)

ii) therapist/carer? (2 points)

iii)assessor/data collector? (1 point) b) If no, i) are reasons given as to why assessment is not blind? (2 points)

ii) Is there discussion of bias resulting from non-blind assessment? (3 points)

Max. = 2 points out of total score of 13

14. Was an attempt made to blind study subjects to the intervention they have received? (1 point)

15. Was an attempt made to blind those measuring the main outcomes of the intervention? (1 point)

Max.= 1 point out of total score of 5

3. Was there a description of withdrawals and dropouts? 1 point)

Max. = 12 points out of total score of 100

6. Has an 'intention-to-treat analysis been performed? i.e. everyone randomized is retained in the study; everyone randomized is included in the final analysis; and no selective dropouts. (8 points)

b) if not, is it clear what was done, its justification and impact on bias? (8 points)

11. Loss to follow-up

a) (<) 20% loss to follow up (2 points)

b) <10% loss to follow-up (2 points)

Max. = 2 points out of total score of 13

25. Was there adequate adjustment for confounding in the analyses from which the main findings were drawn? The questions should be answered no for trials if: the main conclusions of the study were based on analyses of treatment rather than intention to treat;.... (1 point)

26. Were losses of patients to follow-up taken into account? If the number of patients lost to follow-up are not reported, the question should be answered unable to determine. If the proportion lost to follow-up was too small to affect the main findings, the question should be answered yes? (1 point)

no items

Max. = 6 points out of total score of 100

7. Statistical analysis

a) Is the analysis appropriate/specific to the hypothesis and to the data? (1 point)

b) Is the analysis adequately described? (1 point)

c) Do the statistical assumptions hold? (1 point)

d) Are adequate summary statistics provided at:

i) baseline? (0.5 point)

ii) outcome? (05. point)

e) Is the overall significance level reported protected against inflation due to multiple testing? (1 point)

f) If confounders exist, are they adjusted for via multivariate techniques even if differences between groups are not significant? (1 point)

Max. = 4 points out of total score of 13

16. If an of the results of the study were based in 'data dredging', was this made clear? (1 point)

17. Do the analyses adjust for different lengths of follow-up of patients? (1 point)

18. Were the statistical tests used to assess the main outcomes appropriate? (1 point)

25. Was there adequate adjustment for confounding in the analyses from which the main findings were drawn? (1 point)

no items

Max. = 4 points out of total score of 100

14. Has patient compliance been assessed? (4 points)

Max. = 1 point out of total score of 13

19. Was compliance with the interventions reliable? (1 point)

no items

Max. = 14 points out of total score of 100

3. Measurement of outcomes

a) Is the form of measurement stated? (3 points)

b) Has an attempt been made to validate the measures? (3 points)

c) Has an attempt been made to test the reliability of the measures? (2 points)

d) Is the outcome objective as compared to subjective? (2 points)

12. Outcomes

a) How many outcomes are used (1/2 point for each, to a max. of 2)

b) Are they relevant? (1 point)

c) Are they independent? (1 point)

Max. = 1 point out of total score of 13

20. Were the main outcome measures used accurate (valid and reliable)? (1 point)