Table 3 Interpretation of reliability measure scores
Measurement score | Interpretation [33] | Action recommended |
|---|---|---|
< .4 | Insufficient | Examine differences between coders and refine boundaries of inclusion criteria and categories. Perform another round of double-coding on a new data subset. |
.4 - .6 | Moderate | Explore potential systematic differences between coders to further improve the codebook. Perform another round of double-coding on a new data subset. If the score remains > .4 and < .6, continue to coding. Present results with caution. |
.6 - .8 | Substantial | If desired, systematic differences can be explored. |
> .8 | Almost perfect | No |