TY - JOUR AU - Solomon, David J. PY - 2004 DA - 2004/04/29 TI - The rating reliability calculator JO - BMC Medical Research Methodology SP - 11 VL - 4 IS - 1 AB - Rating scales form an important means of gathering evaluation data. Since important decisions are often based on these evaluations, determining the reliability of rating data can be critical. Most commonly used methods of estimating reliability require a complete set of ratings i.e. every subject being rated must be rated by each judge. Over fifty years ago Ebel described an algorithm for estimating the reliability of ratings based on incomplete data. While his article has been widely cited over the years, software based on the algorithm is not readily available. This paper describes an easy-to-use Web-based utility for estimating the reliability of ratings based on incomplete data using Ebel's algorithm. SN - 1471-2288 UR - https://doi.org/10.1186/1471-2288-4-11 DO - 10.1186/1471-2288-4-11 ID - Solomon2004 ER -