Skip to main content

Table 2 Some ideas for the improvement of benchmarking practice

From: Towards evidence-based computational statistics: lessons from clinical research on the role and design of real-data benchmark studies

Clinical research

Treated in

Transfer into computational

Example(s)

  

statistical research?

 

Sample size calculation

[9]

Possible and desirable [9]

[35]

Strict inclusion criteria

Sec. 3

Possible and desirable

[20, 21, 35]

Trial protocol

Sec. 4.1

Principle might be helpful in adapted form

 

Quality control

Sec. 4.2

Principle might be helpful in adapted form

 
  

e.g. via platforms like OpenML [4]

 

Placebo/reference

Sec. 4.3

Principle might be helpful in adapted form

 

Blinding

Sec. 4.4.1

Principle might be helpful in adapted form

 

Intention-to-treat

Sec. 4.4.2

Adequate treatment and reporting of

[29]

  

missing values: possible and desirable

 

Levels of evidence

Sec. 4.5

Principle might be helpful in adapted form