Paper | General information | Assessment methods | Drug format (Number trials), sensory quality | Analysis | Basis for deciding inadequate matching |
---|---|---|---|---|---|
Hill and colleagues, 1976 | 22 trials (11 parallel group trial and 10 cross-over trial), described as “double-blind”, conducted in the UK over 12 months in 1973–74 and published in four journals (e.g. BMJ and Lancet). Out of 32 eligible trials 13 did not participate. Of the 19 remaining trials 3 were multi-armed, providing a total of 22 trial comparisons. In 19 trials the control was placebo. Drug companies produced the trial interventions | Four assessors (medical and administrative staff) were twice provided with 4 interventions from each trial. The assessors were “without knowledge of the experimental design” The 4 interventions consisted of 2 experimental and 2 controls), and assessors conducted 6 comparisons independently: (2 × the same interventions [e.g. control vs. control] and 4 × matched interventions [intervention vs. control]). In total, each assessor did 4 comparisons per trial of identical interventions and 8 comparisons of matched interventions. The number of sensory qualities involved differed depending on drug format, providing a potential number of assessments per trial with tablets or capsules of 64 (identical interventions) and 128 (matched interventions) | Tablet (11), colour, shape and appearance, smell, taste | For each trial, the proportion of assessments reported as different for identical interventions (control vs. control) (false positive rate) was subtracted from the proportion of assessments reported as different for the matched interventions: 35 %, 16 %, 41 %, 34 %, 6 %, 42 %, 43 %, 49 %, 15 %, 43 %, 60 %, 60 %, 42 %, 69 %, 27 %, 0 %, 0 %, 0 %, −2 %, 1 %, 20 %, 59 % | Based on a qualitative assessment the investigators decided which trials were inadequately matched (“obvious differences” were detected by all 4 assessors) |
Capsule (5), colour, shape and appearance, smell, taste | |||||
Aerosol and nasal spray (3), the container, smell and taste of aerosol | |||||
Ointment (2), colour, appearance and consistency, smell | |||||
Liquid (1), the ampoule, colour and liquid consistency | |||||
Blumen-thal and colleagues, 1974 | 6 trials (type not reported), described as “double blind”. Trials were selected from those the investigators had been involved in because research assistants had noted differences between experimental and controls In all six trials the control was placebo. It was not reported who produced the trial interventions | Fifty-two assessors (Students, spouses, lab scientists, secretaries) served as assessors. In one part of the study (“patient-simulated”) 32 assessors examined a container with 6 trial interventions from each trial independently (from 5 trials: 3 experimental and 3 controls, and in one trial: 2 experimental of one type, 2 experimental of another type and 2 controls). Assessors were instructed to decide whether interventions were all one type or not, and to categorize them accordingly. The second part of the study (“experimenter-simulated”) involving 20 assessors was identical to the first part, but they were told specifically that the 6 interventions definitely consisted of both experimental and controls. | Tablet (4), Sensory qualities not specified. | For each trial, the proportion of assessments for which assessors correctly separated all experimental intervention from placebo intervention was calculated. For the “patient-simulated study” the proportions were: 19 %, 36 %, 82 %, 19 %, 0 %, 100 % For the “experimenter-stimulated study” the proportions were: 90 %, 85 %, 85 %, 90 %, 0 %, 100 % | Based on a Chi2 test the authors decided for each trial whether assessors were able to differentiate better than chance between experimental and control interventions. |
Capsule (1), Sensory qualities not specified. | |||||
Liquid (1), Sensory qualities not specified. | |||||
Walter and colleagues, 2005 | 1 trial (parallel group), described as “double-blind”. The study was a pre-trial assessment of blinding integrity. The control was placebo. Drug company produced trial interventions | Four assessors (“study investigators”) were asked to examine independently a package of 10 tablets. The assessors were informed that it contained an equal number of experimental and control interventions. | Tablet (4), Taste and appearance | The number of correctly assigned tablets were recorded for each assessor | Based on a Chi2 test the authors decided whether assessors were able to differentiate better than chance between experimental and control interventions. |
Fai and colleagues, 2011 | 1 trial (parallel group), described as “double-blind”. The study was a pre-trial assessment of blinding integrity. The control was placebo. The trial investigators produced the trial interventions. | 49 “adults” served as assessors. It was not reported how many interventions each assessors examined or what information was provided to assessors. In one part of the study 15 assessors examined the packaging (box and bottle). In a second part of the study 11 assessors examined the capsules. In a third part of the study 23 assessors tested “the overall result” by guessing whether an intervention was control or experimental. | Capsule (1), The appearance and texture of package and capsule, and smell of capsule content. Taste was excluded from the assessment because “the taste was not adjusted with bitter agent” | Each assessor of packaging and capsules scored each property (exactly identical = 3 very close to unanimous = 2; significant difference = 1; inconsistent 0). The “overall result” was based on a comparison between actual intervention status and guessed status. | Based on Fischer’s exact test the authors decided whether assessors were able to differentiate better than chance between experimental and control interventions. |
Dupin-Spriet and colleagues, 1993 | 3 trials (type not reported) The control was placebo Not reported who provided the trial interventions | Four assessors (type not reported) were provided with 8 pairs of tablets from each trial. The information, if any, provided to the assessors on the distribution of experimental and control interventions was not reported. The 8 intervention pairs consisted of 4 matching pairs (experimental vs. controls), and 4 identical pairs (e.g. control vs. control) | Tablet (3), appearance (shape, colour, surface) | For each trial, the proportion of assessments reported as different for the identical interventions (false positive rate) was subtracted from the proportion of assessments reported as different for the matched interventions: | Based on Fischer’s exact test the authors decided for each trial whether assessors were able to differentiate better than chance between experimental and control interventions. |
Wen and colleagues, 2004 | 1 trial (parallel group) Pharmacy provided trial intervention | 32 assessors (e.g. doctors, nurses, graduate students) were provided with two bottles of experimental and control interventions that they examined independently. One group of 10 assessors performed “placebo test”. One group of 22 assessors performed a “simulative test”. It was not reported whether the experimental and control interventions were paired during the assessments | Capsule (1), bottle and capsule appearance, bottle label and notes, size, shape, quality, colour, smell, taste, and capsule content | Proportion of assessments found “not uniform” on specific qualities and overall was calculated. | Based on Fisher’s exact test the authors decided whether assessors were able to differentiate better than chance between experimental and control interventions. |
Farr and colleagues, 1987 | 2 pilot-trials (parallel group) Drug company provided intervention The control was placebo | Two pilot-trials (low and high dose zinc) randomizing assessors (students and university employees) to experimental or control. Assessors were informed that one of the compared substances had shown possible effect, and that they would be asked to decide whether they were receiving an “active” or an “inactive” compound. In the first pilot-trial 224 assessors were allocated to 30 mg zinc or placebo (0.0004 mg denatonium benzoate) In the second pilot-trial, 300 assessors were allocated to 23 mg zinc or placebo A (denatonium benzoate 0.00125 mg) or placebo B (denatonium benzoate 0.0025 mg) Assessors ingested tablets 8 times over one day (14 h). After the fifth tablet assessors filled out a structured questionnaire. | Tablet (2), bitter taste, aftertaste, palatability and guesses as to whether they believed they were receiving active or placebo. | Proportions of each assessment (e.g. bitter taste) was compared between trial groups (i.e. experimental and control groups) | Based on a Chi2 test the authors decided whether proportions of each assessments (experimental and control) differed more between trial groups than expected by than chance. |