Individual differences |
Methods | Statistics | Clinical | Educational | Industrial | Professional items | World psychology |
Initial study syndrome is an idea presented here (but possibly addressed elsewhere) that refers to an observable pattern in clinical research findings where initial studies report significantly elevated effects (in the range of 60-85% cure rates) which are not subsequently confimed. Generally over time the effectiveness of the procedures settles around the 30% mark.
If this pattern is confirmed it suggests that one or more biases are operating, despite the objective methodology of the scientific enterprise.
For example Ioannidis (2005) reported that of 49 highly cited original clinical research studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had reported effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. So the substance of the conclusion in a third of these paper had to be withdrawn
Looking at the methodology : 5 of 6 highly-cited nonrandomized studies had been contradicted or had found stronger effects vs 9 of 39 randomized controlled trials (P = .008). Among randomized trials, studies with contradicted or stronger effects were smaller (P = .009) than replicated or unchallenged studies although there was no statistically significant difference in their early or overall citation impact. Matched control studies did not have a significantly different share of refuted results than highly cited studies, but they included more studies with "negative" results.
Ioannidis,J P. A. (2005)Contradicted and Initially Stronger Effects in Highly Cited Clinical Research. JAMA. ;294:218-228.