Halpern, Karlawish, & Berlin (2002). The continuing unethical conduct of underpowered clinical trials.” - Article summary

Randomized controlled trials without sufficient statistical power may not adequately test the underlying hypotheses. This makes it useless and, thus, unethical. However, underpowered research continues to be conducted.

Arguments in favour of using underpowered research are that meta-analyses may save the small studies by providing a means of combining the results with those of similar studies (1) and that confidence intervals could be used to estimate treatment effects (2).

However, underpowered trials can only be properly used in interventions of rare diseases (1) and in the early-phase of the development of drugs and devices (2). The interventions for rare diseases need to specify that they are planning to use their results in a meta-analysis of similar interventions. However, in both cases, the participants must be informed that their participation may only indirectly contribute to future health care benefits.

The number of participants (1), the expected variability (2) and the chosen probability of a type-I error (3) are used to calculate statistical power. There is no consensus regarding of how small an effect size must be to be clinically significant and thus must be recognized and in cases like this, the expected effect size can be used to calculate the number of participants for any given study. Empirical definitions of clinically meaningful effects (1) and data from earlier trials (2) must be used if it exists or if it doesn’t, the moderate effect sizes described by Cohen (3) should be used in the sample size calculation.

Studies that contain too few participants to detect a positive effect via hypothesis testing will also include unacceptably wide confidence intervals, which cannot properly be used to estimate the effect size of the treatment. Furthermore, problems with synthesizing the results may prevent the calculation of valid treatment effects in a meta-analysis. The ideal conditions for meta-analysis (e.g. comparable research methods among the primary trials) are not often met, which leads different meta-analyses to have different results. Lastly, underpowered trials are more likely to produce negative results and thus less likely to be published, this reduces the probability of it being used in a meta-analysis.

It is necessary to inform the participants of an underpowered study of their limited value to the participants because of ethical concerns. However, this information is often not given to participants because investigators do not do an a priori power analysis (1), investigators do not enrol enough participants in a timely fashion (2) or investigators fear it will reduce enrolment (3).

Only prospectively designed meta-analyses can justify the risks to participants in individually underpowered trials because they provide sufficient assurance that a study’s results will eventually contribute to valuable or important knowledge. Plans for large, comparative trials of experimental intervention can justify the conduct of small studies in earlier phases of drug or device development.