Email updates

Keep up to date with the latest news and content from Genome Medicine and BioMed Central.

Journal App

google play app store
Highly Accessed Musings

Are randomized trials obsolete or more important than ever in the genomic era?

John PA Ioannidis12* and Muin J Khoury34

Author Affiliations

1 Stanford Prevention Research Center, Department of Medicine and Department of Health Research and Policy, Stanford University School of Medicine, 1265 Welch Road, MSOB X306, Stanford, CA 94305, USA

2 Department of Statistics, Stanford University School of Humanities and Sciences, Stanford, CA 94305, USA

3 Office of Public Health Genomics, Centers for Disease Control and Prevention, Atlanta, GA 30333, USA

4 Epidemiology and Genomics Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Rockville, MD 20892, USA

For all author emails, please log on.

Genome Medicine 2013, 5:32  doi:10.1186/gm436

The electronic version of this article is the complete one and can be found online at: http://genomemedicine.com/content/5/4/32


Published:18 April 2013

© 2013 BioMed Central Ltd

Musings

The genomic era has raised the possibility of major changes in the design, conduct, and even the existence of randomized trials as we know them [1-3]. Randomized trials are often seen as a slow, laborious, expensive, and difficult step in the translational process and are associated with a high attrition rate for drugs. Indeed, most tests that are in use for the screening, diagnosis, prognosis, monitoring or management of patients have never been scrutinized by a randomized trial. This has largely been due to a failure to realize that tests can do as much harm and as much good as drugs or devices; thus, a rigorous appraisal of their clinical utility, including both the possible benefits and the possible harms, is necessary. Moreover, numerous new omics-based tests are continually being proposed, especially in the context of targeted preventive or therapeutic interventions. Given rapid development of these new biomarkers, can we make randomized trials more adaptable to a changing landscape? Furthermore, do we still need randomized trials at all? Our answers to these two questions are: yes, to some extent; and yes, definitely. We will explain our reasoning in this article.

Revolutionizing randomized trials in the genomic era

Genomic applications in medicine are highly diverse. They range from diagnostic and carrier tests for many rare Mendelian conditions to complex biomarkers that inform preventive or therapeutic interventions. One relatively new area is the explicit linkage of a treatment and a companion diagnostic test (a molecular assay to indicate the likelihood of a patient responding to this specific treatment or reacting adversely). With this in mind, at least three types of potentially informative designs are available for the conduct of omics-informed clinical trials. The first is the use of enrichment strategies, in which participants are selected for enrollment on the basis of a validated predictive tool to ensure the selection of those who are likely to have the best response rates (this approach is used to test biologically targeted therapies) [1,2]. The second design makes use of surrogate markers (for example, a protein, metabolic, epigenetic, or other marker), especially validated markers, to conduct trials with a smaller sample size and a shorter follow-up period than traditional trials require. The third design incorporates a range of adaptive designs so that the trial can be optimized and modified as it progresses. Various aspects can be altered, such as the inclusion and exclusion criteria and sample size, the data collection processes, and even the definitions of the endpoints and types of analyses. This flexible approach would address the expectation that new information or discoveries will accumulate at such a rapid pace that external information could affect the conduct of trials midstream and could thus effectively guide their modification. Trials could then be completed faster and might yield larger treatment effects and more definitive answers.

Despite these expectations of how genomic discoveries will alter clinical trials, omics research to date has not had the major clinical potential that was heralded. For example, despite decades of pharmacogenetics research, only a limited number of genetic variants have been robustly documented (at a genome-wide significance level) as being associated with drug-related outcomes. Of the several thousand genetic associations that have been validated by genome-wide association studies (GWAS), few pertain to pharmacogenetics, and far fewer have the large effect sizes that would make them readily actionable. In the 50 most recent GWAS that were indexed in the National Human Genome Research Institute GWAS catalog as of 28 February 2013 [4], only 3 of the 290 well-validated variants (with P < × 10-8) pertain to pharmacogenetics, and all 3 have small effect sizes that are unlikely to be useful in clinical practice.

The US Food and Drug Administration records 119 pharmacogenetic associations that are listed on drug labels [5]. Only four, however, pertain to a clear requirement or a strong recommendation for genetic testing (for example, cetuximab (EGFR), trastuzumab (HER2), maraviroc (CCR5), and dasatinib (BCR-ABL)). For the other associations, few randomized trials have proved the clinical utility of routine genetic tests; an exception is HLA-B*5701 testing to assess the suitability of treatment with abacavir [6]. In several additional cases, there is some supporting evidence from randomized trials, but this is based on surrogate outcomes: for example, CYP2C19 testing for patients to be treated with clopidogrel, in which the main outcome assessed was platelet reactivity rather than major bleeding [7]. Most pharmacogenetic associations listed on drug labels have not had their utility tested in clinical trials.

Many other markers, in diverse omics fields, lack even proper validation at the association level. It makes little sense to pursue and incorporate non-validated markers or non-validated molecular signatures in clinical trials. Standards for reproducible research need to be strengthened [8]. As to adaptive confirmatory trials, they remain mostly theoretical constructs. We lack examples of newly arising genomic information leading to an adaptation of the design of already ongoing large trials with major impact on their eventual success. Unjustified adaptations can end up being indistinguishable from spurious manipulations of the data collection and analysis plans.

Getting rid of randomized trials in the genomic era?

A common complaint is that clinical trials are too large and expensive to be practical and, instead, one could wish that omics research would make these trials more efficient or even replace them. The reality is different. Even for blockbuster drugs, randomized trials are rarely large enough to demonstrate conclusively benefits for major clinical outcomes and death [9]. Usually, information is fragmented across many small or modest-sized trials, with selective publication and reporting. Rationalizing the agenda of traditional randomized trials mostly requires common sense, not fancy omics. For example, designing several large, pivotal trials with reliable measurements and clinically relevant outcomes may yield more reliable information and at a lower cost than traditional randomized trials or trials incorporating expensive omics tests. Conversely, the most expensive part of many grant applications that propose to run clinical trials is the exploratory analysis with the latest crop of non-validated biomarkers.

One may argue that perhaps randomized trials could be skipped, especially when it comes to appraising all of the new omics tools and technologies. There are so many new tests that it is expensive to assess them with randomized trials. Moreover, these tools are constantly evolving; thus, if one genomic risk score is tested now, this score will alter or evolve while the trial is being conducted. Alternatively, perhaps other types of designs, such as studies with just one participant (n-of-1 studies) [3] are more commensurate with personalized medicine. These arguments can only go so far. The prostate-specific antigen (PSA) test, for example, was widely overused, and it took decades of debate and millions of misdiagnosed and mistreated patients before extirpating its unwarranted routine use from clinical practice guidelines. Genomic tools can yield numerous biomarkers, all of which could fail in clinical use much like PSA. We agree that the targets for evaluation in randomized trials need to be chosen wisely and, indeed, some tests will evolve; therefore, the trials that are conducted will primarily be proof-of-concept experiments. N-of-1 designs have value [3], but it is difficult to envisage that they could substitute for traditional randomized trials to answer most questions.

Most of the emerging genomic information that is meandering its way toward health applications is still either non-validated noise [10] or true signals with validated small effects, which are not suitable for applying to clinical practice. For the relatively few discoveries that represent more than noise or mere curiosities, randomized trials are indispensable to find out what they can really achieve. Novel trial designs are worth exploring the use of some types of omics information, and modeling, observational, and other non-randomized comparative effectiveness research might be useful in some cases. Nonetheless, such approaches will not eliminate the need for solid evidence from traditional randomized trials of the type that have been known for over 60 years but have rarely been performed and reported properly.

Abbreviations

GWAS: genome-wide association studies; PSA: prostate-specific antigen.

Competing interests

The authors declare that they have no competing interests.

References

  1. Antman E, Weiss S, Loscalzo J: Systems pharmacology, pharmacogenetics, and clinical trial design in network medicine.

    Wiley Interdiscip Rev Syst Biol Med 2012, 4:367-383. PubMed Abstract | Publisher Full Text OpenURL

  2. Simon R: The use of genomics in clinical trial design.

    Clin Cancer Res 2008, 14:5984-5993. PubMed Abstract | Publisher Full Text OpenURL

  3. Lillie EO, Patay B, Diamant J, Issell B, Topol EJ, Schork NJ: The n-of-1 clinical trial: the ultimate strategy for individualizing medicine?.

    Per Med 2011, 8:161-173. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

  4. Catalog of Published Genome-Wide Association Studies. [http://www.genome.gov/gwastudies/] webcite

  5. Table of Pharmacogenomic Biomarkers in Drug Labels. [http:/ / www.fda.gov/ drugs/ scienceresearch/ researchareas/ pharmacogenetics/ ucm083378.htm] webcite

  6. Mallal S, Phillips E, Carosi G, Molina JM, Workman C, Tomazic J, Jägel-Guedes E, Rugina S, Kozyrev O, Cid JF, Hay P, Nolan D, Hughes S, Hughes A, Ryan S, Fitch N, Thorborn D, Benbow A for PREDICT-1 Study Team: HLA-B*5701 screening for hypersensitivity to abacavir.

    N Engl J Med 2008, 358:568-579. PubMed Abstract | Publisher Full Text OpenURL

  7. Roberts JD, Wells GA, Le May MR, Labinaz M, Glover C, Froeschl M, Dick A, Marquis JF, O'Brien E, Goncalves S, Druce I, Stewart A, Gollob MH, So DY: Point-of-care genetic testing for personalisation of antiplatelet treatment (RAPID GENE): a prospective, randomised, proof-of-concept trial.

    Lancet 2012, 379:1705-1711. PubMed Abstract | Publisher Full Text OpenURL

  8. Ioannidis JP, Khoury MJ: Improving validation practices in "omics" research.

    Science 2011, 334:1230-1232. PubMed Abstract | Publisher Full Text OpenURL

  9. Ioannidis JP: Mega-trials for blockbusters.

    JAMA 2013, 309:239-240. PubMed Abstract | Publisher Full Text OpenURL

  10. Janssens AC, Gwinn M, Bradley LA, Oostra BA, van Duijn CM, Khoury MJ: A critical appraisal of the scientific basis of commercial genomic profiles used to assess health risks and personalize health interventions.

    Am J Hum Genet 2008, 82:593-599. PubMed Abstract | Publisher Full Text | PubMed Central Full Text OpenURL

Have something to say? Post a comment on this article!