Is another trial necessary or even ethical?

You don’t have to look too far to see the benefit of systematic reviews and their summary results. The well-known Cochrane logo (pictured) depicts a real example, highlighting the value of systematically pooling data for meta-analysis and in this case demonstrating the clear benefit of corticosteroids in accelerating lung maturation in preterm babies.


Systematic reviews can also protect patients from harm. A systematic review of rosiglitazone, which was developed to treat type 2 diabetes, showed an increased risk of myocardial infarction. The review formed part of the evidence that ultimately led to the suspension of the marketing authorization of rosiglitazone by the European Medicines Agency, despite having been available for over ten years.

So the production and use of systematic reviews to inform clinical decision makers is both appropriate and well supported.

Extending the use of systematic reviews

With clinical needs and finite budgets dictating the priorities in clinical research, systematic reviews can also reduce research waste. As pointed out by Chalmers and Glasziou “New research should not be done unless, at the time it is initiated, the questions it proposes to address cannot be answered satisfactorily with existing evidence.”  There are reasons for this, most importantly that unnecessary clinical trials can harm patients and waste resources.

Take for example the drug rofecoxib (Vioxx). Originally marketed as a safer alternative to existing non-steroidal anti-inflammatory drugs, it was withdrawn in 2004 after concerns emerged of a higher risk of cardiovascular events, notably myocardial infarction. A systematic review of published clinical studies of rofecoxib, conducted before the September 2004 withdrawal, identified 18 randomised controlled trials, all sponsored by the manufacturer. Cumulative meta-analysis of these trials showed that had a systematic review and meta-analysis of accumulating evidence been conducted by the end of 2000, it would have been clear that rofecoxib was associated with a higher incidence of myocardial infarction. So several thousands of participants, in the studies conducted after 2000, were randomised into trials when a clear harm could (and should) already have been detected. Not to mention the wasted costs of running those additional trials.

Research that adds value

Identifying or carrying out a systematic review, before embarking on any new primary research, is increasingly seen by many research funders as an essential early step. One of the largest funders of research in the UK, the National Institute for Health Research (NIHR), makes the production and promotion of systematic reviews a key investment in its infrastructure.

By removing uncertainties in science and research, systematic reviews ensure that only the most effective and best-value interventions are adopted by the NHS and social care providers.”

Professor Dame Sally C. Davies, Director General of Research and Development, Department of Health

Prospective applicants to NIHR funding are offered guidance notes to ensure that all primary research is informed by a review of the existing literature. This may include identifying relevant existing systematic reviews or carrying out an appropriate review and summarising the findings for the application. Researchers who identify a clear need for new studies should use information gained from their systematic review to inform the design, analysis and conduct of their study, as part of the “Adding Value in Research Framework”.

Utilising systematic reviews to inform new research is good practice

An earlier survey on the utilisation of Cochrane reviews in designing new studies showed that the proportion of study investigators using them was very limited. Only 11 of the 24 authors who responded to their survey were aware of the relevant Cochrane review at the time they designed their study.

However, this has improved as a greater understanding of the need to begin (and end) new research with a systematic review has become more apparent.

A recent review of two sets of NIHR HTA funded trials (those funded between 2006 and 2008 and those funded in 2013) sought to identify whether trial planning and design were informed by systematic reviews. The authors extended the definition of a systematic review to include the following:

  • Cochrane systematic reviews;
  • other reviews if “systematic review” was mentioned in the title and the methods specified that a systematic search was conducted;
  • National Institute for Health and Care Excellence (NICE) Technology Appraisal Guidance documents (TA), which include Technology Assessment Reports (TAR) based on reviews of the clinical and economic evidence (i.e. cost-effectiveness assessments).

Only 5 of the 47 trials that were funded in 2006-8 (cohort 1) did not refer to a systematic review, for which the authors found plausible reasons. All the trials that were funded in 2013 (cohort 2) were informed by systematic reviews.

A large change from before, and perhaps not surprising, given the requirements of the funders.

However, few studies have explored how researchers use systematic reviews when planning new trials. The reasons the analysis found were varied, but by far the most common reason was to justify treatment comparisons. Other reasons included obtaining information about adverse events, defining outcomes, and aspects of study design such as recruitment and consent.


The use of systematic reviews to inform new research is not without limitations. For example, we cannot guarantee that using a systematic review to inform new research automatically generates higher quality trials and more reliable outcomes. It has also been argued that reviews of small, poorly conducted, single-center trials exaggerate treatment effects, not seen in subsequent larger well conducted trials. However, as has been pointed out already, funders need assurance, even from reviews of smaller trials, that there is a need to support further research. And a systematic review can provide this, as well as information to inform the design of the new research.

Scientific history already contains examples where a failure to consider, conduct, and use systematic reviews has led to patients being exposed to potential harm as well as the waste of resources in carrying out unnecessary clinical trials. Researchers applying for funding of any new primary study should therefore ensure they are well aware of existing evidence and the implications to their proposed work. Indeed, it would be “ethically, scientifically, and economically indefensible” not to.


Kamal R. Mahtani is a GP, NIHR Clinical Lecturer and Deputy Chair of the NIHR HTA Primary Care, Community and Preventive Interventions Panel (PCCPI).

Disclaimer: The views expressed are those of the author and not necessarily of the any of the institutions or organisations mentioned in the article.



4 thoughts on “Systematic reviews: adding value and reducing uncertainty

  1. On this topic, Sir Iain Chalmers is visiting Cambridge on 27th January – please join us for his 1pm talk at Cambridge Institute of Public Health if you can

    Best wishes


    Lucy Lloyd
    Communications Manager
    Primary Care Unit

    Lucy Lloyd
    01223 330304

  2. Pingback: Utilising systematic reviews: is another trial necessary or ethical? - CEBM

  3. Pingback: Can randomised controlled trials be more efficient? | School for Primary Care Research blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s