Attack On Traditional Ways Of Assessing The Evidence Of Therapeutic Interventions

Main Category: Public Health
Article Date: 20 Oct 2008 - 6:00 PDT

email to a friend   printer friendly   opinions  

In last week's Harveian Oration* Professor Sir Michael Rawlins, Chair of the National Institute for Health and Clinical Excellence, argues that we need a new approach to analysing clinical evidence.

The evidence discussed in Sir Michael's Oration has only one purpose. It is the basis for informing decisions about the appropriate use of therapeutic interventions (including pharmaceuticals, devices and surgical procedures) in routine medical practice.

Such decisions have to be made at various levels but with critical consequences for patients, their families and society. They include the decisions that physicians make for individual patients as well as well as assessing whether interventions are safe, effective and cost effective for healthcare systems as a whole. Mistakes can have repercussions at all levels.

Randomised controlled trials (RCTs), long regarded at the 'gold standard' of evidence, have been put on an undeserved pedestal. Their appearance at the top of "hierarchies" of evidence is inappropriate; and hierarchies, themselves, are illusory tools for assessing evidence. They should be replaced by a diversity of approaches that involve analysing the totality of the evidence-base.

Sir Michael outlines the limitations of RCTs in several key areas:

-- Impossible - with treatments for very rare diseases where the number of patients is too limited

-- Unnecessary - when a treatment produces a "dramatic" benefit - imatinib (Glivec) for chronic myeloid leukaemia

-- Stopping trials early - interim analyses of trials are now commonly undertaken to assess whether the treatment is showing benefit and if the trial can be stopped early. Around 30% of recent trials in oncology have been stopped early for apparent benefit. Although the desire to stop trials early is understandable, the possibility that an interim analysis is a "random high" may be difficult to avoid - especially as there is no consensus among statisticians as to how best to handle this problem

-- Resources - the costs of RCTs are substantial in money, time and energy - a recent study of 153 trials completed in 2005 and 2006 showed a median cost of over £3 million and with one trial costing £95 million. One manufacturer has estimated that the average cost per patient increased from £6,300 in 2005 to £9,900 in 2007

-- Generalisability - RCTs are often carried out on specific types of patients for a relatively short period of time, whereas in clinical practice the treatment will be used on a much greater variety of patients - often suffering from other medical conditions - and for much longer. There is a presumption that, in general, the benefits shown in an RCT can be extrapolated to a wide population; but there is abundant evidence to show that the harmfulness of an intervention is often missed in RCTs.

Sir Michael argues that observational studies are also useful and, with care in the interpretation of the results, can provide an important source of evidence about both the benefits and harms of therapeutic interventions. These particularly include historical controlled trials and case-control studies but other forms of observational data can also reveal important issues.

Sir Michael rejects the trend to grade various kinds of clinical trials and studies on scales of merit which he says has come to dominate the development of some aspects of clinical decision making.

"Hierarchies attempt to replace judgement with an oversimplistic, pseudo-quantative, assessment of the quality of the available evidence."

Sir Michael believes that arguments about the relative importance of different kinds of evidence are an unnecessary distraction. What is needed instead is for "investigators to continue to develop and improve their methodologies; for decision makers to avoid adopting entrenched positions about the nature of evidence; and for both to accept that the interpretation of evidence requires judgement."

Professor Sir Michael Rawlins has been chairman of the National Institute for Health and Clinical Excellence (NICE) since its inception in 1999. NICE is responsible for assisting health professionals, in the NHS, to provide patients with the highest attainable standard of care; as well as advising the public health community on measures that are effective and cost effective in the prevention of ill health. He has seen the Institute grow from an organisation with no staff, no premises, no bank account and a nominal budget of £8.5 million a year to a body now employing over 270 people, with offices in London and Manchester, and an annual budget of £35 million, which is set to more than double over the next few years. From 1973 to 2006, he was the Ruth and Lionel Jacobson Professor of Clinical Pharmacology at the University of Newcastle where he undertook research into the safety and efficacy of new and established pharmacological treatments. At the same time he was consultant physician to the Newcastle University Hospitals where he practised clinical pharmacology and general internal medicine. He is now Emeritus Professor at the University of Newcastle and Honorary Professor at the London School of Hygiene and Tropical Medicine, University of London. He served as chairman of the Committee on Safety of Medicines (1993-8) and of the Advisory Council on the Misuse of Drugs (1998-2008) and he was appointed Knight Bachelor in 1999. He has previously delivered the Bradshaw (1986), William Withering (1994) and Samuel Gee (2006) lectures at the Royal College of Physicians.

* William Harvey, the eminent 17th Century physician who discovered the circulation of the blood, gave an indenture to the RCP in 1656 for an annual dinner to encourage friendship between Fellows and Members of the College at which there should be an oration "with an exhortation to the Fellows and Members of the said College to search and study out the secret of Nature by way of experiment". The College continues the tradition by inviting a leading doctor or scientist to give the Oration on issues relating to his or her own field of work.

The Royal College of Physicians
http://www.rcplondon.ac.uk

• Additional
• References
• Citations