There Is Enormous Waste In The Production And Reporting Of Research Evidence

Editor's Choice
Main Category: Clinical Trials / Drug Trials
Article Date: 16 Jun 2009 - 6:00 PDT

email to a friend   printer friendly   view / write opinions   rate article

In a viewpoint published Online First and in a future edition of The Lancet, Sir Iain Chalmers, of the James Lind Library, Oxford, UK, and Professor Paul Glasziou, of the University of Oxford, UK, debate on the unnecessary waste that exists in medical research. This misuse leads many reports to be of no purpose. If there is a lack of functional reports and if they are not easily accessible, research is of no help to patients or their doctors.

The authors examine the causes and levels of waste in the four following stages:

• The choice of research questions
• The quality of research design and methods
• The adequacy of publication practices
• The quality of research reports

They debate on how public funding of research is moderately linked with disease burden, for the choice of research questions. When analyzing 344 studies, it appears that only nine compared researchers' priorities with those of patients or doctors. Also, these nine studies showed striking mismatches. For example: Patients with osteoarthritis of the knee and the clinicians looking after them considered as a priority additional evaluation of physiotherapy and surgery, and assessment of educational and coping strategies. Only 9 percent of patients wanted more research on drugs, yet over 80 percent of randomised controlled trials in patients with osteoarthritis of the knee were drug evaluations. This example is not limited to commercial research. Controlled trials funded by the UK Medical Research Council and British medical research charities were more probably drug trials than were trials commissioned by what is now the National Institute for Health Research (NIHR).

For quality of research design, the authors explain: "New research should not be done unless, at the time it is initiated, the questions it proposes to address cannot be answered satisfactorily with existing evidence." New research is frequently inadequate as it is unable to deal with other fundamental issues. For example, in a sample of 234 clinical trials in the major medical journals, concealment of treatment allocation sequence was either inadequate (18 percent) or unclear (26 percent).

There has been a great deal of argument lately on the inadequacy of publishing practices, specifically biased under-publication and over-publication of research. The authors mention that the public has become conscious of this matter thanks to exposure of suppressed evidence about serious adverse effects of treatments. In addition, studies showing disappointing results are less likely to be reported promptly, and are less likely to progress from abstracts to full reports. The authors write: "The problem of biased under-reporting of research results mainly from decisions taken by research sponsors and researchers, not from journal editors rejecting submitted reports." They continue by saying that public access to full results of all research is still only a hope. It remains that way because of the resistance of some research sponsors and researchers.

At last, biased or unusable reports of research can be frequent. Some reports do not take into account adverse events. Sometimes there is a change in their primary outcomes between the protocol for the research and the published report. But even with impartiality, a report must still be appropriate for use by the end-user. There are two possible ways to improve studies according to the authors:

1- If clinicians are going to use the evidence showing beneficial effect of an intervention, the intervention needs to be adequately described (particularly when it is not a drug, like for example setting up a stroke unit or details of a low fat diet).

2- Evidence must be set in the context of updated systematic reviews. However, among the world's major medical journals, The Lancet is the only one in requiring reports of new research to be preceded by and to conclude with reference to systematic reviews of other relevant evidence.

The authors comment: "Since research must pass through all four stages...the waste is cumulative. If the losses estimated in the figure apply more generally, then the roughly 50% loss at stages 2, 3, and 4 would lead to a greater than 85% loss, which implies that the dividends from tens of billions of dollars of investment in research are lost every year because of correctable problems."

In conclusion, the authors give several recommendations for each of the four areas, including:

• Growing involvement of clinicians and patients in determining the research.
• All journals should require new studies to be situated in the context of existing published research.
• Once research is completed, there should be support of timely open access to the complete results.
• There should be a supplement of peer review of studies with review by methodologists and end users.
• There should be support for free access repositories (separate from any publications, in order for clinicians and researchers to have the details of the treatments, test or instruments studied).

"Avoidable waste in the production and reporting of research evidence"
Iain Chalmers, Paul Glasziou
DOI: 10.1016/S0140-6736(09)60329-9
thelancet

Written by Stephanie Brunner (B.A.)
Copyright: Medical News Today
Not to be reproduced without permission of Medical News Today