Results from early clinical trials investigating new treatments for chronic diseases may be drastically exaggerated, according to an analysis published this week.
As the average human lifespan slowly expands, the number of people with chronic conditions is steadily rising. In fact,
All these conditions — which include heart and kidney disease, stroke, cancer, and diabetes — can be treated, but many available treatments have unpleasant side effects. None can yet be cured.
Doctors and patients alike eagerly await news of innovative new ways to treat the diseases. At the same time, medical research is at an all-time high. Globally, the number of registered clinical trials increased
Such a boost in research can only be a good thing for people hoping for new treatments. And here at Medical News Today, we cover as many relevant new discoveries as possible.
Poking our fingers into new findings from top journals is what keeps our readers coming back. The importance and excitement of groundbreaking science keeps me in a job.
So, when I read through the report that we are discussing today, I must admit that my heart sank a little. In brief, the researchers conclude that results from early clinical trials should be approached with caution.
Carried out at the Mayo Clinic’s Evidence-Based Practice Center, the analysis investigates the ominously named Proteus effect.
When a new treatment is first trialed, the early results are often much more pronounced than those found in later trials. In other words, the drug or procedure being tested appears to work better at first, and then, when it is re-examined at a later date, the effect size diminishes. This is called the Proteus effect.
Although this effect has been measured before in other fields, lead study author Dr. Fares Alahdab wanted to investigate the phenomenon in regard to clinical trials for chronic conditions.
He wanted to see exactly how many studies were impacted and by how much. The other important question the team asked was, “Why does this happen?”
For their investigation, they reviewed hundreds of articles. These were sourced from the top 10 medical journals, as rated by their impact factor — a universal ranking system for journals. Specifically, they focused on 70 meta-analyses published in 2007–2015.
The findings are published in the journal Mayo Clinic Proceedings. The researchers revealed that effect of the first or second study looking at a device or treatment was 2.67 times larger than the effect that was seen in subsequent trials.
“This phenomenon of exaggerated early results was present in a whopping 37 percent of the studies we reviewed,” reveals Dr. Alahdab.
The effect is interesting and perhaps unexpected. However, it also has serious ramifications for individuals with chronic diseases and those who are treating them.
“Often, patients are living with more than one chronic condition, and they and their doctors watch for research about new treatments. They need to be aware that the effect seen in earlier trials may not bear out over time and may be much more modest.”
Lead researcher Dr. M. Hassan Murad
In short, making a decision about care as a response to early findings might be premature.
The researchers started their study with a number of theories as to why the Proteus effect exists. Some variables that might play a part include the size of study (perhaps later trials involved more subjects), study length (maybe earlier trials were shorter in duration), and the study population (a difference in in-patient versus out-patient, for instance).
Another factor that might play a role is funding. If the researchers are working for a company that manufactures the drug that they are trialing, there could be an incentive to gather positive results. Similarly, a study might be stopped early to produce more favorable results.
When the analysis looked at each of the variables above (and many more), it found no statistically significant effects across all trials. But in each individual case, one or many of these factors could be responsible for the effect.
It seems that there is no single answer. The authors write, “[A]t least for now, the Proteus effect is unpredictable.”
This does not make early clinical trial results irrelevant or pointless. As far as MNT are concerned, they are still well worth reporting. Dr. Murad does not want his findings to be seen as negative.
In fact, he explains, “Some people may think this is an anti-innovation message. To the contrary, we welcome new treatments. We just want people to know that the benefit seen in real practice, when treatments are given to people with various comorbidities and in different settings, may be smaller than what was seen in the earliest clinical trials.”
The take-home message is simple: take early results with a pinch of salt. It is not that early trials are irrelevant — far from it. They are a necessary part of the journey from theory to practice. It is the weight that we put on the findings that might need to be tweaked.
Here at MNT, we’re not quite ready to hang up our keyboards just yet.