Links Between Biomarkers Or Genes To Diseases Exaggerated Too Often, Experts Say

Editor's Choice
Academic Journal
Main Category: Biology / Biochemistry
Also Included In: Genetics;  Public Health
Article Date: 01 Jun 2011 - 5:00 PDT


Research studies that demonstrate an association between specific diseases and biological markers or genes are habitually exaggerated - the links are unrealistically overstated, and generally do not hold up in larger studies, scientists from Stanford University School of Medicine, USA and the University of Ioannina School of Medicine, Greece explained in JAMA (Journal of the American Medical Association). Examples include the overstated link between the BRCA1 breast tumor gene and colon cancer.

The researchers explained:

"Many new biomarkers are continuously proposed as potential determinants of disease risk, prognosis, or response to treatment. The plethora of statistically significant associations increases expectations for improvements in risk appraisal. However, many markers get evaluated only in 1 or a few studies. Among those evaluated more extensively, few reach clinical practice. This translational attrition requires better study. Are the effect sizes proposed in the literature accurate or overestimated?"

John P. A. Ioannidis, M.D., D.Sc., and Orestis A. Panagiotou, M.D., gathered data from medical texts to determine which biomarkers that had been evaluated in at least one highly cited study, continued having the same link to the specific disease or condition after a meta-analysis (larger study) was carried out.

They only reviewed articles published in any of the 24 most highly cited biomedical journals - and only articles with over 400 citations. They searched in MEDLINE for subsequent meta-analyses which had the same biomarker and the same outcome. They identified 35 studies, published between 1991 and 2006.

The biomarkers that were highly cited in those studies included blood biomarkers, infectious agent biomarkers, blood proteins, and genetic risk factors. A large number of them were related to cardiovascular and cancer related outcomes.

The authors found:

• 86% of the highly cited studies overstated the link compared to the larger study (30 of the 35 associations)
• In only three cases did the highly cited study coincide with the larger one
• In only two cases did the larger study estimate a stronger link than in the highly cited one
• In 5 associations, the relative risk estimate was in the opposite direction in the largest study compared to the highly cited one
• In 20 highly cited studies the link was at least 2-fold greater than in the larger study
• In 13 highly cited studies the association was 4-fold greater than in the larger study

The authors say there are several reasons why false-positive and exaggerated results exist in the highly cited studies:

"Many of these studies were relatively small and among the first to report on the association of interest. Discoveries made in small studies are prone to overestimate or underestimate the actual association. Interest in publishing major discoveries leads to selective reporting from chasing significance. ..our study documents that results in highly cited biomarker studies often significantly overestimate the findings seen from meta-analyses.

Evidence from multiple studies, in particular large investigations, is necessary to appreciate the discriminating ability of these emerging risk factors. Rapid clinical adoption in the absence of such evidence may lead to wasted resources."

"Comparison of Effect Sizes Associated With Biomarkers Reported in Highly Cited Individual Articles and in Subsequent Meta-analyses"
John P. A. Ioannidis, MD, DSc; Orestis A. Panagiotou, MD
AMA. 2011;305(21):2200-2210. doi: 10.1001/jama.2011.713

Written by Christian Nordqvist
Copyright: Medical News Today
Not to be reproduced without permission of Medical News Today

• Additional
• References
• Citations