- Researchers developed an A.I. deep learning tool called Sybil to predict lung cancer risk.
- Sybil had an AUC (area under the curve) value of 94%, which shows a high degree of ability to correctly classify people with or without lung cancer within a year of screening, and up to 81% within six years.
- Sybil also reduced the false positivity rate from 14% with current methods of analysis to 8% for the first scan, opening up the possibility of a single scan for lung cancer.
- They noted that further evaluation is needed to ascertain Sybil’s performance, particularly in different ethnic groups.
Lung cancer is the
Cigarette smoking is the
Low-dose computed tomography, also known as a low dose CT scan, is the only recommended way to screen for lung cancer. It involves patients lying on a table while an X-ray machine generates images of their lungs.
The United States Preventive Services Task Force
Studies also suggest that many screened patients do not receive adequate long-term care, including follow-ups. Other research shows lung cancer diagnoses are increasing among never- and light-smokers.
Improving the efficiency of low dose CT scans and expanding them to never- and light smokers could reduce lung cancer mortality rates.
Current low dose CT scan approaches require a combination of demographic information, clinical risk factors, and radiologic annotations for results in addition to 3 or 4 low dose CT scans.
Recently, researchers created a deep-learning cancer risk model named Sybil. Unlike current approaches, Sybil requires just one low-chest computed tomography scan to predict lung cancer risk 1-6 years after screening.
“Sybil gives a risk score, not a diagnosis—so it’s most useful to identify which patients need to be followed closely or screened for cancer,” Dr. Lecia V. Sequist, director of the Center for Innovation in Early Detection at Massachusetts General Hospital and professor of medicine at Harvard Medical School, one of the study’s authors, told Medical News Today.
Sybil’s algorithm is publicly available alongside image annotations to promote further research and clinical applications.
The corresponding study was published in the Journal of Clinical Oncology.
The researchers developed a deep-learning A.I. model using data from 15,000 participants. Altogether, they used 35,001 low dose CT scans to train and develop their model, and 6,282 to test their model.
To help train the model, two thoracic radiologists annotated suspicious lesions on patient scans who developed cancer within a year after the scan.
Using singular low dose CT scans alone, Sybil had a score of correct allocation of lung cancer or not of 92% across all test data sets after 1 year, 86% after 2 years, and a probability (C-index) of 75% after 6 years.
The researchers noted that Sybil’s performance was stable across sex, age, and smoking history.
They next tested Sybil on a dataset from Massachusetts General Hospital (MGH) in Boston, the U.S., and Chang Gung Memorial Hospital (CGMH) in Taiwan. Unlike the primary and MGH datasets, patients from CGMH did not require a positive smoking history for a low dose CT scan.
Sybil correctly predicted 86% of lung cancer cases or healthy lungs within one year from the MGH dataset, alongside 94% of cases in the CGMH data. It also predicted 81% of lung cancers or healthy lungs among the MGH cohort and 80% among the CGHM cohort after six years.
The researchers wrote that Sybil could also predict traditional clinical risk factors such as smoking from scans.
The researchers noted some limitations to their model. They noted, for example, that 92% of Sybil’s training data came from White patients, meaning that their findings may not apply to more diverse populations.
They also noted that the training data scans were obtained between 2002 and 2004, meaning that changes in CT technology over time might adversely affect Sybil’s predictive ability.
As they did not have detailed smoking data from CGMH patients, conclusions about Sybil’s ability to predict lung cancer in nonsmokers are speculative.
“So far, work around Sybil has been retrospective,” Dr. Sheena Bhalla, an assistant professor at the Simmons Cancer Center at U.T. Southwestern, who was not involved in the study, told MNT.
“Moving forward, prospective data, incorporating diverse populations (in terms of race/ethnicity, smokers and non-smokers, etc.), is needed to better understand Sybil’s performance and broader clinical benefit,” she added.
“Though low dose CT-based lung cancer screening per U.S. Preventive Services Task Force guidelines can decrease lung cancer mortality in high risk individuals, low dose CTs can also result in false positive results, which may lead to unnecessary procedures in a subset of patients,” she explained.
These factors in mind, the researchers wrote that further evaluation in a prospective study is needed to assess Sybil’s performance and clinical benefit.
Dr. Jun Zhang, medical oncologist at The University of Kansas Cancer Center, who was not involved in the study, also told MNT:
“Overall, this is a positive finding but nothing to be surprised with the capacity of A.I.”
“[Sybil] certainly has value, for example- telling the probability whether a nodule can be benign and malignant. The most important thing we don’t know yet is whether such a prediction can for sure translate into survival benefit in comparison to LDCT. Other factors such as anxiety, compliance, cost etc. all need to be put into consideration.”
— Dr. Jun Zhang
Rema Padman, professor of management science and healthcare informatics at Carnegie Mellon University’s Heinz College, who was not involved in the study, told MNT that while results from Sybil are exciting, a few issues remain regarding its development.
“One [issue] is about the quality of the image and its impact on the performance of the algorithm. [The s]econd is about the critical features that drive the performance. Are there features in the image that are particularly influential in improving performance and have actionable consequences for clinical decision-making?” she said.
She added that “given the wide skepticism/concerns about the use of AI/ML [machine learning] for clinical care,” there may be translational challenges with this discovery.
The researchers wrote that Sybil could potentially run in the background at radiology reading stations, and predict lung cancer risk as soon as low dose CT scans are available without the presence of radiologists to annotate areas of interest, or demographic and other clinical data.
They hope that Sybil may be able to decrease the need for follow-up scans or biopsies among low risk patients.
“Our hope is that in the future, Sybil may be used to identify which patients need to be screened annually for lung cancer. It may function sort of like a colonoscopy where you get one baseline colonoscopy around age 45-50 (for example), and if no polyps are seen, then you don’t need another one for 10 years,” Dr. Sequist said.
“However, if abnormalities are seen, you need follow-up at shorter intervals,”
“It’s important to know that for screening, one size may not fit all patients – Sybil will help us more towards personalized screening regimens. Sybil also may be very useful in resource-strapped settings because it doesn’t need a radiologist to help it run or interpret the images before Sybil looks at the scan.”
— Dr. Lecia V. Sequist
“These future applications need to be tested in clinical trials and confirmed useful before they are ready for prime time, of course, and we’re preparing to launch these trials as soon as possible!” she concluded.