Endoscopists Outperform CADx in Predicting Polyp Histology

Endoscopists’ predictions of polyp histology were more accurate than those of a computer-aided diagnostic tool (CADx) in real-time colonoscopy, but overall accuracy was highest when the endoscopists and CADx agreed, according to a new study.

“Our results support the role of CADx as an independent second reader in a clinical setting, which can improve the performance of the endoscopist in achieving accurate optical diagnosis of polyp histology in vivo,” write James Weiquan Li, MD, of Changi General Hospital in the Singapore Health Services in Singapore, and colleagues.

Still, more research is needed to establish the tool’s role during real-time colonoscopy, they add.

The study’s primary aim was to clinically validate CADx for characterizing polyp histology in real time. Past research has suggested that CADx may have a role in facilitating decisions about whether or not to resect polyps that are identified during colonoscopy.

The researchers used the CAD EYE system (Fujifilm), with its built-in CADx prediction function, in their evaluation because it “was readily available, required minimal training, and could generate an automated input fast enough for use in a clinical setting,” they write.

The study was published online March 22 in The American Journal of Gastroenterology.

Predictive Capabilities Tested

The researchers prospectively compared predictions of histologies of 661 polyps using CADx with predictions from 21 experienced endoscopists during real-time colonoscopies for 320 patients in four large tertiary referral centers in Singapore. The patients were at least 40 years old and underwent colonoscopies between March 2021 and July 2022.

The endoscopists based their assessments on visual inspection of the polyps. They then recorded automated output from the CADx support tool. The predictions of both were then compared to the actual histology after the imaged polyps were resected and assessed.

The endoscopists’ predictions were accurate 75.2% of the time, compared to 71.6% from the CADx (P = .023).

Endoscopists also outperformed the CADx tool with respect to sensitivity for neoplastic polyps (70.3% vs 61.8%; P < .001). The specificity for neoplastic polyps, however, was lower for endoscopists than for CADx (83% vs 87.4%; P = .022). The positive predictive value for neoplastic polyps was very similar between the two: 87% for endoscopists and 88.7% for CADx. The negative predictive value was 63.4% with endoscopists and 58.6% for CADx.

The interobserver agreement between the endoscopists and CADx in predicting polyp histology was moderate, at 83.1%.

When there was concordance between endoscopist and CADx predictions, the overall accuracy increased to 78.1%. Similarly, concordance increased the sensitivity to 70.5% and the specificity to 88.7%. The positive predictive value with concordance was 89.6%, and the negative predictive value was 68.5%.

The fact that the results showed a lack of perfect concordance between endoscopists and CADx for neoplastic vs hyperplastic polyps “prevents us from currently employing…’resect-and-discard’ and ‘diagnose-and-leave’ strategies, where, potentially with aid from AI, we could make real-time pathological diagnosis aiding the gastroenterologist to determine whether we could resect or leave behind without the cost of pathology,” said Rishi D. Naik, MD, assistant professor of medicine in the Department of Gastroenterology, Hepatology, and Nutrition at Vanderbilt University Medical Center, Nashville, Tennessee, who was not involved in the research.

Subgroup Analysis Findings

In a subgroup analysis, the CADx accuracy was lower (69.8%) than that of the endoscopists (73.%) for smaller polyps (≤5 mm). The accuracy of CADx was also less than that of the endoscopists with respect to “good” vs “excellent” bowel preparation, the researchers report.

While endoscopists outperformed CADx with polyps that were not in a difficult location, performance between the two was not statistically different for polyps that were in difficult locations.

“The suboptimal positioning, lighting, and exposure of polyp surfaces and vessels encountered in a clinical setting may interfere with the digital input needed for analysis by CADx systems to generate an output,” the authors suggest. “Although the results of published retrospective studies are promising, they do not consider the operational environment and clinical context in which such systems are deployed.”

One of the study’s strengths, the authors write, is that they evaluated all detected polyps in all colonic segments, rather than limit the study to diminutive polyps or to the rectosigmoid colon.

“This adds valuable information and clinical context because CADx systems will likely be used by the endoscopist during withdrawal in the entire colon and not just during inspection of the rectosigmoid segments,” they write.

The researchers intentionally excluded from the analysis the 32 sessile serrated lesions/polyps that were found, since they are underrepresented in CADx training data, which could have resulted in misclassification. They also acknowledge that their study was not randomized. In addition, it involved only experienced endoscopists and did not include trainees.

The inclusion of sessile serrated adenomas/lesions would have been worthwhile, Naik told Medscape Medical News.

“Though the authors report it would not have changed their outcomes, I would expect a higher number of sessile lesions in a US population compared to this study, decreasing its generalizability,” Naik said. “The high percentage of hyperplastic polyps in the right colon also gives pause on whether these lesions were misdiagnosed histologically and were truly [sessile serrated adenomas/lesions].”

Highly rigorous research is needed on CADx performance on sessile serrated lesions/polyps, Naik added.

Expectations Not Met

The authors note that they expected better performance from the CADx tool. One reason could be that “CADx systems are often trained on still images or video segments that do not fully represent the wide range in appearances of polyps and background colonic mucosa encountered in a real-world setting,” they write.

The findings indicate “that not all AI are ready to be implemented in clinical practice, but rigorous and transparent research is needed in line with current FDA [US Food and Drug Administration] recommendations,” said Cesare Hassan, MD, PhD, of Humanitas Research Hospital and University in Milan, Italy, who was not involved in the study but who has conducted his own CADx research.

“I understand AI is underperforming in the proximal colon, but I was expecting a better performance for left polyps, also named as PIVI-1,” Hassan told Medscape Medical News. He also pointed out that the study did not include the endoscopists’ level of confidence, which is clinically recommended.

While AI is primed for gastroenterology, “we must think about this in two facets,” Naik said. The first is “real-time recognition in a potentially imperfectly prepped colon or difficult location.”

The second is “histological recognition simultaneously,” he said. “In this manuscript, the endoscopist detected the polyp and then used CADx, but ideally, we are able to use real-time polyp detection and histological detection to aid — not distract — the endoscopist.”

Though the technology is headed “in the right direction for neoplastic detection, we are too early to solely rely on the endoscopist and CADx algorithm,” Naik added.

CADx Is the “Next Frontier”

Advancements in CADx in the coming years hinges on increasing the sophistication of its training, Naik said.

“CADx is the next frontier of gastroenterology, but the technology is outpacing how we think about pattern recognition,” he said. “We are forcing image learning sequences instead of dynamic, video-based learning algorithms, and not utilizing real-time learning to improve the interface. As the proof of concepts have worked for image guidance, we need to leverage dynamic learning and continued learning for our software.”

The technology will eventually “set new boundaries” in gastroenterology beyond polyp detection, such as with detection of Barrett esophagus or neoplastic diseases in the esophagus and with Helicobacter pylori infection and its progression to cancer in the stomach, Naik said. But reaching that point, and thereby improving patients’ lives, requires universal sharing of the training modules and real-time, crowd-sourced learning, he added.

The study did not receive external funding. The authors and Naik have disclosed no relevant financial relationships. Hassan has relationships with Alpha-sigma, Fujifilm (whose equipment was used in this study), Medtronic, Norgine, Olympus, and Pentax.

Am J Gastroenterol. Published online May 22, 2023. Abstract

Tara Haelle is a health/science journalist based in Dallas. Follow her at @tarahaelle.

Follow Medscape on Facebook, Twitter, Instagram, and YouTube. Have a tip for us? Contact us.

Source: Read Full Article