2021 Volume 63 Issue 6 Pages 1232-1240
Background and Aims: We previously showed that the performance of artificial intelligence (AI) in diagnosing deep (≥1mm) submucosally invasive (T1b) colorectal cancer was relatively good after training with non-magnified white light images (sensitivity 80％, specificity 87％). However, the “region of interest” (ROI) within the image that was responsible for the AI diagnosis is a black box. Recently, the class activation mapping (CAM) technique has been developed, enabling identification of the ROI within the image that AI utilized for making the diagnosis. In this study, we aimed to investigate features of the ROI selected by AI using CAM and clarify the similarities and differences between AI and endoscopists.
Methods: We selected endoscopic digital images that were used for training or validation of our AI system in our previous study, comprising histologically proven T1b colorectal cancers (n＝114, 0-Ⅰs 69, 0-Ⅱa 39, 0-Ⅱc 6; maximum diameter 16.5±13.4mm). The application of CAM was limited to a maximum of two images per lesion from which T1b cancer was diagnosed. The CAM images were generated on ResNet50, and the ROI defined by AI was depicted in red. Two expert colonoscopists rated characteristics of the ROI following discussion. The outcome measures were concordance of the ROI defined by AI with the ROI defined by expert endoscopists, and endoscopic features analyzed by AI, including color (red or non-red), surface morphology (depressed, flat, protruding), presence of bleeding, and fold convergence. Concordance of the ROI defined by AI and the ROI that was defined by expert endoscopists was rated by concordance area and classified into excellent (≥75％), fair (≥25％ <75％) and poor (<25％).
Results: CAM images were successfully generated for all 226 images. The level of concordance between the ROI defined by AI and the ROI that was defined by expert endoscopists was excellent in 39％, fair in 34％ and poor in 27％. In images showing poor concordance, the ROI defined by AI was distant from the T1b cancer. After excluding lesions with poor concordance, the vast majority (91％) of the ROI defined by AI was concordant with the ROI containing endoscopists’ identification of red color, and a small proportion (21％) of the ROI defined by AI revealed bleeding. Among the lesions detected by AI, the surface morphology was depressed in 39％, flat in 5％ and protruding in 57％. Fold convergence was observed in 34％ of the ROI defined by AI.
Conclusions: Most of the ROIs identified by AI were concordant with ROIs defined by experienced endoscopists, although AI may diagnose T1b colorectal cancer using different features of the ROI. Since a quarter of ROIs were present within normal mucosa, annotation of the image that the image should be reviewed by expert endoscopists may improve the diagnostic accuracy of AI for T1b colorectal cancer.
