2026 Volume 25 Issue 1 Article ID: mp.2024-0183
Purpose: This study evaluated the performance of artificial intelligence (AI)-based brain aneurysm detection software in clinical settings, aiming to assess its utility as a supportive tool for radiologists. Metrics included sensitivity, positive predictive value (PPV), F1 score, and false positives (FPs) per case.
Methods: A retrospective analysis of 442 cases (March 2023–August 2024) compared AI detections against a reference standard derived from the radiologists’ assessments and image re-review. Findings were categorized into true positives (TPs), FPs, and false negatives (FNs). Subgroup analyses covered aneurysm size, magnetic field strength of the MRI, patient age, and aneurysm location.
Results: The study included 442 cases (226 males, 216 females; median age 72). Out of 94 total aneurysms, the AI detected 73 TP and missed 21 FN. It also identified 520 FP. Overall, sensitivity was 77.7%, PPV was 12.3%, and the F1 score was 0.212. The FPs averaged 1.18 per case. Sensitivity varied by aneurysm size: 85.1% for ≤ 3 mm, 69.2% for 3–5 mm, and 50.0% for > 5 mm. Significant variability in FPs per case was observed across different magnetic field strengths. Performance also varied by patient age and aneurysm location.
Conclusion: The AI software demonstrated moderate sensitivity, especially for smaller aneurysms. Variations in performance across different magnetic field strengths and aneurysm size suggest a need for more robust AI algorithms. Detailed analysis of aneurysm locations provides insights into areas where AI performance could be enhanced. Integrating the AI software as a supportive tool, combined with radiologist expertise, is hypothesized to enhance detection accuracy, though further studies are needed to quantify this combined effect.
