2026 Volume E109.D Issue 5 Pages 683-694
Large Language Models (LLMs) are increasingly used for the critical task of generating AI risk scenarios, yet practitioners lack empirical guidance on model selection. This study addresses that gap through a case study benchmarking 23 LLMs against a real-world AI system to analyze their underlying reasoning patterns. We introduce a novel “Hit Rate” metric based on actual incidents to quantitatively measure performance. The results suggest significant, statistically-verified performance disparities among models and show that this gap is uncorrelated with superficial linguistic fluency. Instead, we indicate that the performance gap appears to be strongly linked to the model’s underlying reasoning pattern, which leaves an unmistakable qualitative signature on the final outputs. A “Systematic Top-Down” approach, which mirrors expert human analysis, consistently produces specific and actionable scenarios, while less structured methods yield generic or contextually flawed warnings. These findings serve as a strong caution against model-agnosticism, establishing that an LLM’s reasoning process—suggested by the specificity and actionability of its outputs—is a critical factor for its efficacy in safety-critical tasks.
