Identifying Relationships between Attack Patterns using Large Language Models

Abstract

The Adversarial Tactics, Techniques, and Common Knowledge (ATT&CK) and Common Attack Pattern Enumeration and Classification (CAPEC) frameworks are essential knowledge bases that catalog traditional attack patterns and their interrelationships (e.g., abstract-concrete relationships). In addition, a knowledge base named Adversarial Threat Landscape for Artificial-Intelligence Systems (ATLAS) focuses on artificial intelligence (AI)/machine learning (ML)-related attack patterns. Newly discovered attack patterns are incorporated into these knowledge bases manually, potentially leading to missed relationships or delayed information updates. This study introduces a methodology that uses large language models (LLMs) to identify abstract-concrete relationships between attack patterns, aiding in rapid classification and in the rapid development of a defensive strategy. We trained BERT, GPT, and SVM models on ATT&CK, CAPEC, and their combined datasets for relation classification among attack patterns. The evaluation results show that the fine-tuned GPT-3.5 model outperformed the other investigated models, showing potential applicability even to AI/ML-related attack patterns and emphasizing the importance of using training data in the same format as test data. This study also finds that GPT-3.5 effectively focuses on critical descriptive terms, bolstering its performance. The proposed methodology is effective in discerning attack-pattern relationships, demonstrating its potential applicability in the AI security domain.