Implementation and Evaluation of a System Call Moving Target Defense Applied Multiple Times at Runtime for Binary Injections

Abstract

We propose and evaluate a system call-based Moving Target Defense (MTD) mechanism as a countermeasure against code injection attacks that exploit unknown vulnerabilities. Although integrating the proposed MTD mechanism into the OS kernel would be more ideal, we implemented it in userland for this study in order to demonstrate its feasibility and evaluate its effectiveness. The proposed system randomizes the mapping between system call numbers and their corresponding functions, thereby invalidating system calls issued by injected malicious code. Since system calls serve as the primary interface through which user applications access system resources, this randomization prevents attackers from achieving their objectives, even if they successfully inject code into a process. This approach, categorized as an MTD technique, is particularly promising against zero-day attacks, where vulnerabilities are exploited before they are patched. By dynamically altering the mapping at each system call invocation, the system increases its runtime diversity and unpredictability. While kernel-level implementation remains a future goal, our evaluation—conducted by remapping system call invocations through a userland wrapper—demonstrates that the proposed method can detect and mitigate code injection attacks in a wide range of existing compiled programs, without requiring specialized hardware support.