A RAT Detection Method Based on Network Behavior of the Communication's Early Stage

Abstract

Remote Access Trojans (RAT) is a spyware which can steal the confidential information from a target organization. The detection of RATs becomes more and more difficult because of targeted attacks, since the victim usually cannot realize that he/she is being attacked. After RAT's intrusion, the attacker can monitor and control the victim's PC remotely, to wait for an opportunity to steal the confidential information. As this situation, the main issue we face now is how to prevent confidential information being leaked back to the attacker. Although there are many existing approaches about RAT detection, there still remain two challenges: to detect RAT sessions as early as possible, and to distinguish them from the normal applications with a high accuracy. In this paper, we propose a novel approach to detect RAT sessions by their network behavior during the early stage of communication. The early stage is defined as a short period of time at communication's beginning; it also can be seen as the preparation period of the communication. We extract network behavior features from this period, to differentiate RAT sessions and normal sessions. For the implementation and evaluation, we use machine learning techniques with 5 algorithms and K-Fold cross-validation. As the results, our approach could detect RAT sessions in the communication's early stage with the accuracy over 96% together with the FNR of 10% by Random Forest algorithm.