GAMPALv2: An Anomaly Detection Mechanism for Internet Traffic by Predicting Flow Size Range from Time Features

抄録

To detect anomalies on an Internet backbone network, we proposed GAMPAL (General-purpose Anomaly detection Mechanism using Prefix Aggregate without Labeled data). For scalability to the number of entries in the BGP RIB (Border Gateway Protocol Routing Information Base), GAMPAL introduces PA (Prefix Aggregate). It adopts an LSTM-RNN (Long Short-Term Memory Recurrent Neural Network) as a model that focuses on the periodicity of Internet traffic patterns at a weekly scale. However, GAMPAL has three issues: (i) computational complexity, (ii) difficulty in defining detection threshold, and (iii) difficulty in detecting when and in which PA anomaly occurred. Therefore, this paper proposes GAMPALv2, which solves these problems for the practical use of GAMPAL. To solve (i), GAMPALv2 reduces the dimension of the input variables from 288 (five-minute slots in a day) to 7 by defining time features. It also adopts the RFR (Random Forest Regressor) as a prediction model. To solve (ii) and (iii), GAMPALv2 defines the predicted range based on the predicted values of the RFR and detects anomalies for each PA by comparing the predicted range with the observed value. As a result, the training and prediction time is reduced from four days using a GPU to 23 minutes using an 8-core CPU. Utilizing semantics such as date, time, and day of the week defined in the time features improves prediction accuracy. The evaluation results show that GAMPALv2 can detect anomalies in the real world, such as connection failure on YouTube, DDoS (Distributed Denial of Service) attacks, and increasing traffic due to an event. In addition, the accuracy evaluation shows that the recall is improved. Although not precisely comparable due to the different calculation methods, the average recall in the previous work is 81.8%, whereas recall improves to 93.1% in GAMPALv2.