Using Machine Learning to Detect APTs on a User Workstation

Adams C*, Tambay AA, Bisses

Abstract

Advanced Persistent Threats (APTs) are explicitly designed to be difficult to detect, but their activities necessarily include some differences from what a regular user might do. We present an analysis and comparison of four machine learning algorithms that were used to first learn a user’s behavior and then to detect APT activity as an anomaly in that behavior. We also present our methodology for each step of the analysis. In particular, for each user, we collected data with Osquery on a clean machine before running the Red Team Automation (RTA) scripts to simulate an APT attack. The four algorithms we tested on each user’s data (neural networks, decision trees, kmeans clustering, and one-class SVM) included supervised, unsupervised and one-class algorithms. This study was undertaken as a proof-of-concept exercise to see if machine learning could be beneficial in APT detection, and our results indicate that looking at user behaviour for APT detection appears to be a promising approach. Previous work focused on APT behaviour (particularly in the context of network traffic), whereas our goal is to detect APTs on the computer where the legitimate user is present and active and to detect the APTs by discovering anomalies with respect to typical user behaviour.