AppFA: A Novel Approach to Detect Malicious Android Applications on the Network

Abstract

We propose AppFA, an Application Flow Analysis approach, to detect malicious Android applications (simply apps) on the network. Unlike most of the existing work, AppFA does not need to install programs on mobile devices or modify mobile operating systems to extract detection features. Besides, it is able to handle encrypted network traffic. Specifically, we propose a constrained clustering algorithm to classify apps network traffic, and use Kernel Principal Component Analysis to build their network behavior profiles. After that, peer group analysis is explored to detect malicious apps by comparing apps' network behavior profiles with the historical data and the profiles of their selected peer groups. These steps can be repeated every several minutes to meet the requirement of online detection. We have implemented AppFA and tested it with a public dataset. The experimental results show that AppFA can cluster apps network traffic efficiently and detect malicious Android apps with high accuracy and low false positive rate. We have also tested the performance of AppFA from the computational time standpoint.