Hypergraph based Unsupervised Contextual Pattern Learning and Anomaly Detection for Global Terrorism Data

Abstract

In a dataset, an event which deviates from the rest of the dataset is a rare event. This rare event can be intrusion or any suspicious activity in the system and is called an anomaly. These anomalies are important to detect because this may be any terrorist attack, outbreak of the disease, malfunctioning or fraud in the system. Anomalies are the deviation from the normal patterns in the dataset. It is important to learn the normal patterns in order to identify the deviation. Labelled data in real life anomaly detection is not available due to rarity of anomalies. It is challenging to identify anomalous combinations and combinatorial patterns of feature instances using conventional machine learning algorithms. We introduce Hypergraph based Unsupervised Contextual Pattern Learning and Anomaly Detection (HUCPLAD) technique for unlabeled datasets and implemented on Global Terrorism Data (GTD). HUCPLAD gets rid of the curse of dimensionality, maintains hierarchy, learns the contextual pattern, detects contextual anomalies and measures the behavior of co-occurring events.