Anomaly Detection in Control Data of Programming Execution by Context Sensitive Hidden Markov Model

Abstract

The inconsistency is a major problem in security of information in computer is two ways: data inconsistency and application inconsistency. These two problems are raised due to bad structure of design in programming and create security breaches, vulnerable entries by exploiting application codes. So we can discover these anomalies by design of anomaly detection system (ADS) models at system programming (coding) levels with the help of machine learning. The security vulnerabilities (anomalies) are frequently occurred at potential code execution by exploitation or manipulation of instructions. So, in this paper we have specified various forms of extensions to our work to detect wide range of anomalies at coding exploits and use of a machine learning technique called Context Sensitive-Hidden Markov Model (CS-HMM) will improve the overall performance of ADS by discovering the correlations between control data instances. In this paper we are going to use Linux OS tracing kits to collect the necessary information such as control data instances (return addresses) collected from system as part of artificial learning. The results evaluated through practice on various programs developed for work and also uses of some Linux commands for tracing, finally compared performance of all those input datasets generated live (artificially). After that, the CS-HMM is applying to datasets to scrutinize the anomalies with similarity-search and correlation of function control data of program and classification process determines the anomalous outcomes.