Machine learning

Abstract

We present an extensive review on the subject of machine learning by studying existing literature. We focus primarily on the main approaches that have been proposed in order to address the problem of machine learning and how they may be categorized according to type and amount of inference. Specifically, the categorization of the various machine learning paradigms according to the type of inference, involves the following two approaches: • Model Identification or Parametric Inference; and • Model Prediction or General Inference. The general framework of the parametric model, in particular, introduces the principles of Empirical Risk Minimization (ERM) and Structural Risk Minimization. On the other hand, the Transductive Inference Model is defined as an extension to the original paradigm of General Inference. The categorization of machine learning models according to the amount of inference includes the following approaches: • Rote Learning; • Learning from Instruction; and • Learning from Examples. Specifically, Learning from Examples provides the framework to analyze the problem of minimizing a risk functional on a given set of empirical data which is the fundamental problem within the field of pattern recognition. In essence, the particular form of the risk functional defines the primary problems of machine learning, namely: • The Classification Problem; • The Regression Problem; and • The Density Estimation Problem which is closely related to the Clustering Problem. Finally, in this chapter we present a conspectus of the theoretical foundations behind Statistical Learning Theory.