Learning from All Types of Experiences: A Unifying Machine Learning Perspective

Abstract

Contemporary Machine Learning and AI research has resulted in thousands of models (e.g., numerous deep networks, graphical models), learning paradigms (e.g., supervised, unsupervised, active, reinforcement, adversarial learning), optimization techniques (e.g., all kinds of optimization or stochastic sampling algorithms), not mentioning countless approximation heuristics, tuning tricks, and black-box oracles, plus combinations of all above. While pushing the field forward rapidly, these results also contributed to making ML/AI more like an alchemist's crafting workshop rather than a modern chemist's periodic table. It not only makes mastering existing ML techniques extremely difficult, but also makes standardized, reusable, repeatable, reliable, and explainable practice and further development of ML/AI products extremely costly, if possible at all. This tutorial presents a systematic, unified blueprint of ML, for both a refreshing holistic understanding of the diverse ML paradigms/algorithms, and guidance of operationalizing ML for creating problem solutions in a composable manner. The tutorial consists of three parts. The first part provides an overview of the current landscape of ML paradigms, with a focus on motivating a systematic perspective. The second part presents the blueprint from three aspects: objective function, optimization solver, and model architecture. We describe standardized formulations of the diverse objectives and algorithms, and a composable view of model structures. On this basis, the third part focuses on the operational side of ML. We describe principled module abstraction of ML building blocks. We show the abstraction enables efficient composition of ML solutions to problems in healthcare, manufacturing, vision/text generation.