Finite Markov Chains and Markov Decision Processes

Abstract

This paper provides a brief introduction to Markov chains and their fundamental properties. Then we use the previous knowledge to construct Markov Decision Processes (MDPs), which give a framework for modeling decision-making processes in optimization problems. Policy iteration and value iteration algorithms are introduced to solve for the optimal result of such models. We finish by introducing the hidden Markov models to solve the part-of-speech tagging problem.