General Theory of Markov Processes

Abstract

Our goal in this chapter is to give a concise introduction to the main ideas of the theory of continuous time Markov processes. Markov processes form a fundamental class of stochastic processes, with many applications in real life problems outside mathematics. The reason why Markov processes are so important comes from the so-called Markov property, which enables many explicit calculations that would be intractable for more general random processes. Although the theory of Markov processes is by no means the central topic of this book, it will play a major role in the next chapters, in particular in our discussion of stochastic differential equations. In fact the whole invention of Itô’s stochastic calculus was motivated by the study of the Markov processes obtained as solutions of stochastic differential equations, which are also called diffusion processes. The present chapter is mostly independent of the previous ones, even though Brownian motion is used as a basic example, and martingale theory developed in Chap. 3plays an important role. After a section dealing with the general definitions and the problem of existence, we focus on the particular case of Feller processes, and in that framework we introduce the key notion of the generator. We establish regularity properties of Feller processes as consequences of the analogous results for supermartingales. We then discuss the strong Markov property, and we conclude the chapter by presenting three important classes of Markov processes.