Parameter Estimation Problems in Markov Random Processes

Abstract

Problems of convergence and stability of Bayesian estimates in the identification of stochastic control systems are considered. The informational measure of the mismatch between the estimated distribution and the estimate is the main apparatus for establishing the fact of convergence. The choice of a priori distribution of parameters is not always obvious. The Kullback-Leibler information number is taken as such measure. The convergence of the estimates of the transition function of the process to the non-stationary transition function is established in this paper. The problem of synthesis of optimal strategies for dynamic systems in which there is no part of the main information needed for constructing the optimal control is also considered. It is assumed that the system contains at least one unknown parameter belonging to some parameter space. Therefore, the class of control systems considered in the article is the class of parametric adaptive systems.