Simulation-based optimization of singularly perturbed markov reward processes with states aggregation

Abstract

We present a simulation-based algorithm to compute the average reward of singulary perturbed Markov Reward Processes (SPMRPs) with large scale state spaces, which depend on some sets of parameters. Compared with the original algorithm applied on these problems of general Markov Reward Processes (MRPs), our algorithm aims to obtain a faster pace in singularly perturbed cases. This algorithm relies on the special structure of singularly perturbed Markov processes, evolves along a single sample path, and hence can be applied on-line. © Springer-Verlag Berlin Heidelberg 2005.