Elementary Continuous-Time Markov Chain-Based Queueing Models

Abstract

In this chapter, we explore the analysis of several queueing models that are characterized as discrete-valued, continuous-time Markov chains (CTMCs). That is, the queueing systems examined in this chapter will have a countable state space, and the dwell times in each state will be drawn from exponential distributions whose parameters are possibly state-dependent. The most elementary queueing systems in this class are characterized by one-dimensional birth and death models. The stochastic behavior of these systems at a particular point in time is completely described by a single number, which we shall think of as the "occupancy" of the system. The dwell times for each state are drawn from exponential distributions independently, but, in general, the parameter of the exponential distribution depends upon the current state of the system. We begin by examining the well known M/M/1 queueing system, which has Poisson arrivals and exponentially distributed service times. For this model, we will consider both time-dependent and equilibrium behavior, with primary emphasis on the latter. In particular, we shall consider both the time-dependent and equilibrium occupancy distributions, the stochastic equilibrium sojourn and waiting time distributions, and the stochastic equilibrium distribution of the length of the busy period. Several related processes, including the departure process, are introduced, and these are used to obtain equilibrium occupancy distributions for simple networks of queues. After discussing the M/M/1 system, we briefly discuss formulation of the dynamical equations for more general birth-death models in Section 3.2. The time-dependent behavior of finite-state general birth-death models is discussed in Section 3.3. A reasonably complete derivation based upon classical methods is presented herein, and the rate of convergence of the system to stochastic