A Stochastic Model with Applications to Learning

Abstract

A stochastic model designed for analyzing data with changing probabilities is presented. On each of a series of trials one of two alternatives occurs and the probabilities of occurrence are changed from time to time by events. Corresponding to each class of events is an operator which represents a linear transformation on the probabilities of the two alternatives. Cases of fixed event probabilities and of changing event probabilities are considered. Recurrence formulas for moments of the resulting distributions of probabilities are provided. These formulas are often tedious to apply, but for the first and second moments several bounds are provided; these bounds are relatively easy to compute. The problem of estimating the parameters of the model is discussed. No general solution is obtained but simplifying assumptions lead to interesting special cases for which detailed procedures of parameter estimation are presented. One such special case arises when there are two event operators which commute, implying that the operators have equal limit points or that one operator is the identity operator. The method of maximum likelihood is applied to this case. Another special case, which arises when the slope parameters of the two operators are equal, is discussed in Section 8. Applications of the model and estimation procedures to certain kinds of data on animal and human learning are described. The examples given are experiments on verbal learning, the avoidance training of dogs, the reward training of rats in a simple T-maze, and the behavior of human subjects in a two-choice situation. CR - Copyright © 1953 Institute of Mathematical Statistics