DOI 10.1007/s10115-002-0094-1 Springer-Verlag London Ltd. �� 2003 Knowledge and Information Systems (2003) 5: 368���386 Modeling Spatial-Temporal Data with a Short Observation History Dragoljub Pokrajac1, Reed L. Hoskinson2 and Zoran Obradovic1 1Center for Information Science and Technology, Temple University, Philadelphia, PA, USA 2Idaho National Engineering and Environmental Laboratory, Idaho Falls, ID, USA Abstract. A novel method is proposed for forecasting spatial-temporal data with a short observa- tion history sampled on a uniform grid. The method is based on spatial-temporal autoregressive modeling where the predictions of the response at the subsequent temporal layer are obtained using the response values from a recent history in a spatial neighborhood of each sampling point. Several modeling aspects such as covariance structure and sampling, as well as identification, model estimation and forecasting issues, are discussed. Extensive experimental evaluation is per- formed on synthetic and real-life data. The proposed forecasting models were shown capable of providing a near optimal prediction accuracy on simulated stationary spatial-temporal data in the presence of additive noise and a correlated model error. Results on a spatial-temporal agricultural dataset indicate that the proposed methods can provide useful prediction on complex real-life data with a short observation history. Keywords: Autoregressive models Forecasting Short observation history Spatial-temporal data Temporal and spatial database processing 1. Introduction Emerging interest in spatial-temporal machine learning and data-mining (Roddick et al., 2001) is a consequence of advances in data acquisition, retrieval, and knowledge discovery technology along with an increased demand for applications in areas such as remote sensing (Kafatos, 1999), precision agriculture (Robert, 1999), and medical imagery (Megalooikonomou et al., 2000). Given historic spatial observations, in spatial- temporal prediction the goal is to find techniques to predict future values of certain attributes or the response (Cressie and Majure, 1997 Pokrajac and Obradovic, 2001a). However, due to specific properties of spatial-temporal datasets, numerous other issues involving prediction techniques arise. Received 11 May 2001 Revised 26 Apr 2002 Accepted 5 June 2002

Modeling Spatial-Temporal Data with a Short Observation History 369 In addition to data domains characterized by long observation history (e.g. daily values of meteorological and oceanographic data exist for a hundred years or more, Jones et al., 1986), spatial-temporal prediction should often be performed on datasets that contain a relatively small number of temporal layers, as is usually the case in precision agriculture and medical imagery. In these domains, in addition to forecasting future attribute values, spatial-temporal prediction can be applied for data compression. Namely, it may be possible to predict certain attribute values based on their historical values and to store the attribute values only on locations where predicted and true values significantly differ (Pokrajac et al., 2002, in press). Similarly, instead of taking samples at each location and at each time instance, it might be possible to reduce spatial and/or temporal sampling resolution, and to approximate missing sample values by suitable interpolation/extrapolation of available data. To properly address these and other emerging problems, it is necessary to establish appropriate methods for attribute prediction on spatial-temporal data based on the exploitation of attribute correlation in space and time. Due to the strong presence of spatial correlation in data, conventional time-series models (Box et al., 1994) cannot provide desirable accuracy for spatial- temporal prediction. As alternatives, methods founded on geostatistical and state-space approaches are emerging. Following geostatistical approach (Chil��s and Delfiner, 1999), the spatial-temporal prediction can be performed through kriging ��� an estimation procedure based on a gener- alized least squares algorithm where an estimated attribute value is a linear combination of available attribute values. Originally, kriging was successfully applied for spatial in- terpolation (Denman and Freeland, 1985 Whelan et al., 1996 Kerry and Hawick, 1998). Recently, there have been numerous efforts to apply the principles of kriging to spatial- temporal domains. The effects of temporal and spatial correlation on attributes can be considered separately (Carrat and Valleron, 1992 Posa, 1995 Campling et al., 2001), but this method lacks firm theoretical foundation (Rouhani and Myers, 1990). An al- ternative is the spatial-temporal kriging where spatial and temporal dependences are modeled simultaneously (Chil��s and Delfiner, 1999 Olea, 1999). The crucial problem here is estimation of valid spatial-temporal attribute statistics when disparity exists in the number of available spatial samples and the length of temporal history (Rouhani and Myers, 1990 Buxton and Pate, 1999). In addition, with a large number of spatial samples, kriging may become computationally prohibitive (Kerry and Hawick, 1998) and, when the number of temporal layers is small, can result in ill-conditioned linear systems (Rouhani and Myers, 1990). Hence, spatial-temporal kriging and its variants (Addink and Stein, 1999) are not applicable for forecasting in domains with a large amount of spatial samples but a comparatively short observation history. Modeling based on the state space paradigm (Harvey, 1989 Brown and Hwang, 1993) is a powerful way of implementing a probabilistic framework to spatial-temporal processes, particularly applicable for domains such as meteorology, where data incre- mentally arrives at the prediction system. In space-time Kalman filtering (Wikle and Cressie, 1999), the observed variable consists of a spatial-temporal process and a spa- tially non-correlated measurement error. The spatial-temporal component is considered dependent on its values at the immediate previous time instance where the involved model coefficients vary in space. Attribute value at the consecutive time instance is predicted using a complicated recursive procedure for computing the optimal current estimate of the state vector based on available historical information (Harvey, 1989). Similar models have been applied by Stroud et al. (1999) for interpolation on meteoro- logical and oceanographic data. The application of the state-space approach combined with stochastic simulations is demonstrated by Wikle et al. (2001) for parameter opti- mization in models with a complex structure (���105 parameters!) on data with a long

370 D. Pokrajac et al. temporal history. A further interesting approach is to combine the Kalman filtering and geostatistical modeling (Huang and Cressie, 1996 Mardia et al., 1998). These and other techniques based on the state-space models are applicable for both uniform and non-uniform grids. They have an advantage over batch prediction models (such as those based on the ordinary least squares principle, Neter et al., 1985), as- suming the availability of proper prior knowledge about the modeled process as well as a large number of temporal layers, so that the recursive nature of the models can be fully exploited (Brown and Hwang, 1993 Haykin, 1996). However, in applications of specific interest to us (precision agriculture and medical imaging), statistical parame- ters of the state-space model (correlation matrices of random shocks and measurement errors) are seldom known. In addition, Kalman filter misadjustment (consequence of poor correlation matrix initialization) can hardly be compensated due to a small num- ber of temporal layers and the transient state of the filter will actually be longer than the available temporal span of the data. Hence, state-space models applied for predic- tion on spatial-temporal data with a short observation history are not likely to provide satisfactory results. Alternative methods to geostatistical and state-space approaches involve predic- tion using spatial-temporal autoregressive models and its derivative spatial-temporal auto-regressive models on a uniform grid (STUG), initially introduced by Bennett (1979) without a proper theoretical underpinning. A special case of STUG, with a non- symmetric neighborhood structure (response at a spatial point dependent on response values only from a few pre-specified neighboring locations) was proposed by Kokaram and Godsill (1996). Similar to state-space models proposed byWikle and Cressie (1999), STUG models assume spatial-temporal dependence in a uniform grid through a con- volution (Smirnov, 1999) of filter coefficients and the response values at previous time instances. However, the STUG model considers only a discrete filtering process and filter parameters are constant in space. In addition, the model does not assume the existence of the prior knowledge about process correlations. Compared to geostatisti- cal models that are essentially based on the moving average principle (Lindkvist and Lindqvist, 1997), shorter temporal history may be sufficient for a proper estimation of STUG ��� an autoregressive model that in contrast has a higher but still moderate number of parameters. In this paper, we introduce a spatial-temporal autoregressive modeling on a uniform grid for phenomena with a short temporal history but a large number of spatial samples per temporal instance. In contrast to previous variants of STUG models, here an attribute depends on its values from all neighboring locations on rectangular lattices within a pre-specified distance. Moreover, unlike ad hoc attempts considered in prior studies (Bennett, 1979 Kokaram and Godsill, 1996), here we provide a compre- hensive theoretical foundation of the STUG model. In addition to the model verification by experiments on real-life data (from the precision agriculture domain), here, by using synthetic datasets, we extensively discuss the influence of various data characteristics (e.g. random shocks correlation, the presence of measurement errors) on forecasting accuracy. In Section 2, a theoretical foundation is provided for spatial-temporal auto- regressive models on a uniform grid, including identification, coefficient esti- mation, and forecasting for different sampling techniques. In Section 3, results of an extensive experimental model evaluation for prediction on synthetic spatial-temporal data are presented. This is followed by reporting results on real-life agricultural data with a short observation history in Section 4 and a discussion in Section 5.