Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 Geoffrey M. Boynton,1 Stephen A. Engel,1 Gary H. Glover,2 and David J. Heeger1 1Departments of Psychology and 2Diagnostic Radiology, Stanford University, Stanford, California 94305 The linear transform model of functional magnetic resonance imaging (fMRI) hypothesizes that fMRI responses are propor- tional to local average neural activity averaged over a period of time. This work reports results from three empirical tests that support this hypothesis. First, fMRI responses in human pri- mary visual cortex (V1) depend separably on stimulus timing and stimulus contrast. Second, responses to long-duration stimuli can be predicted from responses to shorter duration stimuli. Third, the noise in the fMRI data is independent of stimulus contrast and temporal period. Although these tests can not prove the correctness of the linear transform model, they might have been used to reject the model. Because the linear transform model is consistent with our data, we pro- ceeded to estimate the temporal fMRI impulse���response func- tion and the underlying (presumably neural) contrast���response function of human V1. Key words: functional MRI linear systems analysis contrast perception temporal impulse���response function hemodynam- ics calcarine sulcus Functional magnetic resonance imaging (fMRI) measures changes in blood oxygenation and blood volume that result from neural activity (Ogawa et al., 1990 Belliveau et al., 1992) (for review, see Moseley and Glover, 1995). Deoxygenated hemoglo- bin acts as an endogenous paramagnetic agent, so a reduction in the concentration of deoxygenated hemoglobin increases the T2*- weighted magnetic resonance signal. A typical fMRI experiment measures the correlation between the fMRI response and a stimulus. From this, scientists hope to infer something about neural activity. Often it is assumed that there is a simple and direct relationship between neural activity and fMRI response, but the nature of this relationship remains unclear. The goal of the research reported in this article is to understand how the fMRI response relates to neural activity. The vascular source of the fMRI signal places important limits on the tech- nique. Because the hemodynamic response is sluggish, perhaps the fMRI response is proportional to the local average neural activity, averaged over a small region of the brain and averaged over a period of time. We will refer to this as the ���linear transform model��� of fMRI response. The linear transform model, special- ized for a visual area of the brain, is depicted in Figure 1. According to this model, neural activity is a nonlinear function of the contrast of a visual stimulus, but fMRI response is a linear transform (averaged over time) of the neural activity in V1. Noise might be introduced at each stage of the process, but the effects of these individual noises can be summarized by a single noise source that is added to the output. To date, this linear transform model of fMRI response has not been tested, despite the fact that some studies rely explicitly on the linear model for their data analysis (Friston et al., 1994 Lange and Zeger, 1996). The sequence of events from neural response to fMRI response is complicated and only partially understood. It is unlikely that the complex interactions among neurons, hemody- namics, and the MR scanner would result in a precisely linear transform. However, the linear transform model might be a rea- sonable approximation of these complex interactions. The linear transform model is attractive because, if it were correct, it would greatly simplify the analysis and interpretation of fMRI data. Most important, it would provide confidence in infer- ences made about neural activity. In addition, the relationship between neural activity and fMRI response would be character- ized completely and simply by the fMRI ���impulse���response func- tion,��� that is, the fMRI response resulting from a brief, spatially localized pulse of neural activity. The fMRI impulse���response function would allow one to predict the fMRI response evoked by any pattern of neural activity. This would help in experimental design, for example, in choosing the temporal duration of a visual stimulus when measuring fMRI responses in visual cortex. According to the linear transform model, the fMRI impulse��� response function would characterize completely both the spatial and the temporal averaging of the neural activity. This article concentrates on the temporal aspects of fMRI response (for a study on spatial aspects, see Engel et al., 1996). This article also concentrates only on primary visual cortex (V1), although the approach certainly may be used for studying other areas as well. Note, however, that the spatial and temporal averaging may be different in different brain areas, especially since the vasculature seems to be specialized in particular brain areas (e.g., in V1) (Zheng et al., 1991). This article reports fMRI data from experiments designed to test the linear transform model of fMRI responses. Although these tests can not prove the correctness of the linear transform model, they might have been used to reject the model. Because the linear transform model is consistent with our data, we pro- Received Dec. 11, 1995 revised March 29, 1996 accepted April 2, 1996. This research was supported by a National Institutes of Health (NIH) postdoctoral research fellowship (IEQA455) to G.M.B., by a McDonnel-Pew cognitive neuro- science postdoctoral training grant to S.A.E., by an NIH National Center for Research Resources grant (P41 RR09784) to G.H.G., and by a National Institute of Mental Health grant (MH50228), a Stanford University Research Incentive Fund grant, and an Alfred P. Sloan Research Fellowship to D.J.H. We thank Brian Wandell for his insightful advice. Correspondence should be addressed to Geoffrey M. Boynton, Department of Psychology, Stanford University, Jordan Hall, Building 420, Stanford, CA 94305. Copyright q 1996 Society for Neuroscience 0270-6474/96/164207-15$05.00/0 The Journal of Neuroscience, July 1, 1996, 16(13):4207���4221

ceeded to estimate the temporal fMRI impulse���response function and the underlying (presumably neural) contrast���response func- tion of human V1. MATERIALS AND METHODS Data acquisition. Imaging was performed on a standard clinical GE 1.5 T Signa scanner with a 5 inch surface coil. We used a T2*-sensitive gradient-recalled echo pulse sequence (TR 75 msec, TE 40 msec, flip angle 238) with a spiral readout (Meyer et al., 1992). Inplane resolution was 2.4 3 2.4 mm, and slice thickness was 5 mm. A bite bar stabilized the subject���s head. Each experiment consisted of a series of functional images acquired at a rate of 1.5 sec per image, as the subject viewed the stimulus. Data were collected from a single slice through the calcarine sulcus in the right hemisphere of each subject, parallel to and 5 mm from the medial wall. Because data were collected over several sessions, a series of anatomical axial slices was used to localize (nearly) the same slice from one session to the next. An anatomical image was taken in the same plane as the functionals preceding each experimental session. Each fMRI scan was started by hand at the stimulus onset (to within 0.25 sec). Stimuli. Stimuli were presented using a Macintosh Quadra computer (Apple Computer, Cupertino, CA) and a Sanyo PLC300M LCD projec- tor (Sanyo, Chatsworth, CA). Stimuli were focused onto a backlit pro- jection screen inside the bore of the magnet, just above the subject���s chin. A mirror was positioned to allow the subject to view the image from the supine position. Stimuli had a mean luminance of 92 cd/m2 and subtended a visual angle of 218 vertical and 428 horizontal. The LCD projector was gamma-corrected to allow for accurate presentation of contrast stimuli. We used two types of visual stimuli that we will refer to as ���pulse��� stimuli and ���periodic��� stimuli. Both stimuli consisted of flickering (con- trast reversing with a flicker rate of 8 Hz) checkerboard patterns. The periodic stimuli contained flickering checkerboard patterns ar- ranged in slowly moving vertical bars (Fig. 2A). As the bars moved slowly to the left, the time course of stimulation in any part of the image alternated between checks and uniform gray (Fig. 2B) with a period that we refer to as the ���temporal period��� of the stimulus. Note that the temporal period depends on the drift rate of the bars, and it is very different from the flicker rate (that was always fixed at 8 Hz). Subjects viewed periodic stimuli of various contrasts and temporal periods. The ���contrast��� of the stimulus is defined, in the usual way, as the maximum intensity minus the minimum, divided by twice the mean. Twenty-four periodic stimuli were viewed by each of two subjects: the stimuli had one of four temporal periods (10, 15, 30, and 45 sec) and one of six contrasts (0, 0.032, 0.063, 0.16, 0.40, and 1). The stimulus duration was fixed at 192 sec for all conditions, so the number of periodic cycles varied with the temporal period/drift rate of the stimulus. The first 12 sec (8 fMR images) of fMRI data were discarded to avoid magnetic satura- tion effects. The remaining 180 sec (120 images) were analyzed as described below. Figure 2C depicts an example of the time course of a pulse stimulus. Each stimulus cycle began by displaying a full-field flickering checker- board pattern (contrast reversing with a flicker rate of 8 Hz) for a period of time (the ���pulse duration���). Each stimulus cycle was completed by replacing the checkerboard with uniform gray for 24 sec. Six cycles were repeated for each scan. Twenty-four pulse stimuli were viewed by each of two subjects: the stimuli had one of four pulse durations (3, 6, 12, and 24 sec) and one of four contrasts (0, 0.25, 0.5, and 1). The total duration of the scan depended on the pulse duration. Analysis. Figure 3 shows how the periodic data sets were analyzed. For each condition, 120 images were acquired over 180 sec (Fig. 3A). For a given pixel, the image intensity values from all 120 fMRI images comprise a time series of data. This time series was periodic (although noisy) with a period equal to the stimulus temporal period (Fig. 3B). We measured fMRI response as the amplitude of the sinusoid that best fit the time Figure 2. Schematic of visual stimuli used in the experiments. A, One frame of the periodic stimulus consisted of vertical bars of checkerboard patterns alternating with vertical bars of uniform gray (mean). Over time, the checkerboard patterns flickered (contrast reversing with a flicker rate of 8 Hz), and the bars drifted slowly leftward. B, The time course of a single pixel of the periodic stimulus as the bars drifted. C, The time course of pixels for the pulse stimulus. Each stimulus cycle began by displaying a full-field flickering checkerboard pattern (contrast reversing at 8 Hz) for a period of time (the pulse duration). Each stimulus cycle was completed by replacing the checkerboard with uniform gray for 24 sec. Figure 1. Diagram of the linear transform model. The output of the Retinal-V1 Pathway (Neural Response) is a nonlinear function of stimulus���contrast. fMRI signal, mediated by Hemodynamics, is a linear transform of neural activity. That is, fMRI signal is proportional to the local average neural activity, averaged over a small region of the brain and averaged over a period of time. Noise might be introduced at each stage of the process, but the effects of these individual noises on the fMRI Response can be summarized by a single noise source. 4208 J. Neurosci., July 1, 1996, 16(13):4207���4221 Boynton et al. ��� Linear Systems Analysis of fMRI in Human V1

series of each pixel. The best-fitting sinusoid was determined by comput- ing the amplitude and phase of the appropriate component (same as the stimulus temporal period) of the discrete Fourier transform of the time series. The response amplitude was computed in this way for all pixels in the calcarine sulcus. Calcarine pixels were selected by hand from an anatomical image that was taken in the same plane as the functionals preceding each experimental session (Fig. 3C). Finally, the mean and SEM of the amplitudes were used to summarize the fMRI response (Fig. 3D). An alternate measure of fMRI signal strength is the correlation of the fMRI time course with a reference waveform such as a sinusoid. Both amplitude and correlation have been used to quantify fMRI signal strength (Bandettini et al., 1993). Amplitude and correlation are closely related correlation is equal to amplitude divided by the total Fourier energy at all frequencies (Engel et al., 1996). In other words, the corre- lation measure is ���normalized��� with respect to the overall amplitude spectrum in the signal, including the frequency of interest. This means that two time courses that are scaled copies of one another (different amplitudes but with otherwise identical shapes) will have the same correlation coefficients. This is clearly undesirable when quantifying fMRI response as a function of stimulus strength. We therefore used the raw, ���unnormalized��� amplitudes. The response amplitudes were averaged over all of the pixels in the calcarine sulcus. Also, we analyzed a subset of the data by selecting the pixels that resonated most strongly with the stimulus. Averaging over the entire calcarine sulcus has the disadvantage of including many pixels with time courses that correlate poorly with the stimulus, resulting in noisier data. Selecting a region of interest based on a measure of signal strength, however, might be misleading, given that we are trying to characterize the relationship between stimulus contrast and signal strength. Fortunately, our conclusions do not depend on which method was used for selecting the region of interest (see Discussion). Periodic checks for head movements were made by applying an image motion estimation algorithm (Friston et al., 1996) to the functional image series. No significant head movements were discovered, presumably be- cause the subjects were experienced and were using a bite bar. Data from the pulse experiments were analyzed slightly differently. Pixels in the calcarine sulcus again were selected by hand from the aligned anatomical image, and the time course of the fMRI signal again was extracted for each of the selected pixels. Then the time course for each pixel was blocked with the stimulus cycle duration, and the average time course was computed, averaging across all six blocks and across all of the selected pixels. Below, we summarize the percentage of variance in the data accounted for by various models by computing the studentized residual statistic, sometimes called the ���jacknifed��� residual (Atkinson, 1988). The studen- tized residual is the error between the measured data and the predictions (from the model) relative to the SE in the data. Specifically, the studen- tized residual, r, is: r 5 1 2 Oi ~ pi 2 di!2 SEi2 Oi 1 SEi2 , (1) in which pi are the predictions, di are the data points, and SEi are the standard errors. The studentized residual is an ad hoc formula for quantifying the model fits. A large value for r can be obtained either by having a very good fit (small numerator in Eq. 1) or by having very noisy data (large denominator in Eq. 1). Even so, the studentized residual is useful for comparing different models. RESULTS We performed three empirical tests of the linear transform model of fMRI responses. First, we tested whether fMRI responses depend separably on stimulus timing and stimulus contrast. Sec- ond, we tested whether responses to long-duration stimuli can be predicted from responses to shorter duration stimuli. Third, we tested whether the noise in the fMRI data is independent of stimulus contrast and temporal period. Because the results of these tests are consistent with the linear model, we proceeded to estimate the temporal fMRI impulse���response function and the underlying (presumably neural) contrast���response function of V1. Time���contrast separability The linear transform model predicts that the fMRI response should be a separable function of stimulus contrast and pulse duration (see Appendix for a formal statement and derivation of this prediction). In other words, the linear transform model holds only if the responses to pulses of different contrasts are scaled copies of one another. The fMRI responses to the pulse stimuli for subject GMB are shown in Figure 4. Similar data were obtained from the second subject, SAE. Each curve in these figures is the time course of the fMRI response (pixel intensity) averaged across cycle repetitions and averaged across all pixels in the calcarine sulcus. The raw Figure 3. Analysis of data for periodic stimuli. A, Sequence of fMR images. B, Time course of response at a single pixel (dashed curve) superimposed with the best-fitting sinusoid. C, Aligned anatomical image with pixels in the calcarine sulcus highlighted. D, Mean and SE of the response amplitudes of the selected pixels. Boynton et al. ��� Linear Systems Analysis of fMRI in Human V1 J. Neurosci., July 1, 1996, 16(13):4207���4221 4209