Predictive remapping of attention across eye movements.
- PubMed: 21186360
Abstract
Many cells in retinotopic brain areas increase their activity when saccades (rapid eye movements) are about to bring stimuli into their receptive fields. Although previous work has attempted to look at the functional correlates of such predictive remapping, no study has explicitly tested for better attentional performance at the future retinal locations of attended targets. We found that, briefly before the eyes start moving, attention drawn to the targets of upcoming saccades also shifted to those retinal locations that the targets would cover once the eyes had moved, facilitating future movements. This suggests that presaccadic visual attention shifts serve to both improve presaccadic perceptual processing at the target locations and speed subsequent eye movements to their new postsaccadic locations. Predictive remapping of attention provides a sparse, efficient mechanism for keeping track of relevant parts of the scene when frequent rapid eye movements provoke retinal smear and temporal masking.
Predictive remapping of attention across eye movements.
20
10
N
at
ur
e
A
m
er
ic
a,
In
c.
A
ll
rig
ht
s
re
se
rv
ed
.
nature neurOSCIenCe advance online publication
a r t I C l e S
The visual system has to deal with large displacements of the image
on the retina every time the eyes move to bring potentially relevant
target objects into high-acuity foveal vision. In stark contrast with
what we see when a camera is quickly swept across a visual scene,
these retinal image shifts escape conscious perception1. More
importantly, we do not lose track of those parts of the scene that are of
current interest and may be the targets of future eye movements. The
inability to perceive changes in unattended parts of the scene, seen in
inattentional blindness and change blindness procedures2,3, indicates
that attention restricts this displacement problem to a small number
of locations4. Some hundred milliseconds before an eye movement5,6,
visual attention is focused at the upcoming target locations7–11,
shifting the activations in saccade and attention areas of the brain12.
These activations can be considered to be pointers specifying the
locations of currently attended objects, whether targets of upcoming
saccades or not, enabling both the planning of actions toward them
and enhanced processing at those locations13. We found that these
attentional pointers to saccade targets are updated by a predictive
remapping process briefly before the eyes start moving. This process
shifts attention in the direction opposite the saccade to locations that
correspond to the current targets neither in retinotopic nor in world-
centered coordinates, anticipating, before the eyes arrive, the locations
the targets will have on the retina after the saccade lands. Our results
lend strong behavioral support to the proposal that predictive remap-
ping14,15, the fact that many cells in retinotopically organized brain
areas show anticipatory responses if a pending saccade will bring
a stimulus into its receptive field (Fig. 1a), is a critical and rapid
mechanism for keeping track of the locations of attended targets as
the eyes move16.
The discovery of predictive remapping launched intense scientific
activity exploring the different brain areas and pathways involved and
revealing the requirements for this process4,17. Until now, however,
only two studies have targeted behavioral correlates of remapping18,19
and neither of these tested the appropriate locations to determine
the functional correlates of remapping (Fig. 1b and Supplementary
Fig. 1). To the best of our knowledge, this is the first study to directly
investigate the consequences of remapping of eye movement targets
on pre-saccadic perception and post-saccadic action. We adapted
the classic double-step saccade task20,21 that has been the central
procedure of the physiological studies of remapping. In this task,
observers make two consecutive eye movements to pre-specified
target locations and, critically, the vector for the eye movement to
the second target is not given by its current retinal location, but by its
updated location when the first saccade has been executed (Fig. 1a).
If this vector is pre-computed and attention is deployed to that retinal
location before the first saccade, then the second saccade will be pre-
pared even before the first lands. This procedure tests the appropriate
locations for functional remapping (Fig. 1b) and our results provide
strong evidence for two key roles of this predictive process: updat-
ing the retinal location of attended parts of the scene and facilitating
subsequent movements to them.
RESULTS
We assessed the dynamics of perceptual performance in a difficult
visual-discrimination task that examines the allocation of attention
in a stimulus array (Fig. 2a) while subjects prepare a sequence of
two saccades. We probed several locations in space at different times
following the onset of the central movement cue, which indicated
the locations of the two targets. The probe was a tilted Gabor grating,
briefly presented for 20 ms at the end of a flickering stream of vertical
gratings. This procedure allowed for high resolution of temporal
probes of visual performance, a gold standard for the measurement
1Université Paris Descartes, Laboratoire Psychologie de la Perception, Paris, France. 2New York University, Department of Psychology, New York, New York, USA.
3Ludwig-Maximilians-Universität, Department Psychologie, München, Germany. Correspondence should be addressed to M.R. (martin.rolfs@gmail.com) or
P.C. (patrick.cavanagh@parisdescartes.fr).
Received 24 September; accepted 8 November; published online 26 December 2010; doi:10.1038/nn.2711
Predictive remapping of attention across
eye movements
Martin Rolfs1,2, Donatas Jonikaitis3, Heiner Deubel3 & Patrick Cavanagh1
Many cells in retinotopic brain areas increase their activity when saccades (rapid eye movements) are about to bring stimuli into
their receptive fields. Although previous work has attempted to look at the functional correlates of such predictive remapping,
no study has explicitly tested for better attentional performance at the future retinal locations of attended targets. We found
that, briefly before the eyes start moving, attention drawn to the targets of upcoming saccades also shifted to those retinal
locations that the targets would cover once the eyes had moved, facilitating future movements. This suggests that presaccadic
visual attention shifts serve to both improve presaccadic perceptual processing at the target locations and speed subsequent eye
movements to their new postsaccadic locations. Predictive remapping of attention provides a sparse, efficient mechanism for
keeping track of relevant parts of the scene when frequent rapid eye movements provoke retinal smear and temporal masking.
20
10
N
at
ur
e
A
m
er
ic
a,
In
c.
A
ll
rig
ht
s
re
se
rv
ed
.
advance online publication nature neurOSCIenCe
a r t I C l e S
of attentional deployment. This fine-scale temporal structure was
necessary to reveal the short-lived perceptual consequences of pre-
saccadic remapping. After having executed the two successive eye
movements, observers reported the direction of the tilt (clockwise
or counterclockwise), regardless of its location. We ensured that the
perceptual task could only be solved when observers deployed atten-
tion to a particular location by adjusting the stimulus tilt in a pre-test
such that observers were 82% correct at the two target locations for
probes presented 150 ms after the movement cue, ~100 ms before the
saccade. On each trial (each observer ran a minimum of 3,000 trials),
we probed one of four different locations (Fig. 2a): the first saccade
target, the second saccade target, its remapped location or its opposite
location, representing a neutral control location. Note that the
remapped location of the second saccade target corresponds to the
retinal position that the second saccade target will have only following
the first saccade. It does not match either the spatial or retinal location
of the second target before the saccade.
We plotted the average performance of nine observers as a func-
tion of the timing of the probe presentation relative to the first
saccade, superimposed for all four probe locations tested (Fig. 2b
and Supplementary Fig. 2). We found the expected advantage in
discrimination performance at both the first and second saccade
target locations9–11 increasing from around 150 ms before the first
saccade5,6, with a somewhat more shallow slope for the advantage at
the second saccade target. We also found a marked increase in per-
formance at the remapped location for the second target, emerging
a bFixation
1 2
3 4
Fovea
Planning two movements
After the first movementJust before first movemen t
Saccades
planned
Saccade
target
Saccade
target
Attended
second
target
Reversed remapping Functional remappin g
Reversed
remapping
Fovea Fovea
Cue location Cue location
Probe
location
Imminent
saccade
Remapped
location
Remapping
vector
Next saccade
location already
prepared
Retinal location of probe
Retinal location of probe
Unrelated to spatial location of cue Aligned with spatial location of cue
T
im
e
Cue
location
Probe
location
before the
saccade
Saccade
Retinal
position
of probed
location
after the
saccade
Functional
remapping
Probe
location
Figure 1 Predictive remapping across eye movements. (a) If two saccades are planned, first from the red to the blue kite and then to the kite handles
visible near the surfer’s left elbow, the second target (red circle) is attended in parallel to the first9–11. Remapping triggers a predictive activation of
cells responding to the future retinotopic location of the second target, offset from its current location in the direction opposite the saccade vector
(black circle)16. We found that this predictive activation was accompanied by an attention shift to that retinotopic location, specifying the location for
the subsequent saccade. (b) The functional direction of remapping. Two previous studies have targeted behavioral correlates of remapping18,19, but
actually examined a reversal of remapping that has no functional correlate (see also Supplementary Fig. 1). In these studies, the effect of a spatial
cue18 (or, equivalently, an adaptor19) on subsequent pre-saccadic tests was assessed at a location offset from the cue location in the same direction
as the saccade vector (middle left). This location is the opposite of the actual remapped location (middle right) and corresponds instead to the future
world-centered location of the cue’s current retinal location. After the saccade, this reversed remapped location covers retinotopic cortex that is far from
the spatial location of the cue.
Probe time relative to saccade (ms)
P
er
for
m
an
ce
(% c
or
re
ct
)
–150 –100 –50 0
50
60
70
80
90
Chance level
Time
20 ms
Probe
Distr.
Probe
stream
Distractor
stream
First saccade
target
Second saccade
target
Remapped
location
Control
ba
1º
Second target
First target
Remapped
Control
Mask Stim.
Mask Stim.
Figure 2 Predictive remapping of attention in the
double-step task. (a) Stimulus layout. Six stimuli,
arranged in a regular hexagon, displayed a
flickering stream of grating-mask pairs. Following
a central movement cue, subjects quickly made
two eye movements, the first one left or right
(here, right), the second one up or down (here,
up). One of the six gratings changed orientation
(probe stream; here at remapped location)
50–400 ms after the movement cue, whereas
all others remained vertical (distractor streams).
After the eye movements, subjects reported
the direction of tilt that they had seen (\ or /),
regardless of its location. Using performance
in this task, we measured the deployment of
attention at four locations (dashed frames) during
the latency of the first saccade. (b) Performance
as a function of probe offset relative to the
saccade, superimposed for the probe locations
tested. Error bars represent s.e.m.
20
10
N
at
ur
e
A
m
er
ic
a,
In
c.
A
ll
rig
ht
s
re
se
rv
ed
.
nature neurOSCIenCe advance online publication
a r t I C l e S
just 75 ms before the saccade. This benefit reached a magnitude
comparable to that observed at the second saccade target itself
and all nine observers showed it consistently (analyzed at a reso-
lution of 75 ms to counteract the additional noise). In fact, across
observers, the performance at the remapped location in the last 75 ms
preceding the saccade correlated significantly with the perform-
ance at the second saccade target in the same time window (r = 0.91,
P < 0.001), suggesting that the two allocations are strongly linked: an
observer who successfully allocates attention to the second target also
allocates substantial attention to its predicted post-saccadic location.
Note that before the saccade, when this benefit is seen, the remapped
location does not correspond to the second saccade target location
in either retinotopic or world-centered coordinates. Contrary to the
time course of perceptual facilitation at the second saccade target, per-
formance at the remapped location revealed a drop almost to chance
level in the time between 125 to 100 ms before the saccade (while
attention is allocated to the saccade targets), excluding the possibil-
ity that the pre-saccadic enhancement is a general attentional cuing
effect resulting from the movement cue. The stable performance at the
control location indicates that the observed benefits do not represent
a spatially nonspecific increase in performance.
The remapped location for the second target corresponds to the
position this target will have on the retina following the first saccade.
Activity in the saccade control regions of the brain is required at
this location to send the second saccade to its target once the first
saccade has landed. In the absence of this remapped activity, the
second target could be rediscovered following the first saccade
(assuming it is still present); however, if its location has been suc-
cessfully remapped to the appropriate location, the second saccade
should be ready to execute as soon as the first saccade lands. Indeed,
we estimated that the second saccade had a minimum latency benefit
of 19.2 ± 14.8 ms (corresponding to a 9.2 ± 7.1% difference in second
saccade latency; mean ± 95% confidence interval) when there was
evidence that an observer had successfully shifted his or her atten-
tion to the remapped location (for example, correctly identified the
probe stimulus orientation at that location; see Online Methods).
This effect was most pronounced just before the saccade; taking only
trials when observers correctly identified a probe presented in the
last 25 ms before the saccade (rather than the whole pre-saccadic
period as in the main analysis), the conditional benefit was 28.3 ±
23.0 ms. Thus, an attention shift to the remapped location before the
first saccade was associated with a speeded execution of the second
saccade or, equivalently, the preprogramming of the second saccade
was associated with a deployment of attention to the remapped loca-
tion before the eyes moved to the first target. A similar speeding of
the second saccade was also seen for trials in which observers success-
fully identified the target at the pre-saccadic location of the second
target (9.9 ± 6.6 ms), but not at all seen contingent on performance at
other locations (first target, −10.3 ± 7.1 ms; control location, −29.2 ±
32.7 ms). As previously mentioned, attention must be allocated first
to the target before it can be transferred to the remapped location.
The remapped location cannot be computed without first localizing
its current location.
Although the specificity of these effects is marked and consist-
ent with the remapping of attention to the future location of targets
on the retina, we have to rule out two alternative explanations. The
benefit at the remapped location may have arisen simply from an
attentional spread to locations adjacent to the saccade targets or a stra-
tegic deployment of attention to the cued side of the display. In two
separate control experiments, we ruled out both. To test for the spatial
extent of attentional benefits around saccade targets, we repeated the
double-step experiment, but, in addition to testing the saccade target
locations as well as the remapped location of the second saccade tar-
get, we also examined visual performance at a new control location,
the one adjacent to the first saccade target that was not the target
of the second saccade. If attentional benefits extend around saccade
targets, this control location should also show a change in performance
across time, as it is next to the first saccade goal. We found that it did
not. Performance at the first saccade target increased strongly across
time (Fig. 3 and Supplementary Fig. 3), starting more than 150 ms
before the first saccade, while it remained consistently low across
that whole interval at the adjacent location controlling for attentional
spread. Performance at the second saccade target location also showed
a strong increase (Fig. 3) and we again found a strong performance
increase occurring about 100 ms before the first saccade is executed
at the remapped location of the second target.
To exclude the possibility that the local performance increase at the
remapped location resulted from a strategic deployment of attention to
the pre-cued side of the display, we ran a second control experiment.
In this single-cue version of the double-step experiment (Fig. 4a),
a central cue indicated the target of the first saccade (any of the
six locations in the display), whereas the second target was always
the next location in the clockwise direction. Otherwise, the experi-
ment was identical to the control experiment described above. Again,
performance at the first saccade target increased strongly across time
(Fig. 4b and Supplementary Fig. 4), starting ~150 ms before the first
saccade, whereas it remained constantly low throughout that whole
period at the adjacent location controlling for attentional spread.
Performance at the second saccade target location also showed a
strong increase (Fig. 4b). At the remapped location of the second
target, we again found a substantial performance increase, occurring
just 50 ms before the first saccade is executed. The spatiotemporal
Chance level
Probe time relative to saccade (ms)
P
er
fo
rm
an
ce
(
%
c
or
re
ct
)
−200 −160 −120 −80 −40 0
50
60
70
80
90
Chance level
P
er
fo
rm
an
ce
(
%
c
or
re
ct
)
50
60
70
80
90
Second target
Remapped
First target
Spread around first target
Figure 3 Controlling for the spread of attention in the double-step task.
We repeated the double-step task in a new set of subjects, probing the
location adjacent to the first saccade target to test whether attentional
benefits extend around saccade targets, an alternative interpretation of
the effect at the remapped location. Performance is shown as a function
of probe offset relative to the saccade. Attention did not spread around
saccade targets. Instead, it shifted specifically to the remapped location
of the second saccade target. Error bars represent s.e.m.
20
10
N
at
ur
e
A
m
er
ic
a,
In
c.
A
ll
rig
ht
s
re
se
rv
ed
.
advance online publication nature neurOSCIenCe
a r t I C l e S
specificity of this effect indicates that it is
not a result of attentional spread around the
saccade target locations. Moreover, as probe
locations were balanced around the saccade targets and the move-
ment cue pointed nowhere near the remapped location, our results
can neither be explained by strategic nor cue-based deployment of
attention, which in any case may occur only early during the prepara-
tion of a saccade6.
In our first three experiments, we found that, before a saccade,
attention to a second saccade target is updated in a retinotopic frame
of reference. Using the same general procedure, we next studied the
dynamics of attention at the remapped location for a single saccade
target; this remapped location is at (or near) the fovea22, where the
target will land after the saccade. Although this situation is maximally
ecologically valid (the fovea is the future retinotopic location of every
imminent saccade target), it has not yet been tested in neurophysio-
logical studies of remapping. Probing attention at the fovea is difficult,
as the presentation of a probe stimulus at the fovea is likely to inter-
fere with the preparation of an eye movement. Using constantly
flickering stimuli, our procedure avoids this issue by masking the
transient caused by the probe (Supplementary Figs. 2–5). The dis-
play contained three horizontally aligned and equally spaced object
locations (Fig. 5a): fixation, saccade target (denoted by a line point-
ing away from fixation) and a control location at the opposite side.
We plotted the average performance of nine observers as a function
of probe time before the saccade (Fig. 5b and Supplementary Fig. 5).
Performance strongly increased at the saccade target as the eye move-
ment neared. Although performance was always very high at fixa-
tion, allowing for less variation across time, it showed the same time
course as the benefit for the remapped location in the double-step
task, with a continuous increase starting around 75 ms before the
saccade. We observed no variation of performance at the control
location, again excluding the possibility of a nonspecific pre-saccadic
performance increase.
DISCUSSION
We studied the functional correlates of predictive remapping of
targets of saccadic eye movements16. Using a sensitive perceptual
probe, we assessed the dynamics of spatial attention before a saccade
without interrupting saccade programming. The probe revealed a
robust increase in visual performance at the remapped, future retinal
locations of a sequence of movement goals, occurring less than 100 ms
before the eye started moving. This benefit was short-lived and
spatially constrained to the remapped locations and is explained by
a local attentional facilitation (rather than other well-established
changes in visual performance such as perceptual learning23, lateral or
temporal facilitation24,25). Moreover, it did not result from a general
spread of attention deployed to the saccade targets themselves. In fact,
the perceptual benefit at the remapped location was associated with a
decrease of saccade latencies to subsequent targets, emphasizing the
functional consequences of remapping of attention.
Predictive remapping has often been associated with phenomenal,
visual stability across saccades4,13,17,26. Although the proposal that
First saccade
target
Second saccade
target
Remapped
location
Spread around
first target
ba
1º
Time
24 ms
Mask Probe
Probe
stream
Distractor
stream
Probe time relative to saccade (ms)
−200 −150 −100 −50 0
50
60
70
80
90
P
er
fo
rm
an
ce
(
%
c
or
re
ct
)
50
60
70
80
90
P
er
fo
rm
an
ce
(
%
c
or
re
ct
)
Second target
Remapped
First target
Spread around first target
Chance level
Chance level
Stim.
Mask Stim. Distr.
Figure 4 Controlling for cue-based facilitation
in the double-step task. (a) In this version of the
double-step task, we used only one cue, excluding
the possibility of a cue-based attentional facilitation
at the remapped location. The single cue indicated
the first saccade target (any of the six; here upper
right); the second saccade target was always the
next stimulus in the clockwise direction (here,
right). We measured the deployment of attention at
four locations (dashed frames) during the latency of
the first saccade. Testing the location adjacent to
the first saccade target, this experiment also again
tested whether attentional benefits extend around
saccade targets, an alternative interpretation
of the effect at the remapped location (see also
Fig. 3). (b) Performance as a function of probe
offset relative to the saccade. Again, briefly before
the saccade, attention shifted specifically to the
remapped location of the second saccade target.
Error bars represent s.e.m.
Probe time relative to saccade (ms)
P
er
fo
rm
an
ce
(
%
c
or
re
ct
)
−150 −100 −50 0
50
60
70
80
90
Control
Saccade target
Fixation
FixationControl Saccade target
a
b
Chance level
Figure 5 Predictive remapping of attention to the fovea. (a) Stimulus
layout in the single-step task. We presented three stimuli, arranged at
equal distances in a line. Otherwise the procedure was identical to that
in the double-step task (Fig. 2a). Following a movement cue (here, right),
subjects quickly made an eye movement to the indicated target and
reported the direction of the tilted stimulus, regardless of its location.
(b) Performance at the probed locations as a function of probe offset
relative to the saccade. Error bars represent s.e.m.
20
10
N
at
ur
e
A
m
er
ic
a,
In
c.
A
ll
rig
ht
s
re
se
rv
ed
.
nature neurOSCIenCe advance online publication
a r t I C l e S
remapping helps maintain feature information in world-centered
coordinates19 has been challenged recently13,27, our data suggest
that pre-saccadic shifts of activations that index only the locations
of attended targets may be sufficient for visual stability. In fact, our
data provide immediate behavioral evidence for the recent proposal
that these shifting attentional pointers are the essence of trans-
saccadic remapping13, providing an efficient and sparse mechanism
to keep track of relevant locations in space as the eyes explore the
visual scene4. Based on efference copy (or corollary discharge) of the
upcoming saccade22,28,29, neurons in the retinotopic areas controlling
saccades and attention pre-activate in anticipation of a soon-to-arrive
stimulus14,15. This activation projects to the corresponding locations
in lower level visual areas30,31, alerting those parts of the retinotopic
visual cortex that will analyze targets of interest after the saccade.
The results presented here reveal two functional consequences of the
predictive remapping process, the attentional benefits at the remapped
location just before the saccade (subserving attentional facilitation of
world locations once the saccade has landed18,32) and preprogram-
ming of future action.
METhODS
Methods and any associated references are available in the online version
of the paper at http://www.nature.com/natureneuroscience/.
Note: Supplementary information is available on the Nature Neuroscience website.
AcknowledgmentS
We thank C. Buß for help with data acquisition. This work was supported
by the 7th Framework Program of the European Commission (Marie Curie
International Outgoing Fellowship 235625 awarded to M.R.), by Deutsche
Forschungsgemeinschaft (GRK 1091, as a fellowship to D.J.) and by a Chaire
d’Excellence grant to P.C.
Author contributionS
M.R., D.J., H.D. and P.C. designed the experiments. M.R. and D.J. conducted
the experiments and analyzed the data. M.R. and P.C. wrote the manuscript.
P.C. and H.D. supervised the project. All of the authors discussed the results
and commented on the manuscript.
comPeting FinAnciAl intereStS
The authors declare no competing financial interests.
Published online at http://www.nature.com/natureneuroscience/.
Reprints and permissions information is available online at http://www.nature.com/
reprintsandpermissions/.
1. Castet, E., Jeanjean, S. & Masson, G.S. Motion perception of saccade-induced
retinal translation. Proc. Natl. Acad. Sci. USA 99, 15159–15163 (2002).
2. Mack, A. & Rock, I. Inattentional Blindness (MIT Press, Cambridge, Massachusetts,
1998).
3. O’Regan, J.K., Rensink, R.A. & Clark, J.J. Change-blindness as a result of
‘mudsplashes’. Nature 398, 34 (1999).
4. Wurtz, R.H. Neuronal mechanisms of visual stability. Vision Res. 48, 2070–2089
(2008).
5. Deubel, H. The time course of presaccadic attention shifts. Psychol. Res. 72,
630–640 (2008).
6. Montagnini, A. & Castet, E. Spatiotemporal dynamics of visual attention during
saccade preparation: independence and coupling between attention and movement
planning. J. Vis. 7, 1–16 (2007).
7. Deubel, H. & Schneider, W.X. Saccade target selection and object recognition:
evidence for a common attentional mechanism. Vis. Res. 14, 1827–1837
(1996).
8. Kowler, E., Anderson, E., Dosher, B. & Blaser, E. The role of attention in the
programming of saccades. Vision Res. 35, 1897–1916 (1995).
9. Baldauf, D. & Deubel, H. Properties of attentional selection during the preparation
of sequential saccades. Exp. Brain Res. 184, 411–425 (2008).
10. Gersch, T.M., Schnitzer, B.S., Sanghvi, P.S., Dosher, B. & Kowler, E. Attentional
enhancement along the path of a sequence of saccades. Vis. Cogn. 14, 104–107
(2006).
11. Godijn, R. & Theeuwes, J. Parallel allocation of attention prior to the execution of
saccade sequences. J. Exp. Psychol. Hum. Percept. Perform. 29, 882–896
(2003).
12. Awh, E., Armstrong, K.M. & Moore, T. Visual and oculomotor selection: links, causes
and implications for spatial attention. Trends Cogn. Sci. 10, 124–130 (2006).
13. Cavanagh, P., Hunt, A.R., Afraz, A. & Rolfs, M. Visual stability based on remapping
of attention pointers. Trends Cogn. Sci. 14, 147–153 (2010).
14. Duhamel, J.-R., Colby, C.L. & Goldberg, M.E. The updating of the representation
of visual space in parietal cortex by intended eye movements. Science 255, 90–92
(1992).
15. Sommer, M.A. & Wurtz, R.H. Influence of the thalamus on spatial visual processing
in frontal cortex. Nature 444, 374–377 (2006).
16. Gottlieb, J.P., Kusunoki, M. & Goldberg, M.E. The representation of visual salience
in monkey parietal cortex. Nature 391, 481–484 (1998).
17. Berman, R. & Colby, C.L. Attention and active vision. Vision Res. 49, 1233–1248
(2009).
18. Mathôt, S. & Theeuwes, J. Evidence for the predictive remapping of visual attention.
Exp. Brain Res. 200, 117–122 (2010).
19. Melcher, D. Predictive remapping of visual features precedes saccadic eye
movements. Nat. Neurosci. 10, 903–907 (2007).
20. Becker, W. & Jürgens, R. An analysis of the saccadic system by means of double
step stimuli. Vision Res. 19, 967–983 (1979).
21. Hallett, P.E. & Lightstone, A.D. Saccadic eye movements to flashed targets. Vision
Res. 16, 107–114 (1976).
22. Collins, T., Rolfs, M., Deubel, H. & Cavanagh, P. Post-saccadic location judgments
reveal remapping of saccade targets to non-foveal locations. J. Vis. 9, 1–9
(2009).
23. Karni, A. & Sagi, D. Where practice makes perfect in texture discrimination:
evidence for primary visual-cortex plasticity. Proc. Natl. Acad. Sci. USA 88,
4966–4970 (1991).
24. Polat, U. & Sagi, D. Lateral interactions between spatial channels: suppression and
facilitation revealed by lateral masking experiments. Vision Res. 33, 993–999
(1993).
25. Solomon, J.A. The effect of spatial cues on visual sensitivity. Vision Res. 44,
1209–1216 (2004).
26. Melcher, D. & Colby, C.L. Trans-saccadic perception. Trends Cogn. Sci. 12, 466–473
(2008).
27. Knapen, T., Rolfs, M., Wexler, M. & Cavanagh, P. The reference frame of the tilt
aftereffect. J. Vis. 10, 1–13 (2010).
28. Guthrie, B.L., Porter, J.D. & Sparks, D.L. Corollary discharge provides accurate eye
position information to the oculomotor system. Science 221, 1193–1195
(1983).
29. Sommer, M.A. & Wurtz, R.H. A pathway in primate brain for internal monitoring of
movements. Science 296, 1480–1482 (2002).
30. Chen, Y. et al. Task difficulty modulates the activity of specific neuronal populations
in primary visual cortex. Nat. Neurosci. 11, 974–982 (2008).
31. Macknik, S.L. & Martinez-Conde, S. Chapter 81: The role of feedback in visual
attention and awareness. in The Cognitive Neurosciences 4th edn. (ed. Gazzaniga,
M.S.) 1165–1179 (MIT Press, Cambridge, Massachusetts, 2009).
32. Golomb, J.D., Chun, M.M. & Mazer, J.A. The native coordinate system of spatial
attention is retinotopic. J. Neurosci. 28, 10654–10662 (2008).
20
10
N
at
ur
e
A
m
er
ic
a,
In
c.
A
ll
rig
ht
s
re
se
rv
ed
.
nature neurOSCIenCe doi:10.1038/nn.2711
ONLINE METhODS
Participants. We tested nine observers (age 19–32, 2 female, 7 right-eye
dominant, 7 right-handed, 2 authors) in Paris for the two main experiments and
nine observers (age 22–28, 2 female, all right-eye dominant, 8 right-handed,
1 author) in Munich for the two double-step control experiments (7 in the single-
cue and 6 in the two-cue control). Observers had normal or corrected-to-normal
vision and gave informed consent before study participation. We conducted the
experiments in accordance with the Declaration of Helsinki.
Setup. Observers sat in a silent and dimly lit room with the head positioned on a
chin rest. We presented stimuli at a distance of 63 cm on a 22-inch Sony GDM-
F520 screen (1,050 × 1,400 pixels, 100-Hz vertical refresh rate) and recorded the
dominant eye’s gaze position using an EyeLink 2000 Desktop Mount (SR Research).
An Apple MacPro computer running MATLAB (MathWorks) with standard tool-
boxes33–35 controlled stimulus presentation and response collection.
double-step experiment. During each trial, we presented a green fixation dot
(0.3 × 0.3°) at the center of a uniform gray display. Six object locations, highlighted
by green square outlines (1.5 × 1.5°), were arranged 5° from central fixation to
form the corners of a regular hexagon. In each of these boxes, a stream of stimuli
flickered, alternating between vertical Gabor patches (2.5 cpd, 100% contrast,
random phase on each presentation) and white noise, each presented for 20 ms.
After a normally distributed random interval (M = 1,000 ms, s.d. = 300 ms, cutoff
at 3.3 s.d.), a saccade cue appeared consisting of two lines (0.2° long, one left or
right, one up or down) pointing away from the fixation dot. Participants per-
formed this saccade task as quickly and accurately as possible. One of the Gabors
changed orientation 50–400 ms after the onset of the saccade cue. After this probe
presentation, all Gabor patches disappeared and noise patches flickered on and
off at 25 Hz. After finishing the saccade task, participants reported by a button
press whether the probe was tilted clockwise or counterclockwise, regardless of
its location (that is, we never asked for the probe location itself). In each trial, the
probe appeared randomly at either the first saccade goal, the second saccade goal,
the remapped location of the second saccade goal (left or right to the second sac-
cade goal) or at an irrelevant location (same distance from the first saccade goal
as remapped location, but in the other direction). If participants failed to look at
both target locations within 1,500 ms, we gave a visual feedback and the trial was
repeated later in the block. There was no feedback for the perceptual task.
Participants ran a minimum of 3,000 trials in 6 1-h sessions. Before each ses-
sion, we obtained three 82% orientation-discrimination thresholds for probe
patches presented in the upper (25.1 ± 8.8°, M ± s.d. of tilt at threshold, across
participants), middle (16.7 ± 5.3°) and lower (25.0 ± 6.8°) parts of the visual field
using interleaved QUEST staircases36 in the same task. We presented probes
only at the saccade target locations and in a time window of 150–200 ms after
saccade cue onset and provided auditory feedback on performance in the per-
ceptual task.
double-step control experiments. Designed to control for attentional spread
around saccade targets as an explanation of our remapping effect, both the
two-cue and the single-cue control experiments were identical to the double-
step experiment except for the following differences. We presented stimuli at
a distance of 70 cm on a 22-inch Lacie Electron 22 Blue screen (1,024 × 1,280
pixels, 85-Hz vertical refresh rate) and recorded eye movements using an
EyeLink 1000 tower mount. We presented probes either at the first saccade
target, the second saccade target, the remapped location of the second target
or at a control location, the location adjacent to the first saccade target, but
opposite the second target. As a result of the different refresh rate, stimuli
in the flickering streams changed at 21.5 Hz; hence, the probe duration was 23.5 ms.
In addition, in the single-cue control, a single cue indicated the first saccade goal
(any of the six stimulus locations) and the second saccade goal was always the
next target in the clockwise direction.
Participants ran a minimum of 1,920 trials in 4 1-h sessions in the single-cue
control and a minimum of 3,000 trials in 6 1-h sessions in the two-cue control.
In the two-cue control, orientation discrimination thresholds for probe patches
presented in the upper, middle and lower parts of the visual field were 14.6 ± 5.4°,
13.5 ± 5.1° and 14.5 ± 5.5°, respectively. In the single-cue control, we obtained
separate orientation discrimination thresholds for the first and the second saccade
target and for each of them separately for the upper (12.5 ± 7.5° at first and 18.0 ± 9.5°
at second saccade target), the middle (11.9 ± 3.7° at first and 23.2 ± 5.4° at second
target) and the lower (8.0 ± 2.1° at first and 19.3 ± 5.9° at second target) visual
field. We used discrimination thresholds obtained for the first and second saccade
targets for the control location (adjacent to first saccade target) and the remapped
location (adjacent to second saccade target), respectively.
Single-step experiment. The single-step task was different from the double-step
task only in the following ways. We used three object locations, one at the center
of the screen (fixated at trial start) and two at a horizontal distance of 6°. The
saccade cue was a 0.5° line pointing away from either the left or the right side of
the central square, denoting the saccade target. Failure to look at the target within
1,000 ms triggered a feedback and the trial was repeated later in the block.
Participants ran a minimum of 2,000 trials in 4 1-h sessions. In the pre-test,
we obtained two separate orientation discrimination thresholds, one for probes
at fixation (17.0 ± 5.7°) and one for the saccade target (13.3 ± 7.6°). We presented
the probes at fixation 150–200 ms before saccade cue onset, while this location
was still attended.
data pre-processing. We detected saccades with a velocity-based algorithm37 and
defined a response saccade as the first saccade that left a circular fixation region
and landed inside a target-centered circular region (radii of 2°). We rejected trials
with blinks, no response saccades starting 100–400 ms after saccade cue onset,
saccades larger than 1° before a response saccade, or saccades to the remapped
location (circular region with radius of 2°) by 500 ms after the response saccades.
We included a total of 23,318 trials (or 86.4%) in the double-step experiment,
a total of 16,136 trials (or 89.6%) in the two-cue control experiment, a total of
10,532 trials (or 78.4%) in the single-cue control experiment, and a total of 15,409
trials (or 85.6%) in the single-step experiment in data analyses.
data analysis. We used a permutation method38 to generate confidence
intervals, testing whether performance changed across time before a saccade
(Figs. 2b, 3b, 4b and 5b). The method is based on the idea that temporally invari-
ant variables are indistinguishable from their random permutations across time.
In an observer’s original dataset, each response (correct or incorrect) is linked to
a particular probe time. We randomly reassigned responses to the probe times
(without replacement) for each observer separately and, subsequently, computed
an average surrogate time course of performance as for the original data. We
repeated that 1,000 times and computed means and 95% confidence intervals
from these surrogate samples. If the average performance differed from the time
course of the original data, we could be confident that performance varied as a
function of time (Supplementary Figs. 2a, 3a, 4a and 5a).
estimating latency benefits contingent on high performance at a given loca-
tion. We know the distribution of second saccade latencies for trials where
an observer’s perceptual report of a probe presented at a given location was
correct, fc, or incorrect, fi. Neglecting lapses, fi contains only trials with low
perceptual performance (incorrect guesses), whereas fc contains trials with high
perceptual performance, fc|high and correct guesses fc|low. To decompose fc, we
first fitted an ex-Gaussian distribution Fi(t; µi, σi, τi) to fi. Because, by defini-
tion, Fc|low(t; µc|low , σc|low, τc|low) = Fi(t; µi, σi, τi), we fitted fc with a mixture of
p Fi(t; µi, σi, τi) and a second ex-Gaussian (1 − p) Fc|high(t; µc|high, σc|high, τc|high),
where p = pi/pc, that is, the proportion of correct trials that were guesses. The
reported latency differences between high and low performance at a given loca-
tion represent the difference between the means of the distributions, (µc|low
+ τc|low)−(µc|high + τc|high). Because of relatively low performance, few trials
were available for fitting Fc|high for the control location and the procedure did
not converge for four observers. We computed results for that condition over
the remaining five observers. Note that the four remaining observers showed
slightly longer latencies for correct trials at the control location, in agreement
with the average data. These estimates imply that attending to a stimulus and
correct performance go hand in hand, which is certainly not the case. That is,
even if the probe location was attended, presumably decreasing saccade latency,
observers were correct only on a proportion of trials. And, conversely, if the
probe location was not attended (predicting longer saccade latencies), subjects
may still have seen and correctly reported the probe. Therefore, this procedure
results in a conservative estimate of the real latency difference between high
and low performance trials.
20
10
N
at
ur
e
A
m
er
ic
a,
In
c.
A
ll
rig
ht
s
re
se
rv
ed
.
nature neurOSCIenCedoi:10.1038/nn.2711
33. Brainard, D.H. The Psychophysics Toolbox. Spat. Vis. 10, 433–436 (1997).
34. Pelli, D.G. The VideoToolbox software for visual psychophysics: transforming
numbers into movies. Spat. Vis. 10, 437–442 (1997).
35. Cornelissen, F.W., Peters, E.M. & Palmer, J. The Eyelink Toolbox: eye tracking with
MATLAB and the Psychophysics Toolbox. Behav. Res. Methods Instrum. Comput. 34,
613–617 (2002).
36. Watson, A.B. & Pelli, D.G. QUEST: a Bayesian adaptive psychometric method.
Percept. Psychophys. 33, 113–120 (1983).
37. Engbert, R. & Mergenthaler, K. Microsaccades are triggered by low retinal image
slip. Proc. Natl. Acad. Sci. USA 103, 7192–7197 (2006).
38. Rolfs, M., Engbert, R. & Kliegl, R. Crossmodal coupling of oculomotor control and
spatial attention in vision and audition. Exp. Brain Res. 166, 427–439 (2005).
Sign up today - FREE
Mendeley saves you time finding and organizing research. Learn more
- All your research in one place
- Add and import papers easily
- Access it anywhere, anytime






