Comparing the prioritisation of items and feature-dimensions in visual working memory

Selective attention can be directed not only to external sensory inputs, but also to internal sensory representations held within visual working memory (VWM). To date, this has been studied predominantly following retrospective cues directing attention to particular items, or their locations in memory. In addition to item-level attentional prioritisation, recent studies have shown that selectively attending to feature dimensions in VWM can also improve memory recall performance. However, no study to date has directly compared item-based and feature-based attention in VWM, nor their neural bases. Here, we compared the benefits of retrospective cues (retro-cues) that were directed either at a multi-feature item or at a feature-dimension that was shared between two spatially segregated items. Behavioural results revealed qualitatively similar attentional benefits in both recall accuracy and response time, but also showed that cueing benefits were larger following item cues. Concurrent EEG measurements further revealed a similar attenuation of posterior alpha oscillations following both item and feature retro-cues when compared to non-informative, neutral retro-cues. We argue that attention can act flexibly to prioritise the most relevant information – at either the item or the feature-level – to optimise ensuing memory-based task performance, and we discuss the implications of the observed commonalities and differences between item-level and feature-level prioritisation in VWM.


Introduction
attentional prioritisation of feature dimensions that are shared across multiple items 79 held in VWM.

81
In the current study, we therefore compared and contrasted behavioural and neural 82 effects of internal shifts of attention to multi-feature items and to single feature 83 dimensions that were shared across multiple items. Through the behavioural data, the 84 aim was to test whether there is a clear primacy of the object-level information in VWM.

85
If the representational format in VWM organises items as integrated objects, this should 86 also be the primary level at which attention can operate. Accordingly, benefits from 87 item-directing retro-cues should be substantially larger. If, however, attention has 88 similar access to multiple levels of information in VWM, then retro-cueing benefits for 89 feature dimensions and individual items may be similar. By recording EEG and 90 measuring alpha oscillations, we further tested whether a similar alpha modulation 91 occurs when attention is directed to a cued item or to a visual feature dimension that is 92 distributed across multiple items held in VWM.

94
To address these questions, we used a task in which participants were presented with 95 two Gabor gratings, each of which contained both colour and orientation information. 96 On half of the blocks, participants were presented with an item-directing retro-cue and 97 on the other half with a feature-dimension-directing retro-cue. Both blocks contained 98 neutral (uninformative) retro-cues, against which we compared the effects of both types 99 of informative retro-cues. We observed qualitatively comparable retro-cueing effects, 100 though benefits following item cues were larger. Both effects were accompanied by 101 similar alpha attenuation following the cues, and both item-and feature-level benefits 102 on behaviour were highly correlated across participants. 103 104 Methods 105 Participants 106 The study was approved by the Central University Research Ethics Committee of the 107 University of Oxford. Thirty-two healthy volunteers (19 female; mean age 28.3; range 18-108 35) took part. Participants had normal or corrected-to-normal vision and were not 109 colour blind. Participants provided written informed consent before participating in the 110 study and were paid £15 per hour. Data from two participants were excluded from 111 analysis, one for terminating the experiment early and the other due to hardware failure.

113
Experimental set-up & stimuli 114 Participants were seated in front of a 23-inch monitor (1920 × 1080, 100 Hz). Stimuli were 115 generated using Psychophysics Toolbox version 3.0.11 (Brainard, 1997) in MATLAB 2014b 116 (MathWorks, Natick, MA). Head position was set at 90 cm from the monitor, and 117 participants used a chinrest. The stimuli consisted of luminance-defined sinusoidal 118 Gabor gratings generated in MATLAB 2014b. Fourty-eight evenly spaced colours were 119 drawn from a circle in CIE L*a*b colour space (center at L = 54, a = 18, b = -8, radius 120 =59). Gratings were presented using one of 48 different orientations (3.75 to 180 degrees 121 in steps of 3.75) and 48 different colours.

123
Task & design 124 Participants performed a visual working memory task (Figure 1) in which they were 125 asked to reproduce the colour or orientation of one out of two memory items at the end 126 of a memory delay of 2.3 seconds. At the start of each trial, two Gabor stimuli with a 127 radius of 2.2 degrees positioned left and right from fixation (centred 3.1 degrees of visual 128 angle) were presented simultaneously for 300 ms. Participants were instructed to 129 remember the colour and the orientation of both items. At the end of the trial, they 130 were probed to report the orientation or the colour of one of the items. The to-be-131 reported feature was indicated with the probe circle that was either a colour wheel 132 (colour report) or a white wheel (orientation report), while the to-be-reported item was 133 indicated by the location of the probe circle (left/right, corresponding to the original 134 location of the probed memory item). Orientation and colour values varied 135 independently between the two items, with the constraint that no two equal 136 orientations or colours were presented on the same trial. Colours and orientations were 137 counterbalanced so that each was presented equally often across trials.

139
Two events occurred during the memory delay: first a retro-cue appeared, which could 140 provide information about the item or feature dimension that would be probed. Second, 141 700 ms after the retro-cue offset, an irrelevant 'distractor' stimulus was presented that 142 contained either colour or orientation information (Figure 1 for examples). about what item would be probed. We thus cued either a single item that contained two 149 relevant features, or a single feature-dimension that was shared between two relevant 150 items. When informative, the retro-cue was always 100% valid. Both runs also contained 151 50% non-informative neutral cues ('X') that provided no information about what item 152 or feature would be probed at the end of the trial. Participants were encouraged to use 153 the informative retro-cues to select the relevant item or feature dimension. 154 155 Following another fixation period of 700 ms (after the retro-cue), a bilateral distractor 156 was presented for 100 ms. The distractor consisted of either a single colour or a black-157 and-white oriented Gabor grating. Sixteen colours and 16 orientations were used (11.25 158 to 180 degrees in steps of 11.25 degrees). These varied randomly from the 159 orientation/colour of the items in the memory array. However, we ensured that the 160 three types of cues (C/O/X or L/R/X) all contained the same range of 16 distractor 161 features. Out of these 16 distractor features, we randomly assigned 8 as colour features 162 and 8 as orientation features.

164
After another fixation period of 700 ms (after the distractor), the colour-wheel or a 165 white-circle probe appeared. To keep the orientation and colour recall as similar as 166 possible, we presented the colours at a fixed position on the colour wheel. Participants 167 were instructed to respond as accurately as possible by using the 'J' and 'F' key to rotate 168 the probe counter-clockwise and clockwise, respectively. Participants were instructed 169 to use their left index finger to press the 'J' key and the right index finger to press the 'F' 170 key. Although there was no explicit time limit for the response time, we logged reaction 171 times as the time between the onset of the probe and the first button press that initiated 172 the 'dial-up' report. Reaction time therefore serves as a proxy for the time it took 173 participants to access the relevant memory information before commencing their 174 reproduction report.

182
Behavioural analysis 183 We computed the error for each trial for each participant by subtracting the target 184 orientation or colour (in radians around the colour circle in CIE L*a*B space) from the 185 probe response. All error scores were mapped onto a -½p to ½p space. All trials for 186 which the reaction time was more than 4 standard deviations above a participant's mean 187 decision time were discarded (0.9% ± 0.3%). To calculate the retro-cue benefit, we 188 subtracted the absolute error on cued trials from the absolute error on neutral trials. In 189 all our analyses, we only compared trials of one retro-cueing condition with neutral 190 trials from the same blocks.

192
A mixture model was fitted separately for each retro-cueing condition and respective 193 neutral condition, modelling target response rate, guess rate, swap responses, and 194 precision to the error data of each subject ( When comparing more than two conditions, we applied a repeated-measures analysis 203 of variance (rmANOVA) and report h 2 as a measure of effect size. When evaluating 204 retro-cueing benefits, we applied dependent samples t-test, comparing informative vs 205 neutral cues, as well as the cueing effects between item and feature retro-cues. When 206 the assumption of normality was violated we instead applied a Wilcoxon signed-rank 207 test. We report Cohen's d as a measure of effect size for parametric tests and matched 208 rank biserial correlation for non-parametric effect size. For evaluation we two-sided 209 tests with a critical alpha value of 0.05. 210 211 EEG acquisition 212 EEG data were collected using Synamps amplifiers and Neuroscan software 213 (Compumedics). We used a 61 Ag/AgCl sintered electrodes (EasyCap, Herrsching, 214 Germany), laid out according to the international 10-10 system, with mastoids behind 215 the left and right ear. The left mastoid was used as an active reference during the 216 recordings. Offline, an average-mastoids reference was derived using the left and right 217 mastoids. The ground electrode was placed on the left arm above the elbow. Horizontal 218 EOG was measured using lateral electrodes next to both eyes while vertical EOG was 219 measured above and below the left eye. Data were sampled at 1000 Hz, and stored for 220 subsequent analysis.

228
EEG data were down sampled to 200 Hz to reduce computational demands and storage 229 space (ft_resampledata).

231
Next, EEG data were further de-noised using Independent Component Analysis (ICA; 232 ft_componentalanalysis) applying the FastICA algorithm (Hyvärinen, 1999)  To characterise the onset of alpha attenuation after the retro-cue, we extracted the time 261 course of 7-12 Hz power modulation (in the specified informative vs. neutral cue 262 contrast) and focused on the 0-1000 ms period post retro-cue onset. On these data, we 263 then identified the earliest timepoint in which the power modulation reached half of its 264 minimal value for each condition. This latency was used as a measure to compare neural 265 modulation by feature retro-cues and item retro-cues.

267
To depict the topography of the power modulations analysed in the predefined set of 268 posterior electrodes (depicted in Figure 4) Topographies were intended solely to portray the nature of the modulation and were 276 not subjected to further statistical testing.

278
Results 279 Figure 2A shows behavioural performance as a function of experimental condition 280 (collapsed over distractor type, as this did not yield consistent results as discussed 281 below). To analyse the effects of item and feature-dimension retro-cues, we quantified 282 retro-cueing benefits as the difference between the trials with informative and neutral 283 retro-cues ( Figure 2B).

285
To quantify formally the effects of retro-cue informativeness (valid or neutral) and 286 retro-cue block type (item retro-cue block or feature retro-cue block) we used a 2 x 2 287 rmANOVA. We ran this separately for RT and response error, and separately for both 288 colour and orientation recall reports. We observed a significant main effect of retro For completeness, we also considered the third factor 'distractor congruence' (i.e., when 305 the distractor contained the same or the other feature dimension as the to-be-recalled 306 memory feature), but found no systematic effects of distractor congruence across our 307 four dependent variables, nor interactions with the factors of interest -see 308 supplementary table 1.

310
In the following, we describe in more detail the item-and feature retro-cueing effects 311 of interest, in accordance with the data presented in Figure 2.

313
For orientation recall reports, participants significantly benefitted from item retro-cues. Benefits of item-based and feature-based retro-cueing showed strong positive 333 correlations across individuals for both colour and orientation reports ( Figure 2C). For 334 orientation reports, we found significant correlations between retro-cueing benefits 335 following item cues and feature cues for both error (r = .709; p < .001) and reaction time 336 (r = .539; p = .002). Likewise, for colour reports, we found significant correlations 337 between retro-cueing benefits following item cues and feature cues for both error (r = 338 .697; p < .001) and reaction time (r = .596; p < .001). Thus, participants who benefitted 339 most from item retro-cues also benefitted most from feature retro-cues.

351 352
Mixture modelling 353 In addition to the raw behavioural scores, we also modelled sources of error using a 354 mixture model (Figure 3AB; Bays et al., 2009). We modelled four components 1) 355 precision, characterised by width (1/STD) of the target centred response distribution, 2) 356 proportion of target responses modelled by the gaussian centred around the target, 3) 357 proportion of random responses characterised by the height of the uniform response 358 distribution, 4) proportion of responses to the non-cued feature of the same dimension 359 as the cued feature (non-target report or 'swap' errors). Figure 3A and B show the retro-360 cueing effects (informative vs. neutral) on each of these four parameters, separately for 361 item and feature-cues (collapsed over colour and orientation reports, after fitting the 362 model for each condition separately; see Supplementary figure 1 for mixture model 363 parameters separated for colour and orientation reports). As depicted in Figure 3A,B 364 informative (vs. neutral) retro-cues significantly increased precision for item retro-cues 365 Direct comparisons between item and feature retro-cue benefits showed a significantly 373 greater reduction in the rate of swap errors by item retro-cues relative to feature retro-374 cues (Wilcoxon signed-rank test; Z29 = 95; p = 0.002; rrb = 0.591; see Figure 3AB). Effects 375 for the other three parameters were not statistically different between item and feature 376 retro-cues (all p > .10).  were presented compared to neutral trials. Hence, valid retro-cues positively influenced target response proportions 382 and negatively influenced guess rate and swap rate. The green and purple asterisks indicate significant differences 383 of, respectively, item-or feature benefits from zero (i.e., benefits following informative vs. neutral cues). Black 384 asterisks indicate significant differences between item and feature retro-cueing benefits. Error bars indicate 95% 385 confidence intervals. * p < .05, ** p < .01, *** p < .001.

387
Alpha attenuation following feature and item retro-cues 388 Figure 4 shows the time-and frequency-resolved modulations in spectral EEG power 389 in posterior electrodes following item and feature retro-cues, expressed as a difference 390 from the neutral retro-cueing condition (neutral retro-cue minus informative retro-391 cue). After both item retro-cues and feature retro-cues, we observe an attenuation of 392 alpha power starting at around 400 ms after presentation of the retro-cue (clusters all 393 conditions p < .001). The alpha attenuation in trials with informative retro-cues re-394 emerges after the distractor onset, in the window just prior to the probe. To reveal the 395 spatial layout of the significant clusters, we visualised the EEG topographies of the 396 alpha-band power at 400 to 800 ms after the retro-cue. Similar topographies were 397 associated with the later alpha modulation after the distractor and with the early 398 modulation in the higher 13-30 Hz band (topographies not depicted).

400
In addition to this 'global effect' when comparing informative to neutral retro-cues, we 401 also evaluated the difference between left and right item cues, and between colour and 402 orientation feature cues (Figure 4AB  2017), following item cues, alpha attenuation was most pronounced contralateral to the 406 memorised location of the cued item. In contrast, following feature retro-cues no clear 407 differences were observed between colour and orientation cues, which directed 408 attention to a single feature-dimension that was shared between the left and right items. 409 Finally, we found that the alpha attenuation had very similar latencies following item-410 directing and feature-directing retro-cues (t29 = 1.273; p = .213; Supplementary Figure  411 3). 412 413

424
Discussion 425 We demonstrate that both item-based and feature-based attentional prioritisation 426 during VWM maintenance decreases recall error and speeds response initiation times 427 following the probe. Hence, we replicate the finding that selective attention can work, our experimental design uniquely allowed us to compare the magnitudes of both 432 types of behavioural retro-cue benefits within a single experiment, and to correlate their 433 strengths across participants. While the item benefit was larger than feature benefit, 434 both were both highly robust. They were each evident across both colour and 435 orientation reports and in both recall accuracy and response initiation times. Moreover, 436 we found strong correlations between the benefits that followed item and feature cues, 437 and qualitatively similar neural modulations, which suggest that the two types of retro-438 cueing benefits may share similar cognitive operations and resources.

440
The notion that both retro-cueing types yield behavioural benefits that are qualitatively 441 similar was further supported by the similar retro-cueing effects on guess-rate, and 442 target-response rate parameters estimated by the mixture model. At the same time, we 443 observed that only item cues significantly enhanced precision and reduced the 444 probability of swaps (non-target responses) -the latter being the only parameter that 445 also differed significantly between item and feature retro-cue benefits. both item and feature levels, while also revealing an additional benefit when attention 463 is directed at two features of a single item (following item cues), compared to a single 464 feature across two items (following feature cues).

466
At the same time, we note that attentional benefits in behavioural performance in VWM 467 tasks need not only reflect changes in the quality of representational information. 468 Factors related to prospective task preparation may also contribute (Myers, Stokes, & 469 Nobre, 2017). Therefore, while our data provide clear evidence for the benefit of feature 470 retro-cues -which is qualitatively similar to, and correlated with, the benefit following 471 item cues -it remains possible that at least part of these benefits are due to factors other 472 than a change in the underlying mnemonic representation (and this holds for both item 473 and feature retro-cueing benefits).

475
In addition to the behavioural performance data, we also observed commonalities in the 476 neural modulation following item and feature cues; both cases showing robust alpha 477 attenuation over posterior electrodes, arising around the same time, with a similar 478 magnitude. The neural responses therefore provide important relevant complementary 479 data to our behavioural performance data. They provide more direct evidence for an 480 early modulation in posterior (putatively visual) brain areas following both types of 481 retro-cues; compatible with a modulation at the level of the memorised visual 482 representations. However, because we used visual retro-cues, we cannot fully rule out 483 the possibility that at least part of this modulation may be driven by differential visual 484 processing of informative vs. a neutral retro-cues per se -though we note how our 485 neutral retro-cues were designed to be similar to our informative retro-cues, ruling out 486 more obvious differences due to bottom up visual features such as retro-cue size and 487 saliency.

489
In conclusion, retro-cueing studies have typically shown that internally directed 490 attention can prioritise a subset of mnemonic representations ( reveal that such feature cues yield qualitatively similar (albeit weaker) behavioural 497 benefits and neural modulations or latency, as do item cues, and that item and feature 498 cueing benefits are correlated across individuals. We argue that retro-cues help place 499 memorised visual stimuli into a goal-oriented format, such that relevant information at 500 both the item and the feature-level can be optimised for upcoming task performance. 501 502