Iconic memory

Does iconic memory contain the gist of multiple scenes? Three experiments were conducted.
In the first, four scenes from different basic-level categories were briefly presented
in one of two conditions: a cue or a no-cue condition. The cue condition was designed to
provide an index of the contents of iconic memory of the display. Subjects were more sensitive
to scene gist in the cue condition than in the no-cue condition. In the second, the
scenes came from the same basic-level category. We found no difference in sensitivity
between the two conditions. In the third, six scenes from different basic level categories
were presented in the visual periphery. Subjects were more sensitive to scene gist in the
cue condition. These results suggest that scene gist is contained in iconic memory even
in the visual periphery; however, iconic representations are not sufficiently detailed to distinguish
between scenes coming from the same category.

Whether or not awareness entails attention is a much debated question. Since iconic memory has been generally assumed to be attention-free, it has been considered an important piece of evidence that it does not (Koch & Tsuchiya, 2007). Therefore the question of the role of attention in iconic memory matters. Recent evidence (Persuh, Genzer, & Melara, 2012), suggests that iconic memory does depend on attention. Because of the centrality of iconic memory to this debate, we looked again at the role of attention in iconic memory using a standard whole versus partial report task of letters in a 3  2 matrix. We manipulated attention to the array by coupling it with a second task that was either easy or hard and by manipulating the probability of which task was to be performed on any given trial. When attention was maximally diverted from the matrix, participants were able to report less than a single item, confirming the prior results and supporting the conclusion that iconic memory entails attention. It is not an instance of attention-free awareness.

Does scene incongruity, (a mismatch between scene gist and a semantically incongruent object), capture attention and lead to conscious perception? We explored this question using 4 different procedures: Inattention (Experiment 1), Scene description (Experiment 2), Change detection (Experiment 3), and Iconic Memory (Experiment 4). We found no differences between scene incongruity and scene congruity in Experiments 1, 2, and 4, although in Experiment 3 change detection was faster for scenes containing an incongruent object. We offer an explanation for why the change detection results differ from the results of the other three experiments. In all four experiments, participants invariably failed to report the incongruity and routinely mis-described it by normalizing the incongruent object. None of the results supports the claim that semantic incongruity within a scene invariably captures attention and provide strong evidence of the dominant role of scene gist in determining what is perceived.

For the second time Bachmann and Aru (2016) published a brief paper criticizing both our procedure and conclusions. This time in response to our most recent paper, Mack, Erol, Clarke, and Bert (2016) which was our answer to their critique of our earlier paper (Mack, Erol, & Clarke, 2015). Both of our papers explored the question of whether attention was necessary for iconic memory (IM) and the results of both led us to conclude that it was. The procedure we used in the experiments reported in both papers was almost identical and was based on a divided attention task in which participants had to report either whether there was an oddly colored circle among 4 circles which were located in the corners of a notional square, or the letters in a 3 Â 2 letter matrix that was at the center of the array. The cue that signaled which task to perform occurred immediately after the array disappeared and in our first set of experiments in order to be sure that we were assessing IM of the letters, we compared two conditions, one in which the letter cue required a whole report (all the letters in the matrix) and the other which required only a partial report, i.e., only the 3 letters in one of the 2 rows and included a single task condition which provided the baseline against which we could assess the effects of dividing attention. We also and most importantly manipulated the probability of having to report the letters in order to manipulate attention to the letter matrix and thus to see its effect on IM. In the earlier paper the probability of having to report the letters ranged from 60% to 20% over 100 trials in separate conditions. What we found was that as the probability of having to report the letters decreased, the number of letters reported decreased as well. We attributed this to the progressive withdrawal of attention from the letters so that when the probability was only 20%, participants were able to report less than a single letter indicating to us that attention was necessary for iconic memory. Bachmann and Aru (2015) criticized our conclusion arguing that it was possible that the withdrawal of attention affected only the ability to read out the contents of IM but not IM itself, but, as we pointed out in our response to their criticism (Mack, Clarke, & Erol, 2015), it was not clear how one could ever definitively distinguish between our conclusion and theirs, but we intended to come as close as we could with a new experiment. In this new experiment (Mack et al., 2016) we used the same procedure but made the probability of having to report the letters even more extreme. It was now only 10% and we added a 101st trial in which the letters were entirely absent and while we first cue signaled the participants to report the odd circle, we then immediately cued them to report the letters so that they never had to report the odd circle on this final trial. We compared the responses in this condition with those in our control condition in which the probabilities of having to perform the circle or the letter task were reversed so that now the letter task was required on 90% of the trials, ensuring that attention would largely be focused on the letters. We reasoned that if IM is independent of attention as B & A and others have argued, participants should be equally aware that there were no letters in both the control and the experimental conditions since this should have been fully evident in the iconic representation of the array, but that is not what we found. What we found was that while all the participants were aware that there were no letters in the control condition, in the experimental condition which largely removed attention to the letters in the display, 53% of the participants were unaware of the absence of the letters and reported letters when there were none. Moreover, these participants were surprised to learn that there were none. We took and continue to take this as very strong evidence that attention was necessary for IM. Bachmann and Aru (2016), however, once again take issue with our conclusion. This time they argue that we are not measuring the effects of attention on IM but rather the effect of expectation on perception. According to them, it is expectation which is leading participants to perceive ''illusory stimuli " in the experimental condition and they assert that, ''Conscious perception is not a faithful representation of sensory data but is often influenced by prior knowledge and expectations " (p. 199). While we agree with B & A and have no doubt that expectation is playing a role here, we do not think it invalidates our conclusion. First, because the immediate cue to report the letters should have drawn attention to the iconic representation in which there were no letters, but also because 100% of the control participants, for whom the probability of having to report the letters was 90% were aware that the letters were absent on the 101st trial, despite the fact that the letter matrix had been present on all the 100 prior trials so that they too must have had a strong and clear expectation that letters would be present

The part-report advantage has been used to identify short-lived forms of visual storage (Sperling, 1960). We adopt the part-report paradigm to test whether visual memory can preserve, for a brief time, successive inputs and their temporal order. In our experiments, two successive arrays, each of 4 digits, were presented on each trial. The two arrays were spatially coincident, and each was followed by a random pattern-mask. In the part-report conditions, an auditory cue indicated whether the participant should report the first array or the second array. The results consistently showed a part-report advantage, which ranged in size from 16% to 37%. Delaying the cue by 500 ms abolished most of this advantage, in that performance was then similar to that in whole-report conditions. Subsequent experiments confirmed that the part-report superiority we measure is not achieved by (a) making eye movements that spatially displace the second array relative to the first; (b) extracting information from a single snapshot containing an integrated representation of the targets and masks; or (c) transferring a subset of material to a phonological store. We propose instead that observers have access to a limited, rapidly decaying representation of successive visual inputs stored in temporal sequence.