A Computational Model of Context Processing

To appear in Jean-Arcady Meyer, Alain Berthoz, Dario Floreano, Herbert L. Roitblat, Stewart W. Wilson, (Eds) From Animals to Animats 6: Proceedings of the 6th International Conference on the Simulation of Adaptive Behaviour, Cambridge, Mass., 2000. The MIT Press. A Computational Model of Context Processing Christian Balkenius Jan Morén Lund University Cognitive Science Kungshuset, Lundagård SE-222 22 Lund, Sweden christian.balkenius@lucs.lu.se jan.moren@lucs.lu.se Abstract A computational model of the context processing is presented. It is shown in computer simulations how a stable context representation can be learned from a dynamic sequence of attentional shifts between various stimuli in the environment. The mechanism can automatically create the required context representations, store memories of stimuli and bind them to locations. The model also shows how an explicit matching between expected and actual stimuli can be used for novelty detection. The novelty detection system is used to decide when new binding nodes should be created and when the context representation should be shifted from one context to another. The role of context in conditioning and habituation is illustrated in two simple simulations where context learning is combined with conditioning or habituation. 1. Introduction In learning theory the concept of a context is often used in an apparently unproblematic way. Traditionally, contextual stimuli are assumed to have the same properties as other stimuli with the exception that they somehow code the entire experimental situation (Mackintosh, 1983). Another way of defining contextual stimuli are as stimuli that are not manipulated by the experimenter (Donahoe and Palmer, 1994). More generally, many cognitive phenomena are regarded as being context sensitive, but very seldom is the concept of a context explained. Given the important role of context in many theories, it is surprising how little attention this concept has received. In descriptions of experiments, the contexts often enters in the same way as other stimuli, but what justification do we have for including such a hypothetical stimulus in a model of learning? While a context certainly exists around the animal, what evidence do we have that supports that it could be regarded as a single stimulus? Consider, for example, a situation where the experimental apparatus is seen as the context. It is possible that a single stimulus such as the smell of the box or the feeling of the floor is used to code the situation internally. A more likely explanation, however, is that a contextual representation reflects a combination of several stimuli. The context must be seen as a configurational stimulus. A computational account for contextual processing must address how several external stimuli can be combined into a contextual representation. We would like to suggest that a contextual representation is constructed from the sequential scanning of external stimuli and can be equated with a sequence of attentional shifts. As attention moves across a scene, a number of stimuli are perceived in sequence. When each stimulus is attended, the animal has access both to the location of the stimulus and to the stimulus itself. It is then in a position to bind the stimulus to the location and store the result in memory. By combining several stimuluslocation bindings, a contextual representation emerges over time. It is clear that the construction of a contextual representation must proceed sequentially since an animal is not able to perceive all stimuli around it simultaneously. If for nothing else, because it has to turn around to see objects behind it. A mechanism is needed that can somehow integrate several sensory impressions. Since the context is the same regardless of the order in which the environment is scanned, this mechanism must have the ability to disregard this order. However, if the order in which the scene is scanned is controlled by external events, then it is possible that the order has some significance. In this case, it would be useful if the contextual representation contained this information even if it could be ignored when not needed. Very often, the context is the location where the experiment takes place. Given that a location is defined by local or distal stimuli and that these stimuli must be attended in sequence, no difference exists between a representation of a context and that of a place. A reasonable assumption is that the same learning mechanism could be used in both cases. Assuming that stimuli in a context are bound to locations, we may consider in what form these locations are specified. There exists ample evidence that multiple coordinate systems are possible for the representation of locations. These coordinates can be egocentric, i. e. anchored in the body, for example, in the retina, the head, the mouth or arm. It is also possible that allocentric coordinates can be used that are anchored in the environment. A complicating factor is that the context itself may contribute to these coordinate representations. Whatever coordinate system is used to represent locations, the binding of two stimuli to locations in the same coordinate system will also implicitly represent the relation between these two stimuli. In summary, the concept of a context covers many areas (figure 1). For example, a context can be a place indicated by a number of landmarks. It can also be a sequence of events or actions. In most cases, of course, a context is both spatial and temporal since stimuli are usually located and must be attended in sequence. In the limiting case, a context can consist of a single event such as the presentation of stimulus some time ago. In this case, the context essentially acts as a stimulus trace. A more interesting context occurs when the learning experiment itself is the context. The stimuli for the context could even be internally generated such as thought and the like. A l 0 p B C l 2 A t0 B t1 C t2 l1 Figure 1. Two types of contexts. left: A spatial context given by three stimuli at three locations. right: A temporal context consisting of three events A, B and C. 2. The Role of Context in Learning Many types of learning are context dependent. Here we will mainly consider the role of context in habituation and conditioning. Habituation can be defined as a learning process where an animal learns to ignore a stimulus that does not predict anything of value to it. Usually this decreasing interest in a stimulus is studied through its effect on the orienting response toward the stimulus. This reaction can be operationally defined as any response that (1) is elicited by novel stimuli of any modality, and (2) habituates upon repetition of the stimulus (Gray, 1975). Figure 2a shows the basic habituation experiment. A stimulus S is shown in context CX1, and generates an orienting reaction OR. After a number of presentation, the orienting reaction disappears. The orienting reaction reappears when a new novel stimulus is presented (Thompson and Spencer, 1966). This new stimulus will make the organism more likely to attend to the original stimulus again. This situation is called dishabituation (Gray, 1975). This is shown in figure 2b. A novel stimulus N is presented together with the original stimulus S and the previously habituated orienting reaction reappears. A possible explanation for this phenomenon is that there is a direct link between the detection of novelty and the temporary shut down of the habituation system. It is important to realize that for dishabituation to occur, the habituation system must be able to react to novelty. This implies that it must learn expectations to which it can match every stimulus to detect whether it is novel or not. A possibly related fact is that a new context will also cause the orienting reaction to reappear (Gray, 1975, O’Keefe and Nadel, 1978). Since this is similar to the situation above it may be explained by the same mechanisms. However, it does not seem entirely unreasonable that a novel object in an old contexts should be distinguished from a known object in a novel context. As we will see below, the proposed model of context processing is able to function in conditioning in precisely this way. The context also influences conditioning. However, in a normal conditioning experiment, the main influence of conditioning appears in extinction. While the initial learning of a response is associated mainly with the conditioned stimulus, the extinction of the response appears to be controlled by the context (Bouton and Nelson, 1998). Figure 2d-e shows the structure of a typical acquisitionextinction experiment. Like habituation, the effect of extinction can also be temporarily removed by the presentation of a novel stimulus (Pavlov, 1927). This is called disinhibition (figure 2f) and allows the conditioned stimulus to elicit the conditioned response again. The conditioned response is also restored by a context change (figure 2g). a. Habituation CX1 + S -> OR => CX1 + S -> no OR d. Conditioning CX1 + CS + US => CX1 + CS -> CR e. Extinction CX1 + CS => CX1 + CS -> no CR b. Dishabituation CX1 + S + N -> OR c. Context Change CX2 + S -> OR f. Disinhibition CX1 + CS + N -> CR g. Context Change CX2 + CS -> CR Figure 2. Habituation and Conditioning Paradigms. CX1 and CX2 : contexts, S: stimulus, N: novel stimulus, CS: conditioned stimulus, US: unconditioned stimulus, OR: orienting reaction, CR: conditioned response. It is interesting to note that these properties are very similar to the ones described for habituation above. It suggests that both processes can be explained in terms of a contextually controlled learning system that acquires an inhibitory influence on either the innate orienting reactions or on classically conditioned responses. This requires that the context system can produce an output that can be associated with inhibition of a response. The relation between context and memory is also very interesting. On one hand, memory is often context dependent. On the other hand, a temporally extended context will act in much the same way as a memory. Donahoe and Palmer (1994) have suggested that working memory could be equated with context in such tasks as matching-tosample where a subject has to remember an object and later match it to one of several presented stimuli. The remembered stimulus has been incorporated in the context and can later control responding or attention. Another interesting relation between context and attention is that attention can sometimes be seen as contextual discrimination (Donahoe and Palmer, 1994), that is, one stimulus is attended and not the others and this is controlled by the context. Similarily, habituation can be seen as the active process of inhibiting the orienting reaction to stimuli that are of no value to the animal (Gray 1975, Balkenius, 2000). The prefrontal cortex has also been implicated in this process (Fuster 1997). It is likely that the frontal cortex receives information about the current context from the hippocampus. Working together, the hippocampus and prefrontal cortex could be responsible for the inhibition that occurs in habituation and extinction (Rolls, 1995, Fuster, 1997). Several structures in the brain have been implicated in conditioning. Especially interesting is the role of the amygdala in emotional conditioning. This structure is known to be under inhibitory control of the prefrontal cortex. It appears that the amygdala is involved in the initial learning of an emotional response while the prefrontal cortex is necessary for extinction (Rolls, 1995). Another structure that is inhibited by the prefrontal cortex is the basal ganglia (Fuster, 1997). This system in the brain may be involved with the learning of responsereward associations (Houk, Davis and Beiser, 1995) and the inhibition from frontal cortex could be used to select among different motor patterns. 3. Neural Correlates CONT reset Several brain regions have been associated with context processing and learning. This section briefly reviews the relation between the different learning processes described above and various areas of the mammalian brain. The neural structure most closely connected with contextual processing appears to be the hippocampus. The hippocampus must be intact for normal habituation and extinction (O’Keefe and Nadel, 1978). Many different roles have been assigned to the hippocampus in different theories and models. The perhaps most influential theory of the hippocampus is the cognitive map theory of O’Keefe and Nadel (1978). They suggest that the hippocampus is responsible for the mapping of the environment mainly based on environmental cues. Other suggestions include the hippocampus as a memory for sequences or events (Solomon 1979, Rawlins 1985, Olton, 1986), working memory (Olton and Samuelson, 1976) or configurational codes (Solomon, 1980). It has been suggested that the representation of a location of a stimulus and the stimulus itself that are segregated in neocortex are bound together in memory by the hippocampus (Mishkin, Ungerleider and Macko, 1983). Another function associated with the hippocampal system is the comparison between stored regularities and actual stimuli (Gray, 1995). The role of the hippocampus in contextual control of memory and learning is also well known (Hall and Pearce, 1979). The prefrontal cortex is often described as a structure whose role is to inhibit responses that are inappropriate in a certain context or situation (Shimamura, 1995). It has been argued that extinction is controlled by the inhibition from this area (Rolls 1995, Balkenius and Morén, 2000). context representation BIND reset MATCH Location recall MEM Stimulus Figure 3. A Model of Contextual Processing. BIND: stimulus location binding. MEM: context dependent memory. CONT: contextual representation. MATCH: matching between actual and recalled stimuli. 4. A Model of Contextual Processing We have seen above that a model of context processing must include a number of components. The most important being an explicit context dependent memory that stores stimuli or events that can be matched with the current perceptual state and a subsystem that constructs a representation for the current context. This section presents a model of context processing in the restricted case of a spatial context. Figure 3 shows the proposed architecture for the context processing system. It consists of four main components. These modules perform computations in a neural network like fashion, but we do not attempt to be biologically realistic. The input to each module as well as its output uses vector representations. This allows for distributed representations of stimuli and contexts and makes it easy to interface the model with other neural network based models of learning. However, in its present state, the model is best viewed as a mathematical model rather than as a neural network since some of the equations below can not directly be interpreted as local computations in a network. The external input is assumed to consist of two parts, a stimulus representation and a place representation. These two representations correspond to the content of the current focus of attention (Balkenius, 2000). The BIND system contains representations for the binding of a stimulus to a location. Every combination of a stimulus and a location can recruit a compact representation in this system. Once a binding node is activated it stays on until it is explicitly reset. A sequence of inputs of stimulus-location bindings will activate a set of nodes in BIND. Together, these nodes can represent several stimuli and their locations although only one enters the system at each time. The representation in BIND is context independent in the sense that the representation of a stimulus-location binding is independent on the context where it occurs. The MEM system has complementary properties. Each stimulus is stored directly in this subsystem and can be explicitly recalled when cued by a location. This memory is context dependent in the sense that the location to stimulus mapping depends on the current context as received from the CONT module. This implies that there can in principle exists one specific memory for each combination of a context and a location. MEM is thus essentially a heteroassociator where context and location is input and the expected stimulus at that location is output. The output from the MEM module is thus used by the MATCH module as the expected stimulus at the currently attended location. This expectation is compared with the actual stimulus at this location as represented in the input, and if a mismatch occurs the modules BIND and CONT are reset. The output from the BIND module is used by the CONT to create a representation for the current context. A context representation is a compact code for the whole set of binding nodes active in BIND. The context representation evolves in two ways. As long as the context is not reset by the MATCH module, every new BIND node that is activated will be included in the current context. This implies that the context code can be gradually learnt as the animal scans its environment. In the second case, if the MATCH module detects a mismatch and resets CONT simultaneously with the recruitment of a new BIND node, a new context will be created. Together, the mechanisms described above will develop context representations based on sequences of stimuluslocation inputs. It will also detect novel stimuli and reset the context when appropriate. 5. Formal Description This section presents a formal description of the context model and its computer implementation. The input to the context system is assumed to vary over time and is not controlled in any way by the context system itself. In this respect, the system passively processes whatever signals reaches its input structures. The input is divided into two types: locations and stimuli. Both inputs are coded as vectors to allow distributed representations. The current stimulus is represented by a vector S and the location is coded by the vector L. For computational purposes, the location is also represented by the index l of the maximum element of L. When the two vectors L and S are appended together they are called X. The state of the binding module BIND uses two vectors B and W, where B is the activity of all the binding nodes and W are the weights on the connections from the input to these nodes. The context module CONT uses the vectors C and U to represent the activity of the context nodes and the weights on the connections from BIND respectively. Finally, the MEM module uses the structure M to store the memory of each context-dependent stimulus place binding. The state of the different modules are calculated in the following way: First, the binding module activates a binding node for the current input. Second, the memory of the currently attended location is recalled and compared with the current stimulus input in the matching module. Third, if novelty is detected the context and binding nodes are reset otherwise the new context is calculated. Fourth, the activity of the context nodes are updates, and finally, working memory is updated. 5.1 Binding First the distance to each bound pattern is calculated, Di = d(Wi , X), (1) where d() is the Euclidean distance, W are the weights from the input X to the binding nodes B. The index of the best matching binding is stored in b, b = arg max Di . (2) i If Db > 0.5 then a new binding node is recruited, i. e., W new = X , (3) Bnew = 1, and If Db ≤ 0.5 then the best binding node is activated, Bb = 1. (4) In the version of the model described here, the order of the bindings is disregarded. 5.2 Matching 6. Simulation Studies In the second step, the match between the input stimulus and the expected stimulus at the current location l in the current context c is calculated, [ N = ∑ Mc,l,i − Si i ]+ , (5) where [x]+ = max(0, x). The value N expresses the novelty of the stimulus and M c,l is the memory of a stimulus at location l in the current context. 5.3. Reset or Update If N > 0 then the stimulus is novel and the context nodes Ci as well as the binding nodes Bj must be reset. For all i, Ci = 0, and, (6) 1 if i = new Bi =  0 otherwise As the current input is not reset, B new is assumed to resist being reset and is kept active. If a new binding node was created in equation (3) above, a new context representation is recruited, Unew = B. (7) Here, U c is the pattern of the active binding nodes for context c. Otherwise if N = 0, no novelty was detected and the current context is updated with the new stimuluslocation binding, for all i, Uc,i = max(Bi ,Uc,i ) (8) Equation (8) implies that a context can gradually be expanded to include more and more bindings between stimuli and locations. 5.4 Context Calculation In the next step, unless a reset has occurred, the activity of the context nodes are calculated, Ci = BUi , (9) and the current maximum context is found, c = arg max Ci (10) i 5.5 Memory Finally, the memory of the currently attended stimulus is stored, Mc,l = S . (11) This completes one cycle of the context system. This section described a number of simulations of the context model. First the basic properties of the context mechanism is demonstrated. The subsequent simulations illustrates how a context representation can be used in habituation, classical conditioning, and extinction. The aim is here only to show how a context system can interact with other learning processes. The other systems are kept at a minimal complexity to illustrate the role of context in various learning paradigms. We do not pretend to present full-fledged models of habituation or conditioning. 6.1 Context Learning Figure 4 illustrates the basic operation of the model. A contextual representation is built from a number of stimulus-location bindings. In the simulation, attention alternates between a location to the left, called LL, and a location to the right, called LR . In the first context, stimulus A is to the left and stimulus B is to the right. In the next context, both locations contains instances of stimulus B. Finally, in the last context, stimulus B is to the left and stimulus A is to the right. In the beginning of the simulation, the simulated animal first attends to stimulus A to the left. This will lead to the recruitment of a binding node coding for this conjunction. The node B0 is activated to represent this binding. An explicit memory of stimulus A at location LL is also stored in the working memory MEM. After a few time-steps the animal attends the location to the right with stimulus B which will recruit a new binding node, B1. The combination of binding node B0 and B1 will subsequently be learned by a context node CX0 which will become active and represent the current context. In the following timesteps attention shifts back and fourth between stimulus A and B which will not lead to any new internal state. In the second phase of the simulation, the context is changed by placing a second stimulus B to the left. When the simulated animal attends to this location, the stored memory of this location will no longer match the actual stimulus and novelty will be detected. This novelty will cause the context and binding representations to be reset. A new binding node will simultaneously be recruited to represent the novel stimulus and a new context node will become active. When the animal again attends to the stimulus to the right, the binding node for this stimulus will be reactivated and included in the new context. Since one of the stimuli are common to both contexts, the original context will also be partially active. This distributed context-representation allows for better generalization between similar contexts (Balkenius, 1996). Context 0 Context 1 Context 0 Context 1 Context 2 A B A B B B B B B A LL LR SA SB B0 B1 B2 B3 CX0 CX1 CX2 Nov Bind Figure 4. Simulation of context learning, novelty detection and context change. Ll and LR : location representations. SA and SB: two stimuli. B0-B3: binding nodes. CX0-CX 2: context nodes. Nov: novelty detection. Bind: recruitment of binding node. In the next phase, the context is changed back again. This will again trigger a mismatch event and the binding and context nodes will be reset. This time, however, no new binding nodes are recruited since there already exist representations for both A to the left and B to the right. Instead, the original context node will become active again. As above, the previous context will also receive some activation since it has one binding in common with the current context. The following phase shows the effect of a second context change without any learning. Finally, a new context is encountered with B to the left and A to the right. In this context, one new binding node will be recruited for representing A to the right since it has not been encountered before. The distributed context representation includes some activation of context 1 but none of context 0 since it has no bindings in common with the current context. It should be noted that context learning is an entirely passive process, it does not depend on what the animal does. Here it was assumed that the attention alternated between two locations, but does not matter whether these attentional shifts are controlled by external stimulus events or internally by an active scanning. Nor is it necessary that attention shifts in the orderly way used in this simulation. In fact, the order in which attention shifts is not used by the system. The system also ignores the time between each attentional fixation. This is a simplification since ideally the context system should represent this information. 6.2 Context in Habituation We now turn to how the context system can support context dependent habituation. It is not the aim of this section to develop a complete model of habituation. The model below is intended as an elementary example of how the context system can interact with other learning processes. The magnitude of the orienting reaction to a novel stimulus is assumed to be equal the intensity of the stimulus reduced with the total inhibition from the context system, OR = N − ∑ wi CXi , i (12) Habituation Dishabituation LL LR SA SB N OR Inhib CX0 CX1 Nov Bind Figure 5. Habituation and dishabituation by context change. where, OR is the orienting reaction, N is the intensity of the novel stimulus. CX i are the activity of the context nodes as above and wi are the inhibition from each context node to the orienting reaction. Habituation is modelled in the following way, ∆wi = α CXi OR, (13) where, α is the habituation rate. Figure 5 shows a simulation of habituation in one context and a subsequent context change. The simulation can be seen as two distinct tasks where one is the context learning task as described above and the second is a habituation task. The two tasks interact in several ways however. In the habituation phase, the simulated animal alternates between looking at the context stimuli and the novel stimulus N. Since the model does not know whether a stimiulus is part of the context or not, all stimuli will influence the processing in the context system. As can be seen in figure 5, the context system will recruit binding nodes for the novel stimulus N as well as for the context stimuli SA and SB. An orienting reaction occurs when the novel stimulus is attended. This reaction will gradually vanish as the inhibition from the context representation grows in strength. In the second phase, the context changes and as a result, the orienting reaction to stimulus N is reinstated. In this phase, however, habituation is quicker as a result of generalization from the previous context. 6.3 Context in Classical Conditioning and Extinction In the final simulation, we show how the context system can be used to control extinction in classical conditioning. A basic equation describing conditioning is the following, CR = CSu − ∑ wi CXi , (14) i where CR is the conditioned response, CS is the intensity of the conditioned stimulus and u is the excitatory connection from CS to CR. The sum is identical to the case of habituation except that the inhibition now acts on the conditioned response and not on the orienting reaction. Admittedly, this is not much of a conditioning model. As above, the conditioning equation described here is not intended as a full-fledged model. Instead, the aim is to show the operation of the context system in a classical conditioning paradigm. Acquisition of the conditioned response is modelled by the following equation, ∆u = β [US − CR], (15) where β is the learning rate. Extinction is modelled in a similar way as habituation above, ∆wi = α CXi [ CR − US ] (16) Acquistion Extinction Disinhibition LL LR SA SB CS US CR Excit Inhib CX0 CX1 Nov Bind Figure 6. Acquisition, extinction and disinhibition in classical conditioning embedded into a context learning experiment. Figure 6 shows a simulation of context in extinction. In the acquisition phase, the CS is paired with the US and this causes the excitatory connection between the CS and the CR to grow. In the second phase, the US is omitted which will cause the inhibition from the context to increase in strength according to equation (16). After a few presentations, the CR will almost disappear. Finally, the context changes which resets the context. As a consequence, the inhibition will be removed making the animal react to the CS again. As for habituation, the CR is now extinguished again, but with a faster rate than before. 7. Discussion The model presented is limited in several respects. The most severe shortcoming is that there exist cases when it will not be able to resolve which context it is currently in. For example, assume that the model first encounters a context with stimulus A both to the right and to the left, then a second with B at both locations, and finally a context with A to the left and B to the right. When the final context is presented it will first look at A then B and so on. Neither A nor B will trigger the binding system since both bindings have been seen before. When the system looks at A it will start to activate the first context, but when attention is shifted to B, it will be reset again. The same happens when the system starts to attend B and then moves on to A. In effect, the context representation will oscillate between the first and second context, and a new one will not be created. In a sense, this is correct since the system does not know that the context does not change with each attentional shift. On the other hand, it is clearly incorrect since we know that the context is the same. To resolve this situation additional assumption have to be made. For instance, a mechanism similar to that used in the ART model could be used, where a node that has been reset cannot be activated for some time (Carpenter and Grossberg, 1986). This however requires an ad hoc time constant that we did not want to include in the model. Another possibility is that a series of novelty detections are somehow accumulated in such a way that too much novelty will trigger the creation of a new context. Again, this will require an ad hoc constant defining what too much novelty is. An intriguing possibility would be to code the new context in a recursive way using the previous contexts. In this case, the novel context would be represented as a context that resembles both the first and the second context. For more complicated situations, it appears that smaller contexts should be embedded into larger contexts. It would be useful if a context could be represented at a number of levels simultaneously (Balkenius, 1996). For example, a context could represent the spatial location of an animal, the experimental situation, and the fact that a stimulus was presented two seconds ago. These are all different components of a hierarchical context. In the future, we want to develop an expectancy matching mechanism that can handle such hierarchical context. When an expectation is not met, only the part of the context that produced the expectation need to be reset. However, it is far from clear how such a mechanism should work and the experimental data on animals are sparse. Another limitation of the model is that sequential information is not represented. In the future, we want to extend the model with a temporal coding of bindings. This would allow the output of the context system to be used for sequence recognition and sequential discrimination. A final direction for future research is to incorporate more physiological data into the model. It was hinted above that the task solved by the model is similar to that ascribed to the hippocampal system in the brain, but we did not try to model this structure directly. It is interesting to note that the context system presented above, especially with the extensions discussed here, could account for many of the seemingly disparate properties of the hippocampus. If the inputs are visual landmarks, then the context nodes will have properties similar to place cells. Since they store previously attended stimuli, they can also act as a working memory. The grouping of several stimuli into a context will make the context act as a configurational code. Finally, if a sequential code is used rather than the one described above, the model could conceivably describe some the sequence effects and stimulus-trace properties sometimes associated with the hippocampus. References 8. Conclusion Donahoe, J. W., Palmer, D. C. (1994). Learning and complex behavior. Boston, MA: Allyn and Bacon. A computational model of the context processing has been presented. It was shown in compute simulations how a stable context representation could be learned from a dynamic sequence of attentional shifts between various stimuli in the environment. The mechanism can automatically create the required context representations, store memories of stimuli and bind them to locations. The model also shows how an explicit matching between expected and actual stimuli can be used for novelty detection. The novelty detection system is used to decide when new binding nodes should be created and when the context representation should be shifted from one context to another. The role of context is conditioning and habituation was shown in two simple simulations where context learning was combined with conditioning or habituation. It has also been suggested that the function of the context processing system is similar to that carried out by the hippocampus. Balkenius, C. (1996). Generalization in instrumental learning. In Maes, P., Mataric, M., Meyer, J.-A., Pollack, J., Wilson, S. W. (Eds.) From Animals to Animats 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior . Cambridge, MA: The MIT Press/Bradford Books. Balkenius, C. (2000). Attention, Habituation and Conditioning: Toward A Computational Model, submitted to Cognitive Science Quarterly. Balkenius, C., Morén, J. (1999) Dynamics of a Classical Conditioning Model. Autonomous Robots, 7 (1). Balkenius, C., Morén, J. (2000) Emotional Learning: A Computational Model of the Amygdala, Cybernetics and Systems, in press. Bouton, M. E. and Nelson, J. B. (1998). Mechanisms of feature-positive and feature negative discrimination learning in an appetitive conditioning paradigm. In N. A. Schmajuk and P. C. Holland (Eds.), Occasion Setting: Associative Learning and Cognition in Animals (pp. 69-112). Washington, DC: American Psychological Association. Carpenter, G. A., Grossberg, S. (1986). Neural dynamics of category learning and recognition: attention, memory consolidation and amnesia. In Grossberg, S. (Ed.) The adaptive brain (pp. 238-308). Amsterdam: NorthHolland. Fuster, J. M. (1997). The prefrontal cortex: anatomy, physiology, and neuropsychology of the frontal lobe. Philadelphia: Lippincott-Raven. Gray, J. A. (1975). Elements of a two-process theory of learning. London: Academic Press. Gray, J. A. (1995). A Model of the Limbic System and Basal Ganglia: Application to anxiety and schizophrenia. In Gazzaniga, M. S. (Ed.) The cognitive neurosciences (pp. 1165-1176). Cambridge, MA: MIT Press. Hall, G., Pearce, J. M. (1979). Latent inhibition of a CS during CS-US pairings. J. exp. Psychol. anim. Behav. Process., 5, 32–42. Houk, J. C., Davis, J. L., Beiser, D. G. (1995). Models of information processing in the basal ganglia. Cambridge, MA: MIT Press. Mackintosh, N. J. (1983). Conditioning and associative learning. Oxford: Oxford University Press. Mishkin, M., Ungerleider, L. G., Macko, K. A. (1983). Object vision and spatial vision: two cortical pathways. 6, 414-417. O’Keefe, J., Nadel, L. (1978). The hippocampus as a cognitive map. Oxford: Clarenon Press. Olton, D. S. (1986). Hippocampal function and memory for temporal context. In R. L. Isaacson and K. H. Pribram (Eds.), The hippocampus, Vol 4 (pp. 281-298). New York: Plenum Press. Olton, D. S., Samuelson, R. J. (1976). Remembrance of places past: spatial memory in rats. 2, 97-116. Pavlov, I. P. (1927). Conditioned reflexes. Oxford: Oxford University Press. Rawlins, J. N. P. (1985). Associations across time: The hippocampus as a temporary memory store. The Behavioral and Brain Sciences, 8, 479-96. Rolls, E. T. (1995). A theory of emotion and consciousness, and its application to understanding the neural basis of emotion. In Gazzaniga, M. S. (Ed.) The cognitive neurosciences (pp. 1091-1106). Cambridge, MA: MIT Press. Shimamura, A. P. (1995). Memory and frontal lobe function. In Gazzaniga, M. S. (Ed.) The cognitive neurosciences (pp. 803-813). Cambridge, MA: MIT Press. Solomon, P. R. (1979). Temporal versus spatial information processing theories of hippocampal function. Psychol Bull, 86, 6, 1272-9. Solomon, R. L. (1980). The opponent process theory of acquired motivation. American Psychologist, 35, 691– 712. Thompson, R. F. and Spencer, W. A. (1966). Habituation: A model phenomenon for the study of neuronal substrates of behavior. Psychological Review, 73, 16– 43.