Coupled Maps as Tool for Modeling Human Information Processing: Issues of Readout Daan Archer* Maarten van Someren** Cees van Leeuwen*** *Laboratory for Perceptual Dynamics, BSI RIKEN, Wako-shi, Japan (e-mail: daan@juice.nl) ** Human-Computer Studies Laboratory, University of Amsterdam, the Netherlands (e-mail: maarten@science.uva.nl) *** Laboratory for Perceptual Dynamics, BSI RIKEN, Wako-shi, Japan (e-mail: ceesvl@brain.riken.jp) Abstract: In trying to establish neurally plausible models of human information processing, the connectionist movement failed because their models lacked interesting dynamics. We propose that coupled maps offer neurodynamically more plausible tools for modeling cognitive phenomena. Activity in coupled maps shows dynamic synchronization behavior that is phenomenally similar to large-scale brain activity. It is possible to tweak this behavior to enable reading-in of spatially patterned information, and to engineer mechanisms for encoding storage of these patterns. Previous work showed how these solutions lead to simple explanations of a number of phenomena observed in human visual perception, including some visual illusions and multi-stability; and in memory, such as iconic storage, slow decay, and spontaneous re-instantiation. The present work explores dynamic weight adjustment at multiple time scales simultaneously, in particular its possible utility for memory retrieval and read-out of patterned information. Keywords: Logistic maps, dynamical synchronization, neural networks, iconic storage, forgetting curves, spontaneous recall, priming, retrieval. 1. INTRODUCTION Coupled maps (CM) are systems of globally or locally coupled, continuously-valued, nonlinear difference equations, for which often a one-maximum function is chosen (tent map or logistic map). Individual unimodal nonlinear maps show a universal dynamics of regular behavior, via period doubling, to chaos, intermittency and, ultimately ergodicity, depending on the value of an oscillation parameter, for instance on A in the logistic map: xt+1 = F(xt) = Axt(1-xt); 0 ≤ A≤. Amax = 4 The return plot of a logistic map bears coarse resemblance to that of a neural mass model (Breakspear, Terry, & Friston, 2003). When coupled, these systems engage in a variety of spontaneous synchronization behaviors. The properties of uniformly, linearly coupled logistic maps of degree n: xi,t+1 = (1-C)F(xi,t) + C∑F(xj,t)/n, where j = 1,...,n; j ≠ i, have been studied extensively (e.g. Kaneko, 1983; 1990). In combination with the parameter A, coupling strength index C, with 0 ≤ C ≤ 1., determines whether the system will show regular synchronization behavior, dynamic clustering of increasing complexity, or turbulence. Over a large domain of A and C, as a rule of thumb: the larger A and smaller C values yield the more irregular behavior When Amin = 3.7 ≤ A ≤ Amax and 2 ≤ C ≤ Cmax = .265 the system resides in an intermittent regime between stable and unstable activity. The behavior of global and regularly locally coupled maps (Coupled Map Lattices, CML) are analogous in this respect (Kaneko, 1984). The activity patterns in CML show a coarse resemblance with the ongoing, dynamic synchronization characteristic of brain activity. The question, therefore, arises, whether CML models can contribute to the understanding of brain function. A related question is, whether CML can offer models of information processing that are closer to brain activity than, for instance, conventional neural networks. To establish neural network functionality within CML, it is possible to supply local values Ai and Ci, in combination with local connection weights wij. We developed such systems in a number of papers, in which we introduced highly simplified visual input functions (van Leeuwen, Steyvers, & Nooter, 1997; van Leeuwen, Verver & Brinkers 2000), network growth (Gong & van Leeuwen, 2003); Hebbian Learning (Gong & van Leeuwen, 2004; van den Berg & van Leeuwen, 2004) & memory storage (van Leeuwen & Raffone, 2001). Here we shall deal with retrieval. We will argue that retrieval is a matter of signal enhancement in part of the system, corresponding to its output units. We will illustrate how our systems provide this. 2. INPUT AND STORAGE Consider System (1-5). It consists of N variables xi [0,1] R that vary as a function of discrete time t. Each xi is updated according to logistic map (1), coupled to its neighbours according to its net-input function neti (2). with global connectivity (n-1) and periodic boundary conditions (torus). All xi were initially set to homogeneously distributed random values in the interval [0, 1]. For a rectangular array of units input was given, meaning that Ai was lowered from Amax to Amin, from iteration 1 till 124 and from 300 till 400. We observe that the stimulated units reach full synchronization, which represents a perfect image of the stimulus, at t = 61. After offset of the stimulus, the perfect image remains intact for another 65 iterations (iconic storage) until decay set in at t = 190. This decay is typically slower than exponential. This is the case, because exponential weight decay is counteracted by spontaneous piecemeal reinstantiation of the stimulus pattern (van Leeuwen & Raffone, 2001). Thus, the system retains a trace of the stimulus such that, at its second presentation, at t = 300, it takes only 25 iterations to recreate a perfect representation. We observe that this simple model already has a number of features characteristic of the experimental memory literature: a very short-lived iconic storage, slow decay after breakdown of the icon, spontaneous recall, and repetition priming. The coupling bias parameter Ci that belongs to a net-input, with 0≤ Ci ≤ Cmax, determines the likelihood that an xi will come to oscillate in synchrony with the aggregated activity of its neighbours, expressed through the weighted local field WLFi in (3). The parameters Ci and the weights wij in the WLFi vary, respectively, in the ranges [0,0.3] R and [0,1] R and are set initially to .1. The if-then-else clause in (4ab) determines the extent to which coupling bias depends on the weights in the WLF. These weights only determine coupling bias if their collective strength remains below the criterion Wcrit; otherwise it is set a-specifically to Cmax Noticeable values are Wcrit = 0, for which coupling bias is always unspecific, and Wcrit = n, for which it is always specific. To realize a memory function, the weights are updated according to (5) based on the momentary synchrony between pairs of units xi and xj, which is expressed in terms of the absolute value of the difference between them. Speed of update is regulated by a parameter G. When G = 1, the weights are fixed; within the range [0,1> R larger values of G make adaptation proceed at a slower rate, making it sensitive to synchrony patterns at increasingly larger timescales. System (1-5) needs an input operator that will enable it to receive patterned information. Given the strongly nonlinear character of Eq 1, an additive input function would be meaningless. Van Leeuwen et al., (1997; 2000) explored the utility of a simple, modulatory, input operator that modifies the local parameter Ai, depending on input signal strength. Signal strength is scaled between Amax and Amin. Without input, Ai is set to Amax and with maximum input strength to Amin. As a result, stimulated regions are more likely to synchronize, and the more so to the extent that the input is strong and consistent between connected units. Figure 1 illustrates the working of the input and memory function of System (1-5). Consider a CML of 25 by 25 units Fig. 1. Top-left: the “rectangle” input image. Middle: “blank” input. Right: activation values after 400 iterations show a perfect synchronization image of the input; meaning that the Pattern Coherence (van Leeuwen & Raffone, 2001) of this image equals the maximum value, PC = 1. Bottom graph: Pattern Coherence as a function of t, in which the rectangle input was presented during iterations 1-124 and 300-400 and blank input during the remaining ones. PC for the rectangle equals 1 for iterations 61-190 and 325-400. Note that the system keeps a perfect, iconic, representation for approx. 65 iteration after stimulus offset, and retains a trace of the stimulus afterwards, so that the second time the input arrives, it takes 25 instead of 61 iterations to reach a perfect representation. Whereas all these memory occur as a consequence of system dynamics, in the brain different memory functions are distributed across widely different areas. We could think of the G parameter as a factor in distinguishing these areas. A low value of G will make the system capable of rapid adjustments to the momentary input pattern but liable to quick forgetting, meaning it will predominantly have an iconic function; higher values of G will lead to slow adjustments, at the scale of working memory; still lower ones may represent long-term memory effects. In Figure 2, we show how areas with different memory functions can be combined into a system of cascaded CMLs. Each CML represents an area; in-between there are longrange connections that perform a transformation T. This transformation can be performed in a variety of ways. Here, we assume that communication between layers is more effective with synchronized input (Fries, 2005). Synchronization between neighbors was expressed as in (9). The larger the synchronization of two units at the source, the larger the input signal received at the destination. Fig. 2. Cascaded Coupled Map Lattices. T stands for a transformation module that mediates the transfer of information between individual CML modules, connected by a phase difference rescaling rule (6), in a 1-1 feed-forward manner. Fig 3. Two-level cascaded CML (Wcrit=8) Top: An initially perfectly synchronized CML module with static weights (G=1) is perturbed by a small input for 4 iterations. This gives rise to a transient wave of synchronization activity. Bottom: at the next-higher level layer module, in a module with adaptive weights (G=0.25), the transient lower-level activity pattern is stabilized into a period-two oscillatory output code. Figures 4-6 illustrate that the cascaded system is able to retrieve a pattern independently of how fragmented its lower level representation is. Sustained input (a black square) was given to a 3-level cascades system. In one case, Wcrit = 0 in all layers, meaning an unspecific synchronization bias throughout the system, in the other the input layer used Wcrit = 8 implying a bias against global synchrony in the input layer. As a result, whereas in Fig 5 the input layer shows a wave pattern characteristic of the stimulus, in Fig 6 the input pattern is represented in the first layer in a highly fragmented manner. Nevertheless, in the higher layers of the system, the input pattern is represented in a way that is almost identical between the two systems. 3. RETRIEVAL Whereas the model in Figure 1 shows spontaneous reinstantiation of a previous stimulation, this re-instantiation is partial and fleeting. It does not capture spontaneous retrieval of a pattern output code. The cascaded system proposed in Figure 2 enables the construction of such a code. Partial, spontaneous re-instantiations at lower levels could be picked up and accumulated in one or more of the higher levels of the system, where they provide stable output representations. An example (Figure 3) shows how a small, transient perturbation in the input layer is augmented and gives rise to a travelling wave of spreading across the system. While this pattern quickly decays and submerged in the lower-level system, the augmented pattern remains a stable presence at the next higher layer of the system. Fig. 4. Input pattern of 50 x 50 pixels consisting of a black square (Malevich, 1913) on a white ground. REFERENCES Fig. 5. Three cascaded modules (G = 1, 0.25, and 0.95, respectively) with black-square input. Left: first module: Wcrit = 0, Middle: second module, Wcrit = 0; Right: third module, Wcrit = 0. Top: t = 150, bottom: t = 151. Fig. 6. Three cascaded modules (G = 1, 0.25, and 0.95, respectively) with black-square input. Left: first module: Wcrit = 8, Middle: second module, Wcrit = 0; Right: third module, Wcrit = 0. Top: t = 150, bottom: t = 151. 6. CONCLUSIONS Patterned information encoding, storage, and retrieval is possible in cascaded CML with different modules, which dynamically adjust their weights at different time scales. Patterns can be retrieved from integrally synchronized or fragmented input representations. The retrieval mechanism extends earlier models of pattern encoding and storage into a complete, albeit extremely simplified, model of visual pattern memory. By putting forward such a model, we show how these processes can be embedded in a system characterized by ongoing activity of dynamic synchronization. We expect that future developments of this model will combine these elementary mechanisms with greater neural and behavioral plausibility. Breakspear M., Terry, J.R., & Friston, K.J. (2003). Modulation of excitatory synaptic coupling facilitates synchronization and complex dynamics in a biophysical model of neuronal dynamics. Network, 14, 703-732. Fries, P. (2005). A mechanism for cognitive dynamics: neuronal communication through neuronal coherence. Trends in Cognitive Science, 9, 474-480. Gong, P. & van Leeuwen, C. (2004). Evolution to a smallworld network with chaotic units. Europhysics Letters, 67, 328–333. Gong, P. & van Leeuwen, C. (2003). Emergence of scale-free network with chaotic units. Physica A, Statistical mechanics and its applications, 321, 679–688. Kaneko, K. (1983). Transition from torus to chaos accompanied by frequency lockings with symmetry breaking. Progress of Theoretical Physics, 69, 14271442. Kaneko, K. (1984). Period-doubling of kink-antikink patterns, quasiperiodicity in antiferro-like structures and spatial intermittency in coupled logistic lattice. Progress of Theoretical Physics, 72, 480-486. Kaneko, K. (1990). Clustering, coding, switching, hierarchical ordering, and control in a network of chaotic elements. Physica D, 41, 137-172. van den Berg, D. & van Leeuwen, C. (2004). Adaptive rewiring in chaotic networks renders small-world connectivity with consistent clusters. Europhysics Letters, 65, 459–464. van Leeuwen, C., & Raffone, A. (2001). Coupled map lattice models of perceptual pattern and memory trace dynamics. Cognitive Processing, 2, 67–116. van Leeuwen, C., Steyvers, M., & Nooter, M. (1997). Stability and intermittency in large-scale coupled oscillator models for perceptual segmentation. Journal of Mathematical Psychology, 41, 319–344. van Leeuwen, C., Verver, S., & Brinkers, M. (2000). Visual illusions and outline-invariance in non-stationary activity patterns. Connection Science, 12, 279–298.