Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Review Article
  • Published:

State-dependent computations: spatiotemporal processing in cortical networks

Nature Reviews Neuroscience volume 10, pages 113–125 (2009)Cite this article

Key Points

  • All forms of sensory processing require sense to be made of the complex spatiotemporal patterns of action potentials that are generated in our sensory organs by external stimuli.

  • Any general model of cortical processing must account for the brain's ability to process both the spatial and the temporal features of stimuli, and thus must account for spatiotemporal processing in general.

  • State-dependent classes of neural network models propose that the temporal information is inherently encoded in the state of the network.

  • The internal state can be divided into the active state, which reflects ongoing neural activity that interacts with incoming external inputs, and the hidden state, which reflects neural properties that change in time even when a network is silent (for example, short-term synaptic plasticity).

  • In vivo electrophysiological recordings show that the neural population response of a network is strongly influenced by preceding activity, and thus that networks behave in a state-dependent manner.

  • A prediction that emerges from the proposed framework is that the neural network response to a given stimulus encodes not only the current stimulus, but also previous stimuli.

Abstract

A conspicuous ability of the brain is to seamlessly assimilate and process spatial and temporal features of sensory stimuli. This ability is indispensable for the recognition of natural stimuli. Yet, a general computational framework for processing spatiotemporal stimuli remains elusive. Recent theoretical and experimental work suggests that spatiotemporal processing emerges from the interaction between incoming stimuli and the internal dynamic state of neural networks, including not only their ongoing spiking activity but also their 'hidden' neuronal states, such as short-term synaptic plasticity.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Figure 1: Trajectories of active and hidden states.
Figure 2: Active and hidden network states.
Figure 3: Discrimination of complex spatiotemporal patterns.
Figure 4: Population activity from the cat visual cortex encodes both the current and previous stimuli.

Similar content being viewed by others

References

  1. Ahissar, E. & Zacksenhouse, M. Temporal and spatial coding in the rat vibrissal system. Prog. Brain Res. 130, 75–87 (2001).

    Article  CAS  PubMed  Google Scholar 

  2. Mauk, M. D. & Buonomano, D. V. The neural basis of temporal processing. Annu. Rev. Neurosci. 27, 304–340 (2004).

    Article  CAS  Google Scholar 

  3. Drullman, R. Temporal envelope and fine structure cues for speech intelligibility. J. Acoust. Soc. Am. 97, 585–592 (1995).

    Article  CAS  PubMed  Google Scholar 

  4. Shannon, R. V., Zeng, F. G., Kamath, V., Wygonski, J. & Ekelid, M. Speech recognition with primarily temporal cues. Science 270, 303–304 (1995).

    Article  CAS  PubMed  Google Scholar 

  5. DeAngelis, G. C., Ohzawa, I. & Freeman, R. D. Spatiotemporal organization of simple-cell receptive fields in the cat's striate cortex. II. Linearity of temporal and spatial summation. J. Neurophysiol. 69, 1118–1135 (1993).

    Article  CAS  PubMed  Google Scholar 

  6. deCharms, R. C., Blake, D. T. & Merzenich, M. M. Optimizing sound features for cortical neurons. Science 280, 1439–1444 (1998).

    Article  CAS  PubMed  Google Scholar 

  7. Theunissen, F. E., Sen, K. & Doupe, A. J. Spectral-temporal receptive fields of nonlinear auditory neurons obtained using natural sounds. J. Neurosci. 20, 2315–2331 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  8. Margoliash, D. & Fortune, E. S. Temporal and harmonic combination-sensitive neurons in the zebra finch's HVc. J. Neurosci. 12, 4309–4326 (1992).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Brosch, M. & Schreiner, C. E. Sequence sensitivity of neurons in cat primary auditory cortex. Cereb. Cortex 10, 1155–1167 (2000).

    Article  CAS  PubMed  Google Scholar 

  10. Kilgard, M. P. & Merzenich, M. M. Distributed representation of spectral and temporal information in rat primary auditory cortex. Hear. Res. 134, 16–28 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Doupe, A. Song- and order-selective neurons in the songbird anterior forebrain and their emergence during vocal development. J. Neurosci. 17, 1147–1167 (1997).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Bartlett, E. L. & Wang, X. Long-lasting modulation by stimulus context in primate auditory cortex. J. Neurophysiol. 94, 83–104 (2005).

    Article  PubMed  Google Scholar 

  13. Rosenblatt, F. The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65, 386–408 (1958).

    Article  CAS  PubMed  Google Scholar 

  14. Rumelhart, D. E., McClelland, J. L. & University of California, San Diego PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructure of Cognition (MIT Press, Cambridge, Massachusetts, 1986).

    Google Scholar 

  15. Haykin, S. Neural Networks: a Comprehensive Foundation (Prentice Hall, 1999).

    Google Scholar 

  16. Elman, J. L. Finding structure in time. Cogn. Sci. 14, 179–211 (1990).

    Article  Google Scholar 

  17. Elman, J. L. & Zipser, D. Learning the hidden structure of speech. J. Acoust. Soc. Am. 83, 1615–1626 (1988).

    Article  CAS  PubMed  Google Scholar 

  18. Song, S. & Abbott, L. F. Cortical development and remapping through spike timing-dependent plasticity. Neuron 32, 339–350 (2001).

    Article  CAS  PubMed  Google Scholar 

  19. Somers, D. C., Nelson, S. B. & Sur, M. An emergent model of orientation selectivity in cat visual cortical simple cells. J. Neurosci. 15, 5448–5465 (1995).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  20. Maass, W., Natschläger, T. & Markram, H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Comput. 14, 2531–2560 (2002). Introduced the liquid-computing model and analysed its computational power theoretically and through computer simulations.

    Article  PubMed  Google Scholar 

  21. Destexhe, A. & Contreras, D. Neuronal computations with stochastic network states. Science 314, 85–90 (2006).

    Article  CAS  PubMed  Google Scholar 

  22. Buonomano, D. V. & Merzenich, M. M. Temporal information transformed into a spatial code by a neural network with realistic properties. Science 267, 1028–1030 (1995). This paper showed that the hidden states created by short-term synaptic plasticity and slow IPSPs could underlie interval discrimination and spatiotemporal processing. This was the first theoretical study to propose that temporal and spatiotemporal processing could rely on the interaction between external stimuli and both the active and the hidden internal states of a network.

    Article  CAS  PubMed  Google Scholar 

  23. Borgdorff, A. J., Poulet, J. F. A. & Petersen, C. C. H. Facilitating sensory responses in developing mouse somatosensory barrel cortex. J. Neurophysiol. 97, 2992–3003 (2007).

    Article  PubMed  Google Scholar 

  24. Broome, B. M., Jayaraman, V. & Laurent, G. Encoding and decoding of overlapping odor sequences. Neuron 51, 467–482 (2006).

    Article  CAS  PubMed  Google Scholar 

  25. Churchland, M. M., Yu, B. M., Sahani, M. & Shenoy, K. V. Techniques for extracting single-trial activity patterns from large-scale neural recordings. Curr. Opin. Neurobiol. 17, 609–618 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  26. Buonomano, D. V., Hickmott, P. W. & Merzenich, M. M. Context-sensitive synaptic plasticity and temporal-to-spatial transformations in hippocampal slices. Proc. Natl Acad. Sci. USA 94, 10403–10408 (1997).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Engineer, C. T. et al. Cortical activity patterns predict speech discrimination ability. Nature Neurosci. 11, 603–608 (2008).

    Article  CAS  PubMed  Google Scholar 

  28. Schnupp, J. W., Hall, T. M., Kokelaar, R. F. & Ahmed, B. Plasticity of temporal pattern codes for vocalization stimuli in primary auditory cortex. J. Neurosci. 26, 4785–4795 (2006).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Rabinovich, M., Huerta, R. & Laurent, G. Transient dynamics for neural processing. Science 321, 48–50 (2008). Discussed the question of whether attractors or trajectories of transient states of networks of neurons should be viewed as the principal form of neural computations in the brain. Also presented a mathematical method formalism for constructing dynamical systems with given trajectories of network states.

    Article  CAS  PubMed  Google Scholar 

  30. Zucker, R. S. Short-term synaptic plasticity. Annu. Rev. Neurosci. 12, 13–31 (1989).

    Article  CAS  PubMed  Google Scholar 

  31. Reyes, A. & Sakmann, B. Developmental switch in the short-term modification of unitary EPSPs evoked in layer 2/3 and layer 5 pyramidal neurons of rat neocortex. J. Neurosci. 19, 3827–3835 (1999).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  32. Markram, H., Wang, Y. & Tsodyks, M. Differential signaling via the same axon of neocortical pyramidal neurons. Proc. Natl Acad. Sci. USA 95, 5323–5328 (1998).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  33. Dobrunz, L. E. & Stevens, C. F. Response of hippocampal synapses to natural stimulation patterns. Neuron 22, 157–166 (1999).

    Article  CAS  PubMed  Google Scholar 

  34. Marder, C. P. & Buonomano, D. V. Differential effects of short- and long-term potentiation on cell firing in the CA1 region of the hippocampus. J. Neurosci. 23, 112–121 (2003).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Newberry, N. R. & Nicoll, R. A. A bicuculline-resistant inhibitory post-synaptic potential in rat hippocampal pyramidal cells in vitro. J. Physiol. 348, 239–254 (1984).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  36. Buonomano, D. V. & Merzenich, M. M. Net interaction between different forms of short-term synaptic plasticity and slow-IPSPs in the hippocampus and auditory cortex. J. Neurophysiol. 80, 1765–1774 (1998).

    Article  CAS  PubMed  Google Scholar 

  37. Batchelor, A. M., Madge, D. J. & Garthwaite, J. Synaptic activation of metabotropic glutamate receptors in the parallel fibre-Purkinje cell pathway in rat cerebellar slices. Neuroscience 63, 911–915 (1994).

    Article  CAS  PubMed  Google Scholar 

  38. Johnston, D. & Wu, S. M. Foundations of Cellular Neurophysiology (MIT Press, Cambridge, Massachusetts, 1995).

    Google Scholar 

  39. Berridge, M. J., Bootman, M. D. & Roderick, H. L. Calcium signalling: dynamics, homeostasis and remodelling. Nature Rev. Mol. Cell Biol. 4, 517–529 (2003).

    Article  CAS  Google Scholar 

  40. Burnashev, N. & Rozov, A. Presynaptic Ca2+ dynamics, Ca2+ buffers and synaptic efficacy. Cell Calcium 37, 489–495 (2005).

    Article  CAS  PubMed  Google Scholar 

  41. Lester, R. A. J., Clements, J. D., Westbrook, G. L. & Jahr, C. E. Channel kinetics determine the time course of NMDA receptor-mediated synaptic currents. Nature 346, 565–567 (1990).

    Article  CAS  PubMed  Google Scholar 

  42. Buonomano, D. V. Decoding temporal information: a model based on short-term synaptic plasticity. J. Neurosci. 20, 1129–1141 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Harris, K. Neural signatures of cell assembly organization. Nature Rev. Neurosci. 6, 399–407 (2005).

    Article  CAS  Google Scholar 

  44. Medina, J. F., Garcia, K. S., Nores, W. L., Taylor, N. M. & Mauk, M. D. Timing mechanisms in the cerebellum: testing predictions of a large-scale computer simulation. J. Neurosci. 20, 5516–5525 (2000).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  45. Jaeger, H. & Haas, H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304, 78–80 (2004). Demonstrated that echo-state networks — that is, implementations of liquid-state machines through recurrent networks of sigmoidal neurons with static synapses — can be trained quite easily and perform extremely well for time series prediction benchmark tasks.

    Article  CAS  PubMed  Google Scholar 

  46. Buonomano, D. V. & Mauk, M. D. Neural network model of the cerebellum: temporal discrimination and the timing of motor responses. Neural Comput. 6, 38–55 (1994). Using a computer simulation of the cerebellum, this paper demonstrated that the timing of motor responses could emerge from the interaction between the internal active state and a steady-state incoming stimulus. This paper was the first to propose that a temporal code for time could be generated as a result of the recurrent connections (negative feedback in the case of the cerebellum) in local neural networks.

    Article  Google Scholar 

  47. Jaeger, H. The “echo state” approach to analysing and training recurrent neural networks. GMD Report No. 148 (German National Research Center for Computer Science, 2001).

  48. Haeusler, S. & Maass, W. A statistical analysis of information-processing properties of lamina-specific cortical microcircuit models. Cereb. Cortex 17, 149–162 (2007).

    Article  PubMed  Google Scholar 

  49. Knüsel, P., Wyss, R., König, P. & Verschure, P. F. M. J. Decoding a temporal population code. Neural Comp. 16, 2079–2100 (2004).

    Article  Google Scholar 

  50. Maass, W., Natschläger, T. & Markram, H. Fading memory and kernel properties of generic cortical microcircuit models. J. Physiol. (Paris). 98, 315–330 (2004).

    Article  Google Scholar 

  51. Jaeger, H., Maass, W. & Principe, J. Special issue on echo state networks and liquid state machines. Neural Netw. 20, 287–289 (2007).

    Article  Google Scholar 

  52. Cover, T. M. Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition. IEEE Trans. Electronic Computers 14, 326–334 (1965).

    Article  Google Scholar 

  53. Mazor, O. & Laurent, G. Transient dynamics versus fixed points in odor representations by locust antennal lobe projection neurons. Neuron 48, 661–673 (2005). This experimental paper elegantly demonstrated that the presentation of an odour generated complex time-varying neural trajectories in a population of recurrently connected neurons in the locust. Furthermore, it showed that the downstream neurons encode information about the odour best when the trajectory is in motion as opposed to in a point attractor.

    Article  CAS  PubMed  Google Scholar 

  54. Hung., C. P., Kreiman, G., Poggio, T. & DiCarlo, J. J. Fast readout of object identity from macaque inferior temporal cortex. Science 310, 863–866 (2005).

    Article  CAS  PubMed  Google Scholar 

  55. Nikolic, D., Haeusler, S., Singer, W. & Maass, W. in Advances in Neural Information Processing Systems 19 (eds Schölkopf, B., Platt, J. & Hofmann, T.) 1041–1048 (MIT Press, 2007). Showed that the current firing activity of networks of neurons in the cat primary visual cortex contains information not only about the most recent frame of a sequence of visual stimuli, but also almost as much information about the frame before that. It demonstrated in this way that the primary visual cortex acts not as a pipeline for visual processing, but rather as a fading memory that accumulates information over time.

    Google Scholar 

  56. Duda, R. O., Hart, P. E. & Stork, D. G. Pattern Classification (Wiley, 2001).

    Google Scholar 

  57. Gutig, R. & Sompolinsky, H. The tempotron: a neuron that learns spike timing-based decisions. Nature Neurosci. 9, 420–428 (2006).

    Article  CAS  PubMed  Google Scholar 

  58. Legenstein, R. A., Pecevski, D. & Maass, W. A learning theory for reward-modulated spike-timing-dependent plasticity with an application to biofeedback. PLoS Comp. Biol. 4, e1000180 (2008).

    Article  CAS  Google Scholar 

  59. Izhikevich, E. M. Solving the distal reward problem through linkage of STDP and dopamine signaling. Cereb. Cortex 17, 2443–2452 (2007).

    Article  PubMed  Google Scholar 

  60. Sprekeler, H., Michaelis, C. & Wiskott, L. Slowness: an objective for spike-timing-dependent plasticity? PLoS Comput. Biol. 3, e112 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  61. Wiskott, L. & Sejnowski, T. J. Slow feature analysis: unsupervised learning of invariances. Neural Comput. 14, 715–770 (2002).

    Article  PubMed  Google Scholar 

  62. Vapnik, V. N. Statistical Learning Theory (Wiley, New York, 1998).

    Google Scholar 

  63. Bartlett, P. L. & Maass, W. in The Handbook of Brain Theory and Neural Networks (ed. Arbib, M. A.) 1188–1192 (MIT Press, Cambridge, 2003).

    Google Scholar 

  64. Legenstein, R. & Maass, W. Edge of chaos and prediction of computational performance for neural circuit models. Neural Netw. 20, 323–334 (2007).

    Article  PubMed  Google Scholar 

  65. van Vreeswijk, C. & Sompolinsky, H. Chaos in neuronal networks with balanced excitatory and inhibitory activity. Science 274, 1724–1726 (1996).

    Article  CAS  PubMed  Google Scholar 

  66. Banerjee, A., Series, P. & Pouget, A. Dynamical constraints on using precise spike timing to compute in recurrent cortical networks. Neural Comput. 20, 974–993 (2008).

    Article  PubMed  Google Scholar 

  67. Izhikevich, E. M. & Edelman, G. M. Large-scale model of mammalian thalamocortical systems. Proc. Natl Acad. Sci. USA 105, 3593–3598 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  68. Brunel, N. Dynamics of networks of randomly connected excitatory and inhibitory spiking neurons. J. Physiol. (Paris) 94, 445–463 (2000).

    Article  CAS  Google Scholar 

  69. Greenfield, E. & Lecar, H. Mutual information in a dilute, asymmetric neural network model. Phys. Rev. E Stat. Nonlin. Soft Matter Phys. 63, 041905 (2001).

    Article  CAS  PubMed  Google Scholar 

  70. Wessberg, J. et al. Optimizing a linear algorithm for real-time robotic control using chronic cortical ensemble recordings in monkeys. Nature 408, 361–365 (2000).

    Article  CAS  PubMed  Google Scholar 

  71. Maass, W., Joshi, P. & Sontag, E. D. Computational aspects of feedback in neural circuits. PLoS Comput. Biol. 3, e165 (2007). Showed that the computational power of liquid-state machines increases drastically when not only read-outs but also neurons in the otherwise-generic recurrent network (the 'liquid') are trained for a specific computational task. In particular, virtually any computation on time series that requires a non-fading memory (such as computations that depend on the current internal state of the network) can be implemented in this way.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  72. Skowronski, M. D. & Harris, J. G. Automatic speech recognition using a predictive echo state network classifier. Neural Netw. 20, 414–423 (2007).

    Article  PubMed  Google Scholar 

  73. Jaeger, H., Lukosevicius, M., Popovici, D. & Siewert, U. Optimization and applications of echo state networks with leaky- integrator neurons. Neural Netw. 20, 335–352 (2007).

    Article  PubMed  Google Scholar 

  74. White, O., Lee, D. & Sompolinsky, H. Short term memory in orthogonal neural networks. Physical Rev. Lett. 92, 148102 (2004).

    Article  CAS  Google Scholar 

  75. Ganguli, S., Huh, D. & Sompolinsky, H. Memory traces in dynamical systems. Proc. Natl Acad. Sci. USA 105, 18970–18975 (2008).

    Article  PubMed  PubMed Central  Google Scholar 

  76. Karmarkar, U. R. & Buonomano, D. V. Timing in the absence of clocks: encoding time in neural network states. Neuron 53, 427–438 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  77. Abbott, L. F. & Nelson, S. B. Synaptic plasticity: taming the beast. Nature Neurosci. 3, 1178–1183 (2000).

    Article  CAS  PubMed  Google Scholar 

  78. Dan, Y. & Poo, M. M. Spike timing-dependent plasticity of neural circuits. Neuron 44, 23–30 (2004).

    Article  CAS  PubMed  Google Scholar 

  79. Song, S., Sjostrom, P. J., Reigl, M., Nelson, S. & Chklovskii, D. B. Highly nonrandom feature of synaptic connectivity in local cortical circuits. PLoS Biol. 3, 508–518 (2005).

    Google Scholar 

  80. Cheetham, C. E. J., Hammond, M. S. L., Edwards, C. E. J. & Finnerty, G. T. Sensory experience alters cortical connectivity and synaptic function site specifically. J. Neurosci. 27, 3456–3465 (2007).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  81. Feldman, D. E. & Brecht, M. Map plasticity in somatosensory cortex. Science 310, 810–815 (2005).

    Article  CAS  PubMed  Google Scholar 

  82. Buonomano, D. V. & Merzenich, M. M. Cortical plasticity: from synapses to maps. Annu. Rev. Neurosci. 21, 149–186 (1998).

    Article  CAS  PubMed  Google Scholar 

  83. Gilbert, C. D., Sigman, M. & Crist, R. E. The neural basis of perceptual learning. Neuron 31, 681–697 (2001).

    Article  CAS  PubMed  Google Scholar 

  84. Karmarkar, U. R. & Dan, Y. Experience-dependent plasticity in adult visual cortex. Neuron 52, 577–585 (2006).

    Article  CAS  PubMed  Google Scholar 

  85. Buonomano, D. V. A learning rule for the emergence of stable dynamics and timing in recurrent networks. J. Neurophysiol. 94, 2275–2283 (2005).

    Article  PubMed  Google Scholar 

  86. Florian, R. V. Reinforcement learning through modulation of spike-timing-dependent synaptic plasticity. Neural Comput. 19, 1468–1502 (2007).

    Article  PubMed  Google Scholar 

  87. Durstewitz, D. & Deco, G. Computational significance of transient dynamics in cortical networks. Eur. J. Neurosci. 27, 217–227 (2008).

    Article  PubMed  Google Scholar 

  88. Hopfield, J. J. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl Acad. Sci. USA 79, 2554–2558 (1982).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  89. Little, W. A. The existence of persistent states in the brain. Math. Biosci. 19, 101–120 (1974).

    Article  Google Scholar 

  90. Fuster, J. M. & Jervey, J. P. Inferotemporal neurons distinguish and retain behaviorally relevant features of visual stimuli. Science 212, 952–955 (1981).

    Article  CAS  PubMed  Google Scholar 

  91. Goldman-Rakic, P. S. The neural basis of working memory. Neuron 14, 477–485 (1995).

    Article  CAS  PubMed  Google Scholar 

  92. Wang, X. J. Synaptic reverberation underlying mnemonic persistent activity. Trends Neurosci. 24, 455–463 (2001).

    Article  CAS  PubMed  Google Scholar 

  93. Laurent, G. Olfactory network dynamics and the coding of multidimensional signals. Nature Rev. Neurosci. 3, 884–896 (2002).

    Article  CAS  Google Scholar 

  94. Garcia, K. S. & Mauk, M. D. Pharmacological analysis of cerebellar contributions to the timing and expression of conditioned eyelid responses. Neuropharmacology 37, 471–480 (1998).

    Article  CAS  PubMed  Google Scholar 

  95. Ivry, R. B., Keele, S. W. & Diener, H. C. Dissociation of the lateral and medial cerebellum in movement timing and movement execution. Exp. Brain Res. 73, 167–180 (1988).

    Article  CAS  PubMed  Google Scholar 

  96. Perrett, S. P., Ruiz, B. P. & Mauk, M. D. Cerebellar cortex lesions disrupt learning-dependent timing of conditioned eyelid responses. J. Neurosci. 13, 1708–1718 (1993).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  97. Mauk, M. D. & Donegan, N. H. A model of Pavlovian eyelid conditioning based on the synaptic organization of the cerebellum. Learn. Mem. 3, 130–158 (1997).

    Article  Google Scholar 

  98. Yamazaki, T. & Tanaka, S. The cerebellum as a liquid state machine. Neural Netw. 20, 290–297 (2007).

    Article  PubMed  Google Scholar 

  99. Raymond, J., Lisberger, S. G. & Mauk, M. D. The cerebellum: a neuronal learning machine? Science 272, 1126–1132 (1996).

    Article  CAS  PubMed  Google Scholar 

  100. Shepherd, G. M. The Synaptic Organization of the Brain (Oxford Univ. Press, New York, 1998).

    Google Scholar 

  101. Lewicki, M. S. & Arthur, B. J. Hierarchical organization of auditory temporal context sensitivity. J. Neurosci. 16, 6987–6998 (1996).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  102. McKenna, T. M., Weinberger, N. W. & Diamond, D. M. Responses of single auditory cortical neurons to tone sequences. Brain Res. 481, 142–153 (1989).

    Article  CAS  PubMed  Google Scholar 

  103. Brosch, M., Schulz, A. & Scheich, H. Processing of sound sequences in macaque auditory cortex: response enhancement. J. Neurophysiol. 82, 1542–1559 (1999).

    Article  CAS  PubMed  Google Scholar 

  104. Yin, P., Mishkin, M., Sutter, M. L. & Fritz, J. B. Early stages of melody processing: stimulus-sequence and task-dependent neuronal activity in monkey auditory cortical fields A1 and R. J. Neurophysiol. 100, 3009–3029 (2008). This paper provides robust examples of state-dependent neural responses in the auditory cortex of monkeys: some cells exhibited very weak responses to presentations of single tones but strong responses to the same tones presented in sequence. Such responses were more prevalent in neurons from monkeys that were trained on a task using these sequences.

    Article  PubMed  PubMed Central  Google Scholar 

  105. Sejnowski, T. J. & Rosenberg, C. R. NETtalk: a parallel network that learns to read aloud. Johns Hopkins Univ. Elec. Eng. Compu. Sci. Tech. Rep. JHU/EECS-86/01, 1–32 (1986).

  106. Moore, J. W., Desmond, J. E. & Berthier, N. E. Adaptively timed conditioned responses and the cerebellum: a neural network approach. Biol. Cybern. 62, 17–28 (1989).

    Article  CAS  PubMed  Google Scholar 

  107. Zipser, D. A model of hippocampal learning during classical conditioning. Behav. Neurosci. 100, 764–776 (1986).

    Article  CAS  PubMed  Google Scholar 

  108. Fiala, J. C., Grossberg, S. & Bullock, D. Metabotropic glutamate receptor activation in cerebellar Purkinje cells as substrate for adaptive timing of the classically conditioned eye-blink response. J. Neurosci. 16, 3760–3734 (1996).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  109. Bishop, C. M. Pattern Recognition and Machine Learning (Springer, 2006).

    Google Scholar 

Download references

Acknowledgements

We would like to thank D. Nikolić and W. Singer for providing the experimental data shown in figure 4. We would also like to thank S. Haeusler and S. Klampfl for help with data analysis and figures. For providing helpful comment on earlier versions of this manuscript we would like to thank T. Carvalho, L. Dobrunz, S. Haeusler, M. Kilgard, S. Klampfl, D. Nikolić, F. Sommer and T. Zador. D.V.B.'s work is supported by the National Institute of Mental Health (MH60163). The work by W.M. on this article was partially supported by the research project FACETS of the European Union, and the Austrian Science Fund FWF projects P17229-N04 and S9102-N04.

Author information

Authors and Affiliations

  1. Departments of Neurobiology and Psychology, and Brain Research Institute, University of California, Los Angeles, 90095, California, USA

    Dean V. Buonomano

  2. Institute for Theoretical Computer Science, Graz University of Technology, Graz, A-8010, Austria

    Wolfgang Maass

Authors
  1. Dean V. Buonomano
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wolfgang Maass
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dean V. Buonomano.

Related links

Related links

FURTHER INFORMATION

Dean V. Buonomano's homepage

Wolfgang Maass's homepage

Glossary

Perceptron

A simple linear neuron model that computes a weighted sum of its inputs, and outputs 1 if the weighted sum is larger than some threshold, and 0 otherwise. Weights and thresholds can be learned by the perceptron learning rule.

Multi-layer perceptron

A feedforward network of units, the computational function of which is similar to that of a perceptron, except that a smooth function (instead of a threshold) is applied to the weighted sum of inputs at each unit. Weights and thresholds can be learned by the back-propagation learning rule.

Integrate-and-fire neuron

A simple model of a spiking neuron. It integrates synaptic inputs with a passive membrane time constant. Whenever the resulting membrane voltage reaches a firing threshold, it generates an output spike.

Retinotopy

A spatial arrangement in which neighbouring visual neurons have receptive fields that cover neighbouring (although partly overlapping) areas of the visual field.

Somatotopy

A spatial arrangement in which neighbouring sensory neurons respond to the stimulation of neighbouring receptors in the skin.

Membrane time constant

A physical measure that reflects the time it takes the voltage of a neuron to achieve 63% of its final value for a steady-state current pulse.

Liquid-state machine

A class of computational model that is characterized by one or several read-outs applied to some generic dynamical system, such as a recurrent network of spiking neurons. Whereas the dynamical system contributes generic computational operations, such as fading memory and nonlinear combinations of features that are independent of concrete computational tasks, each read-out can be trained to extract different pieces of the information that is accumulated in the dynamical system.

Echo-state network

A class of artificial neural network model that is based on recurrent connections between analogue units, in which the connection weights are random but appropriately scaled to generate stable internal dynamics. These models can encode temporal information as a result of the active state but do not have hidden states.

State-dependent network

A class of model that is based on the characteristics described in this Review. The state-dependent network model proposes that cortical networks are inherently capable of encoding time and processing spatiotemporal stimuli as a result of the state-dependent properties imposed by ongoing activity (the active state) and as a result of time-dependent neural properties (the hidden states).

Reservoir computing

A general term used primarily in machine learning to refer to models that rely on mapping stimuli onto a high-dimensional space in a nonlinear fashion. Such models include echo-state machines, liquid-state machines and state-dependent networks.

Linear discriminator

A type of classifier that can be computed by a perceptron.

Synaptic weights

The strength of synaptic connections between neurons.

Learning rule

A rule that governs the relationship between patterns of pre- and postsynaptic activity and long-term changes in synaptic strength. For example, spike timing-dependent plasticity.

Recurrent network

A network in which any neuron can be directly or indirectly connected to any other — the flow of activity from any one initial neuron can propagate through the network and return to its starting point. By contrast, in a feedforward network information cannot return to the point of origin.

Spike timing-dependent plasticity

(STDP). Traditionally, a form of synaptic plasticity in which the order of the pre- and postsynaptic spikes determines whether synaptic potentiation (pre- and then postsynaptic spikes) or depression (post- and then presynaptic spikes) ensues.

Invariant pattern classification

The discrimination of patterns in a manner that is invariant across some transformation. For example, recognition of the same word spoken at different speeds or by different speakers.

Chaos

In theoretical work this term is applied only to deterministic dynamical systems without external inputs, and characterizes extreme sensitivity to initial conditions. In neuroscience it is also applied more informally to systems that receive ongoing external inputs (and that are subject to noise and hence are not deterministic), and characterizes neuronal systems with a trajectory of neural states that is strongly dependent on noise and less dependent on external stimuli.

Hyperplane

A hyperplane is a generalization of the concept of a plane in a three-dimensional space to d-dimensional spaces for arbitrary values of the dimension d. A hyperplane in d dimensions splits the d-dimensional space into two half spaces.

Attractor

The state of a dynamical system to which the system converges over time, or the state that 'attracts' neighbouring states.

Sparse code

A neural code in which only a small percentage of neurons is active at any given point in time.

Psychophysics

Studies based on perceptual decisions regarding the physical characteristics of stimuli, such as the intensity or duration of sensory stimuli.

Tonotopy

A spatial arrangement in which tones that are close to each other in terms of frequency are represented in neighbouring auditory neurons.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Buonomano, D., Maass, W. State-dependent computations: spatiotemporal processing in cortical networks. Nat Rev Neurosci 10, 113–125 (2009). https://doi.org/10.1038/nrn2558

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/nrn2558

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing