Natural gradient learning for spiking neurons

Due to their simplicity and success in machine learning, gradient-based learning rules represent a popular choice for synaptic plasticity models. While they have been linked to biological observations, it is often ignored that their predictions generally depend on a specific representation of the synaptic strength. In a neuron, the impact of a synapse can be described using the state of many different observables such as neutortransmitter release rates or membrane potential changes. Which one of these is chosen when deriving a learning rule can drastically change the predictions of the model. This is doubly unsatisfactory, both with respect to optimality and from a conceptual point of view. By following the gradient on the manifold of the neuron's firing distributions instead of one that is relative to some arbitrary synaptic weight parametrization, natural gradient descent provides a solution to both these problems. While the computational advantages of natural gradient are well-studied in ANNs, its predictive power as a model for in-vivo synaptic plasticity has not yet been assessed. By formulating natural gradient learning in the context of spiking interactions, we demonstrate how it can improve the convergence speed of spiking networks. Furthermore, our approach provides a unified, normative framework for both homo- and heterosynaptic plasticity in structured neurons and predicts a number of related biological phenomena.