Explicit covariance-learning PCNs

The explicit covPCNs [16, 17] were proposed to extend the PC model for the visual system [11] to a general model of how the brain performs representation learning. Following the probabilistic framework of PC, it introduced the covariance matrix by encoding it explicitly into the network’s recurrent connections. We denote the activity of neurons in a single-layer explicit covPCN by a vector x, and throughout the paper we denote vectors with a bold font. The model assumes that the set of patterns to memorize of dimension d is a set of samples from a Gaussian distribution with true mean μ_true and covariance matrix Σ_true, where i indicates the ith sample within the dataset. The dynamics of the network model, parameterized by mean μ and covariance Σ, thus aim to maximize the following log likelihood of observed neural activity x given this Gaussian distribution (or negative free energy): (1)

An explicit covPCN learns its parameters by iteratively setting neural activity to training patterns x = x(i) and then modifying parameters by directly computing the derivative of with respect to μ and Σ [17]: (2) where α denotes the learning rate for parameters. Notice that this learning rule is fully online as it updates the parameters every time a single training pattern is received. This is closer to learning by biological systems. Here, to derive subsequent analytical properties of the covPCNs, we define the following full-batch learning rules, which can approximate the fully online learning if α is small enough: (3) (4)

The above full-batch learning rules have a property that both parameters will converge to the maximum likelihood estimate (MLE) of μ_true and Σ_true based on the training data points, i.e., and . We denote these MLE estimates of mean and covariance by and S, respectively. Therefore, the parameter matrix Σ will explicitly encode the sample covariance of the data S, thus the name explicit covPCNs. This can be shown by noting that at convergence μ and Σ do not change, so setting Δμ = 0 and ΔΣ = 0 and solving Eqs 3 and 4 for μ and Σ, respectively, gives the above MLE estimates. It is worth noting that although we refer to these estimates MLE, in more general formulations they would correspond to maximum a posteriori (MAP) estimates [16, 18]. However, because we have not placed any prior constraints on the generative model implied by Eq 1, the MLE and MAP become equivalent.

After learning, we fix the parameters μ and Σ and provide a single cue to the network, i.e., we initialize the neural activity to , and the explicit covPCN performs inference on the cue by updating it according to the derivative of the log likelihood: (5) where β defines step size along the gradient direction, which we refer to as “integration step”. For example, if is a corrupted or noisy data point, the equation above will drive it towards the mean in proportion to the precision or inverse variance.

To see how the above equations could be implemented in a neural circuit, it is useful to note that they greatly simplify if we define a vector of prediction errors: (6)

Then, the dynamics of neurons (Eq 5) becomes: (7)

Moreover, the learning rules for this model (Eqs 3 and 4) become: (8) (9)

The above neural dynamics (Eqs 6–9) can be implemented within the network shown in Fig 1A. In this network, ε and x are encoded in activities of neurons, and parameters μ and Σ in synaptic weights. The neurons x, called the value neurons, receive inhibition from prediction error neurons (cf. Eq 7), while the neurons ε receive excitatory inputs from the value neurons, and top-down inhibitory inputs encoding the prior expectation, as well as lateral inhibitory inputs encoding the weight Σ, to compute the weighted prediction error of Eq 6 (for details of how this computation arises see [18]).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Neural implementations of the single-layer covPCNs using error neurons.

x_a and ε_a denotes the a-th value/error neuron in the networks. For simplicity we show the case where the input patterns have only 2 dimensions, corresponding to 2 value and error neurons. A: explicit covPCN originally proposed in [16]. B: Implicit covPCN. C: Dendritic covPCN with direct structural mapping to a pyramidal cell. Unlabeled connections have strengths 1.

https://doi.org/10.1371/journal.pcbi.1010719.g001

Notice that the learning for the top-down connections encoding the predictions μ (Eq 8) follows the Hebbian rule, as it is simply the product of the pre-synaptic activity (1) and post-synaptic error activity (ε). However, the learning rule for the lateral connections Σ (Eq 9) is biologically implausible, since to compute that is needed for the update ΔΣ_ab of the synapse connecting neuron a and b, non-local information from all other entries in Σ is needed, due to the nature of the inverse operation. That is, to compute the synaptic update between neuron a and b, synaptic weights from all over the neural circuit are needed. Moreover, as we will show in the Results section, this numerically unstable inverse term poses significant computational problems that make this model inapplicable to complex patterns, such as the image datasets MNIST [19] and CIFAR10 [20], which are not generated by sampling from a Gaussian distribution.

Implicit covariance-learning PCNs

The biological implausibility of the explicit covPCNs raises the question whether there exists an alternative and more realistic PC model that performs AM via covariance learning. Notice that another way of encoding interactions between neurons, while preserving the predictive nature of the network, is to let the neurons predict each other. With this intuition, we propose the implicit covPCN, which is also a recurrently connected, single-layer network with weight matrix W, with error neurons encoding the prediction errors. The implicit model aims to maximize the following negative energy function: (10) where x and ν are both d-dimensional vectors and W is a d × d 0-diagonal matrix, where W_ab will be encoded in the synapse connecting the a-th and b-th neurons in the recurrent network. In the implicit model, we define the errors as ε = x − W x − ν. The 0-diagonal property of W implies that the predictions W x come from all value neurons in the vector of neurons x except each neuron itself. Like the explicit model, the implicit covPCN first updates its parameters W and ν by computing the derivative of given the dataset (again, we present the full-batch learning rules here for subsequent analytical derivations): (11) (12) where the ()_diag=0 notation means “enforcing the diagonal elements to be 0” as we want to keep the 0 diagonal elements of W unchanged. Notice that by setting Δν and ΔW to 0, we obtain the following relationships at the convergence of learning: (13) (14)

Recall that and S denote the MLEs of the mean and covariance matrix , and that the parameters of the explicit covPCN, μ and Σ, converge to and S. Therefore, the implicit covPCN also learns the mean and the elements of the covariance matrix, without encoding them explicitly in individual connections.

Like the explicit model, after learning, the implicit model performs inference after initializing the activity to a cue input following the derivative of the objective function with respect to x: (15)

The above dynamics can be implemented in the network model in Fig 1B, by replacing the lateral connections Σ with W that projects the predictions from all other value neurons into each error neuron. Notice that the learning rule for the bias term ν is Hebbian, and more importantly, the learning rule for the connections W is also Hebbian, as it is simply a product ε x^T of the pre- and post-synaptic activities. We show in the Results section, that this biologically more plausible implicit model will converge to exactly the same retrieval of a memory as the explicit model, making it a perfect alternative to the implausible explicit model.

Dendritic covariance-learning PCNs

Inspired by the dendritic model in [21, 22], we push the biological plausibility of the implicit model further by imposing a “stop gradient” (sg) operation on the objective function Eq 10: (16) where (17)

Notice that the parameter learning rules in this model is exactly the same as Eqs 13 and 14. Only the dynamics of the value nodes during inference becomes: (18)

The above dynamics differs from the implicit model (Eq 15) that it no longer includes W^Tε term that corresponded to the backward excitatory connections from the error neurons to the value neurons in Fig 1B. This results in the neural network implementation shown in Fig 1C. This implementation is simpler, as the error neurons in this model only receive predictions from external sources, so they can thus be absorbed into the dendrites of the value neurons x, as shown in the soma-dendrite demonstration in Fig 1C. This architecture could be viewed as more biologically plausible because it does not include the special one-to-one connections of strength 1 between value and corresponding error neurons present in Fig 1A and 1B, for which there is no evidence in cortical circuits. Such an error-encoding dendritic model relates to the model proposed in [21, 22]. As we show in the Results section, since this stop gradient operation does not affect the learning of parameters in the implicit covPCN, and the fixed points of dynamics for the implicit and dendritic models are the same, the equivalence to the explicit model in converged memory retrieval is retained.

Hybrid PCNs

We now use the obtained results to propose an architecture that mimics the behavior of the hippocampus as a memory index and generative model. The recurrent, single-layer network models introduced above provide a model for the recurrent dynamics in the hippocampus. However, raw sensory inputs are first processed via a hierarchy of sensory and neocortical layers, allowing the hippocampus to memorize representations of raw signals, instead of the signal itself. Moreover, according to a recent theory, the hippocampus functions as a generative model that accumulates prediction errors from sensory and neocortical neurons lower in the hierarchy, and sends descending predictions to the neocortex, to correct the prediction errors in the neocortical neurons. The generative model of the hippocampus is updated until the hippocampal predictions correct the prediction errors by suppressing neocortical activity [9]. Hierarchical PCNs have already been proposed as a generative model that learn representations of the sensory inputs [17]. Particularly, it has been shown that a purely hierarchical PCN, without any recurrent structure, is able to perform associative memory tasks on highly complex datasets [12]. Here, we combine the hierarchical PCN with our proposed recurrent architecture, obtaining an hierarchical model with recurrent dynamics at the topmost layer. What results is an hybrid network that models the whole pathway from sensory neurons to hippocampal neurons.

Consider an hierarchical PCN with L layers, with a recurrent implicit or dendritic network connected to the last layer. The first layer corresponds to the sensory layer, where sensory signals are presented to be processed, and the last layer corresponds to the hippocampal layer. Then, the intermediate layers 1 < l < L represent the hierarchical structure present in the neocortex that connects the sensory neurons to the hippocampus. We denote the synaptic weight matrix connecting the neurons in the lth layer and the neurons in the (l + 1)th layer as Θ^(l). Within the hippocampal layer L, we use W to denote the recurrent synaptic weight matrix. The vector of value nodes in each layer, denoted as x^(l), is coupled with the error node vector ε^(l), computed as the following prediction error: (19) where ρ^(l) denotes the descending prediction signals into the neurons, computed as: (20) where f(⋅) is a nonlinear function. For simplicity of illustration, we ignore the top-down connections ν in the recurrent covPCNs. The aim of the learning dynamics of this hybrid network is to maximize the negative sum of squared errors: (21)

Like purely hierarchical PCNs, the learning of the hybrid PCN consists of two stages: inference and weight update. During inference, the value neurons relax to maximize following the gradient ascent rule: (22) where ⊙ denotes the element-wise product between two vectors, and γ = 1 if the topmost layer is an implicit covPCN, or 0 if it is a dendritic covPCN. The update at the sensory layer (l = 1) is 0 because during learning, the sensory neurons are fixed at the raw sensory inputs. The inference dynamics are carried out for a certain number of iterations until the value nodes reach an equilibrium. The weight update is then performed, also to maximize using gradient ascent: (23)

In each iteration of learning, the inference will be performed for multiple iterations and then the weight update will take place for one step. The whole learning process will be iterated multiple times until reaches the minimum.

The proposed architecture also models the reconstructions of past memories: as a memory index, the hippocampus sends top-down information to the neocortical neurons to reinstate activity patterns that replicate previous sensory experience [9]. In the reconstruction phase of our generative model, the hippocampal layer provides descending inputs to the sensory neurons to generate stored data points. Assume we have a hybrid PCN already trained on a dataset of memories. To retrieve a stored datapoint, the synaptic weights Θ and W are fixed, and the retrieval process is triggered by providing a partly corrupted version of a memorized pattern to a subset sensory neurons. During retrieval all other value nodes will relax following the same rule specified in Eq 22. The only difference is that, the value nodes in the sensory layer will also experience relaxation: (24) where the subscript c denote the corrupted dimensions of the input pattern, as we keep the intact part of the patterns unchanged during retrieval. Notice that all the dynamics above requires only local computations. Fig 2 shows the general structure of the hybrid PCN, as well as detailed implementations of neural computations.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Multilayer hybrid PCN.

We use the single-layer implicit/dendritic covPCN to model the hippocampus, and a hierarchical PCN from [12] to model the sensory cortex and neocortex. Neurons and synapses in the hierarchical layers follow the dynamic rules in [12]. For clarity of demonstration, only one layer of our neocortex model is shown. Expanded boxes show the detailed computations within individual neurons and related synapses specified in Eqs 20 and 22, where denotes the a-th neuron in the lth layer, and Θ_ab and W_ab denote the individual weights from the ath to the bth neurons. Dog image in this figure is obtained from Wikimedia Commons under a CC BY 4.0 license.

https://doi.org/10.1371/journal.pcbi.1010719.g002

The explicit covPCN performs associative memory

The explicit covPCN was originally proposed as a generative model of how the cortical responses are evoked given observations [16, 17]. To our knowledge, it has never been employed to perform AM tasks. To verify our hypothesis that after training, the explicit covPCN can perform AM using its inferential dynamics and learned covariance, we designed a simple task where multiple 5 × 5 random Gaussian patterns are memorized. We then covered the bottom two rows of the 5 × 5 patterns with 0s and run inference on these corrupted entries only. Examples of the original patterns, along with the corrupted and retrieved patterns, can be found in Fig 3A. In this task, the models can be considered as memorizing the association between the top 3 rows and bottom 2 rows of the random patterns. As can be seen in the third column of panel A, the single-layered explicit covPCN can retrieve the covered part of the original pattern well: it performs AM. We then measured the MSEs between the retrieved and original 5 × 5 random patterns by these models, and plotted them as a function of corrupted/masked pixels in Fig 3B (Fig 3A shows examples when there are 10 masked pixels). As can be seen, the explicit covPCN has slightly higher retrieval MSEs on average, although this performance gap is not visually observable from the retrieved patterns (e.g., Fig 3A). As we will show in the following sections, this performance gap will be exacerbated when these models are trained to memorize structured and more complex datasets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Performance of covPCNs in AM of random patterns, and the equivalence between them.

A: A subset of 5 × 5 random patterns memorized by all 3 models. After training, we corrupted the bottom 2 rows (10 pixels) and let the networks run inference on the corrupted parts for retrieval. B: Retrieval MSEs of the models when corrupted with different mask sizes. Experiments in A and B are performed with networks with d = 25 neurons. C: Sample covariance of a random 2-dimensional dataset and the learned weight matrices of an explicit model and an implicit/dendritic model on this dataset. D: The random 2-dimensional dataset to memorize, and the linear retrieval obtained by masking the second dimension x₂ by all 3 models, as well as the theoretical retrieval line. All the lines overlap as they are equivalent in theory. Experiments in C and D are performed with networks with d = 2 neurons.

https://doi.org/10.1371/journal.pcbi.1010719.g003

We now show that a theoretical result can be derived to analytically describe the pattern retrieved by an explicit covPCN. We will use this result later to establish equivalence between explicit and implicit covPCNs. Consider a corrupted pattern , which can be divided into the corrupted part and “intact” part , where k + m = d: (25)

At the time of retrieval, the training of the covPCN has already finished, during which the network has memorized the association between x_k and x_m. We assume that the training has converged, so that the connections μ and Σ explicitly encode the MLE, and S, of the mean and covariance respectively. In this case, the inference dynamics of the explicit model follows (Eq 5). To correspond to the division of the corrupted and intact parts of x, we also divide S and into blocks and set Δx = 0 to study the convergence of the inference stage: (26) where S_pq, p, q ∈ {k, m} denotes the p × q submatrices of the covariance matrix S. Using Eq 26 above, we can have the following theorem:

Theorem 1 After training, an explicit covPCN retrieves the corrupted x_k using the intact x_m following the dynamics , and the retrieval dynamics on x_k converge to: (27) where denotes the retrieval of the corrupted part. This theorem describes the activities in the network after retrieval given by the explicit covPCN, using the learned parameters and the intact x_m. Details of the derivation can be found in S1 Appendix.

The equivalence between explicit and implicit covPCNs

Having established the theoretical foundation of the retrieval dynamics in the explicit covPCN, we next consider the theoretical aspects of the implicit and dendritic models: what is the retrieval of memorized patterns given by these models, based on learned associations? Again consider the case of a vector x consisting of the top k corrupted entries x_k and bottom m intact entries x_m, and assume that the learning has converged when the corrupted x is presented to the network, which gives the conditions in Eqs 13 and 14 for both implicit and dendritic models (notice that these two models do not differ in parameter learning). We now write the model parameters ν and W into block matrices: (28) where W_pq, p, q ∈ {k, m} denotes the p × q submatrices of the weight matrix W. Notice that for both implicit and dendritic models, the retrieval dynamics converge if and only if all error nodes become zero, that is, ε = x − W x − ν = 0. For the dendritic model this is obvious (Eq 18), whereas for the implicit model this comes from the fact that W has all its diagonal entries equal to 0 and thus cannot be an identity matrix (Eq 15). We can thus derive the following theorem on the converged retrieval dynamics of the implicit and dendritic covPCNs:

Theorem 2 After training, both the implicit and dendritic models retrieve the corrupted x_k given intact x_m following the dynamics specified in Eqs 15 and 18, which converge to the following equilibrium: (29) where I_kk is the k × k identity matrix. Using Eq 14, this equation is equivalent to: (30)

The details of the derivations can be found in S1 Appendix. Notice that the equilibrium condition for implicit and dendritic covPCNs is exactly the same as that for the explicit covPCN, specified in Eq 27. This equivalence was also verified empirically in a simple 2-dimensional example shown in Fig 3D. In this example both x_k and x_m are single-dimensional scalars, and all three models retrieved the corrupted x₂ following the same linear equation (Eq 27). Notice that this line is also the least squares regression line, where x₂ is the dependent variable. It is also worth noting that this line implies that the memory “attractors” learned by the model is a line, instead of individual points, in the 2-dimensional case, which is different from the attractors learned by classical models such as Hopfield Networks [4]. In the final section of the Results, we provide an empirical investigation into this difference. Fig 3C also shows that the weight matrix Σ of the explicit covPCN directly encodes the sample covariance matrix, whereas the other two models encode it implicitly. Due to their equivalence, the implicit and dendritic models also perform AM on the random Gaussian patterns in Fig 3A.

At this point, we have established the complete equivalence between the explicit and implicit/dendritic covPCNs, showing that the latter models can serve as perfect substitutes for the explicit model. However, in contrast to the explicit covPCN, where the learning rule (Eq 4) for the connections Σ employs non-local information, the plasticity rules (Eq 12) for the implicit and dendritic models are entirely Hebbian. Moreover, as was shown above, the neural implementation of a dendritic covPCN can be mapped to the structure of hippocampal pyramidal cells. Therefore, our proposed models are equivalent to but biologically more plausible than the explicit covPCNs.

Model performance in AM with structured image data

In each learning iteration, the explicit covPCN needs to compute the inverse of the current weight matrix. In practice, this works well when the underlying dataset has some specific regularities, but becomes problematic when dealing with structured data, such as images of handwritten digits and natural objects. In this section, we show that the explicit model can no longer perform stable memorization and retrieval of more complex and structured data, whereas the implicit and dendritic models are able to perform AM on such data with a high level of precision, and are hence interesting from an application perspective. To do that, we replicate the pattern completion experiment performed in Fig 3, but using images sampled from the MNIST [19] and grayscale CIFAR10 [20] datasets. Fig 4A shows some retrieved examples from both datasets by all three types of single-layer covPCNs, which were trained to memorize 64 images. The visual results suggest that the explicit covPCN can no longer retrieve clear images, whereas both implicit and dendritic models can obtain visually perfect retrievals of the memories. The failure of the explicit covPCN with these structured images is due to the need to compute the inverse of Σ in each iteration (Eq 4): for low-dimensional patterns with some specific regularities such as Gaussian distribution, the inverse can be precisely computed. However, when encountered with high-dimensional data such as structured images, the inverse computation can become imprecise, and the computational errors may accumulate when the model is trained iteratively, leading to blurry retrievals.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Performance of the single-layer covPCNs in AM of structured images.

A: Examples of retrieved MNIST (top) [19] and grayscale CIFAR10 (bottom)[20] images by explicit, implicit and dendritic models. All models here are trained to memorize 64 images. For MNIST, the networks have d = 784 neurons; for grayscale CIFAR10, d = 1024. B: Retrieval mean squared errors (MSEs) of the single-layer models across multiple numbers of training memories (N). C: Evolution of the retrieval MSEs of the implicit and dendritic models when N = 256. D: Example eigenspectra of the weight matrices defining the inferential dynamics for the dendritic (left) and implicit (right) covPCNs. Error bars obtained by 5 different seeds for image sampling. Please see main text for an explanation of the matrix M.

https://doi.org/10.1371/journal.pcbi.1010719.g004

We next perform a quantitative analysis of the performance of these models, by measuring the mean squared errors (MSEs) between the original and retrieved grayscale CIFAR10 images across different numbers of images we train the models to memorize (N). Fig 4B shows that the explicit model indeed has much larger retrieval MSEs than the implicit and dendritic model, confirming our visual observations. The MSE gap is also larger than that in Fig 3A, where the random Gaussian patterns makes the learning of the explicit model more stable when inverting Σ. Interestingly, this plot also shows that despite the equivalence between the implicit and dendritic models when their inferential dynamics (Eqs 15 and 18) have converged, the dendritic model has larger retrieval MSEs than the implicit model, especially when N is large. We investigate this phenomenon by plotting the evolution of the retrieval MSEs of both implicit and dendritic models during inference, when N = 256. As can be seen in Fig 4C, the MSE of the dendritic model fails to decrease to the same (lower) value as that of the implicit model, and it also fails to converge. To understand the distinction between the implicit and the dendritic covPCNs in inference, recall that their inferential dynamics are described by two linear differential equations (Eqs 15 and 18). We now define: (31) where W is the learned weight matrix in the implicit/dendritic models (note that the learning of these two models are identical) and I is the identity matrix of the same size as W. We can therefore rewrite the inferential dynamics as: (32) where “im” stands for implicit, and “den” stands for dendritic. Notice that in general, the stability of a linear dynamical system Δx = Ax for some matrix A is determined by the eigenvalues of A: the system is stable if and only if the real part of all eigenvalues of A are non-positive [23]. As we show in Fig 4D, the largest eigenvalue of −M^TM is 0, suggesting stable inferential dynamics for the implicit covPCN. This is also generally true as −M^TM defines a negative (semi-)definite matrix for any M ≠ 0, which has all eigenvalues smaller than or equal to 0. On the other hand, Fig 4D shows that the largest eigenvalue of M is greater than 0, indicating unstable dynamics for the dendritic model. Thus, although the implicit and dendritic models have the same fixed point of their dynamics during retrieval, the neural dynamics defined by the dendritic model may become unstable, preventing itself from the convergence. For the dendritic model there are no theoretical reasons that guarantee negative eigenvalues of M, so they may become positive as the values in that matrix grow with learning more patterns. The experiments with MNIST [19] yielded similar results, and we describe them in S1 Fig.

Performance of hybrid models with natural images

While performing well on small subsets of structured data, the purely recurrent implicit/dendritic models present some problems: first, the recurrent structure only takes account of the hippocampal structure, but does not correctly represent the hierarchical structure that connects it to the neocortex; second, the number of parameters is quadratic to the dimension of the inputs, which does not allow it to work on high-dimensional images, such as high quality images. For example, consider a recurrent network trained on 224 × 224 pictures of the ImageNet dataset [24]. Every image consists of ∼ 150000 pixels, and hence needs a network with the same number of neurons. A recurrent network of this size would be impossible to train without an exceptional amount of computational power, as it has ∼ 40 billions of parameters. On the other hand, a hierarchical structure that precedes the hidden implicit layer guarantees the flexibility of choosing the number of parameters. For example, a network with 7 hidden layers, followed by a recurrent implicit layer, all of dimension 1024, would be more than 200 times smaller than the above implicit model for ImageNet, and hence feasible to be trained. In fact, we have successfully trained this model on high-dimensional ImageNet pictures. Particularly, we trained this 7-layer network with 100 samples from the ImageNet dataset. We then defined successful retrievals as those with retrieval MSEs smaller than 5e⁻³, and found that when presenting the network a partial cue consisting of 1/2, 1/4 and 1/8 of the original pixels, the model has successfully recovered, respectively and on average, 100, 97, and 44 of the original memories. Similar performance was also obtained with the colored CIFAR10 dataset [20] and examples of colored CIFAR10 images can be found in S2 Fig.

We then examine whether the hybrid structure leads to better retrieval performances compared to the single-layer models. To do that, we trained an implicit covPCN and a hybrid model with a topmost implicit covPCN to memorize the same subsets of grayscale CIFAR10 images (1024 pixels in total) of varying number of images N, and initialized the retrieval with half-covered partial cues. To compare with earlier works [12] fairly, we also trained a purely hierarchical PCN to perform the same AM task. The sizes of the networks are chosen such that they have approximately the same number of parameters. The hybrid model we use has 3 layers: the sensory layer has 1024 neurons corresponding to the pixel space, with 512 neurons in the hidden layer, and 512 neurons in the topmost implicit layer. We construct the hierarchical model by replacing the topmost implicit recurrent layer with 2 hierarchical layers of size 512. We chose such a configuration of hidden sizes and number of layers because in general, an implicit covPCN with d neurons (for d-dimensional inputs) will have d(d − 1) parameters. A hybrid model with d sensory neurons, one feedforward hidden layer of size d/2 and a topmost implicit layer of size d/2 will have approximately the same number of parameters. Replacing the final implicit layer with two feedforward layers of the same size results in a hierarchical PCN with roughly the same number of parameters. This ensures the fairness of comparison across models, and is illustrated in Fig 5A, where we also included the number of neurons in each layer used in our experiments next to each layer, as well as the number of connections at the top. As Fig 5B shows, the performance of the implicit model is in fact slightly better than the hybrid implicit model and the purely hierarchical model when they have the same number of parameters. Indeed, the retrieval MSE of the implicit model should be the smallest among all linear retrieval functions: notice that the retrieval of the implicit model defined in Eq 27 is the least squares solution if we regress x_k on x_m. Although the exact retrieval function of the hybrid and hierarchical models is unclear due to their nonlinearity, it has been shown that a two-layer linear PCN with 1 neuron in the hidden layer yields linear retrievals following the principal components of the sensory data [15], shedding light on the slightly worse performance of the multi-layer models. However, we note that the similar performance in retrieval fidelity between the implicit and hybrid models does not make the hierarchical structure in the hybrid model redundant, as the functionality of the brain is far more heterogeneous than merely memorization, and many functions require the hierarchical structure. For example, the hierarchical structure can support the learning of meaningful representations of the sensory inputs, which can be utilized to perform other tasks such as classification, while paying only a negligible cost in memory retrieval tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Performance of the multi-layer models.

A: Demonstration of how we keep the number of parameters across different models to be roughly the same. B: Retrieval mean squared errors (MSEs) of the multi-layer models across multiple numbers of training memories (N). The curve for the implicit model is the same as the one in Fig 4. C: Evolution of the retrieval MSEs of the implicit and dendritic hybrid models when N = 256. Error bars obtained by 5 different seeds for image sampling.

https://doi.org/10.1371/journal.pcbi.1010719.g005

We further compared the implicit and dendritic models when they are “plugged in” as the topmost recurrent layer in the hybrid model. Fig 5B shows that the hybrid dendritic model performs identically to the hybrid implicit model, contrasting their performance difference in the single-layer case. During the inferential iterations, the retrieval MSEs of both models also converge stably to the same value (Fig 5C). In other words, the introduction of the neocortical layers remedies the unstable inferential dynamics due to the biologically more plausible dendritic structure in our models. We suspect that the hierarchical pre-processing of the sensory inputs regularize the covariance matrix, such that the topmost recurrent weight matrix W defines stable inferential dynamics for the dendritic layer. However, since the top-layer inference is no longer a simple linear dynamical system (Eq 22), the connections between a regularized W and the stability of the dynamics may not be straightforward.

Nonlinear covPCNs learn individual attractors

The implicit covPCN we have investigated so far assumes that the relationship between neurons in the recurrent network is linear, i.e., projections into neuron i is ∑_j≠iW_ijx_j. Here, we introduce an ad hoc nonlinearity into the definition of the free energy—as motivated by previous heuristics in neural networks and machine learning: (33) where f(⋅) is a nonlinear function. One way to understand these heuristics more formally is to appreciate that the free energy is a statement about the underlying generative model that generates inputs (e.g., images) from some latent causes. Although linear covPCNs have provided nice analytical results discussed above, nonlinearities can speak to more expressive generative models similar to the hierarchical and hybrid schemes demonstrated above. The plasticity rules of this model will be the same as Eqs 11 and 12, with the error ε changing to x − Wf(x) − ν. The neural dynamics during inference follows: (34)

We first train this nonlinear implicit model to perform the same AM task as before, where the model is trained to memorize different numbers of grayscale CIFAR10 images, and then retrieve these memories from images with covered bottom half. For these experiments with CIFAR10 we used tanh() as the nonlinear function f(). Fig 6A shows that the performance of the nonlinear implicit covPCN is worse than that of the linear model in this completion task, especially when N is large. To examine the reasons for this performance difference, we then corrupt the memories differently by adding Gaussian noise with variance 0.1, and initialize the retrieval dynamics with these noisy memories. Fig 6B shows that with this denoising task, the linear and nonlinear implicit covPCNs perform similarly, although the retrieval MSE of the nonlinear model is overall lower. We hypothesize that the difference across different retrieval tasks results from the attractor structures learned by the linear and nonlinear models. Classical nonlinear models, such as the Hopfield Networks [4], usually learn individual attractors corresponding to each memorized item. On the other hand, as we have demonstrated in Fig 3D, the linear implicit covPCN appears to learn a line attractor in the 2-dimensional space. We generalize this observation by training both linear and nonlinear models to memorize 3 random data points in range [−1, 1] in . After training, we initialize the inference with test data in a grid from the range [−1, 1], and examine where these test data points are attracted to at the convergence of the inferential dynamics. Fig 6C shows that the attractor formed by the linear model is a hyperplane, where any point on this plane defines the lowest energy. On the other hand, the test data points all converge to the memories learned by the nonlinear model, suggesting that the nonlinear model indeed learns individual attractors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Comparison of linear and nonlinear implicit covPCN.

A: Performance of linear and nonlinear implicit models in the completion task with varying Ns. B: Same as A, but with the denoising task, where cues are memories with Gaussian noise of variance 0.1. C: A simple 3-dimensional example, where stars are data points the networks were trained to memorize. After training we ran inference on both linear and nonlinear models, initialized with grid test data drawn from the range [−1, 1]³. The position of the test data at convergence of inference indicates the shape of attractors. Images taken from the CIFAR10 dataset [20].

https://doi.org/10.1371/journal.pcbi.1010719.g006

The observation above in turn helps explain the performance difference in completion and denoising tasks between the linear and nonlinear models. Notice that the completion task is inherently more challenging than the denoising task in terms of retrieval, as the corrupted pixels contain no information about the original images. Therefore, in the completion task, individual attractors formed by the nonlinear model will confound with each other more severely than in the denoising task, so that more retrievals will converge into the wrong attractors. The retrieval MSE of the nonlinear model thus comes from these incorrect attractor choices. With the linear model, the retrieved data points are always pulled towards the least squares regression hyperplane, such as the regression line in Fig 3. The MSE of the linear model therefore only comes from the least squares prediction, which is the minimum across all linear solutions. Thus, when we average across the whole dataset, a relatively large number of totally wrong retrievals will produce a higher MSE than the retrievals from the least squares predictions. On the other hand, in the less challenging denoising task, the confounding effect between attractors of the nonlinear model will be less severe, yielding a smaller retrieval MSE across the whole dataset than in the completion task. Moreover, the least squares prediction from the linear model is less reliable in the denoising task, as all entries of the data are corrupted, i.e., the independent variables in the regression problem are also changing. Therefore the retrieval MSE of the linear model will increase, resulting in the observation in Fig 6A and 6B.

Summary

This work introduces a family of covPCNs for AM tasks, providing a possible mechanism of how the recurrent hippocampal network may perform AM via predictive processing, which also unifies two disparate modelling approaches to the hippocampal network, namely covariance learning and PC. First, we identified that a previously proposed PCN already considered the learning of parameters encoding covariance in an explicit manner [16, 17], but was never tested for AM tasks. We showed that, both theoretically and empirically, this model is able to perform AM on random patterns. However, this explicit covPCN is neither biologically plausible nor numerically stable, due to the inverse term in its learning rule. We address both limitations by proposing a model we call implicit covPCN, which also learns the covariance matrix, but in an implicit manner. More importantly, it adopts a Hebbian learning rule that is biologically more plausible than the learning rule of the explicit covPCN. We then showed that this model can be further simplified by introducing an apical dendritic architecture, which is even more biologically realistic. We named it the dendritic covPCN. In theory, we showed that both the implicit and the dendritic models are exactly equivalent to the explicit covPCN in AM tasks if their inference converges, although the dendritic model may suffer unstable inferential dynamics. Nevertheless, in practice, both implicit and dendritic models can be trained to memorize complex patterns such as handwritten digits and natural images. However, these models are computationally and memory expensive, a drawback that limits its applicability when confronted with more complex datasets. We hence solve this problem by combining our implicit covPCN with a hierarchical PCN [11, 12], to model the predictive interaction between the hippocampus and neocortex [9]. We showed that this hybrid network can memorize and retrieve large-scale images in a parameter-flexible manner. We further conducted empirical analysis of the memory attractors learned by our linear implicit model, and showed that the memorized patterns are all stored on a hyperplane attractor of the inputs space. We found that nonlinearities are needed to obtain individual point attractors corresponding to each memorized pattern, similar to those observed in Hopfield Networks [4]. Table 1 below summarizes the property of each model discussed in this work. Overall, our models benefit the theoretical understanding of the computational principles adopted by the hippocampal network, by providing a unitary account for two disparate theories of how the hippocampus performs AM tasks.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Summary of covPCNs discussed in this work.

https://doi.org/10.1371/journal.pcbi.1010719.t001

Relationship to other models

Relationship to experimental data

Our proposed models all belong to the family of PCNs, which assumes that computations in the cortex are carried out by generating predictions of activities representing sensory inputs. In general, it has been shown that this assumption is consistent with the known anatomy of the cortex, and that several cortical microcircuits can implement the PC algorithm [59–62].

More specifically, experimental evidences have suggested mechanisms of predictive processing within the hippocampus and related areas. It has long been postulated that the hippocampus predicts upcoming sensory inputs, in both spatial and non-spatial experimental setups [7, 8, 63, 64]. The special feedforward circuitry of the hippocampus may also be an evidence for predictive processing: Lisman [65] suggested that the CA1 region of the hippocampus serves as a “mismatch” detector, encoding the error between sensory reality and the internally generated prediction based on past events communicated through the feedforward pathway from CA3 to CA1. However, although hippocampal cell populations that fire exclusively following erroneous trials in associative learning tasks have been found in animals [66, 67], it is unclear, at the single-cell level, whether prediction error representations exist in the hippocampus, and in what form they exist: are there neurons encoding them, or are they encoded in specific neuronal structures such as apical dendrites? Our models inform future experimental works to answer these questions by making the prediction that hippocampal neurons can representation prediction errors, via somatic or dendritic activities.

It is also worth pointing our that the experiments in our work focused mainly on “spatial” predictions e.g., predicting each pixel using other pixel values. Although predictions made through different pixels or patches of a static image can be interpreted as eye saccade across a large visual scene, prediction of sequential sensory inputs is more realistic and consistent with experiments mentioned above. Modelling temporal PC models while retaining the plausible implementation discussed in this work will be a focus of our future explorations.

Future directions

Following the discussions above, there are several directions in which this work should develop. The theory needs to be extended to modelling sub-regions of the hippocampus using PC. Rather than a simple, fully-connected recurrent circuit, the hippocampal network contains multiple sub-regions with different possible functionalities. For example, the DG area is postulated to perform pattern separation through its sparse activities, while the CA3 is believed to be the associative area and CA1 acts as a decoder and mismatch detector [5, 6, 65]. Moreover, the connectivity between these regions also has a unique pattern. For example, the EC-DG-CA3 connection is fully feedforward and unidirectional, whereas CA3 also receives direction input from EC. It will be important to investigate how these other regions could be included in the PC framework. Notably, we have shown in this work that a simple stop-gradient operation in PCNs could yield unidirectional connections in a recurrent network, which provides a potential approach of modelling the hippocampal sub-region connectivity.

Another direction in which the work can develop is temporal prediction in the hippocampus. While abundant experimental evidences have shown the predictive nature of the hippocampus, the observed predictions are mainly sequential or temporal, i.e., predicting sensory input in the future. This temporal property of the hippocampal prediction is not reflected in our PC models. Earlier and recent works have attempted to model cortical temporal prediction using a Kalman filter [10, 28], where the hierarchical prediction of sensory inputs made by the hidden state can potentially be implemented within the hierarchical PCNs [11]. Another track of research assumes that both sensory input and neural response dynamics along the temporal dimension are contained by attractor manifolds, but on different levels [68, 69]. Thus, higher-level attractor of neural responses can generate controls over lower-level attractors of responses or sensory input streams, which can be described within the hierarchical PC/free energy framework [11, 16]. However, it remains an open question whether these temporal PC models can be implemented within simple and plausible neural circuits similar to those discussed in this work. It is also possible to combine this direction of research with sub-regions of the hippocampus, as the CA1 area was postulated to encode the mismatch between inputs from CA3 and those from EC [65], whereas recent experimental works have found that the EC-CA1 inputs is prioritized over the CA3-CA1 inputs when a new sensory stimulus is presented, suggesting that the mismatch encoded in CA1 results from a temporal delay [70]. These findings shed light on a hippocampal sub-region approach to temporal PC.

Moreover, in our current work, we have focused on learning covariance or precision matrices inherent in a batch of training images, which can be implemented within a single-layer network. On the other hand, hierarchical implementations of PC often use more expressive generative models, in which the covariance or precisions at various hierarchical levels can change over time. This means that instead of learning the covariances, they are inferred, leading to the notion of dynamic precision weighting of prediction errors [71, 72]. This is also particularly interesting from a machine learning point of view, because it provides a model of attention from a neuroscience perspective—that may be related to the kinds of attention found in transformers and the like [73].