\chapter{Recurrent Predictive Coding for Associative Memory}\label{chap:am}

\section{Introduction}
Memory systems in the brain often store information about the relationships or associations between objects or concepts. This particular type of memory, referred to as \emph{Associative Memory} (AM), is ubiquitous in our everyday lives. For example, we memorize the smell of a particular brand of perfume, the taste of a kind of coffee, or the voice of different singers we like. After a memory is formed, AM will support its retrieval when a related cue is presented to our senses using the learned association between the provided cues and missing components.



It has long been argued that the hippocampus and adjacent cortical areas, located in the medial temporal lobe of the brain, are crucial for AM \citep{scoville1957loss,squire1991medial,squire1992memory}. Historically, various theoretical and computational works have been developed in an effort to model the hippocampus in AM tasks \citep{hopfield1982neural, o1994hippocampal,treves1994computational}. In this chapter, we are interested in a particular approach to modelling, which assumes that the hippocampus employs its \emph{predictive activities} to perform AM. Predictive processing is thought to be a key computational principle underlying various hippocampal activities: Experimentally, abundant evidence has suggested that the hippocampus is capable of predicting ongoing sensory inputs \citep{lisman2009prediction}. High-level theories and computational models have also been proposed to explain how predictive coding (PC) may support various properties of the hippocampus, including the formation of cognitive maps \citep{stachenfeld2017hippocampus} and memory \citep{barron2020prediction}. In particular, \citet{barron2020prediction} proposed that the hippocampus sits at the top of a hierarchical generative model that generates predictions of neocortical activities based on past experiences, thus enabling retrieval of activities from memory at lower neocortical levels.
The ability of predictive coding networks (PCNs) to complete previously learned patterns have been already demonstrated in pioneering work by \citet{rao1997correlates}, although the model in this work was not formally introduced within the PC framework.
More recently, the ability of the hierarchical PCNs introduced by \citet{rao1999predictive} in performing AM has been computationally analysed \citep{salvatori2021associative, ororbia2022neural}. 
%This proposal directly inspired a recent generative network model \citep{salvatori2021associative} that employed the hierarchical predictive coding networks (PCNs) developed originally by Rao and Ballard \citep{rao1999predictive} to model the hippocampo-neocortical interactions during AM, although the ability of PCNs to complete previously learned patterns have already been suggested in pioneering works by Rao \citep{rao1997correlates}. 
Specifically, \citet{salvatori2021associative} showed that hierarchical PCNs can store training data points as memories, and can retrieve these memories given partial or noisy cues. Under this PC framework, the memorization of a sensory input is driven by the Hebbian learning dynamics \citep{hebb1949organization} that minimize the error between the input and a prediction generated by the network. The retrieval of this input is performed by the inferential neural dynamics, also to minimize the error between the internal predictions and the corrupted sensory input. The error signals are represented by prediction error neurons in the network implementation.
% It is worth mentioning that memory retrieval in this PC-based model can be understood as \emph{static pattern completion}, where the internal model of the PCN tries to predict the external world during and after learning, instead of predicting into the future. Thus, the temporal dimension of stimulus inputs is ignored. This will also be the scope of AM that we discuss in this chapter.
% It is worth mentioning that AM in this PC-based model is defined as the memorization and recall of \emph{static} inputs, where the associations are learned between individual components of static patterns, and the temporal dimension is ignored. In this scenario, memory retrieval is equivalent to pattern completion. On the other hand, AM can also be defined as memorizing the association along the temporal dimension, where the associations between two (or multiple) inputs along time are memorized \citep{rao1997correlates}. In this chapter we focus on the first i.e., static type of AM.

This hierarchical PC model provides a possible network mechanism that the hippocampo-neocortical entity employs to support AM, thus offering a possible implementation of the theory of \citet{barron2020prediction}. However, this model only included feedforward and feedback connections between layers of neurons (representing different cortical areas), and thus failed to provide an account for how the \emph{recurrent} hippocampal network performs AM. Furthermore, the absence of recurrent connections in the hierarchical PC models for AM dissociates them from earlier recurrent models of AM, such as the Hopfield Network \citep{hopfield1982neural}. Hopfield Networks assume that the recurrent connections in the hippocampal network learn a covariance matrix representing the associations between individual neurons activated by a memory item. A missing link to Hopfield Networks thus hinders a unified understanding of the computational principles adopted by the hippocampus to support AM. We are thus brought to ask the following two questions: {\it 1. How can the recurrent hippocampal network employ PC to perform AM, in a biologically plausible and computationally stable manner? 2. Can recurrent connections in such a network encode the covariance of neural activity?
% that is also reconcilable with covariance learning?
} Here, by computationally stable, we mean the ability of the model to steadily converge to a fixed point both during learning (memory) and inference (retrieval), and the criteria for biological plausibility include:

\begin{itemize}
    \item[1.] \textbf{Local computation}: A neuron's computation is only dependent on its input neurons and weights connecting itself to its input neurons.
    \item[2.] \textbf{Local plasticity}: The plasticity rule of synapses in a model only depends on quantities encoded by pre- and post-synaptic neurons.
    \item[3.] \textbf{Architectural similarity}: Components of a model resembles architectures of real neurons, such as the recurrent connections and the apical dendrites of the pyramidal neurons in the hippocampus.
\end{itemize}

The hierarchical PC models for AM are stable and satisfy the first two criteria of plausibility \citep{salvatori2021associative}. However, as we pointed out above, they fail to meet the third criterion, architectural similarity, due to the missing recurrent connections. In this chapter, we propose a family of PCNs with {\bf r}ecurrent connections between neurons, which we call rPCNs, as candidate models satisfying these criteria. In particular, we first identify that an earlier type of PCN has already incorporated recurrent synaptic connections encoding the covariance information of inputs \citep{friston2003learning, friston2005theory, bogacz2017tutorial}, and can thus be considered as a PC model meeting the criterion of architectural similarity. We refer to it as the {\it explicit rPCN}, as it encodes the covariance matrix explicitly into its recurrent synapses. The explicit rPCN was originally proposed as a model for learning representations of sensory inputs, and we show in this chapter that its covariance-learning nature can be utilized to perform simple AM tasks. However, we note that the learning rule for the recurrent connections in this model is non-Hebbian, and poses significant computational issues, which makes it fail to satisfy the local plasticity and stability conditions. To address these issues, we propose in this chapter a novel recurrent PCN, which also encodes the covariance matrix via its recurrent connections, but in an implicit manner, thus referred to as {\it implicit rPCN}. We show that the new implicit model also performs AM via covariance learning, and it is equivalent to the explicit rPCN both theoretically and empirically in simple AM tasks, while only employing local Hebbian plasticity. We also show that the implicit model can be further modified to achieve biological resemblance to the hippocampal pyramidal cells by incorporating a dendritic structure while retaining the theoretical and empirical equivalence to the explicit rPCN at convergence. We name it the {\it dendritic rPCN} in this chapter. Importantly, we show that both the implicit and dendritic models can perform more complex AM tasks in which the explicit rPCN would fail due to its unstable dynamics.  Finally, we propose a {\it hybrid PCN} that combines the implicit rPCN with a hierarchical PCN \citep{rao1999predictive, salvatori2021associative} to model the whole hippocampo-neocortical region, and show that it performs challenging AM tasks efficiently. 

In this chapter, we describe a series of more stable and more plausible implementations of recurrent PC to model AM in the hippocampus, to shed light on  how the hippocampal \emph{structure} (apical dendrite and recurrent network) may support its \emph{computations} (PC and covariance learning) underlying its \emph{functions} (AM). The key contribution of this chapter is a reparameterization of the (weighted) prediction errors in the explicit rPCN \citep{friston2003learning}, which leads to simplified forms of a free energy. Crucially, the simplified forms of free energy that we consider all share the same minima or fixed points with that of the explicit rPCN, thereby leading to the same inference and learning. Practically, this allows us to drop certain terms from the gradients, leading to more robust convergence to free energy minima and, crucially, affording more biologically plausible implementation. 
% To unpack the basic ideas of how PC can be implemented in CA3-like recurrent network to support AM, we will focus on a simple kind of PC, in which we ignore temporal predictions (i.e., predicting into the future). Rather, in our models, the prediction of one neuron's activities is from all other neurons in the recurrent network i.e., ``spatial'' prediction. This particular focus enables us to derive interpretable analytical results that enhance our understanding of what is encoded in the synaptic weights in our models i.e., the covariance matrix and the relationship between models. 
% Furthermore, we will restrict our initial analyses to linear systems, under standard Gaussian parametric assumptions. In this setting, a memory is the ability of the PCN to recognize the most likely cause of a particular pattern of inputs (e.g., an image), which is similar to the AM tasks discussed in \citep{salvatori2021associative}. We are not addressing episodic memory --- of the sort sometimes associated with hippocampal function --- rather, we are focusing on how statistical regularities in a series of inputs are learned and then used to predict the missing or noisy part of an input. This contrasts our linear recurrent PCNs with recurrent AM models such as the Hopfield Network \citep{hopfield1982neural}, where the memories are stored as point attractors of the network dynamics. At the end of the Results section, we provide results of an empirical analysis of the attractor behavior of our model, showing that adding nonlinearities to our model will enable it to store memories as point attractors. 

\section{Models}

In this section, we introduce single-layer covariance-learning PCNs, specifically the explicit, implicit, and dendritic rPCNs, arranged in order of increasing biological plausibility. To perform AM tasks, rPCNs first memorize a set of patterns by optimizing their model parameters through gradient descent to minimize an objective function—specifically, the (variational) free energy \citep{friston2003learning}. After memorization, rPCNs are presented with cues, such as corrupted memories, and retrieve the stored patterns by performing inference (or relaxation) of the neurons to minimize the energy function again. Our focus is on the computational processes and corresponding neural implementations of these models, which are designed to simulate the recurrent networks in the CA3 subfield of the hippocampus. We then extend this framework to describe a complete model of the hippocampo-neocortical system. In this model, implicit or dendritic rPCNs represent the hippocampal recurrent network, while the hierarchical PCN \citep{salvatori2021associative} captures the structure of the neocortical hierarchy.

\subsection{Explicit rPCN}

The explicit rPCN \citep{friston2003learning, friston2005theory} was proposed to extend the PC model for the visual system \citep{rao1999predictive} to a general model of how the brain performs representation learning. Following the probabilistic framework of PC, it introduced the covariance matrix by encoding it explicitly into the network's recurrent connections. 
We denote the activity of neurons in a single-layer explicit rPCN by a vector $\mathbf{x}$. The model assumes that the set of patterns to memorize $\{\mathbf{x}(i)\}_{i=1}^N$ of dimension $d$ is a set of samples from a Gaussian distribution with true mean $\bm{\mu}_{\textnormal{true}}$ and covariance matrix $\bm{\Sigma}_{\textnormal{true}}$, where $i$ indicates the $i$th sample within the dataset.  The dynamics of the network model, parameterized by mean $\bm{\mu}$ and covariance $\bm{\Sigma}$, aim to minimize the following negative log likelihood (or free energy) of observed neural activity $\mathbf{x}$ given this Gaussian distribution:
\begin{eqnarray}\label{eq:explicit-energy}
    \mathcal{F}=\frac{1}{2}\log|\bm{\Sigma}|+\frac{1}{2}
    %\sum_{i=1}^N
    (\mathbf{x}-\bm{\mu})^{\top}\bm{\Sigma}^{-1}(\mathbf{x} - \bm{\mu})
\end{eqnarray}

An explicit rPCN learns its parameters by iteratively setting neural activity to training patterns $\mathbf{x}=\mathbf{x}(i)$ and then modifying parameters by directly computing the derivative of $\mathcal{F}$ with respect to $\bm{\mu}$ and $\bm{\Sigma}$ \citep{friston2005theory}:
\begin{eqnarray}
    \bm{\mu} \gets \bm{\mu} + \Delta\bm{\mu} = \bm{\mu} - \alpha\left.\frac{\partial\mathcal{F}}{\partial\bm{\mu}} \right|_{\mathbf{x}=\mathbf{x}(i)}, \quad \bm{\Sigma} \gets \bm{\Sigma} + \Delta\bm{\Sigma} = \bm{\Sigma} - \alpha\left.\frac{\partial\mathcal{F}}{\partial\bm{\Sigma}} \right|_{\mathbf{x}=\mathbf{x}(i)}
\end{eqnarray}

\noindent where $\alpha$ denotes the learning rate for parameters. Notice that this learning rule is fully online as it updates the parameters every time a single training pattern is received. This is closer to learning by biological systems. Here, to derive subsequent analytical properties of the rPCNs, we define the following full-batch learning rules, which can approximate the fully online learning if $\alpha$ is small enough: 
\begin{eqnarray}\label{eq:learning-mu}
    \Delta\bm{\mu} = -\alpha
    \sum_{i=1}^N \left.\frac{\partial\mathcal{F}}{\partial\bm{\mu}} \right|_{\mathbf{x}=\mathbf{x}(i)}
    = \alpha\sum_{i=1}^N \bm{\Sigma}^{-1}(\mathbf{x}(i)-\bm{\mu}),
\end{eqnarray}
\begin{eqnarray}\label{eq:learning-sigma}
    \Delta\bm{\Sigma} = -\alpha
    \sum_{i=1}^N \left.\frac{\partial\mathcal{F}}{\partial\bm{\Sigma}} \right|_{\mathbf{x}=\mathbf{x}(i)}
    =  \alpha \left(-N\bm{\Sigma}^{-1} + \sum_{i=1}^N\bm{\Sigma}^{-1}(\mathbf{x}(i)-\bm{\mu})(\mathbf{x}(i)-\bm{\mu})^{\top}\bm{\Sigma}^{-1} \right)
\end{eqnarray}

With the above full-batch learning rules, both parameters will converge to the maximum likelihood estimate (MLE) of $\bm{\mu}_{\textnormal{true}}$ and $\bm{\Sigma}_{\textnormal{true}}$ based on the training data points, i.e.,  $\bm{\mu} \rightarrow \frac{1}{N} \sum_{i=1}^N \mathbf{x}(i)$ %$\{\mathbf{x}_i\}_{i=1}^N$, 
and $\bm{\Sigma} \rightarrow \frac{1}{N} \sum_{i=1}^N (\mathbf{x}(i)-\bm{\mu})(\mathbf{x}(i)-\bm{\mu})^{\top}$.
We denote these MLE estimates of mean and covariance by $\bar{\mathbf{x}}$ and $S$, respectively.
Therefore, the parameter matrix $\bm{\Sigma}$ will {\it explicitly} encode the sample covariance of the data $S$, thus the name explicit rPCN. 
This can be shown by noting that at convergence $\bm{\mu}$ and $\bm{\Sigma}$ do not change, so setting $\Delta\bm{\mu}=0$ and $\Delta\bm{\Sigma}=0$ and solving Eqs \ref{eq:learning-mu} and \ref{eq:learning-sigma} for $\bm{\mu}$ and $\bm{\Sigma}$, respectively, gives the above MLE estimates. It is worth noting that although we refer to these estimates MLE, in more general formulations with a latent variable $\bfz$ they would correspond to maximum a posteriori (MAP) estimates \citep{friston2003learning,bogacz2017tutorial}; see also Chapter \ref{chap:intro}. However, because we have not placed any prior constraints on the generative model implied by Eq~\ref{eq:explicit-energy}, the MLE and MAP become equivalent. This single-layer formulation of the free energy also circumvents the need for performing optimization over the neuron activities, as there is no hidden neurons. Therefore, unlike hierarchical predictive coding discussed in Chapter \ref{chap:intro} that has a two-step optimization process during training, one for hidden neuron and one for weights, rPCNs discussed in this Chapter only optimize over the weights during training.

After learning, we fix the parameters $\bm{\mu}$ and $\bm{\Sigma}$ and provide a single cue $\tilde{\mathbf{x}}$ to the network, i.e., we initialize the neural activity to $\mathbf{x}=\tilde{\mathbf{x}}$, and the explicit rPCN performs inference on the cue by updating it according to the derivative of the log likelihood:
\begin{eqnarray}\label{eq:explicit-inference}
    \Delta \mathbf{x} = -\beta\frac{\partial\mathcal{F}}{\partial \mathbf{x}} =  \beta \left(-\bm{\Sigma}^{-1}(\mathbf{x} - \bm{\mu})\right)
\end{eqnarray}

\noindent where $\beta$ defines step size along the gradient direction, which we refer to as ``integration step''. For example, if $\tilde{\mathbf{x}}$ is a corrupted or noisy data point, the equation above will drive it 
%to its nearest point within the learned Gaussian density. One can see this from Eq~\ref{eq:explicit-inference} in which the estimate is pulled 
towards the mean in proportion to the precision or inverse variance.

To see how the above equations could be implemented in a neural circuit, it is useful to note that they greatly simplify if we define a vector of prediction errors:
\begin{eqnarray}\label{eq:error-def}
\bm{\epsilon} := \bm{\Sigma}^{-1}(\mathbf{x} - \bm{\mu})
\end{eqnarray}

Then, the dynamics of neurons (Eq \ref{eq:explicit-inference}) becomes:
\begin{eqnarray}\label{eq:explicit-inference-epsilon}
    \Delta \mathbf{x} = -\beta\bm{\epsilon}
\end{eqnarray}

Moreover, the learning rules for this model (Eqs~\ref{eq:learning-mu} and \ref{eq:learning-sigma}) become:
\begin{eqnarray}\label{eq:learning-mu-hebb}
    \Delta\bm{\mu} = \alpha \sum_{i=1}^N \bm{\epsilon}(i)
\end{eqnarray}

\begin{eqnarray}\label{eq:learning-sigma-epsilon}
    \Delta\bm{\Sigma} = \alpha\left(-N\bm{\Sigma}^{-1} + \sum_{i=1}^N \bm{\epsilon}(i)\bm{\epsilon}(i)^{\top}\right)
\end{eqnarray}

The above neural dynamics (Eqs \ref{eq:error-def}-\ref{eq:learning-sigma-epsilon}) can be implemented within the network shown in Fig~\ref{fig:chap3-pcn-all}A. In this network, $\bm{\epsilon}$ and $\mathbf{x}$ are encoded in activities of neurons, and parameters $\bm{\mu}$ and $\bm{\Sigma}$ in synaptic weights. The neurons $\mathbf{x}$, called the value neurons, receive inhibition from prediction error neurons (cf.\ Eq \ref{eq:explicit-inference-epsilon}), while the neurons $\epsilon$ receive excitatory inputs from the value neurons, and top-down inhibitory inputs encoding the prior expectation, as well as lateral inhibitory inputs encoding the weight $\bm{\Sigma}$, to compute the weighted prediction error of Eq \ref{eq:error-def} (for details of how this computation arises see \citep{bogacz2017tutorial}). 

% For figure citations, please use "Fig" instead of "Figure".
% Place figure captions after the first paragraph in which they are cited.
\begin{figure}[ht]

\includegraphics[width=\textwidth]{figs/chap3/PCNs1.pdf}
\caption{{\bf Neural implementations of the single-layer rPCNs using error neurons.} $x_a$ and $\epsilon_a$ denotes the $a$-th value/error neuron in the networks. For simplicity we show the case where the input patterns have only 2 dimensions, corresponding to 2 value and error neurons. A: explicit rPCN originally proposed in \citep{friston2003learning}. B: Implicit rPCN. C: Dendritic rPCN with direct structural mapping to a pyramidal cell. The removal of excitatory connections between $\epsilon$ and $x$ from panel B to C corresponds to the removal of the $\bfW^\top \bm{\epsilon}$ term from Eq. \ref{eq:implicit-inference} to Eq. \ref{eq:dendritic-inference}. Unlabeled connections have strengths 1.}
\label{fig:chap3-pcn-all}
\end{figure}

%We call $\bm{\epsilon}$ the vector of error neurons since they encode the weighted prediction error of the network. The value neurons also receives inhibitory inputs from the corresponding error neurons to perform the inferential dynamics specified in Eq~\ref{eq:explicit-inference}. This network implementation can be considered as a biological neural circuit that generates an internal prediction $\bm{\mu}$ of the inputs $\mathbf{x}$, while the prediction errors are encoded in the error neurons $\bm{\epsilon}$. 

Notice that the learning for the top-down connections encoding the predictions $\bm{\mu}$ (Eq~\ref{eq:learning-mu-hebb}) follows the Hebbian rule, as it is the product of the pre-synaptic activity ($1$) and post-synaptic error activity ($\epsilon$). However, the learning rule for the lateral connections $\bm{\Sigma}$ (Eq~\ref{eq:learning-sigma-epsilon}) is biologically implausible: to compute ${\Sigma}^{-1}_{ab}$ needed for the update $\Delta{\Sigma}_{ab}$ connecting neuron $a$ and $b$,  non-local information from all other entries in $\bm{\Sigma}$ is needed, due to the nature of the inverse operation. That is, to compute the synaptic update between neuron $a$ and $b$, synaptic weights from all over the neural circuit are needed. Moreover, as we will show in the Results section, this numerically unstable inverse term poses significant computational problems that make this model inapplicable to complex patterns, such as the image datasets MNIST \citep{lecun2010mnist} and CIFAR10 \citep{krizhevsky2009learning}, which are not generated by sampling from a Gaussian distribution. 

\subsection{Implicit rPCN}
The biological implausibility of explicit rPCN raises the question of whether a more realistic PC model could perform AM through covariance learning. An alternative approach to encoding interactions between neurons, while preserving the predictive nature of the network, is to allow the neurons to predict each other. Guided by this intuition, we propose the implicit rPCN—a recurrently connected, single-layer network characterized by a weight matrix $\bfW$ and error neurons that encode prediction errors. The implicit model aims to minimize the following energy function:
\begin{eqnarray}\label{eq:implicit-obj}
    \mathcal{F} = \frac{1}{2}\Vert \mathbf{x} -\bfW\mathbf{x} - \bm{\nu} \Vert_2^2,\quad s.t. \quad diag(\bfW)=0
\end{eqnarray}

\noindent where $\mathbf{x}$ and $\bm{\nu}$ are both $d$-dimensional vectors and $W$ is a $d\times d$ 0-diagonal matrix. $W_{ab}$ will be encoded in the synapse connecting the $a$-th and $b$-th neurons in the recurrent network. In the implicit model, we define the errors as $\bm{\epsilon}=\mathbf{x} - \bfW\mathbf{x} - \bm{\nu}$. The 0-diagonal property of $\bfW$ implies that the predictions $\bfW\mathbf{x}$ come from all value neurons in the vector of neurons $\mathbf{x}$ except each neuron itself. Like the explicit model, the implicit rPCN first updates its parameters $\bfW$ and $\bm{\nu}$ by computing the derivative of $\mathcal{F}$ given the dataset $\{\mathbf{x}(i)\}_{i=1}^N$ (again, we present the full-batch learning rules here for subsequent analytical derivations):
\begin{eqnarray}\label{eq:learning-nu}
    \Delta\bm{\nu} = -\alpha\sum_{i=1}^N \left.\frac{\partial\mathcal{F}}{\partial\bm{\nu}} \right|_{\mathbf{x}=\mathbf{x}(i)} = \alpha\sum_{i=1}^N \bm{\epsilon}(i)
\end{eqnarray}
\begin{eqnarray}\label{eq:learning-W}
    \Delta \bfW = -\alpha\sum_{i=1}^N \left.\frac{\partial\mathcal{F}}{\partial \bfW} \right|_{\mathbf{x}=\mathbf{x}(i)} = \alpha\left(\sum_{i=1}^N \bm{\epsilon}(i) \mathbf{x}(i)^{\top}\right)_{diag = 0}
\end{eqnarray}
\noindent where the $()_{diag = 0}$ notation means ``enforcing the diagonal elements to be 0" as we want to keep the 0 diagonal elements of $\bfW$ unchanged. Notice that by setting $\Delta\bm{\nu}$ and $\Delta \bfW$ to 0, we obtain the following relationships at the convergence of learning:
\begin{eqnarray}\label{eq:nu}
    \bm{\nu} = (\mathbf{I}-\bfW)\bar{\mathbf{x}}
\end{eqnarray}
\begin{eqnarray}\label{eq:W}
     \left[(\mathbf{I}-\bfW)\mathbf{S}\right]_{diag = 0} = 0
\end{eqnarray}

Recall that $\bar{\mathbf{x}}$ and $\mathbf{S}$ denote the MLEs of the mean $\bm{\mu}_\textnormal{true}$ and covariance matrix $\bm{\Sigma}_\textnormal{true}$, and
that the parameters of the explicit rPCN, $\bm{\mu}$ and $\bm{\Sigma}$, converge to $\bar{\mathbf{x}}$ and $\mathbf{S}$. Therefore, the implicit rPCN also learns the mean and the elements of the covariance matrix, without encoding them explicitly in individual connections. 

Like the explicit model, after learning, the implicit model performs inference after initializing the activity to a cue input $\tilde{\mathbf{x}}$ following the derivative of the objective function with respect to $\mathbf{x}$:
\begin{eqnarray}\label{eq:implicit-inference}
    \Delta \mathbf{x} = -\beta\frac{\partial\mathcal{F}}{\partial \mathbf{x}} = \beta\left(-\bm{\epsilon} + \bfW^{\top}\bm{\epsilon}\right)
\end{eqnarray}

The above dynamics can be implemented in the network model in Fig~\ref{fig:chap3-pcn-all}B, by replacing the lateral connections $\bm{\Sigma}$ with $\bfW$ that projects the predictions from all other value neurons into each error neuron. Notice that the learning rule for the bias term $\bm{\nu}$ is Hebbian, and more importantly, the learning rule for the connections $\bfW$ is also Hebbian, as it is simply a product $\bm{\epsilon} \mathbf{x}^{\top}$ of the pre- and post-synaptic activities. We show in the Results section, that this biologically more plausible implicit model will converge to exactly the same retrieval of a memory as the explicit model, making it a perfect alternative to the implausible explicit model.


\subsection{Dendritic rPCN}
Inspired by the dendritic model in \citet{sacramento2018dendritic} and \citet{mikulasch2022dendritic}, we push the biological plausibility of the implicit model further by imposing a ``stop-gradient" ($\textnormal{sg}$) operation on the objective function Eq~\ref{eq:implicit-obj}:
\begin{eqnarray}\label{eq:dendritic-energy}
    \mathcal{F} = \frac{1}{2} \Vert \mathbf{x} - \bfW \textnormal{sg}(\mathbf{x}) - \bm{\nu} \Vert_2^2
\end{eqnarray}
\noindent where
\begin{eqnarray}
    \textnormal{sg}(x) = x; \quad \frac{\partial \textnormal{sg}(x)}{\partial x} = 0
\end{eqnarray}

Notice that the parameter learning rules in this model is exactly the same as Eqs~\ref{eq:nu} and~\ref{eq:W}. Only the dynamics of the value nodes during inference becomes:
\begin{eqnarray}\label{eq:dendritic-inference}
    \Delta \mathbf{x} =  
    %-(\mathbf{x} - W\mathbf{x} - \bm{\nu}) = 
    -\beta\bm{\epsilon}
\end{eqnarray}

The above dynamics differs from the implicit model (Eq~\ref{eq:implicit-inference}) that it no longer includes  $\bfW^{\top}\bm{\epsilon}$ term that corresponded to the backward excitatory connections from the error neurons to the value neurons in Fig~\ref{fig:chap3-pcn-all}B. This results in the neural network implementation shown in Fig~\ref{fig:chap3-pcn-all}C. This implementation is simpler,
%could be viewed as more biologically plausible
as the error neurons in this model only receive predictions from external sources, thus can be absorbed into the dendrites of the value neurons $x$, which is shown in the soma-dendrite demonstration in Fig~\ref{fig:chap3-pcn-all}C. 
This architecture could be viewed as more biologically plausible because it does not include the special one-to-one connections of strength $1$ between value and corresponding error neurons present in Fig~\ref{fig:chap3-pcn-all}A and \ref{fig:chap3-pcn-all}B, for which there is no evidence in cortical circuits.
Such an error-encoding dendritic model relates to the model proposed in \citet{sacramento2018dendritic} and \citet{mikulasch2022dendritic}. Since this stop-gradient operation does not affect the learning of parameters in the implicit rPCN, and the fixed points of dynamics for the implicit and dendritic models are the same, the equivalence to the explicit model in converged memory retrieval is retained.
%, and the dendritic model's performance in AM is identical to the performance of the implicit model.


\subsection{Hybrid PCN}
\begin{figure}[ht]
\includegraphics[width=\textwidth]{figs/chap3/full_model.png}

\medskip
\caption{{\bf Multilayer hybrid PCN.}
We use the single-layer implicit/dendritic rPCN to model the hippocampus, and a hierarchical PCN from \citet{salvatori2021associative} to model the sensory cortex and neocortex. For clarity of demonstration, only one layer of our neocortex model is shown. Expanded boxes show the detailed computations within individual neurons and related synapses specified in Eqs~\ref{eq:hy-ff} and \ref{eq:hy-relax}, where $x_a^{(l)}$ denotes the $a$-th neuron in the $l$th layer, and $\Theta_{ab}$ and $W_{ab}$ denote the individual weights from the $a$th to the $b$th neurons. Animal image in this figure is obtained from Wikimedia Commons under a CC BY 4.0 license.}
\label{fig:chap3-hypcn}
\end{figure}

In this section, we propose an architecture that mimics the behavior of the hippocampus as a memory index and generative model. The recurrent, single-layer network models introduced above provide a model for the recurrent dynamics in the hippocampus, specifically the CA3 region that is known to perform pattern completion \citep{o1994hippocampal}. However, raw sensory inputs are first processed via a hierarchy of sensory and neocortical layers, allowing the hippocampus to memorize representations of raw signals, instead of the signal itself. Moreover, according to a recent theory, the hippocampus functions as a generative model that accumulates prediction errors from sensory and neocortical neurons lower in the hierarchy, and sends descending predictions to the neocortex, to correct the prediction errors in the neocortical neurons. The generative model of the hippocampus is updated until the hippocampal predictions correct the prediction errors by suppressing neocortical activity \citep{barron2020prediction}. Hierarchical PCNs have already been proposed as a generative model that learn representations of the sensory inputs \citep{friston2005theory}. Particularly, it has been shown that a purely hierarchical PCN, without any recurrent structure, is able to perform associative memory tasks on highly complex datasets \citep{salvatori2021associative}. Here, we combine the hierarchical PCN with our proposed recurrent architecture, obtaining an hierarchical model with recurrent dynamics at the topmost layer. What results is an \emph{hybrid} network that models the whole pathway from sensory neurons to hippocampal neurons. 

Consider a hierarchical PCN with $L$ layers, with a recurrent implicit or dendritic network connected to the last layer. The first layer corresponds to the sensory layer, where sensory signals are presented to be processed, and the last layer corresponds to the hippocampal layer. Then, the intermediate layers $1<l<L$ represent the hierarchical structure present in the neocortex that connects the sensory neurons to the hippocampus. We denote the synaptic weight matrix connecting the neurons in the $l$th layer and the neurons in the $(l+1)$th layer as $\bm{\Theta}^{(l)}$. Within the hippocampal layer $L$, we use $\bfW$ to denote the recurrent synaptic weight matrix. The vector of value nodes in each layer, denoted as $\mathbf{x}^{(l)}$, is coupled with the error node vector $\bm{\epsilon}^{(l)}$, computed as the following prediction error:
\begin{eqnarray}
    \bm{\epsilon}^{(l)} = \mathbf{x}^{(l)} - \bm{\rho}^{(l)}
\end{eqnarray}

\noindent where $\bm{\rho}^{(l)}$ denotes the descending prediction signals into the neurons, computed as:
\begin{eqnarray}\label{eq:hy-ff}
    \bm{\rho}^{(l)} = \begin{cases}
			      \bfW\mathbf{x}^{(l)}, 
			      & \text{if $l=L$ and implicit}\\
			      \bfW\textnormal{sg}(\mathbf{x}^{(l)}), 
			      & \text{if $l=L$ and dendritic}\\
                  \bm{\Theta}^{(l)}f\left(\mathbf{x}^{(l+1)}\right), & \text{otherwise}
		          \end{cases}
\end{eqnarray}

\noindent where $f(\cdot)$ is a nonlinear function. For simplicity of illustration, we ignore the top-down connections $\bm{\nu}$ in rPCNs. The aim of the learning dynamics of this hybrid network is to minimize the sum of squared errors:
\begin{eqnarray}
    \mathcal{F} = \frac{1}{2}\sum_{l=1}^L \Vert\bm{\epsilon}^{(l)}\Vert_2^2 
\end{eqnarray}

Like purely hierarchical PCNs, the learning of the hybrid PCN consists of two stages: {\it inference} and {\it weight update}. During inference, the value neurons relax to maximize $\mathcal{F}$ following the gradient descent rule:
\begin{eqnarray}\label{eq:hy-relax}
    \Delta \mathbf{x}^{(l)} = \begin{cases}
    \beta\left(-\bm{\epsilon}^{(l)} + f^{\prime}\left(\mathbf{x}^{(l)}\right)\odot \left(\bm{\Theta}^{(l-1)}\right)^{\top} \bm{\epsilon}^{(l-1)} + \gamma \bfW^{\top}\bm{\epsilon}^{(l)}\right), & \text{if $l=L$}\\
    \beta\left(-\bm{\epsilon}^{(l)} + f^{\prime}\left(\mathbf{x}^{(l)}\right)\odot \left(\bm{\Theta}^{(l-1)}\right)^{\top} \bm{\epsilon}^{(l-1)}\right), & \text{if $1<l< L$}\\
    \mathbf{0}, & \text{if $l$=1}
    \end{cases}
\end{eqnarray}

\noindent where $\odot$ denotes the element-wise product between two vectors, and $\gamma=1$ if the topmost layer is an implicit rPCN, or $0$ if it is a dendritic rPCN. The update at the sensory layer ($l=1$) is 0 because during learning, the sensory neurons are fixed at the raw sensory inputs. The inference dynamics are carried out for a certain number of iterations until the value nodes reach an equilibrium. The weight update is then performed, also to maximize $\mathcal{F}$ using gradient descent:
\begin{eqnarray}
    \Delta \bm{\Theta}^{(l)} = \alpha\bm{\epsilon}^{(l)}f\left(\mathbf{x}^{(l+1)}\right)^{\top}
    \ \ \text{and} \ \
    \Delta \bfW = \alpha\bm{\epsilon}^{(L)}\left(\mathbf{x}^{(L)}\right)^{\top}
\end{eqnarray}

In each iteration of learning, the inference will be performed for multiple iterations and then the weight update will take place for one step. The whole learning process will be iterated multiple times until $\mathcal{F}$ reaches the minimum.


The proposed architecture also models the reconstructions of past memories: as a memory index, the hippocampus sends top-down information to the neocortical neurons to reinstate activity patterns that replicate previous sensory experience \citep{barron2020prediction}. In the reconstruction phase of our generative model, the hippocampal layer provides descending inputs to the sensory neurons to generate stored data points. Assume we have a hybrid PCN already trained on a dataset of memories. To retrieve a stored datapoint, the synaptic weights $\bm{\Theta}$ and $\bfW$ are fixed, and the retrieval process is triggered by providing a partly corrupted version of a memorized pattern to a subset sensory neurons. During retrieval all other value nodes will relax following the same rule specified in Eq~\ref{eq:hy-relax}. 
%A sketch of the general structure of the hybrid PCN, as well as detailed implementations of local neural computations is presented in Fig~\ref{fig:chap3-hypcn}. 
The only difference is that, the value nodes in the sensory layer will also experience relaxation:
\begin{eqnarray}
    \Delta \mathbf{x}_{c}^{(1)} = - \beta \bm{\epsilon}_{c}^{(1)}
\end{eqnarray}
\noindent where the subscript $c$ denote the corrupted dimensions of the input pattern, as we keep the intact part of the patterns unchanged during retrieval. Notice that all the dynamics above requires only local computations. Fig~\ref{fig:chap3-hypcn} shows the general structure of the hybrid PCN, as well as detailed implementations of neural computations. 


\section{Results}
\subsection{Explicit rPCN performs associative memory}

The explicit rPCN was originally proposed as a generative model of how cortical responses are evoked given observations \citep{friston2003learning, friston2005theory}. To our knowledge, it has never been employed to perform AM tasks. To verify our hypothesis that, after training, the explicit rPCN can perform AM using its inferential dynamics and learned covariance, we designed a simple task where multiple $5\times 5$ random Gaussian patterns are memorized. We then covered the bottom two rows of the $5 \times 5$ patterns with 0s and run inference on these corrupted entries only. Examples of the original patterns, along with the corrupted and retrieved patterns, can be found in Fig~\ref{fig:chap3-equivalence}A. In this task, the models can be considered as memorizing the association between the top 3 rows and bottom 2 rows of the random patterns. As can be seen in the third column of panel A, the single-layered explicit rPCN can retrieve the covered part of the original pattern well: it performs AM. We then measured the MSEs between the retrieved and original $5 \times 5$ random patterns by these models, and plotted them as a function of corrupted/masked pixels in Fig~\ref{fig:chap3-equivalence}B (Fig~\ref{fig:chap3-equivalence}A shows examples when there are $10$ masked pixels). As can be seen, the explicit rPCN has slightly higher retrieval MSEs on average, although this performance gap is not visually observable from the retrieved patterns (e.g., Fig~\ref{fig:chap3-equivalence}A). 
% As we will show in the following sections, this performance gap will be exacerbated when these models are trained to memorize structured and more complex datasets.


\begin{figure}[t]
\includegraphics[width=\textwidth]{figs/chap3/equivalence.png}
\caption{{\bf Performance of rPCNs in AM of random patterns, and the equivalence between them.} A: A subset of $5\times 5$ random patterns memorized by all 3 models. After training, we corrupted the bottom 2 rows (10 pixels) and let the networks run inference on the corrupted parts for retrieval. B: Retrieval MSEs of the models when corrupted with different mask sizes. Experiments in A and B are performed with networks with $d=25$ neurons. C: Sample covariance of a random 2-dimensional dataset and the learned weight matrices of an explicit model and an implicit/dendritic model on this dataset. D: The random 2-dimensional dataset to memorize, and the linear retrieval obtained by masking the second dimension $x_2$ by all 3 models, as well as the theoretical retrieval line. All the lines overlap as they are equivalent in theory. Experiments in C and D are performed with networks with $d=2$ neurons.}
\label{fig:chap3-equivalence}
\end{figure}

We now show that a theoretical result can be derived to 
analytically describe the pattern retrieved
%show the retrieval of the corrupted part of a pattern given 
by an explicit rPCN.
We will use this result later to establish equivalence between explicit and implicit rPCNs.
% why and how an explicit rPCN retrieves the corrupted part of a pattern.
Consider a corrupted pattern $\mathbf{x}\in\mathbb{R}^d$, which can be divided into the corrupted part $\mathbf{x}_k\in\mathbb{R}^k$ and ``intact" part $\mathbf{x}_m\in\mathbb{R}^m$, where $k+m=d$:

\begin{eqnarray}
    \mathbf{x} = \begin{bmatrix}\mathbf{x}_k \\ \mathbf{x}_m\end{bmatrix}
\end{eqnarray}

At the time of retrieval, the training of the rPCN has already finished, during which the network has memorized the association between $\mathbf{x}_k$ and $\mathbf{x}_m$. We assume that the training has converged, so that the connections $\bm{\mu}$ and $\bm{\Sigma}$ explicitly encode the MLE, $\bar{\mathbf{x}}$ and $\mathbf{S}$, of the mean and covariance respectively. In this case, the inference dynamics of the explicit model follows $\Delta \mathbf{x} = \beta \mathbf{S}^{-1}(\mathbf{x}-\bar{\mathbf{x}})$ (Eq~\ref{eq:explicit-inference}). To correspond to the division of the corrupted and intact parts of $\mathbf{x}$, we also divide $\bfS$ and $\bar{\mathbf{x}}$ into blocks and set $\Delta \mathbf{x} = 0$ to study the convergence of the inference stage:

\begin{eqnarray}\label{eq:blocked-matrices}
    \bfS = \begin{bmatrix}\bfS_{kk} & \bfS_{km} \\ \bfS_{mk} & \bfS_{mm}\end{bmatrix},\quad
    \bar{\mathbf{x}} = \begin{bmatrix}\bar{\mathbf{x}}_k \\ \bar{\mathbf{x}}_m\end{bmatrix}
\end{eqnarray}

%Notice that 
\noindent where $\bfS_{pq}$, $p,q\in\{k, m\}$
denotes the $p\times q$ submatrices of the covariance matrix $\bfS$. Using Eq~\ref{eq:blocked-matrices} above, we can have the following theorem:

\begin{theorem}
    After training, an explicit rPCN retrieves the corrupted $\mathbf{x}_k$ using the intact $\mathbf{x}_m$ following the dynamics $\Delta \mathbf{x} = \beta \bfS^{-1}(\mathbf{x}-\bar{\mathbf{x}})$, and the retrieval dynamics on $\mathbf{x}_k$ converge to:
    \begin{eqnarray}\label{eq:retrieval}
    \hat{\mathbf{x}}_k = \bfS_{km}\bfS_{mm}^{-1}(\mathbf{x}_m - \bar{\mathbf{x}}_m) + \bar{\mathbf{x}}_k
\end{eqnarray}
\end{theorem}

\noindent where $\hat{\mathbf{x}}$ denotes the retrieval of the corrupted part. This theorem describes the activities in the network after retrieval given by the explicit rPCN, using the learned parameters and the intact $\mathbf{x}_m$.
% This theorem gives rise to the retrieval equation describing how the explicit rPCN recovers the corrupted part of the pattern $\mathbf{x}$ using the learned parameters and the intact $\mathbf{x}_m$.
Details of the derivation can be found in Appendix \ref{app:chap3}.

\subsection{Explicit and implicit rPCNs retrieve equivalent memories}

Having established the theoretical foundation of the retrieval dynamics in the explicit rPCN, we next consider the theoretical aspects of the implicit and dendritic models: what is the retrieval of memorized patterns given by these models, based on learned associations? Again consider the case of a vector $\mathbf{x}$ consisting of the top $k$ corrupted entries $\mathbf{x}_k$ and bottom $m$ intact entries $\mathbf{x}_m$, and assume that the learning has converged when the corrupted $\mathbf{x}$ is presented to the network, which gives the conditions in Eq~\ref{eq:nu} and \ref{eq:W} for both implicit and dendritic models (notice that these two models do not differ in parameter learning). We now write the model parameters $\bm{\nu}$ and $W$ into block matrices:

\begin{eqnarray}\label{eq:nu-and-mu}
    \bm{\nu} = \begin{bmatrix}\bm{\nu}_k \\ \bm{\nu}_m
    \end{bmatrix}, \quad \bfW = \begin{bmatrix}\bfW_{kk} & \bfW_{km}\\\bfW_{mk} & \bfW_{mm}\end{bmatrix}
\end{eqnarray}

\noindent where $\bfW_{pq}$, $p,q\in\{k, m\}$ denotes the $p\times q$ submatrices of the weight matrix $\bfW$. Notice that for both implicit and dendritic models, the retrieval dynamics converge if and only if all error nodes become zero, that is, $\bm{\epsilon} = \mathbf{x}-\bfW\mathbf{x}-\bm{\nu} = 0$.
%, because then the negative energy reaches its maximum value $\mathcal{F}=0$ and cannot be further optimized by adjusting neural activity.
For the dendritic model this is obvious (Eq~\ref{eq:dendritic-inference}), whereas for the implicit model this comes from the fact that $W$ has all its diagonal entries equal to $0$ and thus cannot be an identity matrix (Eq~\ref{eq:implicit-inference}). We can thus derive the following theorem on the converged retrieval dynamics of the implicit and dendritic rPCNs:

\begin{theorem}
    After training, both the implicit and dendritic models retrieve the corrupted $\mathbf{x}_k$ given intact $\mathbf{x}_m$ following the dynamics specified in Eqs~\ref{eq:implicit-inference} and~\ref{eq:dendritic-inference}, which converge to the following equilibrium:
    \begin{eqnarray}
        \bfW_{km}(\mathbf{x}_m - \bar{\mathbf{x}}_m) = (\mathbf{I}_{k} - \bfW_{kk})(\mathbf{x}_k - \bar{\mathbf{x}}_k)
    \end{eqnarray}
    where $\mathbf{I}_{k}$ is the $k\times k$ identity matrix. Using Eq~\ref{eq:W}, this equation is equivalent to:
    \begin{eqnarray}\label{eq:retrieval2}
        \hat{\mathbf{x}}_k = \bfS_{km}\bfS_{mm}^{-1}(\mathbf{x}_m - \bar{\mathbf{x}}_m) + \bar{\mathbf{x}}_k
    \end{eqnarray}
\end{theorem}


The details of the derivations can be found in Appendix \ref{app:chap3}. Notice that the equilibrium condition for implicit and dendritic rPCNs is exactly the same as that for the explicit rPCN, specified in Eq~\ref{eq:retrieval}. This equivalence was also verified empirically in a simple 2-dimensional example shown in Fig~\ref{fig:chap3-equivalence}D. In this example both $\mathbf{x}_k$ and $\mathbf{x}_m$ are single-dimensional scalars, and all three models retrieved the corrupted $x_2$ following the same linear equation (Eq~\ref{eq:retrieval}). Notice that this line is also the least squares regression line, where $x_2$ is the dependent variable. This also implies that the memory ``attractors'' learned by the model in $2$-dimension is a line, instead of individual points, which is different from the attractors learned by classical models such as Hopfield Networks \citep{hopfield1982neural}. In the final section of the Results, we provide an empirical investigation into this difference. Fig~\ref{fig:chap3-equivalence}C also shows that the weight matrix $\bm{\Sigma}$ of the explicit rPCN directly encodes the sample covariance matrix, whereas the other two models encode it implicitly. Due to their equivalence, the implicit and dendritic models also perform AM on the random Gaussian patterns in Fig~\ref{fig:chap3-equivalence}A. 

At this point, we have established the complete equivalence between the explicit and implicit/dendritic rPCNs, showing that the latter models can serve as perfect substitutes for the explicit model. However, in contrast to the explicit rPCN, where the learning rule (Eq~\ref{eq:learning-sigma}) for the connections $\bm{\Sigma}$ employs non-local information,
%and is numerically unstable, 
the plasticity rules (Eq~\ref{eq:learning-W}) for the implicit and dendritic models are entirely Hebbian.
%and have no computational issues. 
Moreover, as was shown above, the neural implementation of a dendritic rPCN can be mapped to the structure of hippocampal pyramidal cells. Therefore, our proposed models are equivalent to but biologically more plausible than the explicit rPCN.

\subsection{Relationship between rPCNs and Hopfield Networks}\label{sec:rpcn-hn}

% NEW-CONTENT-IN-THESIS
The ability of explicit and implicit rPCNs to retrieve patterns, as shown in Fig.~\ref{fig:chap3-equivalence}, is a well-known property of Hopfield Networks \citep{hopfield1982neural} when the patterns consist of binary entries. Moreover, we have also demonstrated that, like Hopfield Networks, the implicit rPCN also learns the covariance. What is the exact relationship between these models? The following theorem demonstrates that both types of models perform \emph{metric learning} during the process of parameter optimization:

\begin{theorem}\label{thm:metric_learning}
Consider a collection of memorized patterns $\mathbf{X} = \{\mathbf{x}(i)\}_{i=1}^N$ with zero mean. The energy functions of both rPCNs and Hopfield Networks can be expressed as a metric learning function with a linear embedding transformation:
\begin{equation} \label{eq:general_metric}
d^2_{\mathbf{L},\mathbf{X}}(\mathbf{q}) :=  c\| \mathbf{L}(\mathbf{X})\mathbf{q} \|^2_2
\end{equation}
where $\mathbf{q}$ is a query presented to the model after training (e.g., a corrupted version of a memorized pattern), $c$ is a model-specific scalar constant, and $\mathbf{L}(\mathbf{X})$ is a model specific matrix.
\end{theorem}
The $c$ and $\mathbf{L}$ parameters for rPCNs and Hopfield Network are summarized in the following table:

\begin{table}[h]
    \centering
    \begin{tabular}{|c|c|c|}
        \hline
        \textbf{Model} & $c$ & $\mathbf{L}(\mathbf{X})$ \\
        \hline
        Hopfield Network & $\sfrac{-1}{2}$ & $\bfS^{\frac{1}{2}}$ \\
        Explicit rPCN & $\sfrac{1}{2}$ &  $\bfS^{-\frac{1}{2}}$\\
        Implicit rPCN & $\sfrac{1}{2}$ & $\textnormal{diagMat}(\mathbf{1}\oslash \textnormal{diag}(\bfS^{-1}))\bfS^{-1}$ \\
        \hline
    \end{tabular}
    \caption{AM as metric learning.}
    \label{tab:metric}
\end{table}
\noindent where $\bfS$ is the covariance matrix of the patterns, $\oslash$ is the Hadamard (element-wise) division, $\mathbf{1}$ denotes the $1$-vector, diag extracts the diagonal elements of a matrix and converts them into a $d$-dimensional vector, and diagMat converts a vector into a diagonal matrix. The proof of these results can be found in the Appendix \ref{app:chap3}. In particular, for implicit rPCN, the individual entries of the parameter $\mathbf{L}(\mathbf{X})$ follows:
\begin{eqnarray}
    {L}_{ij}(\mathbf{X}) = (\textnormal{diagMat}(\mathbf{1}\oslash \textnormal{diag}(\bfS^{-1}))\bfS^{-1})_{ij} = 
        \frac{(\bfS^{-1})_{ij}}{(\bfS^{-1})_{ii}},
\end{eqnarray}
i.e. entries of the inverse of $\bfS$ weighted by its own diagonal entries.

\begin{figure}[t]
    \centering
    \includegraphics[width=\linewidth]{figs/chap3/binary.pdf}
    \caption{\textbf{Comparison of the Hopfield Network and rPCNs in binary AM task.} A: Example binary pattern and its corruption and retrievals. B: Retrieval MSE of Hopfield Network and rPCNs across different numbers of masked dimensions. C: Example binary MNIST images and retrievals by different models. D: Retrieval MSEs as a function of number of memorized binary MNIST images.}
    \label{fig:chap3-binary}
\end{figure}

This theorem establishes a formal relationship between Hopfield Networks and rPCNs, showing that both models use the covariance matrix to implement AM. Empirical tests further illustrate this shared mechanism. To evaluate the models' ability to memorize and retrieve binary patterns effectively, we generated randomly two $5$-dimensional binary patterns $\{-1,1\}^5$ for memorization. We then corrupted these patterns by masking the last few dimensions with zeros and examine the retrieval performance of the models. Fig~\ref{fig:chap3-binary}A displays an example of an original pattern, a corrupted version with the last two dimensions masked, and the retrievals produced by a Hopfield Network, implicit rPCN, and explicit rPCN. All three models perform identically in this case. However, as the number of corrupted dimensions increases, Hopfield Networks exhibit higher retrieval mean squared errors (MSEs) compared to rPCNs  (Fig~\ref{fig:chap3-binary}B). Notably, the two rPCNs have identical performances across different numbers of masked dimensions, due to their theoretical equivalence and the simplicity of the patterns. We then repeat the same experiments on binary MNIST images of dimension $28\times 28$, where we masked the bottom half of the images and query the models to retrieve the original images. Fig~\ref{fig:chap3-binary}C presents two examples of the retrievals when the total number of memories is $4$. Implicit and dendritic rPCNs are successful in retrieving the binary memories, whereas the explicit rPCN can only retrieve blurry versions of the memory. On the other hand, Hopfield Network fails in this more challenging task, only able to retrieve noisy versions of the memories. We systematically studied the retrieval MSEs of these models in Fig~\ref{fig:chap3-binary}D. As the number of memories is small, Hopfield Network can almost retrieve the memories perfectly, although the performance deteriorates quickly as the number of images increases. For rPCNs, the implicit and dendritic models have stable and good performance across all memory sizes, whereas the explicit rPCN have higher MSEs. As we will also demonstrate in the next section with continuous memories, this performance difference is due to the numerical instability in the dynamics of explicit rPCN. Also note that binary patterns are used here because the classical Hopfield Network can only process such patterns. To memorize patterns with continuous values, modern variations of Hopfield Networks \citep{ramsauer2020hopfield} are needed. On the other hand, rPCNs can work with both continuously and discretely distributed patterns, without modifying any of their dynamics and architectures.


\subsection{Model performance in AM with structured image data}
\begin{figure}[t]
\includegraphics[width=\textwidth]{figs/chap3/mnist.pdf}
\caption{{\bf Performance of the single-layer rPCNs in AM of structured images.} A: Examples of retrieved MNIST (top) \citep{lecun2010mnist} and grayscale CIFAR10 (bottom)\citep{krizhevsky2009learning} images by explicit, implicit and dendritic models. All models here are trained to memorize $64$ images. For MNIST, the networks have $d=784$ neurons; for grayscale CIFAR10, $d=1024$. B: Retrieval mean squared errors (MSEs) of the single-layer models across multiple numbers of training memories ($N$). C: Evolution of the retrieval MSEs of the implicit and dendritic models when $N=256$. D: Example eigenspectra of the weight matrices defining the inferential dynamics for the dendritic (left) and implicit (right) rPCNs. Error bars obtained by $5$ different seeds for image sampling. Please see main text for an explanation of the matrix $\bfM$.}
\label{fig:chap3-mnist}

\end{figure}

In each learning iteration, the explicit rPCN needs to compute the inverse of the current weight matrix. In practice, this works well when the underlying dataset has some specific regularities, but becomes problematic when dealing with structured data, such as images of handwritten digits and natural objects. In this section, we show that the explicit model can no longer perform stable memorization and retrieval of more complex and structured data, whereas the implicit and dendritic models are able to perform AM on such data with a high level of precision, and are hence interesting from an application perspective. To do that, we replicate the pattern completion experiment performed in Fig~\ref{fig:chap3-equivalence}, but using images sampled from the MNIST \citep{lecun2010mnist} and grayscale CIFAR10 \citep{krizhevsky2009learning} datasets. Fig~\ref{fig:chap3-mnist}A shows some retrieved examples from both datasets by all three types of single-layer rPCNs, which were trained to memorize $64$ images. The visual results suggest that explicit rPCN can no longer retrieve clear images, whereas both implicit and dendritic models can obtain visually perfect retrievals of the memories. The failure of the explicit rPCN with these structured images is due to the need to compute the inverse of $\bm{\Sigma}$ in each iteration (Eq~ \ref{eq:learning-sigma}): for low-dimensional patterns with some specific regularities such as Gaussian distribution, the inverse can be precisely computed. However, when encountered with high-dimensional data such as structured images, the inverse computation can become imprecise, and the computational errors may accumulate when the model is trained iteratively, leading to blurry retrievals.

We next perform a quantitative analysis of the performance of these models, by measuring the mean squared errors (MSEs) between the original and retrieved grayscale CIFAR10 images across different numbers of images we train the models to memorize ($N$). Fig~\ref{fig:chap3-mnist}B shows that the explicit model indeed has much larger retrieval MSEs than the implicit and dendritic model, confirming our visual observations. The MSE gap is also larger than that in Fig~\ref{fig:chap3-equivalence}A, where the random Gaussian patterns makes the learning of the explicit model more stable when inverting $\bm{\Sigma}$. 

Interestingly, Fig~\ref{fig:chap3-mnist}B also shows that despite the equivalence between the implicit and dendritic models when their inferential dynamics (Eqs~\ref{eq:implicit-inference}, \ref{eq:dendritic-inference}) have converged, the dendritic model has larger retrieval MSEs than the implicit model, especially when $N$ is large. We investigate this phenomenon by plotting the evolution of the retrieval MSEs of both implicit and dendritic models during inference, when $N=256$. As can be seen in Fig~\ref{fig:chap3-mnist}C, the MSE of the dendritic model fails to decrease to the same (lower) value as that of the implicit model, and it also fails to converge. To understand the distinction between the implicit and the dendritic rPCNs in inference, recall that their inferential dynamics are described by two linear differential equations (Eqs~\ref{eq:implicit-inference}, \ref{eq:dendritic-inference}). We now define:
\begin{eqnarray}\label{eq:M}
    \mathbf{M} = \bfW - \mathbf{I}
\end{eqnarray}

\noindent where $\bfW$ is the learned weight matrix in the implicit/dendritic models (note that the learning of these two models are identical) and $\mathbf{I}$ is the identity matrix of the same size as $\bfW$. We can therefore rewrite the inferential dynamics as:
\begin{eqnarray}\label{eq:inference-M}
    \Delta \mathbf{x}_{\textnormal{im}} = \beta (-\mathbf{M}^{\top}\mathbf{M}\mathbf{x}), \quad \Delta \mathbf{x}_{\textnormal{den}} = \beta (\mathbf{M}\mathbf{x})
\end{eqnarray}

\noindent where ``im'' stands for implicit, and ``den'' stands for dendritic. Notice that in general, the stability of a linear dynamical system $\Delta \mathbf{x}= \mathbf{A}\mathbf{x}$ for some matrix $\bfA$ is determined by the eigenvalues of $\bfA$: the system is stable if and only if the real part of all eigenvalues of $\bfA$ are non-positive \citep{arnold1992ordinary}. As we show in Fig~\ref{fig:chap3-mnist}D, the largest eigenvalue of $-\bfM^{\top}\bfM$ is $0$, suggesting stable inferential dynamics for the implicit rPCN. This is also generally true as $-\bfM^{\top}\bfM$ defines a negative (semi-)definite matrix for any $\bfM\neq 0$, which has all eigenvalues smaller than or equal to 0. On the other hand, Fig~\ref{fig:chap3-mnist}D shows that the largest eigenvalue of $\bfM$ is greater than $0$, indicating unstable dynamics for the dendritic model. Thus, although the implicit and dendritic models have the same fixed point of their dynamics during retrieval,
%converge to the same retrieval in inference, 
the neural dynamics defined by the dendritic model may become unstable, preventing itself from the convergence. 
%Since the weight matrix $W$ (and thus $M$) is related to the covariance matrix $S$ of inputs by Eq~\ref{eq:W}, we suspect that the unstable dendritic dynamics is due to the irregularities in $S$, akin to the reason for the unstable learning dynamics for the explicit model. 
For the dendritic model there are no theoretical reasons that guarantee negative eigenvalues of $\bfM$, so they may become positive as the values in that matrix grow with learning more patterns. The experiments with MNIST \citep{lecun2010mnist} yielded similar results, and we describe them in \hyperref[S1_Figure]{S1 Fig}.

\subsection{Performance of hybrid models with natural images}
\begin{figure}[t]
\centering
\includegraphics[width=0.9\textwidth]{figs/chap3/multilayer.png}
\caption{{\bf Performance of the multi-layer models.} A: 
Example performance on $100$ images of the ImageNet dataset by a $7$-layer hybrid model with $1024$ hidden neurons per layer, with an implicit rPCN as the topmost layer. B: 
Demonstration of how we keep the number of parameters across different models to be roughly the same. 
B: Retrieval mean squared errors (MSEs) of the multi-layer models across multiple numbers of training memories ($N$). The curve for the implicit model is the same as the one in Fig~\ref{fig:chap3-mnist}. C: Evolution of the retrieval MSEs of the implicit and dendritic hybrid models when $N = 256$. Error bars obtained by $5$ different seeds for image sampling.}
\label{fig:chap3-multilayer}

\end{figure}

While performing well on small subsets of structured data, the purely recurrent implicit/dendritic models present some problems: first, the recurrent structure only takes into account the hippocampal structure, but does not correctly represent the hierarchical structure that connects it to the neocortex; second, the number of parameters is quadratic to the dimension of the inputs, which does not allow it to work on high-dimensional images, such as high-quality images. For example, consider a recurrent network trained on $224 \times 224$ pictures of the ImageNet dataset \citep{deng2009imagenet}. Every image consists of $\sim 150000$ pixels, and hence needs a network with the same number of neurons. A recurrent network of this size would be impossible to train without an exceptional amount of computational power, as it has $\sim 40$ billion parameters. On the other hand, a hierarchical structure that precedes the hidden implicit layer guarantees the flexibility of choosing the number of parameters. For example, a network with $7$ hidden layers, followed by a recurrent implicit layer, all of dimension $1024$, would be more than $200$ times smaller than the above implicit model for ImageNet, and hence feasible to be trained. In fact, 
% as Fig~\ref{fig:chap3-multilayer}A shows, 
we have successfully trained this model on high-dimensional ImageNet pictures. Particularly, we trained this $7$-layer network with $100$ samples from the ImageNet dataset. We then defined successful retrievals as those with retrieval MSEs smaller than $5e^{-3}$, and found that when presenting the network a partial cue consisting of $1/2, 1/4$ and $1/8$ of the original pixels, the model has successfully recovered, respectively and on average, 100, 97, and 44 of the original memories. Similar performance was also obtained with the colored CIFAR10 dataset \citep{krizhevsky2009learning} and examples of colored CIFAR10 images can be found in \hyperref[S2_Figure]{S2 Fig}.

We then examine whether the hybrid structure leads to better retrieval performances compared to the single-layer models. To do that, we trained an implicit rPCN and a hybrid model with a topmost implicit rPCN to memorize the same subsets of grayscale CIFAR10 images ($1024$ pixels in total)  of varying numbers of images $N$, and initialized the retrieval with half-covered partial cues. 
To compare with earlier works \citep{salvatori2021associative} fairly, we also trained a purely hierarchical PCN to perform the same AM task.
The sizes of the networks are chosen such that they have approximately the same number of parameters. 
The hybrid model we use has $3$ layers: the sensory layer has $1024$ neurons corresponding to the pixel space, with $512$ neurons in the hidden layer, and $512$ neurons in the topmost implicit layer. 
We construct the hierarchical model by replacing the topmost implicit recurrent layer with $2$ hierarchical layers of size $512$. We chose such a configuration of hidden sizes and number of layers because in general, an implicit rPCN with $d$ neurons (for $d$-dimensional inputs) will have $d(d-1)$ parameters. A hybrid model with $d$ sensory neurons, one feedforward hidden layer of size $d/2$ and a topmost implicit layer of size $d/2$ will have approximately the same number of parameters. Replacing the final implicit layer with two feedforward layers of the same size results in a hierarchical PCN with roughly the same number of parameters. This ensures the fairness of comparison across models, and is illustrated in Fig~\ref{fig:chap3-multilayer}B, where we also included the number of neurons in each layer used in our experiments next to each layer, as well as the number of connections at the top. As Fig~\ref{fig:chap3-multilayer}C shows, the performance of the implicit model is in fact slightly better than the hybrid implicit model and the purely hierarchical model when they have the same number of parameters. Indeed, the retrieval MSE of the implicit model should be the smallest among all linear retrieval functions: notice that the retrieval $\hat{\mathbf{x}}_k$ of the implicit model defined in Eq~\ref{eq:retrieval} is the least squares solution if we regress $\mathbf{x}_k$ on $\mathbf{x}_m$. Although the exact retrieval function of the hybrid and hierarchical models is unclear due to their nonlinearity, it has been shown that a two-layer linear PCN with $1$ neuron in the hidden layer yields linear retrievals following the principal components of the sensory data \citep{whittington2017approximation}, shedding light on the slightly worse performance of the multi-layer models. However, we note that the similar performance in retrieval fidelity between the implicit and hybrid models does not make the hierarchical structure in the hybrid model redundant,
% However, we note that the better performance of the single-layer implicit model in these exact memorization tasks does not make it a generally superior architecture to the hybrid model,
as the functionality of the brain is far more heterogeneous than merely memorization, and many functions require the hierarchical structure. For example, the hierarchical structure can support the learning of meaningful representations of the sensory inputs, which can be utilized to perform other tasks such as classification, while paying only a negligible cost in memory retrieval tasks.

We further compared the implicit and dendritic models when they are ``plugged in'' as the topmost recurrent layer in the hybrid model. Fig~\ref{fig:chap3-multilayer}C shows that the hybrid dendritic model performs identically to the hybrid implicit model, contrasting their performance difference in the single-layer case. During the inferential iterations, the retrieval MSEs of both models also converge stably to the same value (Fig~\ref{fig:chap3-multilayer}D). In other words, the introduction of the neocortical layers remedies the unstable inferential dynamics due to the biologically more plausible dendritic structure in our models. We suspect that the hierarchical pre-processing of the sensory inputs regularizes the covariance matrix, such that the topmost recurrent weight matrix $\bfW$ defines stable inferential dynamics for the dendritic layer. However, since the top-layer inference is no longer a simple linear dynamical system (Eq~\ref{eq:hy-relax}), the connections between a regularized $\bfW$ and the stability of the dynamics may not be straightforward.

\subsection{Nonlinear rPCN learns individual attractors}
\label{sec:nonlinear}

\begin{figure}[t]
\centering
\includegraphics[width=0.9\textwidth]{figs/chap3/attractors.pdf}
\caption{{\bf Comparison of linear and nonlinear implicit rPCN.} A: Performance of linear and nonlinear implicit models in the completion task with varying $N$s. B: Same as A, but with the denoising task, where cues are memories with Gaussian noise of variance $0.1$. C: A simple 3-dimensional example, where stars are data points the networks were trained to memorize. After training we ran inference on both linear and nonlinear models, initialized with grid test data drawn from the range $[-1, 1]^3$. The position of the test data at convergence of inference indicates the shape of attractors. Images taken from the CIFAR10 dataset \citep{krizhevsky2009learning}.}
\label{fig:chap3-attractors}
\end{figure}

The implicit rPCN we have investigated so far assumes that the relationship between neurons in the recurrent network is linear, i.e., projections into neuron $i$ is $\sum_{j\neq i}W_{ij}x_j$. Here, we introduce an ad hoc nonlinearity into the definition of the free energy – as motivated by previous heuristics in neural networks and machine learning:
\begin{eqnarray}\label{eq:nonlinear-energy}
    \mathcal{F} = \frac{1}{2} \Vert \mathbf{x} - \bfW f(\mathbf{x}) - \bm{\nu} \Vert_2^2
\end{eqnarray}

\noindent where $f(\cdot)$ is a nonlinear function. One way to understand these heuristics more formally is to appreciate that the free energy is a statement about the underlying generative model that generates inputs (e.g., images) from some latent causes. Although linear rPCNs have provided nice analytical results discussed above, nonlinearities can speak to more expressive generative models similar to the hierarchical and hybrid schemes demonstrated above. The plasticity rules of this model will be the same as Eq~\ref{eq:learning-nu} and Eq~\ref{eq:learning-W}, with the error $\bm{\epsilon}$ changing to $\mathbf{x} - \bfW f(\mathbf{x}) - \bm{\nu}$. The neural dynamics during inference follows:
\begin{eqnarray}\label{eq:nonlinear-inference}
    \Delta \mathbf{x} = \beta\frac{\partial\mathcal{F}}{\partial \mathbf{x}} = \beta(-\bm{\epsilon} + f'(\mathbf{x})\odot \bfW^{\top}\bm{\epsilon})
\end{eqnarray}

We first train this nonlinear implicit model to perform the same AM task as before, where the model is trained to memorize different numbers of grayscale CIFAR10 images, and then retrieve these memories from images with covered bottom half. For these experiments with CIFAR10 we used $\texttt{tanh}()$ as the nonlinear function $f()$. Fig~\ref{fig:chap3-attractors}A shows that the performance of the nonlinear implicit rPCN is worse than that of the linear model in this completion task, especially when $N$ is large. To examine the reasons for this performance difference, we then corrupt the memories differently by adding Gaussian noise with variance $0.1$, and initialize the retrieval dynamics with these noisy memories. Fig~\ref{fig:chap3-attractors}B shows that with this denoising task, the linear and nonlinear implicit rPCNs perform similarly, although the retrieval MSE of the nonlinear model is overall lower. We hypothesize that the difference across different retrieval tasks results from the attractor structures learned by the linear and nonlinear models. Classical nonlinear models, such as the Hopfield Networks \citep{hopfield1982neural}, usually learn individual attractors corresponding to each memorized item. On the other hand, as we have demonstrated in Fig~\ref{fig:chap3-equivalence}D, the linear implicit rPCN appears to learn a line attractor in the 2-dimensional space. We generalize this observation by training both linear and nonlinear models to memorize $3$ random data points in range $[-1, 1]$ in $\mathbb{R}^3$. After training, we initialize the inference with test data in a grid from the range $[-1, 1]$, and examine where these test data points are attracted to at the convergence of the inferential dynamics. Fig~\ref{fig:chap3-attractors}C shows that the attractor formed by the linear model is a hyperplane, where any point on this plane defines the lowest energy. On the other hand, the test data points all converge to the memories learned by the nonlinear model, suggesting that the nonlinear model indeed learns individual attractors. 

The observation above in turn helps explain the performance difference in completion and denoising tasks between the linear and nonlinear models. Notice that the completion task is inherently more challenging than the denoising task in terms of retrieval, as the corrupted pixels contain no information about the original images. Therefore, in the completion task, individual attractors formed by the nonlinear model will confound with each other more severely than in the denoising task, so that more retrievals will converge into the wrong attractors. The retrieval MSE of the nonlinear model thus comes from these incorrect attractor choices. With the linear model, the retrieved data points are always pulled towards the least squares regression hyperplane, such as the regression line in Fig~\ref{fig:chap3-equivalence}. The MSE of the linear model therefore only comes from the least squares prediction, which is the minimum across all linear solutions. Thus, when we average across the whole dataset, a relatively large number of totally wrong retrievals will produce a higher MSE than the retrievals from the least squares predictions. On the other hand, in the less challenging denoising task, the confounding effect between attractors of the nonlinear model will be less severe, yielding a smaller retrieval MSE across the whole dataset than in the completion task. Moreover, the least squares prediction from the linear model is less reliable in the denoising task, as all entries of the data are corrupted, i.e., the independent variables in the regression problem are also changing. Therefore the retrieval MSE of the linear model will increase, resulting in the observation in Fig~\ref{fig:chap3-attractors}A and B.

\section{Discussion}
\subsection{Summary}
This chapter introduces a family of rPCNs for AM tasks, providing a possible mechanism of how the recurrent hippocampal network may perform AM via predictive processing, which also unifies two disparate modelling approaches to the hippocampal network, namely covariance learning and PC. First, we identified that a previously proposed PCN already considered the learning of parameters encoding covariance in an explicit manner \citep{friston2003learning,friston2005theory}, but was never tested for AM tasks. We showed that, both theoretically and empirically, this model is able to perform AM on random patterns. However, this {\it explicit rPCN} is neither biologically plausible nor numerically stable, due to the inverse term in its learning rule. We address both limitations by proposing a model we call {\it implicit rPCN}, which also learns the covariance matrix, but in an implicit manner. More importantly, it adopts a Hebbian learning rule that is biologically more plausible than the learning rule of the explicit rPCN. We then showed that this model can be further simplified by introducing an apical dendritic architecture, which is even more biologically realistic. We named it the {\it dendritic rPCN}. In theory, we showed that both the implicit and the dendritic models are exactly equivalent to the explicit rPCN in AM tasks if their inference converges, although the dendritic model may suffer unstable inferential dynamics. Nevertheless, in practice, both implicit and dendritic models can be trained to memorize complex patterns such as handwritten digits and natural images. However, these models are computationally and memory expensive, a drawback that limits its applicability when confronted with more complex datasets. We solve this problem by combining our implicit rPCN with a hierarchical PCN \citep{rao1999predictive, salvatori2021associative}, to model the predictive interaction between the hippocampus and neocortex \citep{barron2020prediction}. We showed that this hybrid network can memorize and retrieve large-scale images in a parameter-flexible manner. We further conducted empirical analysis of the memory attractors learned by our linear implicit model, and showed that the memorized patterns are all stored on a hyperplane attractor of the inputs space. We found that nonlinearities are needed to obtain individual point attractors corresponding to each memorized pattern, similar to those observed in Hopfield Networks \citep{hopfield1982neural}.
% \textcolor{red}{In our future works, we will investigate the formal relationships between the attractors learned by our nonlinear model and those in classical AM models.}
Table~\ref{tab:models} below summarizes the property of each model discussed in this chapter. Overall, our models benefit the theoretical understanding of the computational principles adopted by the hippocampal network, by providing a unitary account for two disparate theories of how the hippocampus performs AM tasks.

% Place tables after the first paragraph in which they are cited.
\begin{table}[htbp]
\centering
\caption{
{\bf Summary of rPCNs discussed in this chapter.} ``Hebbian" denotes whether the model employs the Hebbian learning rule;  ``Parameter flexibility" denotes whether we can freely control the number of parameters of the network, i.e., whether the number of parameters in the model is not constrained to scale quadratically with the input size; ``Random/Strctured pattern" denotes whether the model can perform AM on random/strctured patterns.
}
\begin{tabular}{|c|p{25mm}|p{25mm}|p{25mm}|p{25mm}|}
\hline
& {\bf Hebbian} & {\bf Parameter flexibility} & {\bf Random pattern} & {\bf Structured pattern}\\ \hline
explicit & \xmark & \xmark & \cmark & \xmark \\ \hline
implicit & \cmark & \xmark & \cmark & \cmark \\ \hline
dendritic & \cmark & \xmark & \cmark & \cmark \\ \hline
hybrid & \cmark & \cmark & \cmark & \cmark \\ 

\hline
\end{tabular}
\label{tab:models}
\end{table}

\subsection{Predictive coding}
The explicit rPCN described in this work are essentially predictive coding models with parameters encoding the precision (covariance) matrix \citep{friston2003learning, friston2005theory,bogacz2017tutorial} with a single layer architecture, which corresponds to a model where there is no latent variable $\bfz$. In this case, the MLE estimates of the model are equivalent to the MAP estimates. These models introduced the covariance matrix to represent the uncertainty of different input features, by differentially weighting error signals derived from different input sources. However, to our knowledge, they have never been tested in AM tasks. Earlier works have also identified the implausibility of the learning rule for the covariance parameters, and addressed this issue by introducing additional inhibitory interneurons on top of error neurons \citep{bogacz2017tutorial}. On the other hand, our implicit/dendritic rPCN naturally circumvents the implausible learning rule by introducing a new set of recurrent weights $\bfW$. A natural follow-up of rPCNs is to investigate whether they can be added to all layers of hierarchical predictive coding networks, where the precision matrices were originally included \citep{friston2003learning,friston2005theory}. Furthermore, our nonlinear rPCN is reminiscent of the sparse coding schemes of the kind found in independent component analysis \citep{olshausen1996emergence}. This suggests an interesting relationship between sparse coding, PC and the properties evinced by the numerical studies above.

% In recent years, multiple lines of research have revealed the potential of PC in modeling various brain functions and in performing complex machine learning tasks. Most notably, the exploration of purely hierarchical PCNs in AM tasks by \citet{salvatori2021associative} has directly inspired our work, and their model was also employed in our work to construct the hybrid PCN. More broadly, the approximation of PC to backpropagation \citep{rumelhart1986learning}, the most commonly used learning rule of modern artificial neural networks, has been extensively investigated and discussed in recent works \citep{whittington2017approximation,song2020can,salvatori2022reverse,millidge2022predictive,alonso2022theoretical,rosenbaum2022relationship}. The PC algorithm has also been scaled to adapt to various modern machine learning architectures \citep{salvatori2022learning,pinchetti2022predictive,byiringiro2022robust,salvatori2021predictive,salvatori2022incremental} and studies have also shown that it can approximate backpropagation in arbitrary computational graphs \citep{millidge2022predictive2}. These works suggest that it is possible to implement PC within modern deep neural networks, making our PC-based AM networks potential candidates of efficient memory-storage machines in artificial intelligence.

\subsection{Models of AM and the hippocampus}
In this chapter, we have established a connection between rPCN and Hopfield Networks \citep{hopfield1982neural}. This linkage further associates predictive coding with numerous classical memory models \citep{kanerva1988sparse,kohonen1972correlation}, which share similar computational principles i.e., learning a covariance or correlation matrix to memorize associations between memory features. This in turn relates rPCN to modern variants of these classical memory models \citep{krotov2016dense,ramsauer2020hopfield,bricken2021attention}. While our empirical findings highlight several advantages of rPCN over Hopfield Networks, particularly in terms of memory capacity, exploring the theoretical capacity limits of rPCN presents a compelling avenue for future research \citep{keeler1988comparison,hertz2018introduction,amit1987information,chaudhry2023modern}.

Beyond these analytically tractable models, another noteworthy direction in hippocampal memory research emphasizes anatomical and neurophysiological fidelity. For instance, the models proposed by \citet{treves1994computational} and \citet{o1994hippocampal} incorporate specific hippocampal subregions such as the dentate gyrus, which is involved in orthogonalization before memory storage in the recurrent CA3 region. Additional studies have examined hippocampal-neocortical interactions \citep{gluck1993hippocampal,mcclelland1995there}. Recent studies indicate that predictive coding can effectively train networks of arbitrary structures \citep{salvatori2022learning}. Specifically, implicit rPCN can accommodate arbitrary constraints on the weight matrix $\bfW$ during training, which can reflect the connectivity structure of the hippocampal network more faithfully. This paves the way for future research employing rPCN variants to model the comprehensive hippocampal circuit more realistically.



% It is also worth pointing our that the experiments in our work focused mainly on ``spatial" predictions e.g., predicting each pixel using other pixel values. Although predictions made through different pixels or patches of a static image can be interpreted as eye saccade across a large visual scene, prediction of sequential sensory inputs is more realistic and consistent with experiments mentioned above. Modelling temporal PC models while retaining the plausible implementation discussed in this chapter will be a focus of our future explorations.