Pullback Bundles and the Geometry of Learning

Abstract

Explainable Artificial Intelligence (XAI) and acceptable artificial intelligence are active topics of research in machine learning. For critical applications, being able to prove or at least to ensure with a high probability the correctness of algorithms is of utmost importance. In practice, however, few theoretical tools are known that can be used for this purpose. Using the Fisher Information Metric (FIM) on the output space yields interesting indicators in both the input and parameter spaces, but the underlying geometry is not yet fully understood. In this work, an approach based on the pullback bundle, a well-known trick for describing bundle morphisms, is introduced and applied to the encoder–decoder block. With constant rank hypothesis on the derivative of the network with respect to its inputs, a description of its behavior is obtained. Further generalization is gained through the introduction of the pullback generalized bundle that takes into account the sensitivity with respect to weights.

1. Introduction

Explainable Artificial Intelligence (XAI) is generally described as a collection of methods allowing humans to understand how an algorithm is able to learn from a database, reproduce and generalize. It is currently an active, multidisciplinary area of research [1,2] that relies on several theoretical or heuristic tools to identify salient features and indicators explaining the surprisingly performances of machine learning algorithms, especially deep neural networks. From a statistical point of view, a neural network is nothing but a parameterized regression or classification model, that can be described as a random variable whose probability distribution is known conditionally to external inputs and internal parameters [3]. Unfortunately, even if this approach seems the most natural one, it is not adapted to XAI as no insight is gained on the learning and inference process. Furthermore, it seems that there is a contradiction between the statistical procedure that appeals for models with the smallest possible number of free parameters and the performance of deep learning relying on thousands to millions weights. On the other hand, attempts have been made to design numerical [4] or visual [5] indicators aiming at producing a summary of salient features.

XAI is also related to acceptable AI, that is proving or at least ensuring with a high probability that the model will produce the intended result and is robust to perturbations, either inherent to the data acquisition process or intentional. In both cases, it is mandatory to be able to perform a sensitivity analysis on a trained network. In [6], an approach based on geometry was taken and the need of a metric on the set of admissible perturbations enforced. The problem of the so-called adversarial attacks is treated in several papers [7,8,9] where mitigating procedures are proposed. Adversarial attacks are a major concern for acceptable AI, especially in critical application like autonomous vehicles or air traffic control. From now, most of the research effort was dedicated to the design of such attacks with the idea of incorporating the fooling inputs in the learning database in order to increase robustness. The reader can refer, for example, to Fast Gradient Sign methods [10], robust optimization methods [11] or DeepFool [12,13]. Unfortunately, while these approaches are relevant to acceptable AI, they do not provide XAI with usable tools. Furthermore, they rely on inputs in ${\mathbb{R}}^n$, or generally in a finite dimensional Euclidean space, which is not always a valid hypothesis.

There is also a question on why learning from a high dimension data space is possible, and a possible answer is because data effectively lies on a low dimensional manifold [14,15]. As a consequence, most of the directions in the input space will have a very small impact on the output, while only a few number of them, namely those who are tangent to the data manifold, are going to be of great influence [16]. The manifold hypothesis also justifies the introduction of the encoder–decoder architecture [17] that is of wide use in the field of natural language processing [18] or time-series prediction [19]. The true underlying data manifold, if it exists, is most of the time not accessible, although some of its characteristics may be known and incorporated in the model. In particular, it may be subject to some action by a Lie group or possess extra geometric properties, like the existence of a symplectic structure. Specific networks have be designed to cope with such situations [20,21].

In a general setting, little is known about the data manifold and its geometric features, like metric, Levi-Civita connection and curvature. However, Riemannian properties are the most important ones as they dictate the behavior of the network under moves in the input space. Recalling the statistical approach invoked before, it makes sense to model the output of the network as a density probability parameterized by inputs and weights. Within this frame, there exists a well-defined Riemannian metric on the output space known as the Fisher Information Metric (FIM) originating from a second order expansion of the Kullback–Leibler divergence. The importance of this metric has already been pointed out in several past works [22,23]. The FIM can be pulled back to the input space, yielding, in most cases, a degenerate metric that can nevertheless be exploited to better understand the effect of perturbations [16], or to parameter space to improve gradient-based learning algorithms [24]. In this last case, however, things tend to be less natural than for the input space.

In this work, a unifying framework for studying the geometry of deep networks is introduced, allowing a description of encoder–decoder blocks from the FIM perspective. The pullback bundle is a key ingredient in our approach.

In the sequel, features and outputs are random variables, thus characterized by their distribution functions, or their densities in the absolutely continuous case. Within this frame, a neural network is a random variable:

$$\left\{\begin{array}{c}Y=\mathcal{N}\left(X,W\right)\hfill \\ X:\left(\mathsf{\Omega },\mathcal{T},\mathbb{P}\right)\to \left(E,\mathcal{E}\right)\hfill \\ W:\left(\mathsf{\Omega },\mathcal{T},\mathbb{P}\right)\to \left(\mathsf{\Theta },\mathcal{F}\right)\hfill \end{array}\right.$$

(1)

where $\left(\mathsf{\Omega },\mathcal{T},\mathbb{P}\right)$ is an underlying probability space and $\left(E,\mathcal{E}\right),\left(\mathsf{\Theta },\mathcal{F}\right)$ are, respectively, the input and weight measure spaces Finally, Y is assumed to take its values in the output measure space $\left(O,\mathcal{O}\right).$ Most of the time, the network has a layered structure so that the expression of $\mathcal{N}$ can be factored out as:

$$Y=\mathcal{N}\left(\mathcal{N}\left(\dots ,W_2\right),W_1\right)$$

(2)

In many practical implementations, the weights W are deterministic, that is equivalent to saying that their probability distribution is a Dirac distribution. In this case, a neural network can be described as a parameterized family of random variables ${\mathcal{N}}_W:\omega \mapsto \mathcal{N}\left(X\left(\omega \right),W\right)$. A special case occur when a single decoder is considered [25], that is, a measurable function:

$$f=\mathcal{N}\left(·,W\right):{\mathbb{R}}^d\to {\mathbb{R}}^m,d\le m$$

(3)

where f is a smooth mapping, assumed in [25] to be an immersion; that is, for any x, $D{f}_x$ has maximal rank $d.$ Conversely, one may consider an encoder

$$g=\mathcal{N}\left(·,\tilde{W}\right):{\mathbb{R}}^n\to {\mathbb{R}}^d,d\le n$$

(4)

and assume f to be a submersion. In this paper, the geometry of the complete encoder–decoder network

$$g\circ f=\mathcal{N}\left(\mathcal{N}\left(·,W\right),\tilde{W}\right)$$

(5)

will be considered, as well as the case $d\ge m, d\le n.$

The article is structured as follows: In Section 2, the Fisher information metric is introduced and some formulas, valid when the parameter space is a smooth manifold, are given. In Section 3, the pullback bundle is defined and applied to the encoder–decoder case. Finally, a conclusion is drawn in Section 5. The convention of summation on repeated indices applies in this manuscript.

2. The Fisher Information Metric

In this section, we recall some basic definitions and properties in information geometry. The foundational ideas can be traced back to [26], but the main developments occur quite recently. The reader is referred to [27] for a comprehensive introduction. The exposition below assumes a quite high degree of regularity for the parameterized density families, which is nevertheless a common situation in practice, especially in the field of machine learning we are interested in.

2.1. Definitions and Properties

Definition 1.

A statistical model is a pair $\left(\mathcal{M},p\right)$ where $\mathcal{M}$ is an oriented n dimensional smooth manifold and ${\left(p_{\theta }\right)}_{\theta \in \mathcal{M}}$ is a parameterized family of probability densities on a measured space $(\mathsf{\Omega },\mathcal{T},\mu )$ such that, putting $p(\theta ,\omega )=p_{\theta }\left(\omega \right)$:

• For μ-almost all $\omega \in \mathsf{\Omega }$, the mapping $\theta \mapsto p(\theta ,\omega )$ is smooth;
• For any $\theta \in \mathcal{M}$, there exists an open neighborhood $U_{\theta }$ of θ and an integrable mapping $h:\mathsf{\Omega }\to {\mathbb{R}}^{+}$ such that, for any $\xi \in U_{\theta }$, $\left|{\partial }_{\theta }p(\xi ,\omega )\right|\le h$;
• The mapping $\theta \to p_{\theta }\in L^1(\mathsf{\Omega },\mu )$ is one-to-one;
• The support of $p_{\theta }$ does not depend on θ.

Assuming p never vanishes, one can define the score $l:\mathcal{M}\times \mathsf{\Omega }\to \mathbb{R}$ as:

$$l(\theta ,\omega )=logp(\theta ,\omega )$$

(6)

For any $\theta \in \mathcal{M}$:

$${\int }_{\mathsf{\Omega }}{p}_{\theta }\left(\omega \right)d\mu \left(\omega \right)=1$$

(7)

Thus, using the fact that the assumptions made on family $p_{\theta }$ allow swapping derivatives and integrals, it becomes:

$${\int }_{\mathsf{\Omega }}{\partial }_{i}p\left(\theta ,\omega \right)d\mu \left(\omega \right)=0,i=1\dots n$$

(8)

where ${\partial }_i$ denotes the derivative with respect to the i-th component of $\theta$ in local coordinates. So, the score $l_{\theta }=log{p}_{\theta }$ satisfies by (8):

$$E{\left[{\partial }_i{l}_{\theta }\right]}_{p_{\theta }}=0,i=1\dots n.$$

(9)

A simple computation shows that:

$$E\left[{\partial }_i{l}_{\theta }{\partial }_j{l}_{\theta }\right]={\int }_{\mathsf{\Omega }}\frac{{\partial }_i{p}_{\theta }}{\sqrt{p_{\theta }}}\frac{{\partial }_j{p}_{\theta }}{\sqrt{p_{\theta }}}d\mu \left(\omega \right)=4{\int }_{\mathsf{\Omega }}{\partial }_i\left(\sqrt{p_{\theta }}\right){\partial }_j\left(\sqrt{p_{\theta }}\right)d\mu \left(\omega \right),i,j=1\dots n$$

(10)

proving that:

$$g_{ij}=E\left[{\partial }_i{l}_{\theta }{\partial }_j{l}_{\theta }\right]={\langle {\partial }_i\left(\sqrt{p_{\theta }}\right),{\partial }_j\left(\sqrt{p_{\theta }}\right)\rangle }_{L^2(\mathsf{\Omega },\mu )}$$

(11)

Let g be the section of $T{\mathcal{M}}^{*}\otimes T{\mathcal{M}}^{*}$ defined by:

$$g=g_{ij}d{\theta }^i\otimes d{\theta }^j$$

(12)

Now, given any tangent vector $X=X^i{\partial }_i\in T_{\theta }\mathcal{M}$:

$$\begin{array}{cc}\hfill g(\theta ;X,X)=g_{ij}{X}^i{X}^j& ={\langle {\partial }_i\left(\sqrt{p_{\theta }}\right),{\partial }_j\left(\sqrt{p_{\theta }}\right)\rangle }_{L^2(\mathsf{\Omega },\mu )}\hfill \\ \hfill & ={\langle X^i{\partial }_i\left(\sqrt{p_{\theta }}\right),X^j{\partial }_j\left(\sqrt{p_{\theta }}\right)\rangle }_{L^2(\mathsf{\Omega },\mu )}\hfill \\ \hfill & ={\langle Z,Z\rangle }_{L^2(\mathsf{\Omega },\mu )}\hfill \end{array}$$

(13)

with $Z=X^i{\partial }_i\left(\sqrt{p_{\theta }}\right).$ Given the assumptions made on the family $p_{\theta }$, g is a thus a positive definite symmetric section of $T\mathcal{M}\otimes T\mathcal{M}$, hence a Riemannian metric on $\mathcal{M}$ called the Fisher Information Metric (FIM).

Remark 1.

The mapping $\mathcal{I}:\theta \mapsto \sqrt{p_{\theta }}$ embeds $\mathcal{M}$ as a submanifold of the unit sphere in $L_{\mathsf{\Omega },\mu }^2$ and the Fisher information metric is just the pullback of the ambient metric in $L_{\mathsf{\Omega },\mu }^2$ with respect to $\mathcal{I}$. However, in machine learning applications, it is common to consider parameter spaces for which the one-to-one assumption for $\mathcal{I}$ is non-valid so that g is only positive semidefinite. The study of the rank of the metric in this case is an important research topic.

It is quite fruitful to consider differential forms on $\mathcal{M}$ parameterized by $\mathsf{\Omega }$. The starting point is the definition of parameterized degree 0 forms.

Definition 2.

A parameterized 0-form is a mapping $f:\mathcal{M}\times \mathsf{\Omega }\to \mathbb{R}$ satisfying:

• For almost all $\omega \in \mathsf{\Omega }$, the mapping $\theta \in \mathcal{M}\to f(\theta ,\omega )$ is smooth;
• For all ${\theta }_0\in \mathsf{\Omega }$, and all integers n, there exists a neighborhood $U_{n,{\theta }_0}$ and an integrable positive mapping $h_{n,{\theta }_0}$ such that for all $\theta \in U_{n,{\theta }_0}$ and almost all $\omega \in \mathsf{\Omega }$: $\left|{\partial }_{\theta }^{n}f(\theta ,\omega )\right|\le h_{n,{\theta }_0}\left(\omega \right).$

Proposition 1.

Let X be a vector field on $T\mathcal{M}$ and f a parameterized 0-form in the previous sense. Then:

$$X\left(E\left[f\right]\right)=E\left[X\left(f\right)\right]+E\left[fX\left(l\right)\right]$$

(14)

with $l(\theta ,\omega )=logp(\theta ,\omega )n.$

Proof. 

$E\left[f\right]$ is a degree 0 form on $T\mathcal{M}$. If $\psi$ is the flow of X, then:

$${\psi }^{★}E\left[f\right]={\int }_{\mathsf{\Omega }}f\left(\psi (t,\theta ),\omega \right)p\left(\psi (t,\theta ),\omega \right)d\mu \left(\omega \right)$$

(15)

The assumptions made on f allowing the swapping of derivatives and integrals, so:

$${\frac{\partial }{\partial t}}_{t=0}E\left[f\right]={\int }_{\mathsf{\Omega }}{\partial }_{\theta }f(\theta ,\omega )X\left(\theta \right)p(\theta ,\omega )d\mu \left(\omega \right)+{\int }_{\mathsf{\Omega }}f(\theta ,\omega )\frac{{\partial }_{\theta }p(\theta ,\omega )}{p(\theta ,\omega )}p(\theta ,\omega )d\mu \left(\omega \right)$$

(16)

□

Remark 2.

Applying Proposition 1 to the constant function $f=1$ yields $E\left[X\left(l\right)\right]=0$, a result already known by Equation (9)

A parameterized degree k differential form on $T\mathcal{M}$ can be defined readily by requiring that the coefficients of the elementary forms $d{\theta }_{i_1}\wedge \cdots \wedge d{\theta }_{i_k}$ be parameterized differential forms of degree 0.

Proposition 2.

Let α be a degree k parameterized differential form on $T\mathcal{M}$. Then:

$$dE\left[\alpha \right]=E\left[d\alpha \right]+E\left[dl\wedge \alpha \right]$$

(17)

Proof. 

It is enough to consider a form $\alpha (\theta ,\omega )=f(\theta ,\omega )d{\theta }_{i_1}\wedge \cdots \wedge d{\theta }_{i_k}.$ Then:

$$dE\left[\alpha \right]\left(\theta \right)=\sum _{j=1}^{n}E\left[{\partial }_{{\theta }_j}f\right]d{\theta }_j\wedge d{\theta }_{i_1}\wedge \cdots \wedge d{\theta }_{i_k}+\sum _{j=1}^{n}E\left[f{\partial }_{{\theta }_j}l\right]d{\theta }_j\wedge d{\theta }_{i_1}\wedge \cdots \wedge d{\theta }_{i_k}$$

(18)

since:

$$\begin{array}{cc}\hfill & d\alpha =\sum _{j=1}^n\left({\partial }_{{\theta }_j}f\right)d{\theta }_j\wedge d{\theta }_{i_1}\wedge \cdots \wedge d{\theta }_{i_k}\hfill \end{array}$$

(19)

$$\begin{array}{cc}\hfill & dl\wedge \alpha =\sum _{j=1}^{n}f\left({\partial }_{{\theta }_j}l\right)d{\theta }_j\wedge d{\theta }_{i_1}\wedge \cdots \wedge d{\theta }_{i_k}\hfill \end{array}$$

(20)

the claim follows. □

Proceeding the same way as in Proposition 1, and using Cartan’s homotopy formula, we obtain:

Proposition 3.

Let X be a vector field on $T\mathcal{M}$ and α a degree k parameterized differential form. Then

$${\mathcal{L}}_X\left(E\left[\alpha \right]\right)=E\left[i_{X}d\alpha \right]+E\left[d{i}_X\alpha \right]+E\left[\left(i_{X}dl\right)\wedge \alpha \right]$$

(21)

When $\alpha =dl$, Equation (21) reads as:

$${\mathcal{L}}_{X}E\left[dl\right]=E\left[i_X{d}^{2}l\right]+E\left[d\left(i_{X}dl\right)\right]+E\left[\left(i_{X}dl\right)\wedge dl\right]$$

(22)

Since $E\left[dl\right]=0$, it becomes:

$$E\left[d\left(i_{X}dl\right)\right]=-E\left[\left(i_{X}dl\right)\wedge dl\right]$$

(23)

Given two vector fields $X,Y$:

$$i_{Y}E\left[\left(i_{X}dl\right)\wedge dl\right]=E\left[\left(i_{X}dl\right)\left(i_{Y}dl\right)\right]=g(X,Y)$$

(24)

with g the Fisher metric. Thus:

Proposition 4.

$$g(X,Y)=-E\left[i_{Y}d\left(i_{X}dl\right)\right]$$

(25)

Remark 3.

In coordinates, $i_{Y}d\left(l_{X}dl\right)={\partial }_{ij}{X}^j{Y}^i+{\partial }_{j}l{\partial }_i{X}^j{Y}^i$, and after taking the expectation:

$$g(X,Y)=-E\left[{\partial }_{ij}l\right]X^j{Y}^i$$

(26)

This is a well-known result in the ${\mathbb{R}}^n$ case.

Let ∇ be an affine connection on $T\mathcal{M}$. The same computation as above yields:

Proposition 5.

Let X be a vector field on $T\mathcal{M}$ and α a degree k parameterized differential form. Then:

$${\nabla }_{X}E\left[\alpha \right]=E\left[{\nabla }_X\alpha \right]+E\left[\left(i_{X}dl\right)\wedge \alpha \right]$$

(27)

When $\alpha =dl$, we recover $E\left[{\nabla }_{X}dl\right]\left(Y\right)=-g(X,Y)$, showing that while the parameterized Hessian $\nabla dl$ depends on the connection ∇, it is not the case of its expectation. When $\mathsf{\Omega }=\mathcal{M}={\mathbb{R}}^n,\mu =d{x}_{1}d{x}_2\dots d{x}_n$, the Fisher metric is known to be twice the second order term in the Taylor expansion of the Kullback–Leibler divergence, which can be proved easily by iterating derivatives. More generally, let ∇ be a connection and let $\theta :]-ϵ,ϵ[\to \mathcal{M},ϵ>0$ be a smooth curve with ${\theta }_0=\theta \left(0\right),X={\theta }^{\prime }\left(0\right)$. Recall that the Kullback–Leibler divergence between two probability densities $p,q$ is defined as:

$$\mathrm{KL}\left(p,q\right)=E_p\left[log\left(p/q\right)\right]=\int log\left(\frac{p\left(x\right)}{q\left(x\right)}\right)p\left(x\right)dx$$

(28)

The mapping:

$$t\in ]-ϵ,ϵ[\mapsto \xi \left(t\right)=\mathrm{KL}\left(p_{{\theta }_0},p_{\theta \left(t\right)}\right)=E_{p_{{\theta }_0}}\left[l_{{\theta }_0}\left(t\right)-l_{\theta }\left(t\right)\right]$$

(29)

is smooth, so Taylor formula applies for t close enough to 0:

$$\xi \left(t\right)=\sum _{i=1}^n\frac{{\xi }^{\left(i\right)}\left(0\right)}{i!}{t}^i+o\left(t^n\right)$$

(30)

With:

$${\xi }^{\left(i\right)}\left(0\right)=E_{p_{{\theta }_0}}\left[\underset{i\mathrm{times}}{\underbrace{X\left(X\left(\dots X\left(l\right)\right)\right)}}\right]=E_{p_{{\theta }_0}}\left[\underset{i-1\mathrm{times}}{\underbrace{X\left(X\left(\dots dl\left(X\right)\right)\right)}}\right]$$

(31)

If the curve $t\to \theta \left(t\right)$ is a geodesic for ∇, then:

$$X\left(dl\left(X\right)\right)=\left({\nabla }_{X}dl\right)\left(X\right)+dl\left({\nabla }_{X}X\right)=\left({\nabla }_{X}dl\right)\left(X\right)$$

(32)

And, by recurrence:

$${\xi }^{\left(i\right)}\left(0\right)=E_{p_{{\theta }_0}}\left[\left({\nabla }_X^{(i-1)}dl\right)\right]\left(X\right).$$

(33)

The first derivative ${\xi }^{\left(1\right)}\left(0\right)$ is readily computed as:

$$-E\left[d{l}_{{\theta }_0}\right]\left(X\right)=0.$$

(34)

The second derivative ${\xi }^{\left(2\right)}\left(0\right)$ can be obtained using ∇ as:

$$-E\left[{\nabla }_{X}d{l}_{{\theta }_0}\right]\left(X\right)=g_{{\theta }_0}(X,X).$$

(35)

Since g is symmetric, $g(X,Y)=\left(g(X+Y,X+Y)-g(X-Y,X-Y)\right)/4$, thus (35) characterizes g as ${\theta }_0.$ Higher-order terms can be computed by repeatedly applying Proposition 5 and are expressed thanks to the quantities:

$$E\left[\left(i_{X}dl\right)\wedge {\nabla }_X^{\left(i\right)}dl\right]\left(X\right).$$

(36)

An interesting case occurs when the Fisher metric is non-degenerate and ${\nabla }^{\mathrm{lc}}$ is its associated Levi-Civita connection. Normal coordinates at ${\theta }_0$, denoted by $x^i,i=1\dots N$, are given by taking an orthonormal basis, with respect to the Fisher metric, $(v_1,\dots ,v_N)$ and letting [28] (p. 72):

$$x^i\left({exp}_{{\theta }_0}{t}^j{v}_j\right)=t^i$$

(37)

Using the $x^i,i=1\dots N$ system of coordinates in place of $\theta$, and noting that ${\theta }_0$ corresponds to the origin in normal coordinates, the KL divergence can be approximated at order 2 by:

$$\mathrm{KL}\left(p_0,p_x\right)=\frac{1}{2}{x}^i{x}^j$$

(38)

where $x=\left(x^1,\dots ,x^N\right).$

2.2. The Fisher Information in Machine Learning

In machine learning applications, when the output is a probability distribution, then the Kullback–Leibler divergence is a natural measure for goodness-of-fit. Assuming that the database is given in the form of an iid sample of couples ${(X_i,Y_i)}_{i=1\dots N}$, then one can introduce the error function:

$$E\left(W\right)=\sum _{i=1}^N\mathrm{KL}\left(Y_i,\mathcal{N}\left(X_i,W\right)\right)$$

(39)

That may be approximated by:

$$\tilde{E}\left(W\right)=-\sum _{i=1}^N\frac{1}{2}g\left(Y_i;\overrightarrow{Y_i\mathcal{N}\left(X_i,W\right)},\overrightarrow{Y_i\mathcal{N}\left(X_i,W\right)}\right)$$

(40)

where the notation $\overrightarrow{PQ}$ stands for the tangent vector at P such that a geodesic (for ${\nabla }^{\mathrm{lc}}$) $\theta$ with $\theta \left(0\right)=P,{\theta }^{\prime }\left(0\right)=\overrightarrow{PQ}$ is such that $\theta \left(1\right)=Q$. Taking the derivative with respect to W yields:

$$\frac{\partial \tilde{E}}{\partial W}=-\sum _{i=1}^{N}g\left(Y_i;\frac{\partial \mathcal{N}\left(X_i,W\right)}{\partial W},\overrightarrow{Y_i\mathcal{N}\left(X_i,W\right)}\right)$$

(41)

with $\frac{\partial \mathcal{N}\left(X_i,W\right)}{\partial W}$ being a tangent vector at $Y_i$.

We recall the musical isomorphism $♭:TM\to T{M}^{★}$ defined by:

$$X^{♭}\left(Y\right)=g(X,Y)$$

(42)

and use it to rewrite (41) as:

$$\frac{\partial \tilde{E}}{\partial W}=-\sum _{i=1}^N{\left(\frac{\partial \mathcal{N}\left(X_i,W\right)}{\partial W}\right)}^{♭}\left(\overrightarrow{Y_i\mathcal{N}\left(X_i,W\right)}\right)$$

(43)

In this form, having a critical point of the energy $\tilde{E}$ with respect to W is equivalent to the vanishing of a totally symmetric multilinear form on $T\mathcal{M}\oplus T{\mathcal{M}}^{★}$, the generalized tangent bundle of $\mathcal{M}.$

Finally, if $\psi :\mathcal{N}\to \mathcal{M}$ is a smooth mapping, one can take the pullback the Fisher metric on $\mathcal{M}$ to obtain a semi-definite symmetric bilinear form on $\mathcal{N}$:

$${\psi }^{★}g(\eta ;X,Y)=g(\psi \left(\eta \right);{\psi }^{\prime }\left(\eta \right)X,{\psi }^{\prime }\left(\eta \right)Y)$$

(44)

When $\psi$ is an embedding, ${\psi }^{★}g$ is a Fisher metric on $\mathcal{N}$ with $p_{\psi \left(\eta \right)},\eta \in \mathcal{N}$ as underlying densities. This is the case considered in [25].

As an example of a pullback metric, we are going to investigate the case of the von Mises–Fisher distribution (VMF) on ${\mathbb{S}}^{n-1}$ with density:

$$p_{\kappa ,\mu }\left(x\right)=\frac{{\kappa }^{n/2-1}}{{\left(2\pi \right)}^{n/2}{I}_{n/2-1}\left(\kappa \right)}exp\left(\kappa \langle x,\mu \rangle \right)$$

(45)

where $\kappa \ge 0$ is the concentration parameter, $\mu \in {\mathbb{S}}^{n-1}$ is the location parameter and $I_k$ is the modified Bessel function of the first kind of order k. The Fisher metric in the embedding space ${\mathbb{R}}^n$ can be deduced from the second moment $E\left[x{x}^t\right]$ since $l_{\kappa ,\mu }=log\left(p_{\kappa ,\mu }\right)=f\left(\kappa \right)+\langle x,\mu \rangle$. If $\kappa$ is assumed to be constant, then:

$$E\left[{\partial }_{\mu }{l}_{\kappa ,\mu }{l}_{\kappa ,\mu }^t\right]=E\left[x{x}^t\right]$$

(46)

Although the expression for $E\left[x{x}^t\right]$ has been given in [29], we present here an alternative proof based on the fact that for any integer n, ${\mathbb{S}}^{n-1}$ is a suspension of ${\mathbb{S}}^{n-2}$. If $x=\left(x_1,\dots ,x_n\right)$, then $x{x}^t$ is a matrix whose $(i,j)$ entry is $x_i{x}_j$. By the rotation invariance of the VMF, $\mu$ can be selected as the first vector of an orthonormal basis, with respect to which x is expressed in components as $x=\left(x_1,\dots ,x_n\right)$. If we specialize the first component, then, if $i\ne 1,j\ne 1$:

$${\int }_{{\mathbb{S}}^{n-1}}{x}_i{x}_j{p}_{\kappa ,\mu }\left(x\right)dx=c_{\kappa }{\int }_0^{\pi }exp\left(cos\theta \right){sin}^{n-2}\left(\theta \right){\int }_{{\mathbb{S}}^{n-2}}{\xi }_i{\xi }_{j}d{\sigma }_{n-2}\left(\xi \right)$$

(47)

with $x_i=sin\theta {\xi }_i,i=1\dots n-1$ and ${\sigma }_{n-2}$ the Lebesgue measure on ${\mathbb{S}}^{n-2}$. If $i\ne j$, then the integral vanishes by symmetry, otherwise:

$$\begin{array}{cc}\hfill {\int }_{{\mathbb{S}}^{n-2}}{\xi }_i{\xi }_{j}d{\sigma }_{n-2}\left(\xi \right)& ={\int }_0^{\pi }{cos}^2\left(\psi \right){sin}^{n-3}\left(\psi \right){\int }_{{\mathbb{S}}^{n-3}}d{\sigma }_{n-3}d\psi \hfill \\ \hfill & ={\int }_0^{\pi }{cos}^2\left(\psi \right){sin}^{n-3}\left(\psi \right)d\psi \mathcal{A}\left({\mathbb{S}}^{n-3}\right)\hfill \end{array}$$

(48)

with $\mathcal{A}\left({\mathbb{S}}^{n-3}\right)$ the area of the $n-3$-sphere, which is given by the general relation:

$$\mathcal{A}\left({\mathbb{S}}^n\right)=\frac{2{\pi }^{\frac{n+1}{2}}}{\mathsf{\Gamma }\left(\frac{n+1}{2}\right)}$$

(49)

Now, observing that [30]:

$${\int }_0^{\pi }{cos}^2\left(\psi \right){sin}^{n-3}\left(\psi \right)d\psi =B\left(\frac{3}{2},\frac{n}{2}-1\right)$$

(50)

with B the beta function, the overall expression becomes, after using (49):

$$\begin{array}{cc}\hfill & \frac{{\left(2\pi \right)}^{n/2}\mathsf{\Gamma }\left(\frac{n}{2}-1\right)I_{n/2}\left(\kappa \right){\kappa }^{n/2-1}}{{\kappa }^{n/2}\mathsf{\Gamma }\left(n/2-1\right){\left(2\pi \right)}^{n/2}{I}_{n/2-1}\left(\kappa \right)}\hfill \\ \hfill & =\frac{1}{\kappa }\frac{I_{n/2}\left(\kappa \right)}{I_{n/2-1}\left(\kappa \right)}\hfill \end{array}$$

(51)

When $i=j=1$, then the expression for the second moment becomes:

$$\begin{array}{cc}\hfill & {\int }_0^{\pi }exp\left(\kappa cos\theta \right){cos}^2\left(\theta \right){sin}^{n-2}\left(\theta \right)d\theta \mathcal{A}\left({\mathbb{S}}^{n-2}\right)=\hfill \\ \hfill & {\int }_0^{\pi }exp\left(\kappa cos\theta \right)\left(1-{sin}^2\left(\theta \right)\right)\left(\theta \right){sin}^{n-2}\left(\theta \right)d\theta \mathcal{A}\left({\mathbb{S}}^{n-2}\right)\hfill \end{array}$$

(52)

The integral is a difference of two terms, each of which can be simplified as before to yield:

$$\left(1-\frac{n}{\kappa }\right)\frac{I_{n/2}\left(\kappa \right)}{I_{n/2-1}\left(\kappa \right)}$$

(53)

This procedure can easily be applied to an arbitrary moment, each of the integral involved being expressible using $I_n$ and the Beta function.

Remark 4.

Since μ is not a parameterization of the unit sphere, the Fisher metric defined that way is related to an ambient metric in ${\mathbb{R}}^n$, defined only on the unit sphere.

An obvious embedded dimension $n-2$ submanifold of ${\mathbb{S}}^{n-1}$ is obtained by taking a unit vector $\nu$ and computing the intersection of ${\mathbb{S}}^{n-1}$ with an hyperplane $\mathcal{H}$ defined by:

$$x\in \mathcal{H}\iff \langle x,\nu \rangle =\alpha \alpha \in [0,1]$$

(54)

An elementary computation proves that the intersection locus is a $n-2$ sphere contained in $\mathcal{H}$:

$${|x-\alpha \nu |}^2=1-{\alpha }^2$$

(55)

Without loss of generality, $\nu$ can be taken as $\left(\begin{array}{cccc}1& 0& \dots & 0\end{array}\right)$ and the embedding can be written easily as:

$$\left(\begin{array}{ccc}{x}_1& \dots & x_{n-1}\end{array}\right)\mapsto \left(\begin{array}{cccc}\alpha & \lambda x_1& \dots & \lambda x_{n-1}\end{array}\right),\lambda =\sqrt{1-{\alpha }^2}$$

(56)

The pullback metric is just the original one scaled by $1-{\alpha }^2$. The loss functions related to the VMF distribution are discussed in [31].

3. Pullback Bundles

In this section, a neural network with weights W is a mapping $\mathcal{N}\left(·,W\right):\mathcal{I}\to \mathcal{O}$, where $\mathcal{I}$ (i.e., $\mathcal{O}$) is the input (i.e., output) manifold of dimension n (i.e., m). Both manifolds are assumed to be smooth, and also the mapping ${\mathcal{N}}_W.$ This last assumption is valid when the activation functions are smooth, which is the case for sigmoid functions, but not for the commonly used ReLu function. However, smooth approximations to the ReLu are easy to construct with an arbitrary degree of accuracy, so the framework introduced below can be still applied.

As mentioned in the introduction, $\mathcal{O}$ is further assumed to be a statistical model 1 with Fisher metric $g.$ This setting is the one of a neural network whose output is a random variable with conditional density in a family $p_{\theta },\theta \in \mathcal{O}$.

When the weights are kept fixed, the only free parameters are the inputs and the network is fully described by the mapping:

$$\left\{\begin{array}{cc}\mathcal{N}\left(·,W\right):\hfill & \mathcal{I}\to \mathcal{O}\hfill \\ \hfill & x\mapsto \mathcal{N}\left(·,W\right)=p_{\theta \left(x\right)}\hfill \end{array}\right.$$

(57)

For the ease of notation, the mapping $\mathcal{N}\left(·,W\right)$ will be abbreviated by ${\mathcal{N}}_W\left(·\right)$. When the activation functions in the network are smooth, ${\mathcal{N}}_W\left(·\right)$ is a smooth mapping and its derivative will be denoted by $d{\mathcal{N}}_W\left(\dots \right)$. With this convention, the pullback metric of g by ${\mathcal{N}}_W\left(·\right)$, denoted $\tilde{g}$, is defined by:

$$\tilde{g}\left(X,Y\right)=g\left(d{\mathcal{N}}_W\left(X\right),d{\mathcal{N}}_W\left(Y\right)\right)$$

(58)

Unless the network $\mathcal{N}$ is a decoder, $\tilde{g}$ is generally degenerated and does not provide $\mathcal{I}$ with a Riemannian structure, so an ambient metric h on $\mathcal{I}$ is assumed to exist. The triple $\left(\mathcal{I},h,\tilde{g}\right)$ is called the data manifold of the network. The kernel of $\tilde{g}$, denoted $ker\tilde{g}$, is the distribution in $T\mathcal{I}$ consisting of vectors X such that $\tilde{g}(X,·)$ is the zero mapping. At a point $x\in \mathcal{I}$, the vectors in $T_x\mathcal{I}$ belonging to $ker\tilde{g}$ give directions in which the output of the network will not change up to order $1.$ Figure 1 represents the case of a one dimensional output space and a 2-sphere input space. Since the dimension of the output is less than the one of the input, some moves in the data manifold will not induce any change at the output.

Unless the dimension of $ker\tilde{g}$ is constant, this distribution does not define a foliation. However, this is true locally in the neighborhood of points in $\mathcal{I}$ such that $d{\mathcal{N}}_W\left(·\right)$ has maximal rank. Finally, if $E\stackrel{\pi }{\to }\mathcal{O}$ is an r-vector bundle on $\mathcal{O}$, then its pullback by ${\mathcal{N}}_W\left(·\right)$ will be denoted in short by $E^{{\mathcal{N}}_W}.$ We recall that if E has local charts:

$$\left(V_i,{\xi }_i\right),{\xi }_i:V_i\times {\mathbb{R}}^r\to {\pi }^{-1}\left(V_i\right),i\in I$$

and $\mathcal{I}$ has local charts $\left(U_j,{\varphi }_i\right),j\in J$, then $E^{{\mathcal{N}}_W}$ has local charts:

$$\begin{array}{cc}\hfill & \left(W_{ji}=U_j\cap {\mathcal{N}}_W^{-1}\left(V_i\right),{\psi }_{ji}\right),{\psi }_{ji}:W_{ji}\to W_{ji}\times {\mathbb{R}}^r\hfill \\ \hfill & {\psi }_{ji}\left(x\right)={\xi }_i\circ f\circ {\varphi }_j\hfill \end{array}$$

(59)

The pullback bundle enjoys a universal property that is in fact the main reason for introducing it in our context.

Proposition 6.

Let $\left(\tilde{E},\tilde{\pi },\mathcal{I}\right)$ (i.e., $\left(E,\pi ,\mathcal{O}\right)$) be a vector bundle on $\mathcal{I}$ (i.e., $\mathcal{O})$). For any bundle morphism $\left({\eta }_1,{\eta }_0\right)$, there exists a unique bundle morphism $\left({\tilde{\eta }}_1,Id\right)$ such that the following diagram commutes:

(60)

where ${\pi }^{{\eta }_0}:(x,v)\mapsto x$ and ${\tilde{\eta }}_0:(x,v)\mapsto \left({\eta }_0\left(x\right),v\right).$

This proposition is a classical one and its proof can be found in many textbooks. The one we give below is very simple, using only local charts.

The above construction is constructive and thus gives a practical mean of computation. For a network with fixed weights, e.g., a trained one, the derivative $d{\mathcal{N}}_W$ can be efficiently computed by back propagation, so the bundle morphism:

(61)

has a practical meaning.

Introducing the pullback bundle gives the diagram:

(62)

The bundle mapping $d{\mathcal{N}}_W$ to $T^{{\mathcal{N}}_W}\mathcal{O}$ is then the association:

$$(x,v)\in {\mathbb{R}}^m\times {\mathbb{R}}^n\mapsto \left(x,d{\mathcal{N}}_W·v\right)\in {\mathbb{R}}^n\times {\mathbb{R}}^n$$

(63)

The pullback bundle is thus a mean of representing the action of the network on tangent vectors to the data manifold. As an example, the construction of adversarial attacks given in [32,33] can be revisited in this context, extending it to the general setting of network with manifold inputs.

The general problem of building an adversarial attack is, informally, to find, for an input point in the data manifold, a direction in which a perturbation will have the most important effect on the output, hopefully fooling the network. Following [33], we define:

Definition 3.

Let h be a Riemannian metric on the input space. An optimal adversarial attack at $x\in \mathcal{I}$ with budget $ϵ>0$ is a solution to:

$$\underset{v\in T_x\mathcal{I},h(v,v)\le ϵ}{max}\tilde{g}\left(v,v\right)$$

(64)

Using (38), this optimization program can be viewed as a local approximation to the one based on the Kullback–Leibler divergence:

Definition 4.

A Kullback–Leibler optimal adversarial attack at $x\in \mathcal{I}$ with budget $ϵ>0$ is a solution to:

$$\underset{y\in \mathcal{I},h(x,y)\le ϵ}{max}\mathit{KL}\left(\mathcal{N}\left(x,W\right),\mathcal{N}\left(y,W\right)\right)$$

(65)

The metric g on $T\mathcal{O}$ can be pulled back to $T^{{\mathcal{N}}_W}$ by letting:

$$g^{{\mathcal{N}}_W}\left(x;v,v\right)=g\left(f\left(x\right);v,v\right)$$

(66)

Due to the special form of the criterion, the optimal point is on the boundary, so that finally, the optimal adversarial attack problem may be formulated as:

Definition 5.

An optimal adversarial attack at $x\in \mathcal{I}$ with budget $ϵ>0$ is a solution to:

$$\underset{v\in U{T}_x\mathcal{I}}{max}{ϵ}^2{g}^{{\mathcal{N}}_W}\left(d{\mathcal{N}}_{W}v,d{\mathcal{N}}_{W}v\right)$$

(67)

Where $UT\mathcal{I}$ stands for the unit sphere bundle with respect to the metric $h.$ Please note that due to bilinearity, the problem can be solved for $ϵ=1$, then let the optimal vector be scaled by the original $ϵ.$ From standard linear algebra, if $G_x$ is the matrix of the bilinear form $g^{{\mathcal{N}}_W}$ at x and $H_x$ the one of h, then one can find unitary matrices $A,B$ and diagonal matrices $\mathsf{\Lambda },\mathsf{\Sigma }$ such that:

$$H_x=A^t\mathsf{\Lambda }A,G_x=B^t\mathsf{\Sigma }B$$

(68)

Any vector v in $U{T}_x\mathcal{I}$ can be written as:

$$V=A^t{\mathsf{\Lambda }}^{-1/2}w,w^{t}w=1$$

(69)

So that, finally, the original problem can be rewritten as:

$$\underset{w,w^{t}w=1}{max}{w}^t{M}^{t}Mw,M={\mathsf{\Sigma }}^{1/2}B{d}_x{\mathcal{N}}_W{A}^t{\mathsf{\Lambda }}^{-1/2}$$

(70)

which is solved readily by taking w to be the unit eigenvector of M associated with the largest eigenvalue. This is the solution found in [33] when $H_x=\mathrm{Id}.$

In many cases, as the above example indicates, it is more convenient to work uniquely in the input space, thus justifying the introduction of the pullback bundle $T^{{\mathcal{N}}_W}\mathcal{O}.$ From now, we are going to adopt this point of view.

Remark 5.

Please note that a section in $T^{{\mathcal{N}}_w}\mathcal{I}$ is generally not related to a section of the form (63) in either $T\mathcal{O}$ or $T\mathcal{I}$ due to the fact that $d_x{\mathcal{N}}_W$ may not be a monomorphism or an epimorphism. The next proposition gives condition for the existence of global sections in $T\mathcal{O}$ associated with global sections in $T^{{\mathcal{N}}_W}\mathcal{O}.$

Proposition 7.

In the case of a decoding network, when ${\mathcal{N}}_W$ is an embedding, there is a natural embedding of bundles $\begin{array}{ccc}\hfill T\mathcal{I}& \stackrel{i}{\hookrightarrow }& T^{{\mathcal{N}}_W}\mathcal{O}\hfill \end{array}$ such that the image of $(x,v)$ is $(x,d{\mathcal{N}}_{W}v)$. The pullback bundle then splits as:

$$T^{{\mathcal{N}}_W}\mathcal{O}=i\left(T\mathcal{I}\right)\oplus F$$

(71)

where F has rank $n-m$.

Be careful that in this case, a section of the pullback bundle will not define a global section in $T\mathcal{O}$ since some points of the output space may have no preimage by ${\mathcal{N}}_W.$ However, by the extension lemma [34] (Lemma 5.34, p. 115), local (global if ${\mathcal{N}}_W\mathcal{I}$ is closed) smooth vector fields on $T\mathcal{O}$ exist, extending it.

Proof. 

If ${\mathcal{N}}_W$ is an embedding, ${\mathcal{N}}_W\left(\mathcal{I}\right)$ is a submanifold of $\mathcal{O}$ and in an adapted chart, a vector field in $T{\mathcal{N}}_W\left(\mathcal{I}\right)$ can be written as $v={\sum }_{i=1}^n{v}^i{\partial }_i$, where the ${\partial }_i,i=1\dots n$ are the first n coordinate vector fields. It thus pulls back to a section $\tilde{v}$ of the same form in $T^{{\mathcal{N}}_W}\mathcal{O}$. Now, since $d{\mathcal{N}}_W$ is injective, $\tilde{v}$ is the image of a unique section in $T\mathcal{I}$, hence the claim. □

Proposition 8.

If $kerd{\mathcal{N}}_W$ has constant rank r, then there exists a splitting $T\mathcal{I}=kerd{\mathcal{N}}_W\oplus F$, $T^{{\mathcal{N}}_W}\mathcal{O}=im d{\mathcal{N}}_W\oplus G$ and bundle isomorphism $F\to im d{\mathcal{N}}_W$ that coincides with $d{\mathcal{N}}_W$ on the fibers.

Proof. 

By Theorem 10.34, [34] (p. 266), $kerd{\mathcal{N}}_W$ is a subbundle of $T\mathcal{I}$ and $\mathsf{im}d{\mathcal{N}}_W$ a subbundle of $T^{{\mathcal{N}}_W}\mathcal{O}$. In local charts, the morphism $d{\mathcal{N}}_W$ gives rise to the decomposition:

$$\begin{array}{ccc}\hfill kerd{\mathcal{N}}_W\oplus {\mathbb{R}}^r& \stackrel{d{\mathcal{N}}_W}{\to }& \mathsf{im}d{\mathcal{N}}_W\oplus {\mathbb{R}}^{m-r}\end{array}$$

(72)

with $d{\mathcal{N}}_W$ an isomorphism where restricted to ${\mathbb{R}}^r.$ Passing to local sections yields the result. □

An important case is the one of submersions, corresponding to encoders in machine learning. In this case, $r=m$ and $d{\mathcal{N}}_W$ establishes a bundle isomorphism between F and $T^{{\mathcal{N}}_W}\mathcal{O}.$ The pullback of Fisher–Rao metric g on $T\mathcal{O}$ gives rise to a metric $g^{{\mathcal{N}}_W}$ on $T^{{\mathcal{N}}_W}\mathcal{O}$, but only to a degenerate metric on $T\mathcal{I}$ that can, nevertheless, be quite well understood, as indicated below.

Definition 6.

On the input bundle $T\mathcal{I}$, the symmetric tensor $\tilde{g}$ is defined using the splitting $T\mathcal{I}=kerd{\mathcal{N}}_W\oplus F$, by:

$$\begin{array}{cc}\hfill & g(X,Y)=0,X\in kerd{\mathcal{N}}_W,Y\in T\mathcal{I}\hfill \\ \hfill & g(X,Y)=g^{{\mathcal{N}}_W}\left(d{\mathcal{N}}_{W}X,d{\mathcal{N}}_{W}Y\right),X,Y\in F\hfill \end{array}$$

(73)

Proposition 9.

There exists a symmetric $(1,1)$-tensor on $\mathcal{I}$, denoted by Θ, such that, for any tangent vectors $(X,Y)\in T\mathcal{I}$:

$$h\left(\mathsf{\Theta }X,Y\right)=\tilde{g}\left(X,Y\right)$$

(74)

Proof. 

From standard linear algebra, there exists an adjoint ${}^{t}d{\mathcal{N}}_W$ to $d{\mathcal{N}}_W$, defined by:

$$g^{{\mathcal{N}}_W}\left(d{\mathcal{N}}_{W}v,d{\mathcal{N}}_{W}v\right)=h\left({}^{t}d{\mathcal{N}}_{W}v,d{\mathcal{N}}_{W}v\right)$$

(75)

with, in local coordinates:

$${}^t{N}_i^j=h^{il}{N}_l^k{g}_{ij}^{{\mathcal{N}}_W}$$

(76)

where N (i.e., ${}^{t}N$ ) is the matrix associated with $d{\mathcal{N}}_W$ (i.e., ${}^{t}d{\mathcal{N}}_W$) and, as usual, $h^{il}={\left(h^{-1}\right)}_{il}.$ The $(1,1)$-tensor $\mathsf{\Theta }$ is then the product ${}^{t}d{\mathcal{N}}_{W}d{\mathcal{N}}_W.$ □

Remark 6.

Θ is defined even if $d{\mathcal{N}}_W$ is not full rank.

Remark 7.

All the relevant information concerning $d{\mathcal{N}}_W$ is encoded in $\mathsf{\Theta }.$ As a consequence, the geometry of an encoder is described by this tensor, hence also the one of an encoder–decoder block.

Remark 8.

The tensor Θ has expression $g_{pj}{N}_i^j{N}_k^p$ in a local orthonormal frame, hence is symmetric.

Definition 7.

Let ∇ be a connection on $T\mathcal{I}$. Its dual connection ${\nabla }^{★}$ is defined by the next equation:

$${\nabla }_{Z}h\left(X,Y\right)=h\left({\nabla }_{Z}X,Y\right)+h\left(X,{\nabla }_Z^{★}Y\right)$$

(77)

where Z is any tangent vector in $T\mathcal{I}$ and $X,Y$ are vector fields.

Definition 8.

A $(1,1)$-tensor Θ is said to satisfy the gauge equation [35] if, for all tangent vectors Z:

$${\nabla }_Z^{★}\mathsf{\Theta }=\mathsf{\Theta }{\nabla }_Z$$

(78)

Proposition 10.

If Θ satisfies the gauge Equation (78), then the $(0,2)$-tensor defined by:

$$\left(X,Y\right)\mapsto h\left(\mathsf{\Theta }X,Y\right)$$

(79)

is ∇ parallel.

Proof. 

For any vector fields $X,Y$, and any tangent vector Z:

$$\begin{array}{cc}\hfill & {\nabla }_Z^{★}h\left(\mathsf{\Theta }X,Y\right)=h\left({\nabla }_Z^{★}\mathsf{\Theta }X,Y\right)+h\left(\mathsf{\Theta }X,{\nabla }_{Z}Y\right)\hfill \\ \hfill & =h\left(\mathsf{\Theta }{\nabla }_{Z}X,Y\right)+h\left(\mathsf{\Theta }X,{\nabla }_{Z}Y\right)\hfill \end{array}$$

(80)

hence the claim. □

$\mathsf{\Theta }$, being symmetric, admits a diagonal expression in a local orthonormal local frame $(X_1,\dots ,X_n)$. When there exists a connection ∇ such that ${\nabla }_Z^{★}\mathsf{\Theta }X=\mathsf{\Theta }{\nabla }_{Z}X$ for any vector fields $X,Z$, parallel transport of the $X_i,i=1\dots n$ shows that the eigenvalues are constant and the eigenspaces preserved. The existence of a solution to the gauge equation thus greatly simplifies the study of an encoder, as a local splitting of the input manifold exists. The reader is referred to [35] for more details. In fact, the tensor $\mathsf{\Theta }$ is defined even if for general networks and the splitting may exist in this setting. This is the case when the rank of $d{\mathcal{N}}_W$ is locally constant, hence when it is maximal. A practical computation of $\mathsf{\Theta }$ can be obtained through the singular value decomposition, as Proposition (74) indicates. A numerical integration of the distribution given by the first singular vectors gives rise to a local system of coordinates, defining in turn a connection satisfying the gauge equation (the existence of a global solution has a cohomological obstruction that is outside the scope of this paper).

Finally, we introduce below a construction that takes into account the weight influence. As mentioned in Section 2, the derivative of the network with respect to its weights is adequately described as a 1-form, thus a section of $T^{★}\mathcal{O}.$ In fact, when the inner layers of the network are manifolds, the parameters are no longer real values and a suitable extension has to be introduced. One possible approach is to take a connection ∇ on the layer manifold $\mathcal{L}.$ Considering a point $p\in \mathcal{L}$, the exponential ${exp}^{\nabla }$ defines a local chart centered at $p.$ Given a point q in the injectivity domain of ${exp}^{\nabla }$, one can obtain its coordinates as ${log}_p^{\nabla }q=\overrightarrow{pq}$ and the activation of a neuron with input q as $\alpha \left(\overrightarrow{pq}\right)$, with $\alpha$ a 1-from in $T^{★}\mathcal{L}.$ In this general setting, a manifold neuron will be defined by its input in an exponential chart, a 1-form corresponding to the weights in the Euclidean setting and an activation function. Its free parameters are thus a couple $(q,\alpha )\in T\mathcal{L}\oplus T^{★}\mathcal{L}.$ This particular vector bundle is known as the generalized tangent bundle.

Recalling (43), it is worth to study the pullback of the generalized bundle $T\mathcal{O}\oplus T^{★}\mathcal{O}$. The generalized pullback bundle is then $T^{{\mathcal{N}}_W}\mathcal{O}\oplus T^{★{\mathcal{N}}_W}\mathcal{O}$, whose local sections are generated by the pullback local sections of the form:

$$\left(x,v\left({\mathcal{N}}_W\left(x\right)\right),\alpha \left({\mathcal{N}}_W\left(x\right)\right)\right)$$

(81)

Please note that the pullback can be performed on any layer, internal or input. Most of the previous derivations can be carried out on the generalized bundle, which must be thus considered as a general, yet tractable framework for XAI.

4. A Numerical Example

In this example, the input data are the handwritten digits from the MNIST database. A neural network with the next architecture was coded in torch 2 and trained on the dataset:

• First layer: convolutional, kernel size of 3, nonlinearity sigmoid;
• Second layer: convolutional, kernel size of 3, nonlinearity sigmoid;
• Pooling layer;
• Two linear layers;
• Softmax layer.

The input metric is Euclidean, the output one is the Fisher metric of the multinomial distribution with ten classes, that is given by the matrix:

$$\left(\begin{array}{cccc}{p}_1^{-1}& 0& \dots & 0\\ 0& p_2^{-1}& \ddots & 0\\ ⋮& ⋮& ⋮& ⋮\\ 0& \dots & 0& p_9^{-1}\end{array}\right)+\frac{1}{p_{10}}\left(\begin{array}{ccc}1& \dots & 1\\ ⋮& ⋮& ⋮\end{array}\right)$$

(82)

Since the output space has dimension 9, the pullback bundle also has dimension 9. At an input point x, a point in the pullback bundle is a couple $(x,v)$ with v a vector from ${\mathbb{R}}^9$ at output point ${\mathcal{N}}_W\left(x\right)$. On the other hand, the image of the input tangent bundle (simply a vector space in our case) has points $(x,d{\mathcal{N}}_{W}u)$ with u an input vector. We are thus considering a bundle mapping $(x,d{\mathcal{N}}_{W}u)\mapsto (x,d{\mathcal{N}}_{W}u)$ where the right-hand term has values in the pullback bundle, equipped with the output Fisher metric. Tensor $\mathsf{\Theta }$ is computed via singular value decomposition, already implemented in torch. We selected the rotation rate of the singular vector associated with the largest singular value as an indicator of the complexity of the decision process in the neighborhood of an input point. The code was adapted from https://github.com/eliot-tron/CurvNetAttack (accessed on 12 September 2023). A detection of outliers from a sample of 1000 points was performed. A visual analysis reveals that they correspond to poorly drawn digits, as indicated in Figure 2 where the two digits with the highest curvature indicator are plotted:

The first one is labeled “9”, which is quite obvious for a human operator, although the final stroke is vertical, while the second is labeled “7”, easily confused with a “1”.

5. Conclusions and Future Work

In this paper, several important constructions originating from information geometry were surveyed and some new ones introduced. The pullback bundle on a layer allows to describe the behavior of a network with respect to the Fisher information metric, and a simple description can be obtained when a gauge equation is satisfied. One important feature of this construction is its ability to fit in a general framework where layers take their inputs on a manifold.

Future work involves a companion paper describing computational procedures and examples from real case studies. An study of the properties of the pullback generalized bundle is also in progress. Finally, the case of networks with non constant rank $d{\mathcal{N}}_W$ must be considered. It is believed that they give rise to singular foliations.