biblio.bib
Dynamical Low-Rank Approximation for
Stochastic Differential Equations
Abstract
In this paper, we set the mathematical foundations of the Dynamical Low-Rank Approximation (DLRA) method for stochastic differential equations (SDEs). DLRA aims at approximating the solution as a linear combination of a small number of basis vectors with random coefficients (low rank format) with the peculiarity that both the basis vectors and the random coefficients vary in time.
While the formulation and properties of DLRA are now well understood for random/parametric equations, the same cannot be said for SDEs and this work aims to fill this gap. We start by rigorously formulating a Dynamically Orthogonal (DO) approximation (an instance of DLRA successfully used in applications) for SDEs, which we then generalize to define a parametrization independent DLRA for SDEs. We show local well-posedness of the DO equations and their equivalence with the DLRA formulation. We also characterize the explosion time of the DO solution by a loss of linear independence of the random coefficients defining the solution expansion and give sufficient conditions for global existence.
1 Introduction
This paper is concerned with the theoretical foundation of dynamical low-rank methods for stochastic differential equations (SDEs). Low-rank methods aim to approximate solutions of high-dimensional differential equations in a well chosen low dimensional subspace. Such methods are widely used in computational science and industrial applications [Bachmayr.M_2023_LowrankTensorMethods, lubich2008quantum, quarteroni2014reduced, schilders2008model]. Our focus here is on low-rank methods for high-dimensional SDEs, which is a case of primary interest, for instance in finance [redmann2021low, redmann2022solving], or in various applications such as biology [adamer2020coloured] or machine learning [muzellec2022dimension] where SDEs often appear as discretizations of Stochastic Partial Differential Equations (SPDEs).
Among such low-rank methods, the so-called Dynamical Low-Rank Approximation (DLRA) has gained notable attention and has been successfully applied in many applications in recent years [koch2007dynamical, kieri2016discretized, nonnenmacher2008dynamical]. Unlike traditional reduced basis methods, however, DLRA allows the low-dimensional subspace to evolve over time, enabling it to adapt to possibly rapid temporal variations of the solution. Thus DLRA evolves in a low-rank manifold, not in a fixed linear space of low dimension. In its original form [koch2007dynamical], such evolution in a manifold is achieved by enforcing the temporal derivative of the DLRA to be in the corresponding tangent space, which requires the differentiability of the approximate solution in time.
An equivalent formulation of the DLRA is the Dynamically Orthogonal (DO) approximation [sapsis2009dynamically]. This strategy can approximate random time-dependent equations by using a sum of a small number of products between spatial and stochastic basis functions, thus leading to a low-rank approximation with explicit parametrization of the low-rank factors. In the context of random partial differential equations (RPDEs), the DO/DLRA methodology has been successfully applied in several problems, showing promising computational results [choi2014equivalence, cheng2013dynamically, cheng2013dynamicallyII, feppon2018dynamically, musharbash2015error, musharbash2018dual, musharbash2020symplectic, kazashi2021stability, sapsis2009dynamically].
The goal of this work is to set a theoretical framework of DLRA for SDEs. This is a challenging task, because, unlike the RPDEs setting, the differentiability in time of the solution cannot be exploited due to the presence of the rough diffusion term. Our approach is to first define DO dynamics for a general SDE, specifying conditions for the well-posedness of this framework. From this DO system, we derive an equation that is independent of the parametrization. We define DLRA for SDEs as the solution to this parameter-independent equation, and show that these DO and DLRA methodologies are equivalent.
The DO approximation is given by the solution of a system of equations, which provide the evolution of the deterministic and stochastic basis functions. This system is however highly nonlinear, even if the original dynamics was linear. In addition, its well-posedness is not obvious due to the presence of the inverse of a Gramian, which is not guaranteed to exist at all times. Moreover, in the specific case of SDEs, the DO solution depends on the law of the whole process at every instant of time. These features make the well-posedness study challenging and not-standard, questioning also whether the existence of solutions is global or only local and, in the latter case, what happens to the solution at the explosion time. Besides being of interest in itself, a well-posedness study of the DO equations is also important to derive consistent and stable time discretization schemes, which are eventually needed to apply these techniques in real-life problems.
For RPDEs, theoretical results on the existence and uniqueness of the solution of the DO equations [kazashi2021existence], as well as stability and error estimates of time discretization schemes [kazashi2021stability, musharbash2015error, vidlickova2022dynamical] are available. An existence and uniqueness result is also available for the DLR approximation of two-dimensional deterministic PDEs [bachmayr2021existence], where the sum involves products of basis functions for the different spatial variables. In contrast, for SDEs well-posedness results remain largely unexplored. This lack of theory for SDEs is unfortunate, since DO approximations are very appealing and, despite no results on its well-posedness, have been also used in several problems modeled by SDEs (see e.g. [sapsis2009dynamically, sapsis2013blended, sapsis2013blending, ueckermann2013numerical]). The reader can find other approaches of initiating a DLRA framework starting from the Fokker-Plank equation in [sapsis2009dynamically, cao2018stochastic].
This paper aims at filling this gap between application and theory by:
-
•
(Re-)deriving the DO equations for SDEs. We also derive the corresponding parametrization independent DLRA equation for SDEs.
-
•
Showing the equivalence between DO approximation and DLRA. By exploiting this equivalence, we show that the DO equations and the DLRA equation are well-posed.
-
•
Characterising the finite explosion time in terms of the linear independence of the stochastic DO basis.
-
•
Discussing the extension of DLRA beyond the explosion time. We also provide a sufficient condition under which the DO/DLRA solution exists globally.
More precisely, we start by revisiting the DO equations for SDEs, which were originally derived in [sapsis2009dynamically] relying on the formal assumption that the DO solution is time differentiable, a property that SDE solutions do not possess. Deriving DO equations without using the time derivatives is a fundamental challenge, since the essence of the usual DLRA methodology consists in projecting the time derivative onto the tangent space of the low-rank manifold. We overcome this challenge by pushing the differentiability to the spatial basis, while the stochastic basis remains non-differentiable. It turns out that this alternative approach leads to the same DO equations as in [sapsis2009dynamically], supporting their validity as a correct dynamical low-rank approximation.
In the DLRA literature [koch2007dynamical, kieri2016discretized], it is well known by now that, when applied to RPDEs, the DO formulation is just a specific parametrization of the DLRA, the parameters being the deterministic and stochastic time-dependent bases defining the DO solution [musharbash2015error, kazashi2021existence]. In this paper, we derive an analogous result for SDEs. In particular, starting from the DO equations, we derive a parameter independent low-rank approximation which we name DLRA for SDEs, and show its equivalence with the DO formulation. The main result of this work is to prove local existence and uniqueness to both DO and DLRA equations. As a part of our existence result, we give a characterisation of the interval on which the solution exists. More specifically, we show that, if the explosion time is finite, then the stochastic DO basis must become linearly dependent at .
Although the characterisation of the explosion time provides us a valuable insight into the DO solution, it is not satisfactory from the practical point of view; if the true solution still exists at and beyond, we expect any sensible approximation to exist as well. Under some mild integrability condition on the initial datum, we show that, while the DO solution ceases to exist at the explosion time , the DLRA can be continuously extended up to and beyond. Thus, our findings offer insights on how to continue a DO approximation beyond the explosion time . As a final result, we show that a sufficient condition for global existence of a DO solution (i.e. ) is that the noise in the SDE is non-degenerate.
In passing, we point out that the DO/DLRA equation derived in this work cannot be regarded as a trivial application of the standard theory of SDEs on manifolds (see e.g. [gliklikh2010global, hsu2002stochastic]). Indeed, the projection operator employed in the DO decomposition is not standard, as it also involves a purely stochastic term that depends on the average of all paths.
The rest of the paper is organised as follows. Section 2 introduces the DO equations, together with theoretical justifications for why they are sensible. Furthermore, it introduces a DLRA formulation that does not depend on the parametrization. Section 3 concerns the local well-posedness of the DO equations, where we also show the continuity of the solution with respect to the initial data. In Section 4, we study the behaviour of the solution up to and beyond the explosion time . In Section 5 we draw some conclusions and perspectives.
2 The DO equations
Let be a filtered complete probability space with the usual conditions; see for example [schilling2021brownian, Remark 6.24]. Consider the stochastic differential equation (SDE)
(2.1) |
where is a standard -dimensional -Brownian motion. Here, we used the notation for and . Let and denote the Euclidean norm and the Frobenius norm, respectively. We work with the following general assumptions on the coefficients of the SDE (2.1).
Assumption 0.
The drift coefficient and the diffusion coefficient are jointly measurable and Lipschitz continuous with respect to the second variable, uniformly in time, i.e.
(2.2) |
for some constant .
Assumption 0.
The drift and the diffusion satisfy the following linear-growth bound condition uniformly in time:
(2.3) |
for some constant .
The choice of the Frobenius norm in Assumption 2 is made primarily for convenience when utilizing Ito’s isometry; this is standard practice. Since we are in a finite-dimensional setting, the results presented in this work remain valid under Lipschitz continuity with respect to other norms, with a modified constant. It is also worth mentioning that in the infinite-dimensional case, Lipschitz continuity is typically considered with respect to the Hilbert–Schmidt norm between appropriate spaces; see for example [da2014stochastic]; hence our choice of the Frobenius norm is a first step in this direction. Furthermore, we assume that the initial condition in (2.1) satisfies the following:
Assumption 0.
(2.4) |
Under these assumptions, equation (2.1) has a unique strong solution; see for example [schilling2021brownian, Theorem 21.13].
Let us consider a positive integer such that . To numerically approximate (2.1), in this work we consider dynamically orthogonal approximations of the form
(2.5) |
where is a deterministic absolutely continuous matrix-valued function that gives a matrix in with orthogonal rows for all , whereas is an Itô process with values in with linearly independent components for all . It is worth pointing out that for all and , the approximate process belongs to an dimensional vector space spanned by the rows of , whereas each component , belongs to .
Approximations of the form (2.5) for SDEs have been considered already in [sapsis2009dynamically], where the following evolution equations, hereafter called DO equations were derived for the factors by using the same formal procedure as for ODEs via treating the process as differentiable in time:
(2.6) | ||||
(2.7) |
Here, we let whereas denotes the projection-matrix onto the row space of ; when has orthonormal rows, one has . In Section 2.1, we will give a rigorous justification for (2.6) and (2.7).
Having stated these equations, we now define our notion of strong DO solution and DO approximation for an SDE problem of the type (2.1).
[Strong DO solution] A function is called a strong DO solution for (2.1) if the following conditions are satisfied:
-
1.
the initial conditions are such that is a matrix with orthonormal rows and has linearly independent components;
-
2.
the curve is absolutely continuous on and for a.e. ;
-
3.
the curve has almost surely continuous paths on and it is -measurable for all . Moreover, for any the components are linearly independent in ;
- 4.
We remark that points (1) and (2) in Definition 2 imply that has orthonormal rows for all , i.e. for all . Furthermore, the term “strong” in Definition 2 follows the standard terminology for the solution of SDEs: we seek a solution on the given probability space and Brownian motion. See for example [karatzas2012brownian] for more details. For convenience, given a DO solution we call the product a DO approximation. {defn}[DO approximation] We call a process a DO approximation of (2.1) if there exists a strong DO solution such that for all . Given a DO approximation , the corresponding DO solution is determined only up to a (process of) rotation matrix. Indeed, let and be two strong DO solutions such that for all . Then the orthogonality of the rows of implies with . The matrix is orthogonal for every , but not necessarily an identity. See Section 2.3, in particular Proposition 2.7, for more details.
2.1 Consistency of the DO equations
In this section, we rigorously show the consistency of the DO equations (2.6) and (2.7) in the sense described hereafter. Assume that the exact solution of (2.1) is of the form with deterministic function and an Itô process for some that fulfil the following properties:
-
1.
the function , , is absolutely continuous on and satisfies for almost every ; moreover, for almost every , where is the identity matrix;
-
2.
is an Itô process
(2.8) with coefficients and , where and are progressively measurable and have a continuous path almost surely.
Indeed, since and in (2.8) are progressively measurable, by applying Itô’s formula [ito1951formula, kunita1967square] we have
(2.9) | ||||
(2.10) | ||||
(2.11) |
where , is the quadratic covariation of and . Since is assumed to satisfy (2.1), the uniqueness of the representation of Itô processes implies
(2.12) | ||||
(2.13) |
Now, using and for almost every , these equalities imply
(2.14) |
Moreover, is absolutely continuous and by assumption is a.s. continuous. This implies that and have a continuous path almost surely, and hence (2.7) follows. In turn, from (2.12) we find
with . Hence, from the orthogonality assumption for the rows of we have
This completes our consistency argument.
2.2 DO equations interpreted as a projected dynamics
The DO equations (2.6)–(2.7) are posed as a system of equations for the separate factors and . We now discuss what equation the DO approximation should satisfy. In other words, we aim to derive an equation for in the ambient space, independent of the parametrization .
For this purpose, we substitute (2.6) and (2.7) into (2.9); we obtain
(2.15) | ||||
where is the application of the -orthogonal projection to each component of . To derive a parameter-independent equation, we seek a parameter-free expression of the projections and . Given , , let be the orthogonal projection matrix to the image of the mapping , and let be the -orthogonal projection to the image of the mapping . In Lemma 2.1 we will show , i.e., is a finite-dimensional linear subspace of , and thus is well defined.
In the case of DO approximation , we have and , we conclude that , and , , so that (2.15) can be rewritten as
(2.16) | ||||
This equation, derived from (2.6) and (2.7), does not depend on the parametrization of the pair and could be taken as an alternative definition of DO approximation. More precisely, given any process with for , we can define the following stochastic process:
(2.17) |
It is worth noticing that (2.17) is a McKean-Vlasov-type SDE since the evolution of depends on the law of the process.
Finally, to speak of the rank of the solution to (2.17) we note the following.
Given the mapping defined by for is by definition finite rank. Moreover, the operator is the adjoint of :
The operator is given by for . The following lemma characterises the rank of these operators.
Lemma 2.1.
Given , we have .
Proof.
First, notice that we have , where is the orthogonal complement with respect to the Euclidean inner product. Hence, we have . Thus, the rank-nullity theorem implies
Therefore . Moreover, from (see for example [Kato.T_book_1995_reprint, Theorem III.4.13]), the proof is complete. ∎
In view of the lemma above, we call , equivalently and , rank of . {defn}[DLR solution of rank ] A process is called a DLR solution of rank to (2.1) for an initial datum if it satisfies (2.17), for some for all , and has almost surely continuous paths.
The parameter-independent formulation (2.17) corresponds to the projected dynamics in the DLRA literature; see [koch2007dynamical], also [kazashi2021stability, Proposition2]. Notice however that in our formulation, only the projector is applied to the diffusion term , instead of the full projector
If we naively wrote the projected dynamics following the standard DLRA approach, we would end up with the alternative formal expression
(2.18) |
which would coincide with (2.17) if one could prove that “” for a.e. . However, it is not obvious how to give a rigorous meaning to the term as an Itô integral and for this reason we do not pursue the formulation (2.18) further.
To confirm that (2.17) is nevertheless a sensible projected dynamics, let us ask ourselves the following question: if the solution of (2.17) satisfies
is the right hand side of (2.17) consistent with this structure of ? In the following, we will see that the answer is affirmative, which supports the validity of (2.17), and thus (2.6) and (2.7), as a correct DLRA formulation for SDEs.
In view of Lemma 2.1, we study the rank of the matrix , where is given by (2.17). To do this, we will show that is in a tangent space of the rank- manifold, i.e., the manifold of matrices with rank equal to , at . For , Itô’s formula implies
where and . Thus, we have
(2.19) |
with and .
On the other hand, from [vandereycken2013low, Proposition 2.1] we know that the tangent space of a rank manifold at can be characterized as
To conclude we will use the following result.
Lemma 2.2.
Let the singular value decomposition , with be given, where is a matrix consisting of orthogonal columns . Then, the canonical expansion of the finite rank operator is given by
where , is an orthonormal basis of .
Proof.
From
we have , and thus is an orthonormal basis of .
Thus, with some coefficients we have for any a representation , which implies
The functions , are orthonormal in , and thus form an orthonormal basis in the -dimensional subspace . ∎
Remark 2.3.
Lemmata 2.1 and 2.2 imply , where
i.e. the rank of as in Definition 2.1 gives the dimension of the space spanned by all the realizations of . To see this, take such that . Lemma 2.1 implies that holds, where and is a matrix consisting of orthogonal columns , while Lemma 2.2 tells us that , for , is an orthonormal basis of . Therefore, admits the expansion
(2.20) |
where is the component-wise -inner product. From we have
(2.21) |
By substituting (2.21) into (2.20), we obtain for any ,. This implies that for any , and , i.e. the space spanned by all the realizations of is spanned by . Therefore, .
The equation (2.19) together with Lemma 2.2 shows . Indeed, the first term of the right hand side of (2.19) can be written as
where, using Lemma 2.2, we may write and with some . Then, with and we have
Similarly, we have and . Hence, if is of rank around , then its derivative (2.19), which was derived from the projected dynamics (2.17), lies indeed in . This consistency supports the validity of the formulation (2.17), and in turn that of the DO equations (2.6) and (2.7).
2.3 Equivalence of DO and DLR formulations
In the previous section, we showed that if there exists a strong DO solution to (2.1), then the corresponding DO approximation satisfies (2.17). We now investigate the reverse question: if is a rank- solution of (2.17) (DLR solution of rank of (2.1)) does there exist a DO solution such that ?
First, we need the following bound for the DLR solution.
Lemma 2.4.
Let a rank- DLR solution , , to (2.1) with be given. For all , satisfies
(2.22) |
The following theorem gives the uniqueness of the DLR solution.
Theorem 2.5.
Let be such that . Suppose that two DLR solutions and of rank to (2.1) with initial datum exist on . Then and are indistinguishable.
Proof.
The processes and are assumed to satisfy
and
respectively, for the same initial datum . This implies that
(2.23) | ||||
Denote by
the infimum over time of the maximum between the -th singular value of the matrices and , where for all we are considering the singular values in a decreasing order. We have . To see this, first note that, from Lemma 2.1 the rank of is the same as the rank of . But from the definition of DLR solution, has always rank , and moreover, the continuity of implies continuity of on . Hence follows; see also [kazashi2021existence, Lemma 2.1]. Then, Proposition A.3 gives us the following Lipschitz bounds for the projectors for any :
(2.24) | ||||
Hence (2.23), Itô’s isometry and Jensen’s inequality lead to
(2.25) | ||||
for all in with and as in (2.22). Gronwall’s lemma yields for all and therefore and are versions of each other. Moreover, because they both have a.s. continuous paths, they are indistinguishable; see for example [Protter.P.E_2005_book_StochasticIntegral_2nd, Theorem 2]. ∎
Given a strong DO solution to (2.1), the corresponding DO approximation satisfies the equation (2.16), and thus (2.17). Hence, Theorem 2.5 gives the sought equivalence between the DLR equation (2.17) and the DO equation (2.6)–(2.7), which we state as a corollary.
Corollary 2.6.
Let with . Suppose that a DLR solution of rank to (2.1) and a strong DO solution exist on , both with initial datum . Then, and the DO approximation are indistinguishable on .
Moreover, the DO solution giving the same DLR solution is unique up to a rotation matrix. See also [kazashi2021existence, Section 2.2] for a similar result.
Proposition 2.7.
Assume that a DLR solution of rank to (2.1) with initial datum exists for all . Suppose there exist two strong DO solutions and . Then there exists a unique orthogonal matrix such that .
Proof.
The proof follows closely the arguments in [kazashi2021existence, Lemma 2.3, Corollary 2.4, and Lemma 2.5]. We will show that there exists a unique absolutely continuous curve with orthogonal matrix for all such that , and that such is a constant in . First, we will derive an equation that must satisfy. Suppose that sought exists. Note that, since both and satisfy (2.16), Theorem 2.5 implies for all and a.s. Then since is a strong DO solution, one must have
(2.26) |
As and , from it follows
(2.27) |
Using orthogonality of and , we obtain the following differential equation that has to satisfy:
(2.28) |
where is an orthogonal matrix. But this equation has a unique solution , which is an orthogonal matrix. Hence, going back the argument above, satisfies (2.26). The absolute continuity of and yields for all , with some matrix .
Such that makes can be constructed explicitly. Since , are deterministic, from we must have
Take . Then, from the argument above holds for all . We conclude the proof by noting the orthogonality of :
∎
3 Local existence and uniqueness
The equations (2.6) and (2.7) define a non-standard system of stochastic differential equations. Notice that in (2.6), the matrix-valued function is not defined everywhere in . Moreover, the vector field requires taking the expectation, and thus depends on the knowledge of all the paths of . Hence, the vector fields that define the DO equation are not defined path-wise a priori. This setting makes the existence and uniqueness result of DLR solutions non-trivial.
To establish existence and uniqueness of solutions of (2.6) and (2.7) given an initial datum , we follow a fixed-point argument. We will define a sequence of Picard iterates, which belongs to a specific set of functions, where the first element of the sequence is made of a pair of constant-in-time functions . Then we will show that this sequence converges in this set.
Let be such that the rows of are linearly independent for every (but not necessarily orthonormal), and let be an -adapted process such that has linearly independent components for every . For with and , define
(3.1) |
(3.2) |
where we recall that is the projection-matrix onto the row space of . Note that the stochastic integral is well defined, since is (indistinguishable from) a progressively measurable process.
We will construct a unique fixed point of and . Because of the aforementioned difficulties, defining a suitable sequence of Picard iterates requires some care. Let us consider with having orthogonal row vectors, and , having linearly independent components, -measurable with and . We force iterations to belong to balls in and around and , respectively, of a suitable radius .
For the ball in , to invoke Proposition A.1 in the appendix, we equip with the norm . For orthogonal, we have , and
so with as in (A.1), implies . Hence, the projection is Lipschitz continuous on ; see Lemma A.4.
For the ball in , first note that, since is continuous in the open set (cf. [kazashi2021existence, Proof of Lemma 3.5]), and , via Proposition A.1 there exists a ball in around with radius such that any has linearly independent components and . Abusing the notation slightly, we let
(3.3) |
Proposition A.1 tells us that is non-increasing in both variables and .
We want to define sequences and for with a suitable . To this extent, let , , , and , for with
(3.4) |
Moreover, let
and
The following lemma shows that our Picard sequence takes value in .
Lemma 3.1.
Proof.
We have , , , so, trivially, and , where is built as in (3.3). Moreover, is -adapted thanks to Assumption 3. Thus, . Assume for . Then, from Assumption 2, we see that and are well defined, and that is -adapted. Moreover, and the inequality for , implies
where in the second and in the third lines we employ the Doob’s martingale inequality and Itô’s isometry, respectively, in the penultimate line we used , and in the last line together with the definition of . Similarly, we have
We readily have and hence . Likewise, we have , since
where in the second inequality we have used the fact that is an orthogonal projector. Finally,
Thus, by induction we conclude for . ∎
We now establish a Lipschitz continuity for and on .
Lemma 3.2.
Take as in (3.4). There exists a constant such that for any it holds
(3.5) | ||||
Proof.
For any , from Doob’s martingale inequality and Itô’s isometry applied similarly as in the proof of Lemma 3.1, we have
where is a positive constant. Similarly, we have for a constant that
where we used the Lipschitz continuity of and ; see [kazashi2021existence, Lemma 3.5] and Lemma A.4. ∎
Thanks to the previous results, we have that sequences and not only live in and , respectively, but also converge therein.
Lemma 3.3.
The sequence admits a limit , and the sequence admits a limit almost surely. Moreover, is also an -limit.
Proof.
From Lemma 3.1 we have for all . Let . Then, since is continuous on , so is , and thus is measurable. Similarly, is a.s. continuous on . Noting that implies , from Lemma 3.2 we have
(3.6) |
Chebyshev’s inequality then implies
and thus from the Borel-Cantelli lemma we have
Hence, has a limit , where the convergence is uniformly in on , a.s. Moreover, from the completeness of the underlying probability space, is -adapted. Also, from , , Fatou’s lemma implies
(3.7) | ||||
(3.8) |
and thus . An analogous argument applies to see that has a limit in with .
The sequences converge in as well. Indeed, from (3.6), for we have
and thus Fatou’s lemma implies
hence . ∎
The argument used in Lemmata 3.1-3.3 to show convergence of the Picard iterates resembles the standard one used in establishing well-posedness of SDEs, however containing important differences. First, the map (3.1) to update involves a “stochastic projection” introducing a dependence on the law of the process. Second, the same projection involves the term (inverse of the Gramian of the stochastic basis), which might not be defined for all iterates or it may fail to be Lipschitz continuous with respect to . We thus have to restrict to a well-designed set, containing the initial state, in which the inverse of the Gramian is well-defined and Lipschitz continuous. Lemma 3.1 guarantees that the Picard iterates are contained in such set for all . Lemma 3.2 shows the Lipschitz continuity of the mapping that defines the iterates on this set, and Lemma 3.3 assures that admit a limit in the same set. We are now prepared to establish our existence result for the DO system, employing an argument that closely resembles the conventional approach used for SDEs.
Theorem 3.4 (Existence of a DO solution).
Proof.
To establish uniqueness, first we will show the following norm bound analogous to Lemma 2.4.
Lemma 3.5.
For , suppose that , with having orthonormal rows, and , with having linearly independent components, satisfy and for all . Then, for all we have
(3.9) | ||||
(3.10) |
Proof.
First, from , the function is absolutely continuous on , and thus differentiable almost everywhere. The derivative satisfies
and thus a.e. on . Therefore, from the orthonormality of the initial condition , for all we have
where only if , and otherwise. This shows the identity (3.9).
Now the following uniqueness result follows.
Theorem 3.6 (Uniqueness of DO solutions).
Let the assumptions of Lemma 3.5 hold. For , suppose that and satisfy and ; and for . Then, we have
Proof.
By hypothesis, the solutions and satisfy the stability estimates shown in Lemma 3.5. Moreover, from the continuity of , we have
for some . Then, noting the norm bounds (3.9) and (3.10), by an argument similar to the proof of Lemma 3.2 (see also [kazashi2021existence, Lemma 3.5] and Lemma A.4), with a constant we have
(3.11) |
for . Thus, applying the Gronwall’s lemma yields
Now the proof is complete. ∎
Remark 3.7.
We conclude this section by showing the continuity of the solution with respect to the initial datum.
Lemma 3.8.
Let and be DO solutions on with initial data and , respectively. Define and . Then, under Assumptions and ,
holds. Moreover, let and be DO approximations on with initial data and , respectively, and let
Then, there exists a positive constant such that
(3.12) |
Proof.
Using an argument similar to the proof of Lemma 3.2 and the Lipschitz continuity of the projections as in Lemma A.4, we obtain the following relation for the stochastic basis:
Similarly, noting the Lipschitz continuity of the inverse Gram matrix [kazashi2021existence, Lemma 3.5], for the deterministic basis we have
Then, the Gronwall’s lemma yields the first part of the statement. Finally, to prove (3.12), we proceed as above and as done in Theorem 2.5. Therefore, for all there exists a positive constant such that it holds
where in the last line we use the Gronwall’s lemma. ∎
4 Maximality
In the previous section, we established the existence and uniqueness of strong DO solutions locally in time on an interval . In this section, we investigate how much such an interval can be extended.
We give a characterisation of the maximal interval of existence of the strong DO solution in terms of . It turns out that the DO solution can be extended until explodes. If stays bounded for all , then the DO solution exists globally. If explodes at a finite explosion time , the couple inevitably ceases to exist at ; we will show nevertheless that the DO approximation can be extended beyond .
4.1 Explosion time of the DO solution
Theorem 3.4 guarantees the (unique) existence of a strong DO solution, albeit up to possibly a short time . In this section, we show that the solution can be extended until becomes singular.
Analogous maximality results have been considered in the DLRA literature for deterministic and random PDEs [koch2007regularity, kazashi2021existence]. For the SDE case of this paper, we need to proceed with caution. Indeed, adhering to the definition of SDEs, we need the DO solution to be path-wise continuous a.s. As such, we consider extension with countable number of operations, which we describe in the following.
Let be the interval on which the DO solution exists; such existence is guaranteed by Theorem 3.4. Let us choose such that the following two bounds are satisfied
(4.1) | ||||
We will show that the solution can be extended at least until or hits the bound or . Define by
(4.2) |
where , with defined in (A.1). Then from the proofs of Lemmata 3.1 and 3.3, and Theorem 3.4, we can construct a convergent Picard-iteration for the interval , which yields the extension of the DO solution up to . Set . By construction is independent of , and with the same we can repeat the same argument as long as the bound (4.1) is satisfied. Hence, the solution can be extended until either or gets violated.
From the argument above, the following quantities are well defined for any :
with a convention . With these, we define the sequence
(4.3) |
which is a sequence of stopping times. By continuity of the paths, is a non-decreasing sequence, which allows us to define .
Now we will show that if , then the norm of the inverse of the Gram matrix must blow up.
Proposition 4.1.
We have either or , where is defined in (4.3). If , we necessarily have .
Proof.
From the norm bound (3.10), for sufficiently large we must have
for any , and thus without loss of generality we assume
We will first show that implies . We argue by contradiction, and assume and . Then, we have for some and . But then since , the continuity of implies and thus for any sufficiently large we have
which is absurd. Hence, implies .
To conclude the proof we will show
If this is false, then there exist a sequence and such that for all . But since there is a sequence such that for all . We take a sub-sequence so that for all . From the continuity of on , there exists such that . Now, from (3.10) and [kazashi2021existence, Lemma 3.5] we have for any
which is impossible since as and is continuous on . Hence, the proof is complete. ∎
4.2 Extension up to the explosion time
Even when the explosion time for the DO solution is finite, and thus and cease to exist at , we will show that the product nevertheless admits a continuous extension up to , and beyond , under suitable assumptions.
For any , from (2.17) we have
(4.4) |
and, hence, the Itô’s isometry implies
From Assumption 2, the orthogonality of the rows of and the norm bound (3.10) of it follows that
with Therefore, admits a unique extension .
Thus obtained is continuous from to , but not necessarily path-wise a.s. continuous on . It turns out that for initial data with suitable integrability, the DO approximation actually admits a.s. Hölder continuous paths. Namely, we assume the following -integrability condition.
Assumption 0.
The initial condition satisfies
(4.5) |
Notice that this condition is equivalent to
(4.6) |
where is arbitrary such that for orthonormal. Indeed, for any DO initial condition , the orthogonality of the rows of implies
provided that the -th moment of either or (hence both) exists. Thus, . Similarly, for any other DO initial condition we have
Hence, Assumption 4.2 is equivalent to (4.6), with an arbitrarily fixed stochastic basis.
Analogously, for all we have
(4.7) |
The DO solution preserves the -integrability of the initial datum.
Lemma 4.2 (Even order moments of the solution).
Proof.
From the Itô formula, satisfies the following SDE (see also [kloeden1992stochastic, Theorem 4.5.4]):
We take the expectation of both sides, and, noting the progressive measurability of , , and , use Fubini’s theorem to obtain
Then, by using the relation for , the statement follows by Gronwall’s lemma:
∎
With Lemma 4.2, we can now establish a Hölder-type bound on for
, which by Kolmogorov-Chenstov theorem, implies the existence of an a.s. continuous version of .
Proposition 4.3.
Proof.
The proof follows closely the one in [kloeden1992stochastic, Theorem ]. First, notice that, in the view of (4.7), Lemma 4.2 gives an estimate of the -moments of the DO approximation for some . Now we use the inequality for twice and Jensen’s inequality to (4.4) to obtain
Having proved these results concerning boundedness of -moments of the DO solution, we are now ready to extend it up to the explosion time with a.s. continuous paths.
Theorem 4.4.
Proof.
By Assumption 4.2, Proposition 4.3 holds and admits an extension that is unique in . From for , the extension is unique in for . By construction of , we have
(4.13) |
Therefore, by Kolmogorov-Chenstov Theorem (see for example [karatzas2012brownian]) there exists a version of the DO approximation that is -Hölder continuous for all . By construction, is a process with bounded -moments.
Finally, let us see that and are indistinguishable. Indeed, by construction of we have a.s. Moreover, and have a.s. continuous paths and are versions of each other, and thus indistinguishable. Hence, and are indistinguishable. This completes the proof. ∎
Together with Proposition 4.1, Theorem 4.4 gives us an insight into how to continue a DO approximation beyond the explosion time . Let a DO approximation of (2.1) with explosion time that satisfies (4.5) be given. Suppose for all , with a positive integer . By Theorem 4.4, can be continuously extended up to , while from Proposition 4.1 we know . This implies that . Therefore, one can extend our DO approximation continuously in beyond by considering the DO system (2.6)–(2.7) with initial datum . The corresponding DO solution satisfies and for , where is the new explosion time for .
4.3 The case of uniformly positive-definite diffusion
In the previous section, we have shown that by assuming the boundedness of the moments of the initial condition of the DO approximation for , the DO approximation can be extended up to the explosion time . It turns out that, under the condition that we will introduce in the following, the explosion time is never finite; as a result, rank- DO solution exists globally.
A sufficient condition is that the diffusion matrix is positive definite with a lower bound on the smallest eigenvalue, where the bound is uniform in and . This condition turns out to ensure that the smallest eigenvalue of the Gram matrix remains bounded below uniformly in and , which in turn guarantees the global existence of the DO solution. In the following, we use the notation (respectively ) with square matrices to indicate that is positive definite (respectively positive semidefinite).
Proposition 4.5.
Suppose the rank DO solution exists on . Assume moreover that there exist such that
(4.14) | ||||
(4.15) |
Then for all we have
(4.16) |
where is defined in Lemma 3.5.
Proof.
By introducing the shorthand notation and , the -th component of satisfies
Hence, using Itô’s formula for , we have
Hence,
and taking the expectation of both sides yields
We now aim at analyzing the smallest eigenvalue of through the Rayleigh quotient. For any unit vector , we have
(4.17) |
Thanks to (4.15), the last term can be bounded below as
The first term can be bounded above as
Taking leads to the following estimate on the derivative of :
As a consequence of the proposition above, the following global existence result is obtained.
Theorem 4.6 (Global Existence of DO solution).
Let Assumptions 1–3 hold. Suppose that the assumptions of Proposition 4.5 hold. Then, the DO solution exists for all .
Proof.
From Theorems 3.4 and 3.6, the rank solution uniquely exists up to a certain time . Denote by its explosion time and suppose, to obtain a contradiction, . Then, from Proposition 4.1 we have . But by Proposition 4.5 the Rayleigh quotient with satisfies
and, hence, for some constant we have for all . This contradicts . Therefore, . ∎
Remark 4.7.
It is worth noticing that (4.15) is satisfied in the case of additive non-degenerate noise.
5 Conclusion
In this work, we achieved to set a rigorous DO setting for SDEs under the conditions that the studied drift and diffusion satisfy a Lipschitz condition and a linear-growth bound. First, we (re-)derived the equations which characterize the evolution of the deterministic and stochastic modes, in a DO formulation, and showed how these can be re-interpreted as a projected dynamics leading to a DLRA formulation. Our derivation makes use of the Itô’s formula and avoids the direct use of time derivatives. We proved local-existence and uniqueness of the DO formulation and analyzed the possibility of extending the solution up to and beyond the explosion time. Finally, we gave a sufficient condition that assures the global existence of the DLR approximation.
One natural development of this work would be to extend this DO framework and well-posedness results to accommodate weaker assumptions on the drift and diffusion (e.g. local lipschitzianity and/or weak monotonicity).
Furthermore, it would be interesting to build a DLR formulation as a fully projecting dynamics as in (2.18) by giving a rigorous meaning to the term , possibly in a distributional sense. In case one manages to achieve this goal, the direct connection to the standard DLRA formulation for deterministic or random equations would allow us to apply numerical projector splitting schemes, which have been shown to perform very well in those contexts [ceruti2022unconventional, lubich2014projector].
Acknowledgements
Yoshihito Kazashi acknowledges the financial support of the University of Strathclyde through a Faculty of Science Starter Grant. This work has also been supported by the Swiss National Science Foundation under the Project n. 200518 “Dynamical low rank methods for uncertainty quantification and data assimilation”.
Appendix A Appendix
For a Hilbert space , denote by the product Hilbert space equipped with the norm for . For let be the Gram matrix
If is invertible, then for close to , is also invertible. The following result makes this intuition precise.
Proposition A.1.
Suppose that has linearly independent components , in , and that and hold for . Then, there exists defined as
(A.1) |
such that we have
and is decreasing in both and . Here, is the open ball in of radius around .
Proof.
Any may be written as with and . We will derive an upper bound on that guarantees .
Define and let . Then, we have
Therefore, it holds
Let so that and
which implies . Now we use to obtain the bound . For any , we have and thus this bound yields
Then, is guaranteed by the condition , which also implies above. Finally, the function , defined as in (A.1), is decreasing in and :
∎
Recall that a finite rank function admits a representation reminiscent of the singular value decomposition as in (A.2) below; see [kazashi2021existence, Lemma 2.1], as well as Lemma 2.2 and Remark 2.3. For this decomposition, in Proposition A.3 we show that projector-valued mappings, where the projection is onto the space spanned by the spatial basis of this decomposition, are Lipschitz continuous. In order to achieve this goal, we need the following Lemma A.2. The proofs of Lemma A.2 and Proposition A.3 follow closely [bachmayr2021existence, Proof of Lemma A.2]; see also [wei2016guarantees, Lemmata 4.1 and 4.2] for a finite dimensional version. However, unlike [bachmayr2021existence, Lemma A.2] we do not assume to be separable.
Lemma A.2.
Consider that have the following representations
(A.2) |
with , and , all orthonormal in their respective Hilbert spaces, for . Here, and , , are ordered in the descending order. Define the projections , , , and ; and similarly for , , , and . Then, we have
(A.3) | ||||
Proof.
The mappings and are orthogonal projections. Indeed, the idempotency of and can be easily checked and the -orthogonality of follows from its -orthogonality. To show the orthogonality of , first consider . Noting that is a continuous linear functional on for , from [Leoni.G_2017_book_Sobolev_2nd, Theorem 8.13] we have:
Similarly, and are orthogonal projections.
Let us prove the second identity in (A.3). For , we have
(A.4) |
As is an orthogonal projector, for any . Hence, together with (A.4), for all one has
(A.5) |
Moreover, for any it also holds
(A.6) |
and
(A.7) |
with unit vectors and . From (A.5) , one has
Using (A.6) and , we obtain
To show the opposite inequality we first notice that
Therefore, from (A.5) one has
Similarly, Hence,
Finally, let us prove that . First define the vector space and endow it with the norm for . Thanks to the orthogonality of and the properties of orthogonal projectors, the following chain of equalities holds:
(A.8) | ||||
Considering the Stiefel manifold the following equality holds
(A.9) | ||||
where in the fourth line we defined , in the second-to-last line indicates the -th row of , and in the last line indicates the -th eigenvalue of . The infimum above can be attained by taking in (A.9) such that is the unit-norm eigenvector of associated with , and for . Then, independently of . Coming back to (A.8), we get
where in the last line we just follow the same discussion of (A.8) and (A.9) in a reverse order. Since and are orthogonal projections and thus self-adjoint, we get The first equality in (A.2) follows similarly. ∎
We now come back to the following Lipschitz continuity of projector-valued mappings of finite rank functions.
Proposition A.3.
Suppose that have the representation (A.2) with , and , all orthonormal in their respective Hilbert spaces, for . Here, and , , are ordered in the descending order. Then, the projections and satisfy
(A.10) | ||||
Moreover, we have
(A.11) |
Proof.
First, let us prove that . On the one hand, it holds for all that
On the other hand, since with is trivially in with we have
Fix . Using in yields
We have for ; indeed,
holds because , but [Leoni.G_2017_book_Sobolev_2nd, Theorem 8.13] implies
and thus for . Hence, the Cauchy–Schwarz inequality implies
Recalling the orthonormality assumptions on and , we use Bessel’s inequality to obtain , Moreover, one can also prove that following a similar argument to show Bessel’s inequality. It follows . Therefore, via using Lemma A.2 we conclude that
Likewise, holds. By exchanging the roles of , with ,, respectively, using the homogeneity property of the norm and (A.3), then the inequalities (A.10) follow. Finally, the last claim (A.11) comes from
∎
We conclude this section giving another useful result inherent to orthogonal projections.
Lemma A.4.
Given two positive constants, let
where the invertibility of is implicitly assumed. Given , define the orthogonal projectors onto the rows of and , respectively. Then there exists a constant such that the following holds
Further, given two positive constants, let
Given , let , be the orthogonal projectors onto the subspace spanned by the components of and , respectively. Then there exists a constant such that the following holds
Proof.
For we have
Let . From [kazashi2021existence, Lemma 3.5], there exists a constant such that
and thus
Then, using the Cauchy–Schwarz inequality to and , taking the square of both sides and taking the expectation yields the result. The result for can be analogously shown. ∎