TITLE: What is the relationship between the wave functions on both sides of the one-dimensional local non-zero potential energy?
QUESTION [1 upvotes]: In a one-dimensional space, the potential $V(x)\neq 0$ if $0<x<a$ and $V=0$ if $x <0$ or $x>a$.
If we set the wave function in $x<0$ as $\phi_1 = Ae^{ikx}+Be^{-ikx}$, and the wave function in $x>a$ as $\phi_2 = Ce^{ikx}+De^{-ikx}$.
Now I want to know if there is a general relation between $\phi_1$ and $\phi_2$, or a relation between $A,B,C,D$.
Here is my solution:
Since $\phi_1$ satisfies the stationary Schroedinger's equation,
$$
\frac{d^2\phi_i}{dx^2} + \frac{2m}{\hbar^2}E\phi_i =0, i=1,2 (x<0).
$$
Therefore,
$$
(\frac{2m}{\hbar^2}E-k^2) (Ae^{ikx}+ Be^{-ikx}) = 0,
$$
and
$$
(\frac{2m}{\hbar^2}E-k^2) (Ce^{ikx} + De^{-ikx}) = 0.
$$
Since the energy at both regions should be the same, I just obtain
$$
E=\frac{\hbar^2k^2}{2m}
$$
for both $x<0$ and $x>a$. But this expression does not include the constants $A,B,C$ or $D$.
Since both $Ae^{ikx}$ and $Ce^{ikx}$ describe a plane wave propagates from left to right, in stationary states, they should be the same, therefore, $$A=C.$$ Similarly, I obtain $$B=D.$$
Since $\phi^*_1$ is also the solution in $x<0$, I can obtain
$$
(\frac{2m}{\hbar^2}E-k^2) (B^*e^{ikx} + A^*e^{-ikx})= 0.
$$
Therefore,
$$
A^* = D; B^* = C.
$$
I am not sure this result. Could someone give some suggestion about this problem?

REPLY [0 votes]: It is not, in general, true that $A=C$ and $B=D$. For example, in a scattering problems, we might choose $D=0$, in which case $|A|>|C|$ and $B\neq 0$ (assuming $V(x)\neq 0$ between $0$ and $a$) because some of the right moving probability will be "reflected" off of the potential.
The important thing to note is that once you fix the boundary conditions at one end of the region between $0$ and $a$, there will be only one solution to the Schrödinger equation in that region matching those conditions. That means that the boundary conditions one end uniquely fix the boundary conditions on the opposite end.


TITLE: Numerical upper bound of $o(1)$
QUESTION [1 upvotes]: What can be a numerical upper bound for the $x$ in the following formula ($n \ge 1$):
$$x < ((1 + o(1))n\log n$$
I mean to replace $o(1)$ with some number or constant. Is it possible?

REPLY [2 votes]: No and yes.
You cannot go from
$$
x<(1+o(1))n\log n
$$
to
$$
x < (1+C)n\log n ~~~ \mathrm{for~all}~n\ge1
$$
since the first bound also includes functions like $x(n) = n\log n + 10C\log n$ which will violate the bound for $n\le 10$.
But you can say that for any $C>0$ there is a number $N(C)$ such that
$$
x < (1+C)n\log n ~~~ \mathrm{for~all}~n>N(C)
$$
that is, as @Aryabhata said in the comments, "for sufficiently large $n$." (This is the definition of $o(1)$.)
For example if $x(n)=n\log n + 10\log n$ then $x<2n\log n$ for all $n>10$, and $x<1.001n\log n$ for all $n>10^4$.


text:
\begin{document}
\title{Monodromies of algebraic connections on the trivial bundle}
\author{Byungheup Jun}
\email{byungheup.jun@uni-essen.de}
\address{FB 6, Mathematik,
Universit\"at Essen,
45 117 Essen,
Germany}
\date{25. June, 2000}
\maketitle

\begin{abstract}
In this note, we study monodromies of algebraic connections
on the trivial vector bundle. We prove that on a smooth complex
affine curve, any monodromy arises as the underlying local system of an
algebraic connection on the trivial bundle. We give a
generalization of this for rank one monodromies in higher dimension.
\\[9pt]
{\bf\normalsize Monodromies des connexions alg\'ebriques sur le fibr\'e
trivial}\\[5pt]
{\sc R\'esum\'e.} Dans cette note, nous \'etudions la monodromie
de connexions alg\'ebriques sur le fibr\'e vectoriel trivial.
Nous prouvons que sur une courbe affine, lisse, complexe, toute
monodromie est sous-jacente \`a une connexion alg\'ebrique sur
le fibr\'e trivial. Nous donnons une g\'en\'eralisation de ceci
aux monodromies de rang un en dimension sup\'erieure.

\end{abstract}

\section*{Version fran\c caise abr\'eg\'ee}

Soit $U$ une vari\'et\'e affine lisse sur $\C$. Deligne a
montr\'e dans \cite{D} qu'il y a une correspondence biunivoque
entre les syt\`emes locaux sur $U$ et les fibr\'es vectoriels
alg\'ebriques \`a connexion plate, r\'eguli\`ere singuli\`ere
\`a l'infini. Si on supprime la condition de r\'egularit\'e \`a
l'infini, alors il y a beaucoup de connexions plates alg\'ebriques
\`a monodromie fix\'ee. Dans \cite{DEL}, il prouve que si $\dim U=1$,
toute monodromie de rang 1
est sous-jacente \`a une connexion alg\'ebrique  sur
le fibr\'e trivial. D\`es lors, les singularit\'es \`a l'infini
peuvent \^etre irr\'eguli\`eres.

\begin{fthm}[Deligne \cite{DEL}]
Sur une courbe affine lisse complexe $U$,  tout caract\`ere $\rho$
du groupe fondamental $\pi_1(U_\an)$ est sous-jacent \`a une
connexion alg\'ebrique sur le fibr\'e trivial.

\end{fthm}

{\it Id\'ee de la preuve.}--
On identifie
$H^1_{dR}(U)=H^1(U_\an,\C)$ (\cite{Gd}), ce qui associe \`a la classe
de de~Rham de $\omega$ dans $\Gamma(U,\Omega^1_U)$ son
exponentielle
$\exp[\omega]$ dans $H^1(U_\an,\C^*)$,
qui correspond \`a la monodromie sous-jacente \`a la connexion
$(\sO,d-\omega)$.
L'annulation de  $H^2(U_\an,\Z)$ permet de conclure.
\hfill{\bf c.q.f.d.}

Nous donnons une version l\'eg\`rement diff\'erente du
th\'eor\`eme 1 dans le corollaire suivant.

\begin{fcor}
Soit $U$ une vari\'et\'e affine lisse complexe.
Les conditions suivantes sont \'equivalentes.
\begin{itemize}
\item[1)] $H^2(U_\an,\Z)$ n'a pas de torsion.
\item[2)]
Tout caract\`ere $\rho$
du groupe fondamental $\pi_1(U_\an)$ est sous-jacent \`a une
connexion alg\'ebrique sur le fibr\'e trivial.
\item[3)] Soit $\sL$ un fibr\'e alg\'ebrique de rang 1 avec classe
de Chern triviale dans $H^2_{dR}(U)$. Alors
tout caract\`ere $\rho$
du groupe fondamental $\pi_1(U_\an)$ est sous-jacent \`a une
connexion alg\'ebrique sur $\sL$.
\end{itemize}
\end{fcor}

Dans le th\'eor\`eme suivant, nous donnons un exemple
d'application.
\begin{fthm}
Soit $X$ une vari\'et\'e projective lisse complexe et soit $D$
un diviseur ample supportant le groupe de
N\'eron-Severi  $NS(X)$ de $X$.
Alors tout caract\`ere $\rho$
du groupe fondamental $\pi_1(U_\an)$ est sous-jacent \`a une
connexion alg\'ebrique sur le fibr\'e trivial.
\end{fthm}

Nous renvoyons au diagramme (5) de Remark 1 dans \S{2} pour une
visualisation des r\'esultats pr\'ec\'edents.

Soit $U$ de nouveau une courbe lisse affine complexe.
Nous g\'en\'eralisons le th\'eor\`eme 1
aux repr\'esentations de rang sup\'erieur. Nous partons
de la d\'ecomposition d'une repr\'esentation $\rho$ \`a valeurs dans
 $\GL(r)$ comme produit tensoriel
$\chi\otimes\phi$, o\`u $\chi$ est un caract\`ere
et
$\phi$ est une repr\'esentation \`a valeurs dans $SL(r)$.

Nous consid\'erons la suite exacte
\begin{equation}
0 \to \mu_r \to  \G_m\times\SL(r) \stackrel{\pi}{\to} \GL(r) \to 0,
\end{equation}
o\`u
$\mu_r$ est le groupe des racines $r$-i\`emes de  l' unit\'e,
et o\`u la premi\`ere application est d\'efinie par
$\zeta \mapsto (\zeta, (1/\zeta)I)$ et la seconde est simplement
la multiplication d'une  matrice par un scalaire.

Alors on a la suite exacte longue de cohomologie sur $U_\an$
(d'ensembles point\'es)
\begin{equation}
\cdots \to H^1(U_\an,\G_m\times\SL(r)) \stackrel{\pi_*}{\to}
H^1(U_\an,\GL(r)) \stackrel{\delta}{\to} H^2(U_\an,\mu_r) \to \cdots
\end{equation}

Le groupe $H^2(U_\an,\mu_r)$ est nul car on est en dimension 1.
Donc  $\pi_*$ est
surjective. Par cons\'equent, il suffit de prouver le
th\'eor\`eme pour des repr\'esentations \`a d\'eterminant
trivial.
\begin{fprop}
Toute repr\'esentation  $\rho: \pi_1(U_\an)\to SL(r,\C)$
du groupe fondamental est sous-jacente \`a une
connexion alg\'ebrique sur le fibr\'e trivial.
\end{fprop}
{\it Id\'ee de la preuve.--}
Soit $\rho: \pi_1(U_\an)\to SL(r,\C)$  et soit $(E,\nabla)$ l'unique
fibr\'e \`a connexion r\'eguli\`ere singuli\`ere \`a l'infini.
Sur une courbe affine $U$,  un fibr\'e alg\'ebrique $E$
a une d\'ecomposition
$\sO^{r-1}\oplus\sL$ (cf. \cite{At}, \cite{Bou}).
Le d\'eterminant $\det(E)= \sL$ a la connexion d\'eterminant,
qui est aussi
r\'eguli\`ere singuli\`ere \`a l' infini. Comme
${\rm det \ }\rho = 1$, on a $({\rm det}( E), {\rm det
}(\nabla))=(\sO_U,d)$, donc en particulier
$\det(E)=\sL=\sO$.
\hfill{\bf c.q.f.d.}

Finalement, nous obtenons
\begin{fthm}
Soit $U$ une courbe affine lisse complexe. Alors,
toute repr\'esentation  $\rho: \pi_1(U_\an)\to GL(r,\C)$
du groupe fondamental est sous-jacente \`a une
connexion alg\'ebrique sur le fibr\'e trivial.
\end{fthm}

\section{Introduction}
On a nonsingular projective algebraic variety $X$ over $\C$,
using the existence theorem of Cauchy-Kovalevski and Serre's
GAGA(\cite{GAGA}),
we have the
Riemann-Hilbert correspondence: there exists a unique algebraic
vector bundle with a
connection for a given representation of the fundamental group $\pi_1(X)$
and vice versa.

On a noncomplete variety,
the uniqueness no longer holds: there exist several vector bundles with
connection for a monodromy.
In \cite{D}, Deligne showed the existence and uniqueness with regularity
condition at ininity.

Instead of regularity, in this article, we think about a restriction
of the underlying vector bundle of the connection.
In \cite{DEL},
Deligne showed one can realize any monodromy
representation of the fundamental group of an affine curve $U$
as the underlying monodromy of
a connection $\nabla=d-\omega$ on the trivial bundle of rank
1 on $U$(ie. $\sO_U$).

The main theorem(Theorem 5) generalizes Deligne's result(Theorem 1):
On an affine curve, to any monodromy representation of the fundamental
group, there is a connection on trivial bundle with the prescribed underlying
monodromy.

The aim of this note is to introduce Deligne's result on rank 1 case
in \cite{DEL} and to generalize it in two directions: one is when $U$ is of
higher dimension and the other is the similar to the higher rank
representation of the fundamental group of the affine curve.

{\bf Acknowledgements.}
Theorem 1, on which this note relies, is essentially due to Pierre Deligne.(\cite{DEL}).
I thank to H\'el\`ene Esnault for all the fruitful discussions on of my
work and for sharing her time and ideas.

\section{Rank 1 representations}

\begin{thm}[Deligne \cite{DEL}]
On a smooth affine curve $U$,
to given rank 1 monodromy representation $\rho$ of $\pi_1(U_\an)$,
there exists an algebraic connection $\nabla$ on $\sO$
with the underlying monodromy $\rho$.
\end{thm}

{\bf Proof.}
From the exponential sequence, we have the following
exact sequence of singular cohomology groups.
\begin{equation}
H^1(U_\an,\C) \stackrel{\exp}{\longrightarrow} H^1(U_\an,\C^*) \longrightarrow
H^2(U_\an,\Z) \longrightarrow H^1(U_\an, \C),
\end{equation}
where $U_\an$ denotes $U$ with the strong topology.
We know $H^1(U_\an,\C^*)$ is the group of isomorphism classes
of rank 1 local systems
on $U_\an$
and $H^2(U_\an,\Z)$ vanishes by dimension reason.

Via the identification(cf. \cite{Gd})
$$
H^*_{dR}(U):= \H^*(U,\Omega^\bullet) = H^1(U_\an,\C),
$$
the de~Rham class of $\omega$ in $H^0(U,\Omega^1)$
yields a class $[\omega]$ in $H^1(U_\an,\C)$, such that
$$
\exp[\omega]
\in H^1(U_\an,\C^*)=\Hom(\pi_1(U_\an),\C^*)
$$
is the underlying monodromy of $(\sO, d-\omega)$.

On an affine variety, the higher coherent sheaf cohomology vanishes.
Thus $H^0(U,\Omega^1)$ generates $H^1_{dR}(U)$.
Therefore $H^0(U,\Omega^1)$ surjects onto $\Hom(\pi_1(U_\an),\C^*)$.
This finishes the proof.
\hfill {\bf  q.e.d.}\\[5pt]

Without much difficulty, we generalize the theorem when $\dim U > 1$.

\begin{coro}
Let $U$ be an affine variety over $\C$.
Then the followings are equivalent:
\begin{itemize}
\item[1)] $H^2(U_\an,\Z)$ is torsion-free.
\item[2)] To any rank 1 representation $\rho$ of $\pi_1(U_\an)$, there exists an algebraic
 connection $\nabla$ on $\sO$ with underlying monodromy $\rho$.
\item[3)] If a line bundle $\sL$ has vanishing Chern class in $H^2_{dR}(U)$.
To any rank 1 representation $\rho$ of $\pi_1(U_\an)$ and , there exists an algebraic connection
$\nabla$ on $\sL$ with underlying monodromy $\rho$.
\end{itemize}
\end{coro}
{\bf Proof. }
1) $\Leftrightarrow$ 2)
As in the proof of the theorem 1, one has
the exact sequence arising from the exponential sequence(3).
On an affine variety $U$, $\Gamma(U,\Omega^1_{\rm closed})$ generates
$H^1_{dR}(U) = H^1(U_\an,\C)$.
Then $\exp$ in the exact sequence is surjective if and only if
$\Gamma(U,\Omega^1_{\rm closed})\to H^1(U_\an,\C^*)$ is surjective or equivalently,
$H^2(U_\an,\Z)$ is torsion-free.

2) $\Rightarrow$ 3)
Consider the complex of sheaves on the Zariski topology of $U$:
$$
\sC^\bullet :
	\sO^* \stackrel{d\log}{\longrightarrow} \Omega^1 \to \Omega^2 \to \cdots.
$$
The first hypercohomology of $\sC^\bullet$ is the group of isomorphism classes of line
bundles with integrable connection, which will be denoted by
$\Pic^\nabla(U)$.
The filtration of $\sC$ by its degree gives
\begin{equation}
\begin{CD}
\H^1(U,\sC^\bullet) @>>> H^1(U,\sO^*)	@>>>
		\H^2(U,\Omega^{\bullet \ge 1})	\\
 		@|				@|		@|    \\
\Pic^\nabla(U) @>>>  \Pic(U)		@>c>> H^2_{dR}(U),
\end{CD}
\end{equation}
where $c$ denotes the first Chern class map and the last vertical map is
an isomorphism since
the kernel and the cokernel are $H^1(U,\sO)=H^2(U,\sO)=0$.

Since $c(\sL)=0$, from the exact sequence, there exists an
integrable connection $\nabla_0$ on $\sL$.
Let $\rho_0$ be the monodromy of $(\sL,\nabla_0)$.
As one can realize any monodromy on $\sO$, one has an algebraic connection
$\nabla_1$ with the monodromy $\rho_0^{-1}\cdot\rho$ on $\sO$.
Twisting $(\sL, \nabla_0)$ with $(\sO,\nabla_1)$, one has the connection
$\nabla=\nabla_0\otimes\nabla_1$ on $\sL$ with the desired monodromy
$\rho$.

3) $\Rightarrow$ 2) Clear.
\hfill{\bf q.e.d.}

As a special case of the corollary, we have the following:
\begin{thm}
Let $X$ be a smooth projective variety over $\C$ and $D$ be an ample divisor supporting
the N\'eron-Severi group $NS(X)$ of $X$.
Then to any monodromy $\rho$ in $\Hom(\pi(U_\an),\C^*)$, there exists an integrable algebraic
connection $\nabla=d-\omega$ on $\sO_U$ with underlying monodromy $\rho$.
\end{thm}

{\bf Proof.}
By the equivalence of the corollary 2, we have to show that $H^2(U_\an,\Z)$ is
torsion-free.
But, in the exact sequence (3), torsion comes from $H^1(U_\an,\C^*)$.
Using the existence theorem in \cite{D},
one has a line bundle on $X$ with connection $(\sL,\nabla)$
which is regular singular at $D$ with the underlying monodromy $\rho$.
The image of $\sL$ maps into $NS(X)$, and thus a torsion element in $H^2(U_\an,\Z)$
vanishes.
Therefore $H^2(U_\an,\Z)$ is torsion-free.
\hfill{\bf q.e.d.}\\[5pt]
{\bf Remark 1.}
Using a commutative diagram, one can summarize the result in a simple form
where all rows and columns are exact:

\begin{equation}
\begin{array}{ccccccccc}
  &					& 0 & & 0 & & 0 \\
  &					& \downarrow   &	 & \downarrow & & \downarrow \\
0 & \longrightarrow & \displaystyle{\frac{H^0(U,\Omega^1)_\Z}{d \log H^0(U,\sO^*)}} & \longrightarrow
  					& \displaystyle{\frac{H^0(U,\Omega_{\rm closed}^1)}{d \log H^0(U,\sO^*)}} & \longrightarrow & H^1(U_\an,\C^*) & \longrightarrow & 0\\
  &	   	 	   	 	& \downarrow     & 				  &	\downarrow &  & \downarrow \\
0 & \longrightarrow & \Pic^{\nabla_0}(U) & \longrightarrow & \Pic^\nabla(U) &
  					\longrightarrow & \Hom(\pi_1(U_\an),\C^*) & \longrightarrow & 0\\
  & 	& \downarrow & 	   	   &  \downarrow &	  &  \downarrow \\
0 & \longrightarrow	& \Pic(U)	&	\longrightarrow	 &	    \Pic(U)	& \longrightarrow & 0	 \\
  &		& \downarrow &	 		& \downarrow \\
  &  & H^2_{dR}(U) 	&	=		&	H^2_{dR}(U) &
\end{array}
\end{equation}
In the diagram,
$\Pic^{\nabla_0}(U)$ is the subgroup of $\Pic^\nabla(U)$ with respect to
the trivial monodromy and $H^0(U,\Omega^1)_\Z$ is the global 1-forms
with integral periods.
When $\dim U=1$, one has $H^2_{dR}(U)=0$.
Thus, we have $\Pic^\nabla(U)$ and $\Pic^{\nabla_0}(U)$ as extensions
of the $\Pic(U)$.

\section{Higher rank case on curves}
In this section, $U$ is again a smooth affine curve over $\C$.

One has the short exact sequence:
\begin{equation}
\begin{split}
0 \to \mu_r  \to \G_m\times\SL(r) & \stackrel{\pi}{\to} \GL(r) \to 0,\\
        \zeta \mapsto   (\zeta, (1/\zeta) I) &\\
                         (\ell,M)  &\mapsto \ell M
\end{split}
\end{equation}
where $\mu_r$ denotes the multiplicative group of $r$-th roots of
unity.
Because $\mu_r$ maps into the center of $\G_m \times \SL(r)$,
we have long exact sequence of cohomologies on $U_\an$ (of pointed sets).
\begin{equation}
\cdots \to H^1(U_\an,\G_m\times\SL(r)) \stackrel{\pi_*}{\to} H^1(U_\an,\GL(r))
\stackrel{\delta}{\to}
H^2(U_\an,\mu_r) \to \cdots
\end{equation}
By dimension reason, $H^2(U_\an,\mu_r)$ vanishes and one sees $\pi_*$
surjective.
Thus it suffices to solve the problem for $SL$-representations.

If $A$ is a Dedekind domain and $M$ is a projective $A$-module of finite
rank $r$,
one has $M\cong A^{r-1}\oplus L$ where $r= rank(M)$ and $L \in \Pic(A)$.
As a special case, a vector bundle $E$ of $rank(E)=r$ on an affine curve $U$
is isomorphic to $\sO^{r-1}\oplus \sL$, where $\sL = \det E$ is a line bundle
on $U$ (cf. \cite{Bou}, \cite{At})

\begin{prop}
Let $U$ be a smooth affine curve.
To any $\SL(r)$-representation $\rho$ of $\pi_1(U_\an)$,
there exists an algebraic connection $\nabla$
on the trivial bundle of rank $r$ with underlying monodromy $\rho$.
\end{prop}
{\bf Proof.}
Let $\rho$ be a $\SL(r)$-representation and $X=U\cup D$ be the smooth completion of $U$.
Then there exists a vector bundle $E$ on $X=U\cup D$ the smooth completion
of $U$, with a connection $\nabla$ which is regular singular,
$$
\nabla : E \to E\otimes \Omega^1(\log D),
$$
such that the restriction on $U$ gives the monodromy $\rho$ (cf. \cite{D}).

By the decomposition theorem, one has $E|_U = \sO_U^{r-1}\oplus \sL$,
where $\sL = \det(E|_U)$.
On $X$,
$$
\det \nabla : \det(E) \to \det(E)\otimes \Omega^1(\log D)
$$
is regular at $D$ and has trivial monodromy since the representation
is $SL$-valued.
Using the uniqueness theorem in \cite{D},
we have
$$
(\det (E)|_U,\det \nabla) = (\sO_U,d).
$$
In particular $\sL=\sO_U$ and $E_U = \sO^r_U$.
\hfill {\bf q.e.d.}

After the proposition, we obtain the main theorem.

\begin{thm}
Let $U$ be a smooth affine curve over $\C$.
Then to any rank $r$ representation of
the fundamental group $\rho\in \Hom(\pi_1(U_\an),\GL(r))$,
there exists a connection on the trivial bundle of rank $r$ on an affine
open set $U$ with underlying monodromy $\rho$.
\end{thm}
{\bf Proof.} Clear.
\hfill{\bf q.e.d.}\\[5pt]
\bibliographystyle{plain}
\renewcommand\refname{References}

config: arxiv


TITLE: Sequences proof
QUESTION [1 upvotes]: I'm doing an assignment for my analysis class and need help with the following question:
Let $f$ be a continuous function which is defined on the interval $I = [a, b]$. Assume
that $f$ is not bounded on $I$. Use the fact that $f$ is not bounded to prove that there is a
sequence $(x_n)$ in $I$ for which $|f(x_n)| > n, \forall n \in \mathbb N$
I would really appreciate your help .
Thanks

REPLY [2 votes]: If $ f $ is not bounded on $ I $ then, by definition, given any $ C > 0 $ we can find $ x \in I $ such that $ \vert f(x) \vert > C $. In particular, for each $ n \in \mathbb{N} $ we can pick $ x_n \in I $ with $ \vert f(x_n) \vert > n $.


text:
The training of neural networks amounts to the numerical optimisation of typically smooth, but high-dimensional and highly non-linear objectives as in \eqref{eq1:training}, and as discussed in the optimal control framework in Section \ref{sec:optimalcontrol}. The most widely used numerical method for \eqref{eq1:training} is Adam \cite{kingma2014adam}.

As before let $\Psi: \FeatureSpace\times \GenParamSet\to\FeatureSpace$ be a (deep) neural network that depends on the data and the parameters
$\theta\in\GenParamSet$. The training of $\Psi$ amounts to the optimisation over the parameters $\theta$ with respect to a loss function
$$
    \min_{\theta \in \Theta} \left\{ E(\theta) = \frac1N \sum_{n=1}^N \Loss (\eta\circ \Psi(x_n, \theta), y_n) + R(\theta)\right\},
$$
as in \eqref{eq1:training}. While the loss function $\Loss$ usually is a convex function, the dependency of the parameter $\theta$ on the network $\Psi$ is in general highly nonlinear which makes the overall optimisation problem in \eqref{eq1:training} non-convex. For what follows, let us denote by $\Loss^j(\theta)=\frac{1}{|\mathcal S^j|}\sum_{n\in \mathcal S^j} (\Loss (\eta\circ \Psi(x_n, \theta), y_n) + \frac1N R(\theta))$, for $j=0,1,\ldots$ and where $\mathcal S^j$ is a randomly chosen set of indices from the $N$ training samples. For fixed positive parameters $\alpha$ and $\beta_1,\beta_2<1$, and for appropriate initialisations for the parameters and the auxiliary moment functions $m_0$ and $v_0$, Adam amounts to the following iteration
\begin{equation}\label{eq:stochADAM}
\begin{aligned}
    m^j&=\frac{1}{(1-\beta_1^j)}\left(\beta_1 m^{j-1} + (1-\beta_1)\nabla \Loss^j(\theta^{j-1})\right)\\
    v^j&=\frac{1}{(1-\beta_2^j)}\left(\beta_2 v^{j-1} + (1-\beta_2) \nabla \Loss^j(\theta^{j-1}) \cdot \nabla \Loss^j(\theta^{j-1})\right)\\
    \theta^j&=\theta^{j-1} - \alpha \frac{m^j}{\sqrt{v^j}+\varepsilon},
\end{aligned}
\end{equation}
for a small $\varepsilon>0$ and where $\sqrt{v^j}$ is taken component-wise. Stochasticity, in the form of choosing subsets $\mathcal S^j$ randomly in every iteration, is crucial to deal with high-dimensionality of the problem.

Adam is being used for almost all of neural network training because of its easy implementation, its robustness to rescaling, its computational efficiency and small memory requirements and for its suitability for problems which are large-scale in terms of training data and parameters. On the other hand, its theoretical convergence properties do in general not guarantee convergence to a solution of \eqref{eq1:training}, cf. \cite{reddi2019convergence} where the authors propose a convergent variant in the convex setting and \cite{zaheer2018adaptive} which provides convergence guarantees in the non-convex and stochastic setting.

The literature for optimising smooth (non-convex) objectives is, however, much richer than Adam alone. It is tightly linked to developments in convex analysis and operations research, as well as the numerical discretisation of dynamical systems, ODEs and PDEs as discussed in parts in Sections \ref{sec:ODEPDE} and \ref{sec:optimalcontrol}. Also in the context of neural network training other optimisation schemes have been investigated recently. Here, we will mainly focus on those which have some structure-preserving property such as Hamiltonian descent \cite{maddison2018hamiltonian,o2019hamiltonian} which are guaranteed to dissipate a Hamiltonian energy, optimisation approaches when the features or network parameters are elements in a Riemannian rather than Euclidean space \cite{absil08oao, udriste94cfa,li2019efficient}, and -- as a special case of the latter -- information geometry approaches that describe optimisation on statistical manifolds \cite{amari1998natural}.

\subsection{Conformal Hamiltonian systems}
The most classic approach for minimising $E(\cdot)$ over $\mathbb R^L$ is gradient descent, i.e. to seek a stationary point of $E$ by evolving
\begin{equation}\label{eq:graddescent}
\dot{\theta}=-\nabla E(\theta).
\end{equation}
Several optimisation methods for $E$ can then be derived through different discretisations of \eqref{eq3:gradient}, with the simplest one being explicit gradient descent and its stochastic versions \cite{robbins1951stochastic}. Another route for the derivation of optimisers for $E$ is obtained by replacing the gradient flow \eqref{eq:graddescent} with a Hamiltonian flow as in \eqref{eq1:extended_ode} with dissipation, for example a conformal Hamitonian system, i.e.
\begin{equation}\label{eq:confHamiltonsys1}
\begin{aligned}
    \dot{p}&=-\nabla E(\theta)-\gamma p,\\
    \dot{\theta}&=\frac{p}{\mu},
\end{aligned}
\end{equation}
with $\gamma,\mu>0$. Rewriting the system \eqref{eq:confHamiltonsys1} in one equation gives
$$
\ddot{\theta}+\gamma \dot{\theta} = -\frac1\mu \nabla E(\theta),
$$
which is gradient descent accelerated by momentum. More generally, \eqref{eq:confHamiltonsys1} is a special case of a conformal Hamiltonian system of the form
\begin{equation}\label{eq:confHamiltoniansys2}
\begin{aligned}
    \dot{p}&=-\nabla_\theta H(p,\theta)-\gamma p\\
    \dot{\theta}&=\nabla_p H(p,\theta),
\end{aligned}
\end{equation}
for a \emph{separable} Hamiltonian function $H(\theta,p)=T(p)+E(\theta)$ as in Section \ref{subsubsec:hamilton}.   Taking $T(p)=\frac{1}{2\mu}\|p\|_2^2$ we get \eqref{eq:confHamiltonsys1}. Taking $T(p)=\sqrt{\|p\|^2+\varepsilon}$
we get a new optimisation approach for $E$ which is called relativistic gradient descent \cite{francca2019conformal}
\begin{equation}
\label{eq:relgraddesc}
\begin{aligned}
    \dot p &= -\nabla E(\theta)-\gamma p\\
    \dot\theta &= \frac{p}{\sqrt{ \varepsilon+\|p\|^2}}
    \end{aligned}
\end{equation}
While the gradient system \eqref{eq:graddescent} dissipates $E$, for a conformal Hamiltonian system \eqref{eq:confHamiltoniansys2} the Hamiltonian $H$ is dissipated as
$$
\frac{\ddiff}{\ddiff t} H(p,\theta) = -\frac{\gamma}{2\mu} p^T~p.
$$
For separable Hamiltonians with the kinetic energy $T$ chosen so it has a global minimiser in $0$, limit points of \eqref{eq:confHamiltoniansys2} recover stationary points of $E$. More precisely, for $H=T(p)+E(\theta)$ being separable the equilibria
of \eqref{eq:confHamiltoniansys2} fulfill
\begin{align*}
    \gamma~p&= - \nabla E(\theta)\\
    0&=\nabla T(p).
\end{align*}
If $T$ is chosen so it has a unique global minimum in $p^*=0$ (e.g. in the case of \eqref{eq:confHamiltonsys1}), we have that $(0,\theta^*)$ is a zero of $\nabla H(p,\theta)$ if and only if $u^*$ is a zero of $\nabla E(\theta)$, i.e. $\theta^*$ is a stationary point of $E(\theta)$. Moreover, we have that $(0,\theta^*)$ is the solution of the conformal Hamiltonian system as $t\rightarrow\infty$. Then, using a numerical integration method that preserves the property of Hamiltonian energy dissipation (such a numerical method is called conformal symplectic scheme \cite{bhatt2016second}), we get an approximation of  $(\theta^*,0)$, where $(\theta^*,0)$ is an equilibrium of \eqref{eq:confHamiltoniansys2} and $\theta^*$ a stationary point of $E$ \cite{mclachlan2001conformal}.

In the recent work \cite{francca2019conformal} the authors take advantage of the connection between optimisation schemes for $E$ and conformal symplectic Hamiltonian schemes for $H=T+E$ for the design of new optimisation approaches for $E$ - similar to the connections we have seen before, e.g. between Adam and conformal Hamiltonian descent. See also Figure \ref{fig:HamiltonianDescent} for a comparison between different optimisation methods applied to the camelback function.

\begin{figure}
    \centering
    \includegraphics[height=4.5cm]{pics/ConvergenceHD1e7.png}\hspace*{5mm}
    \includegraphics[height=4.5cm
    ]{pics/contourplot1e7.png}
    \caption{Optimisation of $V(x,y)=2x^2-1.05x^4 +1/6x^6 +xy+y^2$. Comparison of gradient descent, Hamiltonian descent and Adam iterations, initial guess $[-0.5,0.8]$, global minimum at $[0,0]$. Methods and parameters: Gradient descent (GD) learning rate $h=0.01$, Heavy ball (HB) $\mu=0.9$, $h=0.01$, Nesterov Accelerated Gradient (NaG) $\mu=0.012$, $h=0.01$, Relativistic Gradient Descent (RGD) \cite{francca2019conformal} $h=0.0001$, $\mu=0.9259$, Adam $\beta_1=0.9$, $\beta_2=0.999$, $\varepsilon=1e-8$, $\alpha=h=0.1$. Left: value of the loss function versus number of iterations. Right: trajectory of approach to the optimum (NaG $h=0.01$ yellow, RGD $h=0.0001$ magenta, Adam $h=0.1$ green).
    }
    \label{fig:HamiltonianDescent}
\end{figure}

\subsection{Learning in Riemannian metric spaces}\label{subsection:LearningRiemannian}

After identifying the dissipation of an appropriate Hamiltonian as a structure worth preserving for numerical optimisation of neural networks, as discussed in the last section, explicit conditions on the parameter matrices themselves, such as orthogonality, seem to impose structure on the optimisation that can be of advantage and in particular lead to better (at least empirical) convergence rates, better generalisability and accuracy \cite{li2019efficient}. Such conditions usually pose a parameter optimisation problem within a Riemannian metric space rather than Euclidean space. Moreover, there are several important applications, cf. in particular the next subsection, where the training data and the parameters naturally lie in a Riemannian space.

While in section \ref{sec:Riemanniandata} we discussed the case when the training data and features lie on a Riemannian manifold, in this section we consider the setting when the parameters belong to a manifold. This in particular means that ordinary gradient descent a la \eqref{eq:graddescent}, i.e. with the gradient $\nabla$ being the ordinary gradient in $\mathbb R^L$, does in general not describe steepest descent of $E$ in such a Riemannian metric space. Instead descent with respect to a metric-induced gradient needs to be considered \cite{ambrosio2008gradient}. In what follows, we consider some representative examples of how Riemannian optimisation could arise in the context of deep learning.

\subsubsection{Network parameters evolving on manifolds}

Assuming that the parameters to be learned evolve on a manifold arises when additional structure or constraints are imposed on the parameters. This is done to improve stability of the training algorithm and of the trained model. Pioneering work advocating the use of Riemannian gradient and orthogonality constraints can be found in \cite{amari98ngw,amari96anl,cardoso92eas} and there is an extensive follow up literature. Examples of this procedure in the context of deep learning have recently appeared in \cite{li20ero} and earlier in the context of CNNs in \cite{cho17rat,becigneul19rao}. It is crucial to implement efficiently the Riemannian gradient descent making good use of the tensor structure of the data and of the layers of the neural network, avoiding undesired increase in computational complexity.
One way to proceed is to introduce an evolution equation for $\theta$ and replace \eqref{eq1:ode} by the extended system of ODEs
\begin{align}
\dot z &=f(z,\theta ), \label{eq:param0}\\
\dot \theta &= g( \theta).\label{eq:param1}
\end{align}
Of particular importance is the case where the second equation evolves on a Lie group $G$ or on a homogeneous manifold $\mathcal{M}=G/H$.\footnote{Here $H$ is a closed Lie subgroup of $G$, and $\mathcal{M}:=G/H$ the quotient with the manifold structure turning  $\pi: G \rightarrow G/H$  $\pi: g \mapsto gH$ into a submersion. Then $\mathcal{M}$ becomes a homogeneous space for $G$ with respect to the transitive Lie group action induced by left multiplication. }
In \cite{amari96anl} $G$ is simply the general linear group, while orthogonality constraints have been adopted in \cite{hyvarinen00ica} in the context of independent component analysis. Other concrete examples that we have in mind include the affine group, the special Euclidean group $\mathrm{SE}(n)$, the special orthogonal group $\mathrm{SO}(n)$, the Stiefel manifold $\mathrm{V}_{n,p}=\mathrm{SO}(n)/\mathrm{SO}(n-p)$ and the Grassmann manifold $\mathrm{G}_{n,p}=\mathrm{SO}(n)/(\mathrm{SO}(n-p)\times \mathrm{SO}(p))$ including the $n$-sphere.

For matrix Lie groups $G$ (and homogeneous manifolds) \eqref{eq:param1} can simply take the form of a linear matrix differential equation
$$\dot \theta = M( t)\, \theta, \qquad \theta \in \mathcal{M}$$
and $M( t)\in \mathfrak{g}$, and the optimisation can be performed in terms of the variable $M$. To preserve the manifold structure one can consider a discretisations of this equation of the form
$$\theta_{k+1}=\exp (h\,M_k)\theta_k,$$
where $\exp:\mathfrak{g}\rightarrow G$ is the matrix exponential. This discretisation can be seen as the prototype for constructing local coordinates\footnote{Otherwise called retraction maps \cite{absil08oao}.} on the manifold of parameters. Alternatively any approximation of the exponential map $\phi\approx \exp$ (e.g. by a rational approximant) preserving the property $\phi:\mathfrak{g}\rightarrow G$, can be used to construct local parametrisations of the manifold\footnote{An example for the case of $\mathrm{SO}(n)$ and $\mathrm{Sp}(n)$ is the Cayley transformation, used in the context of invertible networks in section
~\ref{linearInvertible}}. On homogeneous manifolds such as $\mathrm{V}_{n,p}$ and $\mathrm{G}_{n,p}$ additional structure on $M(t)$ can be exploited to reduce the computational cost of matrix exponentials and similar mappings \cite{celledoni04nlb}.

After time discretisation, this approach guarantees that the parameters at each layer of the network belong to manifolds which are all naturally equipped with Riemannian metrics and some of which are compact. In particular, a metric on $G$ which is $H$-right invariant (i.e. the right multiplication $R_h$ with $h\in H$ is an isometry)  descends to a Riemannian metric on $\mathcal{M}=G/H$, \cite{gallot04rg}.
Gradient descent techniques to train the network should then exploit the Riemannian structure \cite{amari98ngw, li20ero}.
A sufficient condition for convergence of Riemannian gradient descent is that the manifolds are geodesically complete, \cite{weinmann14tvr,udriste94cfa}. All Riemannian homogeneous manifolds as well as compact Riemannian manifolds are geodesically complete
\cite[IV.4]{kobayashi96fod1}.

\subsubsection{Information Geometry}\label{sec:informationgeo}

A special case of the Riemannian structure discussed in the previous paragraph arises when taking into account the inherent statistical properties of the underlying unknown distributions of training pairs and, connected to this, statistical properties of the network parameters $\theta$.

Treating the parameters $\theta$ as probability distributions, they can be modelled as elements on a statistical manifold with an appropriate metric. A statistical manifold is a Riemannian manifold whose points correspond to probability distributions. Gradient descent on statistical manifolds is studied in information geometry. Here, the so-called natural gradient is the proposed notion for gradient on a statistical manifold \cite{amari98ngw}. For an $L$-dimensional parameter space, equipped with a Riemannian metric tensor $G=G(\theta)=(g_{ij}(\theta))\in \mathbb R^{L\times L}$ that depends on $\theta$, the natural gradient of $E(\theta)$ is defined as
$$
\tilde\nabla E(\theta) = G^{-1}(\theta) \nabla E(\theta),
$$
where $\nabla E(\theta)$ denotes the ordinary gradient of $E$ in $\mathbb R^L$.
The natural metric considered in this context is the Fisher information, with $G$ being the Fisher information matrix of the parameters $\theta$. Natural gradient descent then reads
\begin{equation}\label{eq:ngd}
    \dot{\theta}=-G^{-1}(\theta) \nabla E(\theta).
\end{equation}
In the case of $E(\theta)=\frac{1}{2N}\sum_{n=1}^N \|\Psi(x_n,\theta)-y_n\|^2$ a squared error loss and $G$ being the Fisher information matrix, we have
$$
G(\theta)=\frac1N\sum_{n=1}^N J_n^TJ_n,
$$
where $J_n$ is the Jacobian of $\Psi(x_n,\theta)$ with respect to $\theta$, cf. \cite{martens2014new}. Note that in this case natural gradient descent, discretised with forward-Euler, is equivalent to Gauss-Newton iteration \cite{nocedal2006numerical}. This connection between natural gradient descent and (extended) Gauss-Newton methods can be extended to more general losses as well \cite{pascanu2013revisiting}. In the continuum limit, \eqref{eq:ngd} is a gradient flow with respect to the Fisher-Rao metric, see \cite[Definition 3.1.]{modin2016geometry}.

Several works \cite{amari98ngw,amari1998natural,pascanu2013revisiting} have demonstrated advantages of using the natural gradient over the ordinary gradient in \eqref{eq:graddescent} for neural network training. The Riemannian structure seems to help against the gradient descent being trapped in flat areas of the loss function's surface \cite{yang1997natural}, and as a result the network to feature better generalisation capabilities, cf. \cite{hochreiter1997flat,keskar2016large} for different characterisations of flatness of the loss function's surface.

\subsection{Optimisation of 2-layer ReLU neural networks as Wasserstein gradient flows}\label{sec:2layerwassgrad}

The inherent gradient flow structure of training neural networks also appears when studying global optimality and generalisation properties of trained networks. Bach and Chizat \cite{chizat2018global} pick up the gradient flow formulation of neural network learning and study convergence of the learning problem \eqref{eq1:training} to a global minimiser of $E$ for 2-layer ReLU neural networks $\Phi$ in the $\infty$-width limit. In particular, their analysis makes use of the structure of a Wasserstein gradient flow formulation of \eqref{eq:graddescent} over the space of probability measures in the $\infty$-width network limit. Proving convergence to a global minimiser of $E$ also allows the study of optimal generalisation capabilities of the trained 2-layer ReLU neural network by characterising the limit (for a certain class of loss functions with exponential tails) as a max-margin classifier \cite{chizat2020implicit}.

In \cite{chizat2018global} they consider a 2-layer neural network of the form
\begin{equation}\label{eq:2layer}
\Phi(\theta,x)=\frac1J\sum_{j=1}^J f(\theta_j,x),
\end{equation}
where
$$
f(\theta_j,x)=c_j~\max\{a_j^tx+b_j,0\},
$$
for $x\in\mathbb R^M$ and $\theta_j=(a_j,b_j,c_j)\in \mathbb R^{M+2}$. The weights $\theta$ in \eqref{eq:2layer} are learned by minimising a loss function such as \eqref{eq1:training} with optional regularisation $R(\theta)= \frac{\lambda}{L}\sum_{j=1}^L \|\theta_j\|_2^2$. In this setting, they investigate the performance (generalisation capabilities) of the learned network $\Phi$ by studying the associated gradient flow of the loss function $E$ for initial weights $\theta(0)=\theta_0 ~\sim \mathrm{i.i.d.\,}\mu_0\in P_2(\mathbb R^{n+1})$
\begin{equation}\label{eq:gradflow2layer}
\dot\theta = -m \nabla E(\theta,x).
\end{equation}
In particular, they analyse convergence of \eqref{eq:gradflow2layer} in the $\infty$-width limit, which they prove can be written as a Wasserstein gradient flow. More precisely, they parametrise the network $\Phi$ with probability measures $\mu\in P_2(\mathbb R^{d+2})$ as
$$
\Phi(\mu,x) = \int \Phi(\theta,x) \dint\mu(\theta),
$$
and the associated loss function as
\begin{equation}\label{eq:lossprobparam}
E(\mu) = \frac1N \sum_{n=1}^N \Loss (\Phi(\mu,x_n),y_n) + \lambda \int \|\theta\|_2^2 \dint\mu(\theta).
\end{equation}
In this setting they prove the following results.
\begin{theorem}[\cite{chizat2018global}] Assume that
$$
\mathrm{spt}(\mu_0)\subset \{|c|^2 = \|a\|_2^2 + |b|^2\}.
$$
As $L\rightarrow\infty$, $\mu_{t,L} = \frac1L \sum_{j=1}^L \delta_{\theta_j}(t)$ converges in $P_2(\mathbb R^{d+1})$ to $\mu_t$, the unique Wasserstein gradient flow of $E$ in \eqref{eq:lossprobparam} starting in $\mu_0$.

Moreover, assuming $\mu_0$ is `diverse' enough (cf. \cite{chizat2018global} for details). If $\mu_t$ converges to $\mu_\infty$ in $P_2(\mathbb R^{d+1})$, then $\mu_\infty$ is a global minimiser of $E$.
\end{theorem}

\subsection{Open problems}

\subsubsection{Port-Hamiltonian optimisation methods}
Generally, investigating new optimisation methods for $E$ by considering different instances of Hamiltonian descent is a promising research direction. Here, different choices of Hamiltonians or special cases of Hamiltonian systems might be advantageous for classes of loss functions and network architectures, imposing different descent dynamics. A more concrete example are numerical schemes that arise when symplectic numerical integration is applied to port-Hamiltonian systems (with different Hamiltonians), cf. \cite{van2014port} for an introduction to port-Hamiltonian systems and \cite{celledoni2017energy} for the development of structure preserving numerical integrators for port-Hamiltonian systems by using discrete gradient and splitting approaches. In \cite{massaroli2019port} the authors design a loss function and parameter optimisation dynamics of the network in such a way that the neural network itself behaves like an autonomous port-Hamiltonian system. This in turn allows a proof of convergence of the optimisation algorithm to a minimum of the loss. Taking this a step further, port-Hamiltonian systems also lend themselves to the design of locally adaptive optimisation schemes as the port-Hamiltonian structure is preserved under concatenation of port-Hamiltonians with orthogonal input-output relation.

\subsubsection{Convergence analysis for natural gradient optimisation}

While several papers seem to suggest (mostly empirically or for linear networks) that natural gradient descent helps to mitigate nuisance curvature in the neural network parameter optimisation, very little seems to have been done on this theoretically, in particular for nonlinear neural networks \cite{zhang2019fast}. In general, while for particular choices of the Riemannian metric \eqref{eq:ngd} boils down to classical optimisation schemes, such as Gauss-Newton for $G$ being the Fisher information metric, analytic results on convergence properties of natural gradient flow discretisations are generally open. Indeed, the study of \eqref{eq:ngd} for different choices of $G$ could be very interesting. For instance, if $G$ is a BFGS approximation to the Hessian of $E$, then a stochastic method was proposed and analysed in \cite{Wang2017bfgs}.

\subsubsection{Studying generalization properties of neural networks by metric gradient flows} As the example of the Wasserstein gradient flow in section \ref{sec:2layerwassgrad} has shown, metric gradient flows can serve as a useful tool for studying convergence of the network training and for the study of generalisation properties of the minimisers \cite{chizat2018global,chizat2020implicit}. It would be interesting to see if other metric gradient flows would also lend themselves to such an analysis. In connection with information geometry in section \ref{sec:informationgeo} we have seen the gradient flow with respect to the Fisher-Rao metric appearing. Could this be used to study convergence properties for other network architectures, beyond 2-layer ReLU? Are there other metrics that could be interesting to investigate for that purpose?


TITLE: How do I determine the general solution of $L=\frac{\partial ^2}{\partial t^2}$?
QUESTION [1 upvotes]: How do I determine the general solution of $L=\frac{\partial ^2}{\partial t^2}$? $L\begin{pmatrix}x\\ y\\ z\end{pmatrix}=A\begin{pmatrix}x\\ y\\ z\end{pmatrix}$. $A=\begin{pmatrix} 4 &0 &2 \\ 0&4 &6 \\ 2& 6 &1 \end{pmatrix}$. I calculated eigenvalues and those are -4,4,9, but what is next that I should do?

REPLY [1 votes]: Let $X(t) = \pmatrix{x(t) \\y(t)\\z(t)} = \pmatrix{x_1(t) \\ x_2(t) \\ x_3(t)} $. The equation is equivalent to $X''(t) = AX(t)$.
As you calculated,  the eigenvalues of $A$ are $-4, 4$ and $9$. Three associated eigenvectors are
$$ \begin{pmatrix}-\frac{1}{4}\\ -\frac{3}{4}\\ 1\end{pmatrix},\:\begin{pmatrix}-3\\ 1\\ 0\end{pmatrix},\:\begin{pmatrix}\frac{2}{5}\\ \frac{6}{5}\\ 1\end{pmatrix}. $$
Thus, one has
$$A = \underbrace{\begin{pmatrix}-\dfrac{1}{4}&-3&\dfrac{2}{5}\\ -\dfrac{3}{4}&1&\dfrac{6}{5}\\ 1&0&1\end{pmatrix}}_{P} \underbrace{\begin{pmatrix}-4&0&0\\ 0&4&0\\ 0&0&9\end{pmatrix}}_D P^{-1}.$$
Let $Y(t) = \pmatrix{y_1(t)\\y_2(t)\\y_3(t)} = P^{-1}X(t) \Leftrightarrow X = PY$, hence $Y'' = P^{-1}X''$ and $X'' = PY''$.
$$ \begin{align} X'' & = AX \\  PY'' & = APY \\  Y'' & = P^{-1}APY \\ Y'' &= DY \tag{*}\end{align}$$
The new system (*) is equivalent to
$$ \cases{y_1''(t) = -4y(t) \\ y_2''(t) = 4y_2(t) \\ y_3''(t) = 9y_3(t)} $$
Using for instance each equation's characteristic polynomial, one can easily find its solution:
$$ \cases{y_1(t) = c_1 \cos(2 t) + c_2 \sin(2 t) \\ y_2(t) = c_3 e^{2t}+c_4 e^{-2t} \\ y_3(t) = c_5 e^{3t}+c_6 e^{-3t} } \qquad \text{with} \: c_1, c_2, c_3, c_4, c_5, c_6 \in \mathbb{R}.$$
Using $X = PY$, one can eventually get to the solution of the initial system:
$$\pmatrix{x_1(t) \\ x_2(t) \\ x_3(t)} = \begin{pmatrix}-\dfrac{1}{4}&-3&\dfrac{2}{5}\\ -\dfrac{3}{4}&1&\dfrac{6}{5}\\ 1&0&1\end{pmatrix} \pmatrix{y_1(t)\\y_2(t)\\y_3(t)},$$
which yields, using your initial notation,
$$\bbox[5px,border:2px solid green,lightgreen]{\cases{x(t) = -\frac{1}{4}\left(c_1 \cos(2t) + c_2 \sin(2t)\right) - 3 \left(c_3 e^{2t} + c_4 e^{-2t}\right) + \frac{2}{5} \left(c_5e^{3t} + c_6 e^{-3t}\right) \\ y(t) = -\frac{3}{4}\left(c_1 \cos(2t) + c_2 \sin(2t)\right) + c_3 e^{2t} + c_4 e^{-2t} + \frac{6}{5} \left(c_5e^{3t} + c_6 e^{-3t}\right) \\ z(t) = c_1 \cos(2t) + c_2 \sin(2t) + c_5e^{3t} + c_6 e^{-3t}}}.$$
Of course, to determine the constants $c_1, c_2, c_3, c_4, c_5$ and $c_6$, you'll need initial conditions.


TITLE: Do linear momentum eigenstates exist?
QUESTION [1 upvotes]: Suppose that a particle  is in  a  linear momentum eigenstate $\mid p \rangle$ that is $\hat p \mid p \rangle=p \mid p \rangle$. According to the Heisenberg uncertainty principle we must have  $p\neq 0$. I am not familiar with how detectors work, but they should have finite size. So if a  detector detects a particle with zero momentum then the particle should remain in the detector. To not contradict the Heisenberg uncertainty principle the detector should have infinite size.
Now there is a reference frame where this particle has linear momentum   equal to zero so in  this frame the particle is represented by  $\mid 0 \rangle $.
How can we resolve this apparently contradiction?

REPLY [3 votes]: $\require{cancel}$
Well, they "exist" in our (physicists') imagination, as a limit, but they are not normalizable.
Adopting your notation for the null eigenstate of the momentum operator, $\hat p|0\rangle=0$, you may readily identify it as Dirac's celebrated standard ket, through which he defines his bra-ket formalism in Ch III.20 of his textbook. But, since $\langle p|0\rangle=\delta(p)$, your hyper-localized state has infinite norm, "$\delta(0)$".  You may, if you wished, approximate it as the limit of a suitably normalized Gaussian in momentum space, as its width goes to zero. (Intelligent squeezed coherent states. )

Note it is translationally invariant, since the exponential of the momentum operator acting on it amounts to one.

The x-space wavefunction of it is a plane-wave of zero momentum, so, then, a constant, $\langle x|0\rangle= 1/\sqrt{2\pi \hbar}$, everywhere in space; also unnormalizable, and terminally delocalized: a flatline, translationally invariant. In your Gaussian approximation, it would be a "Gaussian" with infinite width.
But you may translate this state in momentum space by acting on it with the exponential of the position operator,
$$ \exp (ip\hat{x}/\hbar ) | 0\rangle = |p\rangle  ,$$
and further define, with Dirac,
$$|x\rangle= \delta(\hat{x}-x) | 0\rangle  \sqrt{2\pi \hbar} .$$
The uncertainty principle is fine as a freak limit, but you'd better reassure yourself with the Gaussians involving mutually inverse widths. As it stands, it is egregious nonsense, which should never have been written down,$\cancel{
 \frac{\langle 0|\hat p^2 |0\rangle }{\langle 0|0\rangle } \frac{\int dx ~x^2 }{\int dx }  }$. I didn't write this...


text:
\begin{document}

\title{A renormalization approach to the Riemann zeta function at $-1$, $1+2+3+....\sim-1/12.$}

\author{
  Gunduz Caginalp\affil{1,}\corrauth
}

\shortauthors{the Author(s)}

\address{
  \addr{\affilnum{1}}{Mathematics Department, University of Pittsburgh, Pittsburgh, PA 15260, USA}
}

\corraddr{caginalp@pitt.edu; Tel: +1-412-624-8339
}

\begin{abstract} A scaling and renormalization approach to the Riemann zeta
function, $\zeta$, evaluated at $-1$ is presented in two ways. In the first,
one takes the difference between $U_{n}:=\sum_{q=1}^{n}q$ and
$4U_{\left\lfloor \frac{n}{2}\right\rfloor }$ where $\left\lfloor \frac{n}
{2}\right\rfloor $ \ is the greatest integer function. Using the Cesaro mean
twice, i.e., $\left(  C,2\right)  $, yields convergence to the appropriate
value. For values of $z$ for which the zeta function is represented by a
\textit{convergent} infinite sum, the double Cesaro mean also yields
$\zeta\left(  z\right)  ,$ suggesting that this could be used as an
alternative method for extension from the convergent region of $z.$ In the
second approach, the difference $U_{n}-k^{2}\bar{U}_{n/k}$ between $U_{n}$ and
a particular average, $\bar{U}_{n/k}$, involving terms up to $k<n$ and scaled
by $k^{2}$ is shown to equal exactly $-\frac{1}{12}\left(  1-k^{2}\right)  $
for all $k<n$. This leads to another perspective for interpreting
$\zeta\left(  -1\right)  $.
\end{abstract}

\keywords{
\textbf{(Riemann zeta function, sum of natural numbers,
$\zeta\left(-1\right)$, 1+2+3+..., Cesaro mean or sum)}
\newline
\textbf{Mathematics Subject Classification:} 11M99, 40C99}

\maketitle

\section{Introduction}

The Riemann zeta function is defined as the analytic continuation of the
infinite sum, $\zeta\left(  z\right)  =\sum_{q=1}^{\infty}q^{-z}$ where
$z=x+iy\in\mathbb{C}$ for $\operatorname{Re}z=x>1.$ For $x>1$ the series
converges absolutely to an analytic function. For all other values of $z$ it
diverges. Riemann showed \cite{t} that it can be continued analytically for
complex values of $z\in\mathbb{C}\ \backslash\left\{  1\right\}  ,$ i.e.,
except for the value corresponding to the harmonic series. For $z=-1$ one has
the (divergent) sum of natural numbers. The analytic continuation of the
series yields the result $\zeta\left(  -1\right)  =-1/12,$ with the formal
representation that appears to be an obvious contradiction:
\begin{equation}
1+2+3+\ ...\ \sim\zeta\left(  -1\right)  =-\frac{1}{12}\ . \label{sum}
\end{equation}
In this note we examine this relation using an approach that involves scaling
the truncated (finite) sum and renormalizing in order to obtain a finite
result as one takes the infinite limit of the sum.

Renormalization consists of a set of methodologies constituting a philosophy
and approach to problems exhibiting a divergence in some form. Originally
introduced for statistical mechanics and quantum field theory by Ken Wilson in
the 1970's, renormalization was able to yield the exponents with which key
physical properties diverge (see for example, \cite{w}, \cite{cfp}). The basic
idea is first to average spins within a particular geometric configuration,
thereby reducing the size of the system by a factor greater than unity. The
reduction in size must be compensated by adjusting the interaction strengths.
If this were not done, then iteration of this process would yield a trivial
fixed point of zero or infinity. With the appropriate renormalization,
however, one can iterate the procedure repeatedly. The key ansatz is that the
exponent of the divergent quantity should not change due to this averaging
process\ (with the interactions appropriately renormalized) since the
singularity is due to the divergence of the "correlation length" which is the
a measure of the distance at which spins can influence one another.

This approach to statistical mechanics revolutionized many calculations, as
very simple calculations yielded the results previously obtained by a
\textit{tour de force}, and led to its adaptation in a number of other areas.
The text by Creswick,\ Poole and Farach \cite{cfp} describes the
implementation of this approach to classical mathematical problems such as
fractals and random walk. For example, in random walk, the classical result
under robust conditions is that after $n$ steps the random walk has mean
distance $n^{1/2}$ from the original point. This result can also be obtained
by averaging sets of $k$ steps, and readjusting (i.e., renormalizing) the step
size so that one considers a walk of $n/k$ steps with the new step size. The
unique renormalization (i.e., setting the new step size) that leads to a
non-trivial result (i.e., not $0$ or $\infty$) yields the exponent $1/2$ in
$n^{1/2}.$

In this paper we describe methodology along the lines of this approach to
obtain an analog of $\left(  \ref{sum}\right)  $ that is well-defined.

The expression $\left(  \ref{sum}\right)  $ has been of interest in
applications such as string theory \cite{p}. In addition to this perspective,
two physicists \cite{pc} have also provided an explanation of $\left(
\ref{sum}\right)  $ based on shifting infinite sums.

\section{Averaging and re-scaling (Method 1)}
Using the notation $U_{n}=\sum_{q=1}^{n}q$ and $\left\lfloor r\right\rfloor $
as the greatest integer less than or equal to $r$ we define
\[
Y_{n}=U_{n}-4U_{\left\lfloor \frac{n}{2}\right\rfloor }
\]
One has from a simple calculation,
\[
Y_{n}=\left\{
\begin{array}
[c]{ccc}
-n/2 & if & n\ even\\
\left(  n+1\right)  /2 & if & n\ odd
\end{array}
\right.  \ .
\]

Now, let $Z_{N}$ be the Cesaro mean of $\left\{  Y_{n}\right\}  $, which is
also known as the Cesaro sum and plays an important role in Fourier analysis
(see for example, \cite{s}, p.52), i.e.,

\[
Z_{N}:=Avg\left\{  Y_{n}:n\leq N\right\}  =\frac{1}{N}\sum_{n=1}^{N}Y_{n}.
\]
Considering the odd and even terms separately, one can readily observe that
\begin{align}
Z_{2K+1}  &  =\frac{1}{2K+1}\left\{  1-1+2-2+...+\frac{2K+2}{2}\right\}
\nonumber\\
&  =\frac{1}{2}+\frac{1}{4K+2}\ .\nonumber\\
Z_{2K}  &  =0. \label{z}
\end{align}
Let $X_{M}:=Avg\left\{  Z_{N}:N\leq M\right\}  $, i.e., the Cesaro mean of
$Z_{N},$ which is the Cesaro mean of the Cesaro mean, i.e., $\left(
C,2\right)  ,$ of the original $\left\{  Y_{n}\right\}  $.

\begin{theorem}
For $K\in\mathbb{N}$ one has the following:

\begin{equation}
X_{2K}=\frac{1}{4}+\frac{1}{8K}\sum_{j=1}^{K}\frac{1}{j-1/2},\ \ \ \ X_{2K+1}
=\frac{K+1}{4K+2}+\frac{1}{8K+4}\sum_{j=1}^{K+1}\frac{1}{j-1/2} \label{ident}
\end{equation}
\begin{equation}
\left\vert X_{2K}-\frac{1}{4}\right\vert \leq\frac{1}{3K}+\frac{\log\left(
K+1\right)  }{8K} \label{even}
\end{equation}

\begin{equation}
\left\vert X_{2K+1}-\frac{1}{4}\right\vert \leq\frac{16/3+\log\left(
K+1\right)  }{8K+4} \label{odd}
\end{equation}
and thus the limit
\begin{equation}
\lim_{M\rightarrow\infty}X_{M}=\frac{1}{4}, \label{lim}
\end{equation}
which can also be expressed as
\begin{equation}
\lim_{M\rightarrow\infty}Avg_{N\leq M}\left\{  Avg_{n\leq N}\left\{
\frac{U_{n}-4U_{\left\lfloor \frac{n}{2}\right\rfloor }}{1-4}\right\}
\right\}  =-\frac{1}{12}\ .
\end{equation}
\end{theorem}

\begin{proof}
A computation using $\left(  \ref{z}\right)  $ for even and
odd values of $M$ results in $\left(  \ref{even}\right)  $ and odd $\left(
\ref{odd}\right)  .$ The inequalities$\ $
\[
\log\left(  K+1\right)  \leq\sum_{j=1}^{K+1}\frac{1}{j-1/2}=\frac{8}{3}
+\sum_{j=3}^{K+1}\frac{1}{j-1/2}\leq\frac{8}{3}+\log\left(  K+1\right)
\]
yield the result $\left(  \ref{lim}\right)  $.
\end{proof}

\begin{remark}
One can summarize this heuristically as
\begin{align}
E\left[  U_{n}-4U_{\lfloor\frac{n}{2}\rfloor}\right]   &  =P\left\{
n=odd\right\}  \left(  \frac{n+1}{2}\right)  +P\left\{  n=even\right\}
\left(  -\frac{n}{2}\right) \nonumber\\
&  =\frac{1}{2}\left(  \frac{n+1}{2}-\frac{n}{2}\right)  =\frac{1}{4}.
\label{k=2}
\end{align}
\end{remark}

In order to make $\left(  \ref{k=2}\right)  $ precise, one would need to
invoke some basic probabilistic ideas, primarily the Kolmogorov extension, or
existence, theorem (see for example, \cite{b}, p. 514) whereby the probability
on a finite subset of $\mathbb{N}$ can be extended to all of $\mathbb{N}$.
Formally identifying $U_{n}$ and $U_{\lfloor\frac{n}{2}\rfloor}$ as
$n\rightarrow\infty$ as though they were convergent leads to $\left(
\ref{sum}\right)  $.

\bigskip

One can define the analogous relations $U_{n}\left(  z\right)  :=\sum
_{q=1}^{n}q^{-z}$ which, as noted above, converges to $\zeta\left(  z\right)
$ for $\operatorname{Re}z>1$. Note that if a series that converges in the
ordinary sense, the Cesaro mean must converge to the same limit. Moreover, if
a sequence $\left\{  c_{j}\right\}  $ is convergent to $c$, the Cesaro mean
also converges to $c$. Thus, for values of $z\in\mathbb{C}$ in the convergent
region, the infinite sum is equal to the Cesaro mean, so that the Cesaro mean
can be used for both convergent and nonconvergent values.

\bigskip

In particular for a value $z$ for which $U_{n}\left(  z\right)  $ converges in
the usual sense to $\zeta\left(  z\right)  $, one has
\[
\lim_{n\rightarrow\infty}Y_{n}\left(  z\right)  =\lim_{n\rightarrow\infty
}\left\{  U_{n}\left(  z\right)  -4U_{\left\lfloor \frac{n}{2}\right\rfloor
}\left(  z\right)  \right\}  =-3\zeta\left(  z\right)
\]
as $n\rightarrow\infty$ . Since $Y_{n}\left(  z\right)  $ is convergent for
this value of $z,$ it follows that the Cesaro mean $Z_{N}\left(  z\right)
:=N^{-1}\sum_{n=1}^{N}Y_{n}\left(  x\right)  $ also converges to the same
limit, $-3\zeta\left(  z\right)  $. Similarly, the Cesaro mean, $X_{M}\left(
z\right)  :=M^{-1}\sum_{N=1}^{M}Z_{N}\left(  z\right)  $ also converges to
$-3\zeta\left(  z\right)  $. Thus, the interpretation that double Cesaro mean
of
\[
\frac{U_{n}\left(  z\right)  -4U_{\left\lfloor \frac{n}{2}\right\rfloor
}\left(  z\right)  }{1-4}
\]
converges to $\zeta\left(  z\right)  $ is maintained for values of $z$ for
which $U_{n}\left(  z\right)  =\sum_{q=1}^{n}q^{-z}$ is convergent.

For example, setting $z=2,$ so that one has a convergent series,
\[
\zeta\left(  2\right)  =\sum_{n=1}^{\infty}n^{-2}=\frac{\pi^{2}}
{6},\ i.e.,\ \ \lim_{n\rightarrow\infty}Y_{n}\left(  2\right)  =\left(
-3\right)  \frac{\pi^{2}}{6},
\]
and consequently $\lim_{M\rightarrow\infty}X_{M}\left(  z\right)  /\left(
-3\right)  =$ $\pi^{2}/6.$

Hence, this approach using the double Cesaro mean may present another avenue
to extend the sum from the convergent to the nonconvergent regions, and offer
other ways to study the Riemann zeta function.

\section{Averaging and re-scaling (Method 2)}

An alternate approach to averaging (without using the greatest integer
concept) can be implemented by fixing $k$ and letting $n=mk+j.$

Note that for $r\in\mathbb{N}$ one has $U_{r}=\sum_{q=1}^{r}q=\frac{1}
{2}r\left(  r+1\right)  .$ Define a continuous extension of $U_{r}$ by
$U_{r}:=\frac{1}{2}r\left(  r+1\right)  $ to $r\in\mathbb{R}$. \ Then define
the average of over the values of $j\in\left\{  0,...,k-1\right\}  $ as
\begin{equation}
\bar{U}_{\frac{n}{k}}:=\frac{1}{k}\sum_{j=0}^{k-1}U_{\frac{n-j}{k}}.
\label{avg}
\end{equation}

\begin{theorem}
For $n,k\in\mathbb{N}$ and $k<n$ one has the exact
relation $\ $
\begin{equation}
\frac{U_{n}-k^{2}\bar{U}_{\frac{n}{k}}}{1-k^{2}}=-\frac{1}{12}. \label{k}
\end{equation}
\end{theorem}

\begin{proof}
From the definition $\left(  \ref{avg}\right)  $ one observes
\begin{align*}
U_{n}-k^{2}\bar{U}_{\frac{n}{k}}  &  =\frac{1}{2}n\left(  n+1\right)
-k^{2}\frac{1}{k}\sum_{j=0}^{k-1}\frac{1}{2}\left(  \frac{n-j}{k}\right)
\left(  \frac{n-j}{k}+1\right) \\
&  =-\frac{1}{12}\left(  1-k^{2}\right)  .
\end{align*}

Thus, the quotient in $\left(  \ref{k}\right)  $ is thus independent of both
$k$ and $n.$ This averaging and scaling of $U_{n}$ by any factor $k$ together
with renormalization by $k^{2}$ yields the unique number $-1/12.$
\end{proof}

Formally, the identification of $U_{n}$ and $\bar{U}_{n/k}$ with $U$ in the
limit as $n\rightarrow\infty$ leads to $\sum_{q=1}^{\infty}q\sim
\zeta\left(  -1\right)  =-1/12.$

\bigskip

\begin{remark}
(a) The number $k$ can be regarded as the analog of the
"subwalk" of $k$ steps\ in the random walk problem discussed in the
introduction, with $n/k$ the new walk with the mean step size increased by a
factor $k^{2}.$

\bigskip

(b) Setting $k:=n^{p}$ with $p\in\left(  0,1\right)  $ one can write expression
$\left(  \ref{k}\right)  $ in the form
\[
\frac{n^{-p}U_{n}-n^{p}\bar{U}_{n^{1-p}}}{n^{-p}-n^{p}}=-\frac{1}{12}.
\]

\bigskip

(c) Applying this approach for $k=2$ yields,

\begin{align*}
\bar{U}_{\frac{n}{2}}  &  =\frac{1}{2}\left(  \frac{\frac{n}{2}\left(
\frac{n}{2}+1\right)  }{2}+\frac{\frac{n-1}{2}\left(  \frac{n-1}{2}+1\right)
}{2}\right)  =\allowbreak\frac{1}{8}n+\frac{1}{8}n^{2}-\frac{1}{16}\\
U_{n}-4\bar{U}_{\frac{n}{2}}  &  =\allowbreak\frac{1}{4}
\end{align*}
so that formally identifying $U_{n}$ and $\bar{U}_{\frac{n}{2}}$ as $U$ in the
limit $n\rightarrow\infty$ yields $U\sim-\frac{1}{12}.$

\bigskip

(d) To extend this result to other values of $z\in\mathbb{C}$,\ one can utilize
again the analogous quantity, $U_{n}^{\left(  z\right)  }=\sum_{q=1}^{n}
q^{-z}$, determine whether it is possible to formulate a definition analogous
to $\left(  \ref{avg}\right)  ,$ and consider values of $z $ for which these
converges. The left hand side of $\left(  \ref{k}\right)  $ can be considered
in the same manner as described for Method 1.

\end{remark}

\section*{Acknowledgments}
The author thanks Dr. Alban Deniz and Prof. Bogdan
Ion for useful discussions.

\section*{Conflict of Interest} The author declares no conflict of interest.