text:
\begin{document}

\title{Breaking the chain}
\author{Michael Allman and Volker Betz}

\address{Michael Allman \hfill\newline
\indent Department of Mathematics \hfill\newline
\indent University of Warwick \hfill\newline
\indent Coventry, CV4 7AL, England \hfill
}
\email{m.j.allman@warwick.ac.uk}

\address{Volker Betz \hfill\newline
\indent Department of Mathematics \hfill\newline
\indent University of Warwick \hfill\newline
\indent Coventry, CV4 7AL, England \hfill\newline
{\small\rm\indent http://www.maths.warwick.ac.uk/$\sim$betz/}
}
\email{v.m.betz@warwick.ac.uk}

\maketitle

\begin{quote}
{\small
{\bf Abstract.}
We consider the motion of a Brownian particle in $\R$, moving between a particle fixed at the origin and another moving deterministically away at slow speed $\ve>0$.  The middle particle interacts with its neighbours via a potential of finite range $b>0$, with a unique minimum at $a>0$, where $b<2a$.  We say that the chain of particles breaks on the left- or right-hand side when the middle particle is greater than a distance $b$ from its left or right neighbour, respectively.  We study the asymptotic location of the first break of the chain in the limit of small noise, in the case where $\ve = \ve(\sigma)$ and $\si>0$ is the noise intensity.
}

\vspace{1mm}
\noindent
{\footnotesize {\it Keywords:} first-exit from space-time domains, interacting Brownian particles, asymptotic theory}

\vspace{1mm}
\noindent
{\footnotesize {\it 2000 Math.\ Subj.\ Class.:} 60J70}
\end{quote}

\section{Introduction}

We are interested in the behaviour of a chain of interacting particles while it is pulled beyond its breaking point. Obvious real world examples would include the tearing of a band of rubber, or a rope, and obvious questions would be how much strain a given chain can endure before breaking, and where the breakpoint will be located once it occurs.

A model for such a chain is given by a collection of interacting Brownian particles i.e. one investigates solutions of the SDE system
\be \label{long chain}
\dd x_i(t) = - \frac{\partial H}{\partial x_i}(\bsx(t)) \, \dd t + \sigma \dd W_i(t),
\ee
where $(W_i)_{1 \leq i \leq N}$ are independent Brownian motions, $\bsx = (x_1, \ldots x_N) \in \bbR^N$ is the collection of particle positions, and $\sigma$ is the (small) noise intensity. The potential energy of the chain is given by
\[
H(\bsx) = \sum_{1 \leq i < j \leq N} U(x_i-x_j)
\]
for some pair potential $U$. We now exert strain on this chain of interacting Brownian particles.  This is done by solving (\ref{long chain}) only for $2 \leq i \leq N-1$, fixing $x_1=0$ and pulling $x_N$ outwards with (slow) speed $\eps$; the starting configuration of the chain should be a stable equilibrium, ideally a global minimum configuration of the potential energy. The mathematical questions corresponding to the problems above are then about the expected time of a (still to be defined) breaking event, and its location along the chain. In our case, the potential $U$ will have compact support, on $|x| \leq K$, say, and the breaking event will occur when the distance between two given particles is greater than $K$.

The model (\ref{long chain}) is widely used in material science to model the dynamics of crystals, in particular the propagation of cracks. Investigations there are purely numerical, and the main difficulty is the sheer size of the system under consideration. Out of the vast literature on the topic, we only mention \cite{EMcC78,BvG82,PE02,WHV92} and the references therein.

In mathematics, a model of type (\ref{long chain}) has recently been investigated by
T.\ Funaki \cite{fun1, fun2}. He studies the free motion of a, possibly multi-dimensional, crystal of interacting Brownian particles. In the limit of zero temperature, and under suitable assumptions on $U$, he shows  that if the system is initially rigidly crystallized, then it stays so for macroscopic time, and that the crystal as a whole performs Brownian motion both in the translational and in the rotational degrees of freedom. This is, in some sense, the opposite situation to when the crystal is torn apart by force.

It is clear that in the situation of stretching the chain (\ref{long chain}) until two particles are more than $K$ apart, we are looking at a first exit problem from a time-dependent domain. Also, although the chain is one-dimensional, the first-exit problem is not, indeed it is $(N-2)$-dimensional.

The problem of first-exit from a stationary potential well has been studied in great detail.  In \cite{fw}, the expected exit time from such a well is shown to behave asympotically like $\e{2h/\sigma^2}$, where $h$ is the height of potential well to overcome.  This is in agreement with the classical Eyring-Kramers formula \cite{ey,kr}.  In the multidimensional case, a proof of the expected exit time, with prefactor, has been given only recently \cite{ab}.

The case of a moving potential well is even more difficult and thus for the time being we settle for a further simplification: we take $N=3$, and $U$ as a cut-off strictly convex potential. In this case, only $x_2$ is moving, and so the problem to solve now is the exit of a one-dimensional stochastic process from a time-dependent domain, which still is a rather difficult and interesting topic,
and is related to the theory of stochastic resonance \cite{nb:pathwise, nb:beyond, nb:sample-paths}. Additionally, while in \cite{nb:pathwise, nb:beyond, nb:sample-paths} usually only the exit time distribution is of interest, we will need to know on which side of the domain the exit occurs. This question cannot, to our knowledge, be answered by any previously available results. So we solve it by direct investigation of the SDE, using some of the theory from \cite{nb, nb:pathwise}.

Our main result is Theorem \ref{main}. Roughly speaking, it says that for pulling speed $\eps$ and noise level $\sigma$ both going to zero, the chain will almost surely break on the right hand side if $\ve > \sigma \sqrt{|\ln \sigma|}$, while it will break on either side with probability $1/2$ when $\ve < \sigma/ \sqrt{|\ln \sigma|}$. This corresponds to the intuition that pulling too fast will just rip off the final
particle of the chain, as the noise does not have time to bring the configuration back to an equilibrium. Conversely, pulling very slowly corresponds to an adiabatic situation where the chain is in its local energy minimum all the time and the $1/2$ exit probability follows by symmetry. What is surprising is that we obtain this picture with great precision, with both cases separated only by a factor of $|\ln \sigma|$.
We do not know what happens in between the two cases specified in Theorem \ref{main}, although it is likely that the almost-sure law will start to fail before $\sigma = \eps$ due to the fluctuations of Brownian motion. The asymptotic behaviour of the system at this point or, for that matter, at any constellation with $\sigma \sqrt{|\ln \sigma|} \leq \ve \leq \sigma / \sqrt{|\ln \sigma|}$ is an interesting, but probably rather hard, open problem.
\\[1mm]
{\bf Acknowledgements:} We would like to thank Nils Berglund, Barbara Gentz and Anton Bovier for their valuable comments and stimulating discussions.

\section{The model and main result}
Three particles $x_L$, $x$ and $x_R$ in $\R$ interact with each other via a potential, $U$, of finite range satisfying
\begin{itemize}
	\item[(U0)] $U \in \caC(\R)$ with $U(-y)\isdefby U(y)$
	\item[(U1)] $U(y) = 0$ for $|y| \geq b$ and $U \in \caC^3((0,b))$
	\item[(U2)] $b < 2a$, where $U(a) = \min_{y \geq 0} U(y)$
	\item[(U3)] There exists $a_0 \in (0,a)$ such that $U''(y) \geq u_0 > 0$ for all $y \in (a_0,b)$
\end{itemize}
The particle $x_L$ is fixed at the origin and the position of $x_R$ at time $s$ is given by $x_R(s)=2a(1+\ve s)$, where $\ve > 0$ is a small parameter.  We study the behaviour of the middle particle, with position at time $s$ given by $x_s$.  Initially, it has position $x_0 = a$ so that the distance between neighbouring particles is $a$, which is the energetically-optimal configuration for this potential.  The time-dependent potential energy of the particle at position $x$ is given by
\[
H(x,\ve s) = U(x) + U(2a(1+\ve s)-x)
\]
The middle particle $x$ moves according to the SDE
\be\label{eq:sdes}
\dd x_s = -\frac{\partial H}{\partial x}(x_s,\ve s) \, \dd s + \sigma \dd W_s
\ee
where $x_0 = a$, $W_s$ is a standard Brownian motion and $\sigma>0$ is the noise intensity.  Rescaling time as $t=\ve s$, this is the same in distribution as solving
\begin{align}
\dd x_t & = -\frac{1}{\ve}\frac{\partial H}{\partial x}(x_t,t) \, \dd t + \frac{\sigma}{\sqrt{\ve}} \dd W_t \label{eq:xt}\\
& = \frac{1}{\ve}(-U'(x_t)+U'(2a(1+t)-x_t))\, \dd t + \frac{\sigma}{\sqrt{\ve}} \dd W_t \nonumber
\end{align}
This equation is well-defined as long as $2a(1+t) - b  < x_t < b$, which is the same condition that ensures the distance between any neighbouring particles is less than $b$.  As soon as this inequality fails, we consider the chain to be broken as there is no longer any interaction between $x$ and one of its neighbours.  Let
\be\label{tau0}
\tau = \inf\{t \geq 0 : x_t \notin (2a(1+t)-b,b)\}
\ee
We say the chain breaks on the left-hand side if $x_{\tau} = b$ and it breaks on the right-hand side if $x_{\tau} = 2a(1+\tau)-b$.  The chain necessarily breaks when $t = b/a-1$, so $\tau \leq b/a-1$.

Let $\Prob$ denote the law of the process $x_t$ when started from $a$ at time $0$.  We also write $f(\sigma) \ll g(\sigma)$ to mean that $f(\sigma)/g(\sigma) \to 0$ as $\sez$.
\begin{theorem}\label{main}
Let $x_t$ solve (\ref{eq:xt}) and define $\tau$ as in (\ref{tau0}).
\begin{enumerate}
 \item (Fast stretching) If $\sigma |\ln \sigma|^{1/2} \ll \ve(\sigma) \ll 1$ then $\Prob\{x_{\tau} = b\} \to 0$ as $\sez$.
 \item (Slow stretching) If
\[
 \frac{1}{\si^{2/3}}\exp\left\{-\frac{1}{\si^{2/3}}\right\} \ll \ve(\sigma) \ll \sigma|\ln \sigma|^{-1/2}
\]
then $\Prob\{x_{\tau} = b\} \to 1/2$ as $\sez$.
\end{enumerate}
\end{theorem}
 \begin{itemize}
  \item The proof of this theorem will actually yield that when $\sigma |\ln \sigma|^{1/2} \ll \ve(\sigma) \ll 1$, $\Prob\{x_{\tau} = b\} < (C\ve/\si^2)\e{-c \ve^2/\si^2}$.
 \item The lower bound on $\ve$ in (2) arises because our method applies on timescales shorter than Kramers' time, although we expect the result to hold without this lower bound.
 \item If $U$ is quadratic, then (2) is true without the lower bound on $\ve$.  We will comment on this at the end.
 \item The proof can be extended to the case that the chain is stretched according to some non-linear function $p(t)$, that is, $x_R(t) = 2a(1+p(t))$ where $0 < p_0 < p'(t) < p_1$.
 \end{itemize}
The theorem shows that when the stretching is fast, the chain will almost surely break on the right-hand side as $\sez$.  This is the same behaviour as in the deterministic case when $\si = 0$ (see the following section).  However, when the stretching is sufficiently slow, there is an equal probability to break on either side, as when there is no stretching at all.

\section{Proof}
\subsection{An alternative formulation}
For times $t < \tau$, we can replace $U$ with any potential $\tU \in \caC(\R)$ such that $\tU \in \caC^3((0,\infty))$, $\tU(y) = U(y)$ for $|y| \leq b$, $\tU(-y) = \tU(y)$ and $\tU''(y) \geq u_0 > 0$ for all $|y| > a_0$.  Defining $\tH(x,t) = \tU(x) + \tU(2a(1+t)-x)$ we have that for times $t < \tau$, $H(x_t,t) = \tH(x_t,t)$ and $x_t$ also solves
\[
\dd x_t = -\frac{1}{\ve}\frac{\partial \tH}{\partial x}(x_t,t) \, \dd t + \frac{\sigma}{\sqrt{\ve}} \dd W_t
\]
Let $x_t^{\deter}$ be the solution of the deterministic equation
\be\label{eq:ode}
 \dd x_t^{\deter} = -\frac{1}{\ve}\frac{\partial \tH}{\partial x}(x_t^{\deter},t)\dd t
\ee
with $x_0^{\deter} = a$.  This ODE is well-defined as long as $0 < x_t^{\deter} < 2a(1+t)$.  Since its solution can be written
\be\label{eq:det}
 x_t^{\deter} = a(1+t)-\frac{\ve}{\tU''(a(1+t))}+\mcO(\ve^2)
\ee
we see that for small $\ve$ this condition holds, in particular, for all $t \in [0,b/a-1]$.  Furthermore, by taking $\ve$ sufficiently small we also have in this interval that $a_0 < x_t^{\deter} < 2a(1+t) - a_0$.  If we had used $H$ instead of $\tH$ in (\ref{eq:ode}), then $x_t^{\deter}$ would not have been defined on the whole interval $[0,b/a-1]$.  Indeed, there is $t < b/a-1$ such that $2a(1+t)-x_t^{\deter}=b$.

We can now define the deviation process $y_t \isdefby x_t-x_t^{\deter}$ on the interval $[0,(b/a-1 )\wedge \tau]$.   This solves, with initial condition $y_0 = 0$,
\begin{align}
 \dd y_t & = \frac{1}{\ve}[-\tU'(x_t)+\tU'(x_t^{\deter})+\tU'(2a(1+t)-x_t)-\tU'(2a(1+t)-x_t^{\deter})]\dd t+\frac{\sigma}{\sqrt{\ve}}\dd W_t \nonumber \\
& = \frac{1}{\ve}[A(t)y_t+B(y_t,t)]\dd t+ \frac{\sigma}{\sqrt{\ve}}\dd W_t \label{eq:y}
\end{align}
where
\[
 A(t) = -\tU''(x_t^{\deter})-\tU''(2a(1+t)-x_t^{\deter})
\]
and there is a constant $M>0$ such that $|B(y,t)| \leq My^2$ for all pairs $(y,t) \in \caD$, where $\caD$ is given in (\ref{eq:D}).  We can also find constants $A_0,A_1 > 0$ such that $-A_1 \leq A(t) \leq -A_0$ for all $t \in [0,b/a-1]$.

For the chain to be unbroken, $y_t$ must satisfy
\[
 2a(1+t)-b-x_t^{\deter} < y_t < b-x_t^{\deter}
\]
which we write as
\[
 d_{-}(t)<y_t<d_{+}(t)
\]
where
\[
 d_{+}(t) = b-a(1+t)+\frac{\ve}{\tU''(a(1+t))}+\mcO(\ve^2)
\]
and
\[
 d_{-}(t) = a(1+t)-b+\frac{\ve}{\tU''(a(1+t))}+\mcO(\ve^2)
\]
The problem is then to study the first exit of the process $y_t$ from the space-time domain, $\mcD = \mcD(\ve)$, given by
\be\label{eq:D}
 \mcD = \{(y,t):d_{-}(t)<y<d_{+}(t), 0 \leq t \leq b/a-1\}
\ee
The stopping time $\tau$ given in (\ref{tau0}) can be written
\be\label{eq:tau}
\tau = \inf\{t \geq 0 : y_t \notin \caD\}
\ee
Then $y_{\tau} = d_-(\tau)$ corresponds to $x_{\tau} = 2a(1+\tau)-b$, that is, the chain breaking on the right-hand side.
Note that $d_+(t) \geq -d_-(t)$ for all $t \in [0,b/a-1]$ and so the curve $d_{-}(t)$ crosses zero before $d_{+}(t)$.  This means that in the deterministic case, when $y_t \equiv 0$, the curve $d_{-}(t)$ is hit before $d_{+}(t)$ and the chain breaks on the right-hand side.

If we let $\Prob^{t_0,y_0}$ denote the law of the process $y_t$ when started from $y_0$ at time $t_0$, then Theorem \ref{main} can be stated as
\begin{theorem}[Alternative version of Theorem \ref{main}]Let $y_t$ solve (\ref{eq:y}) and define $\tau$ as in (\ref{eq:tau}).
\begin{enumerate}
 \item If $\sigma |\ln \sigma|^{1/2} \ll \ve(\sigma) \ll 1$ then $\Prob^{0,0}\{y_{\tau} = d_+(\tau)\} \to 0$ as $\sez$.
 \item If
\[
 \frac{1}{\si^{2/3}}\exp\left\{-\frac{1}{\si^{2/3}}\right\} \ll \ve(\sigma) \ll \sigma|\ln \sigma|^{-1/2}
\]
then $\Prob^{0,0}\{y_{\tau} = d_+(\tau)\} \to 1/2$ as $\sez$.
\end{enumerate}

\end{theorem}
The main idea when proving this theorem is as follows.  The process $y_t$ is given by
\begin{align*}
 y_t & = \frac{\sigma}{\sqrt{\ve}}\int_0^t \e{\alpha(t,s)/\ve} \dd W_s + \frac{1}{\ve}\int_0^t \e{\alpha(t,s)/\ve}B(y_s,s)\dd s\\
& =: y_t^0 + y_t^1
\end{align*}
where $\alpha(t,s) = \int_s^t A(u)\dd u$ satisfies $-A_1(t-s) \leq \alpha(t,s) \leq -A_0(t-s)$.  We will also write $\alpha(t) = \alpha(t,0)$.  The term $y_t^0$ is Gaussian and so is easier to work with than $y_t^1$.  As long as $y_t$ is not too large, then $y_t^1$ can be bounded using that $|B(y,t)| \leq My^2$.  For example, if $t<\tau$ and $\sup_{0 \leq s \leq t} |y_s|\leq D$, then
\be\label{y13}
|y_t^1| \leq \frac{1}{\ve}\int_0^t |B(y_s,s)| \e{\alpha(t,s)/\ve} \dd s \leq \frac{MD^2}{\ve}\int_0^t \e{-A_0(t-s)/\ve} \dd s = \frac{MD^2}{A_0}(1-\e{-A_0 t/\ve})
\ee
If $D$ is small, then the contribution of $y_t^1$ will be much less than that of $y_t^0$.  The following proposition tells us for which $\ve$ we have this type of bound.
\begin{proposition}\label{prop:yleqD}
Let $\si \ll D(\si) \ll 1$ be such that
\[
 \frac{D^2}{\si^2}\exp\left\{-\frac{D^2}{\si^2}\right\} \ll \ve(\si) \ll 1
\]
Then
\[
 \lim_{\sez}\Prob^{0,0}\left\{\sup_{0 \leq t \leq (b/a-1 )\wedge \tau}|y_t| \geq D\right\} = 0
\]
\end{proposition}
\begin{remark}
 The lower bound on $\ve$ is related to the fact that we cannot bound $y_t$ on timescales larger than Kramers' time.  An excursion of size $D$ corresponds to climbing a potential height of $\mcO(D^2)$, which we expect to occur after a time of order $\e{D^2/\si^2}$.
\end{remark}
To prove this proposition, we will use a lemma which says roughly that the Gaussian term $y_t^0$ stays with high probability in a corridor of width proportional to its variance.  More precisely, the variance of $y_t^0$ is given by
\[
 \Var(y_t^0) = \frac{\sigma^2}{\ve}\int_0^t \e{2\alpha(t,s)/\ve}\dd s = \sigma^2 v(t)
\]
where $v(t)$ is a solution of $\ve \dot{v} = 2A(t)v+1$ with $v(0)=0$.  Following \cite{nb}, we see that since the right-hand side of this ODE vanishes for $v = -1/(2A(t))$, we can find a particular solution of the form
\be\label{eq:xi}
 \xi(t) = -\frac{1}{2A(t)}+\mcO(\ve)
\ee
where the $\mcO(\ve)$ term is uniform in $t$.  So there are constants $0 < \xi_- < \xi_+$ such that for $\ve$ sufficiently small, $\xi_- \leq \xi(t) \leq \xi_+$ for all $t \in [0,b/a-1]$.  The function $\xi(t)$ satisfies $|\xi(t)-v(t)| \leq C\e{2\alpha(t)/\ve}$ and will be used in the following lemma.  The advantage of $\xi(t)$ over $v(t)$ is that it is bounded away from zero.  The following lemma will be applied in cases where $\si \ll H$ and shows how paths of $y_t^0$ are concentrated.
\begin{lemma}[Berglund, Gentz \cite{nb,nb:pathwise}]\label{bglem2}
If $H^2 > 2\sigma^2$ then for any $t \in [0,b/a-1]$, we have
\[
 \Prob^{0,0}\left\{\sup_{0 \leq s \leq t} \frac{|y_s^0|}{\sqrt{\xi(s)}} \geq H\right\} = C_{H/\sigma}(t,\ve)\e{-H^2/2\sigma^2}
\]
with
\[
 C_{H/\sigma}(t,\ve) \leq 2\e{}\left\lceil \frac{|\alpha(t)|}{\ve}\frac{H^2}{\sigma^2}[1+\mcO(\ve)]\right\rceil
\]
\end{lemma}
This lemma is proved by partitioning the interval $[0,t]$ and applying on each subinterval the inequality
\[
\Prob\left\{\sup_{0 \leq s \leq t} \left|\int_0^s \varphi(u)\dd W_u\right| \geq \delta \right\} \leq 2\exp\left\{-\frac{\delta^2}{2\int_0^t \varphi(u)^2 \dd u}\right\}
\]
which is valid for deterministic Borel-measurable functions $\varphi: [0,t] \to \R$.
\begin{proof}[Proof of Proposition \ref{prop:yleqD}]
Consider the stopping time given by
\[
 \tau(h) = \inf\{t \geq 0: |y_t| \geq h\sqrt{\xi(t)}\}
\]
where $\xi(t)$ is given in (\ref{eq:xi}).  If we can find $h$ such that $h \sqrt{\xi(t)} \leq D$ for all $t \in [0,b/a-1]$ then
\begin{align*}
 \Prob^{0,0}\left\{\sup_{0 \leq t \leq (b/a-1 )\wedge \tau}|y_t| \geq D\right\} & \leq \Prob^{0,0}\left\{\tau(h)<(b/a-1 )\wedge \tau \right\}\\
 & = \Prob^{0,0}\left\{\sup_{0 \leq t \leq (b/a-1 )\wedge \tau \wedge \tau(h)} \frac{|y_t|}{\sqrt{\xi(t)}} \geq h\right\}
\end{align*}
As was noted above, $\xi(t) \leq \xi_+$ and so choosing $h = D/\sqrt{\xi_+}$ gives $h \sqrt{\xi(t)} \leq D$.  For $0 \leq t \leq (b/a-1 )\wedge \tau \wedge \tau(h)$, we have $|y_t| \leq D$ and so $|y_t^1| \leq M D^2 /A_0$.  Therefore,
\[
\sup_{0 \leq t \leq (b/a-1 )\wedge \tau \wedge \tau(h)} \frac{|y_t|}{\sqrt{\xi(t)}} \leq \sup_{0 \leq t \leq b/a-1} \frac{|y_t^0|}{\sqrt{\xi(t)}} + \frac{M D^2}{A_0 \sqrt{\xi_-}}
\]
and we have
\begin{multline*}
\Prob^{0,0}\left\{\sup_{0 \leq t \leq (b/a-1 )\wedge \tau \wedge \tau(h)} \frac{|y_t|}{\sqrt{\xi(t)}} \geq \frac{D}{\sqrt{\xi_+}}\right\} \\ \leq \Prob^{0,0}\left\{\sup_{0 \leq t \leq b/a-1} \frac{|y_t^0|}{\sqrt{\xi(t)}} \geq D\left(\frac{1}{\sqrt{\xi_+}} - \frac{M D}{A_0 \sqrt{\xi_-}}\right)\right\}
\end{multline*}
We can apply Lemma \ref{bglem2} with $H = D(1/\sqrt{\xi_+} - MD /(A_0 \sqrt{\xi_-})) = D(1/\sqrt{\xi_+}+\mcO(D))$.
\begin{multline*}
\Prob^{0,0}\left\{\sup_{0 \leq t \leq b/a-1} \frac{|y_t^0|}{\sqrt{\xi(t)}} \geq D\left(\frac{1}{\sqrt{\xi_+}} - \frac{M D}{A_0 \sqrt{\xi_-}}\right)\right\} \\  \leq 2\e{}\left\lceil C_1 \frac{D^2}{\ve \si^2}(1+ \mcO(D+\ve))\right\rceil \exp\left\{-C_2 \frac{D^2}{\si^2}(1+\mcO(D))\right\}
\end{multline*}
for constants $C_1,C_2 > 0$, from which the result follows.
\end{proof}

\subsection{Fast stretching}
In this case, we show that the chain is stretched so fast that the process $y_t$ is almost surely never greater than $d_{+}(b/a-1)$ in absolute value.  Note that the curve $d_{+}(t)$ is decreasing, since its derivative is given by
\[
-\dd x^{\deter}_t/\dd t = \tU'(x_t^{\deter})-\tU'(2a(1+t)-x_t^{\deter}) < 0
\]
using that $\tU''(y) \geq u_0 > 0$ for $|y|>a_0$ and $a_0 < x_t^{\deter} < 2a(1+t) - x_t^{\deter}$ for $t>0$, which can be seen from (\ref{eq:det}).  Since the curve $d_{+}(t)$ is decreasing, this means that it cannot have ever been hit by the process $y_t$ and so the chain must have broken on the right-hand side (Fig \ref{fig:corr}).  This is contained in the following proposition.
\begin{figure}
    \centering
    \includegraphics[width=.8\textwidth]{fast2}
\caption{When the chain is pulled sufficiently fast, the process $y_t$ is unlikely to leave the corridor of width $D = d_{+}(b/a-1)$, so must hit $d_{-}(t)$ first (shown here for $U(y) = y^2-4y+3$ and $\ve = 0.25$).}
\label{fig:corr}
\end{figure}
\begin{proposition}\label{prop:fast}
Let $\sigma |\ln \sigma|^{1/2} \ll \ve(\si) \ll 1$.  Then
\[
 \lim_{\sez}\Prob^{0,0}\left\{\sup_{0 \leq t \leq (b/a-1 )\wedge \tau}|y_t| \geq d_{+}(b/a-1)\right\} = 0
\]
\end{proposition}
\begin{proof}
Apply Proposition \ref{prop:yleqD} with $D = d_+(b/a-1) = \ve/\tU''(b/a-1)+\mcO(\ve^2)$.
\end{proof}
\subsection{Slow stretching}
The strategy is as follows.  Suppose we are given $D$ such that
\[
\lim_{\sez}\Prob^{0,0}\left\{\sup_{0 \leq t \leq (b/a-1) \wedge \tau} |y_t|\geq D\right\} = 0
\]
Then we can assume that $|y_t^1| \leq MD^2/A_0$ for all $t<\tau$, since all other cases have zero probability in the limit.  To simplify notation, we will write this last inequality as $|y_t^1| \leq D^2$.  For all $t < \tau$ we have
\[
 y_t^0 - D^2 \leq y_t \leq y_t^0 + D^2
\]
and, therefore,
\[
 P_L \leq \Prob^{0,0}\{y_{\tau} = d_+(\tau)\} \leq P_U
\]
where
\[
 P_L = P_L(D) = \Prob^{0,0}\{y_t^0 - D^2 \text{ hits $d_+(t)$ before $d_-(t)$}\}
\]
and
\[
 P_U = P_U(D) = \Prob^{0,0}\{y_t^0 + D^2 \text{ hits $d_+(t)$ before $d_-(t)$}\}
\]
The aim of this section is to show that given $\ve(\si)$, we can pick $D(\si)$ such that $P_L$ and $P_U$ tend to $1/2$, which gives the result.  The proof of each limit is similar, so we will show the details for $P_L$ only.  Note that $P_L$ can be written
\[
 P_L = \Prob^{0,0}\{y_t^0 \text{ hits $d_+(t) + D^2$ before $d_-(t)+D^2$}\}
\]
Define the stopping time
\[
 \tau_L = \tau_L(D) = \inf\{t \geq 0 : |y_t^0| \geq -d_-(t)-D^2\} < T
\]
where $T = \inf\{t \geq 0: -d_-(t)-D^2 = 0\}$.  By symmetry,
\[
 \Prob^{0,0}\{y_{\tau_L}^0 = -d_-(t)-D^2\} = \Prob^{0,0}\{y_{\tau_L}^0 = d_-(t)+D^2\} = \frac{1}{2}
\]
We must show that if $y_{\tau_L}^0 = -d_-(\tau_L)-D^2$ then almost surely $y_t^0$ hits $d_+(t)+D^2$ soon after as $\sez$ (see Fig \ref{boundary}).

\begin{figure}
  \centering
    \includegraphics[width=.8\textwidth]{newtriangle}
  \caption{When the chain is stretched slowly, we show that conditional probability of hitting the curve $d_+(t)+D^2$ before $d_{-}(t)+D^2$, when started from $-d_{-}(t)-D^2$, goes to one as $\sez$.}
\label{boundary}
\end{figure}
In the following two lemmas, we establish upper and lower bounds for $\tau_L$.  The upper bound is needed in order that $y_t^0$, when started from $-d_-(t)-D^2$, is much closer to $d_+(t)+D^2$ than to $d_-(t)+D^2$.  If $\tau_L$ is too close to $T$ then this is not the case and $y_t^0$ is more likely to exit in the ``wrong direction''.  The lower bound is required since we cannot expect the conditional probability of hitting $d_+(t)+D^2$, when started from $-d_-(t)-D^2$, to be close to one if it is unlikely that $y_t^0$ has even reached $-d_-(t)-D^2$.

We will now write $\Prob^{t_0,y_0}$ to denote the law of $y_t^0$ when started from $y_0$ at time $t_0$.
\begin{lemma}\label{lem:up}
For any $f(\si) \ll 1$ and $D^2 \ll \si$,
\[
\lim_{\sez} \Prob^{0,0}\left\{\tau_L \leq \frac{b}{a} - 1 -\frac{\sigma f(\sigma)}{a}\right\} = 1
\]
\end{lemma}
\begin{proof}
To prove this upper bound for $\tau_L$, we use a simple fixed-time estimate.  Let $t = b/a-1 - \sigma f(\sigma)/a$.  If $f$ is such that $t \geq T$, then the upper bound is trivial.  Otherwise
\begin{align*}
\Prob^{0,0}\{\tau_L \leq t\} & \geq \Prob^{0,0}\left\{|y_t^0|\geq -d_-(t)-D^2\right\}\\
& = \frac{2}{\sqrt{2\pi \Var(y_t^0)}}\int_{-d_-(t)-D^2}^{\infty}\exp\left\{-\frac{x^2}{2\Var(y_t^0)}\right\}\dd x\\
& = \frac{2}{\sqrt{\pi}}\int_{(-d_-(t)-D^2)/\sqrt{2\Var(y_t^0)}}^{\infty}e^{-z^2}\dd z
\end{align*}
where
\[
\frac{-d_-(t)-D^2}{\sqrt{2\Var(y_t^0)}} \leq \frac{ f(\si)-D^2/\si + \mcO(\ve/\si)}{\sqrt{1/A_1+\mcO(\ve^n)}}, \text{ $n \geq 1$ }
\]
This last inequality follows since $\Var(y_t^0) \geq \si^2 (1-\e{2A_1 t/\ve})/A_1$.  The right-hand side goes to zero as $\sez$.
\end{proof}
\begin{lemma}\label{lem:low}
For any $f$ such that  $1 \ll f(\si) \ll 1/\si$ and
\begin{equation}\label{f2}
 f(\si)^2 \exp\{-f(\si)^2\} \ll \ve \ll 1
\end{equation}
and for any $D^2 \ll \si$, we have
\[
 \lim_{\sez}\Prob^{0,0}\left\{\tau_L \geq \frac{b}{a}-1-\frac{\sigma f(\sigma)}{a}\right\} = 1
\]
\end{lemma}
\begin{proof}
We will use Lemma \ref{bglem2}.  First note that there is $c_1>0$ such that for $\ve(\si)$ sufficiently small,
\[
 -d_-(t) \geq b-a(1+t)-c_1 \ve
\]
holds for all $t$.  Putting $t_1 = b/a-1-\si f(\si)/a$, we get
\[
 \inf_{0 \leq t \leq t_1}(-d_-(t)) \geq \si f(\si)-c_1 \ve
\]
Now put $H= (\si f(\si)-c_1 \ve -D^2)/\sqrt{\xi_+}>0$.  Then
\[
 H\sqrt{\xi(t)} \leq H\sqrt{\xi_+} = \si f(\si)-c_1 \ve -D^2 \leq \inf_{0 \leq t \leq t_1}(-d_-(t)-D^2)
\]
Therefore,
\begin{align*}
 \Prob^{0,0}\left\{\tau_L < t_1 \right\} & \leq \Prob^{0,0}\left\{\sup_{0 \leq t \leq t_1} \frac{|y_t^0|}{\sqrt{\xi(t)}} \geq \inf_{0 \leq t \leq t_1} \frac{-d_-(t)-D^2}{\sqrt{\xi(t)}}\right\}\\
& \leq \Prob^{0,0}\left\{\sup_{0 \leq t \leq t_1} \frac{|y_t^0|}{\sqrt{\xi(t)}} \geq H\right\}
\end{align*}
Now we apply Lemma \ref{bglem2} to show that the right-hand side of this inequality tends to $0$ as $\sez$, which gives the result.
\end{proof}
Suppose that $f_+$ satisfies $1 \ll f_+(\si) \ll \min(\si/D^2,\si/\ve)$.  Then we can apply Lemmas \ref{lem:up} and \ref{lem:low} to $1/f_+(\si)$ and $f_+(\si)$, respectively.  This tells us that
\be\label{eq:t^*}
 \lim_{\sez}\Prob^{0,0}\left\{\frac{b}{a}-1-\frac{\si f_+(\si)}{a} \leq \tau_L \leq \frac{b}{a}-1-\frac{\si }{a f_+(\si)}\right\} = 1
\ee
The next proposition has three parts.  Together, they show that if $y_t^0$ starts from $-d_-(t)-D^2$ for suitable times $t^*$ as given in (\ref{eq:t^*}), then it hits $d_+(t)+D^2$ in a small interval $[t^*,t^*+\Delta]$ afterwards and does not hit the lower curve $d_-(t)+D^2$ in this time.

Recall that $T = \inf\{t \geq 0: -d_-(t)-D^2=0\}$.
\begin{proposition}\label{prop:3}
Let $f_+(\si)$ and $\Delta(\si)$ be chosen so that
\be\label{eq:f+}
 1 \ll f_+(\si) \ll \sqrt{\ve/\Delta} \ll \min(\si/D^2,\si/\ve)
\ee
Then for every $f$ such that $1/f_+(\si) \leq f(\sigma) \leq f_+(\si)$ and $t^* \isdefby b/a-1-\sigma f(\sigma)/a$, we have
\begin{enumerate}
 \item $[t^*,t^* + \Delta] \subset [0,T]$ for $\si$ sufficiently small.\\
 \item $\displaystyle \lim_{\sez}\Prob^{t^*,-d_-(t^*)-D^2}\left\{y_t^0 \geq d_+(t) + D^2 \text{ for some $t \in [t^*,t^*+\Delta]$}\right\} = 1$
 \item $\displaystyle \lim_{\sez}\Prob^{t^*,-d_-(t^*)-D^2}\left\{\inf_{t^* \leq t \leq t^* + \Delta} y_t^0 < 0 \right\} = 0$
\end{enumerate}
\end{proposition}
\begin{remark}
 Note that (1) and (3) together guarantee that $y_t^0$ does not hit $d_-(t)+D^2$ in the interval $[t^*,t^*+\Delta]$.
\end{remark}
\begin{proof}
(1) The upper bound on $f_+$ implies, in particular, that $f_+ \ll 1/\si$ and so $t^* > 0$.  As in the proof of Lemma \ref{lem:low}, for sufficiently small $\ve$ we have the uniform bound
\[
 -d_-(t)-D^2 \geq b-a(1+t) -c_1 \ve -D^2
\]
If $t< b/a - 1 - (c_1\ve +D^2)/a$ then the right-hand side is positive and $t<T$.  Note that
\begin{align*}
 t^*+\Delta & = \frac{b}{a}-1 -\frac{\si f(\si)}{a} + \Delta\\
& = \frac{b}{a}-1 -\frac{\si f(\si) - a\Delta}{a}
\end{align*}
If $\si f(\si) - a\Delta > c_1 \ve+D^2$ for sufficiently small $\si$ then $t^*+\Delta < T$.  By (\ref{eq:f+}), this is indeed satisfied.
\\[2mm]
(2) We show that $y_t^0$ hits $d_+(t)+D^2$ in the interval $[t^*,t^*+\Delta]$, which is the same as the process $\e{-\alpha(t)/\ve}y_t^0$ hitting the curve $\e{-\alpha(t)/\ve}(d_+(t)+D^2)$.  The latter will be more convenient to show, since it will lead to a probability involving a Gaussian martingale, for which the reflection principle can be applied.  The process $y_t^0$, when started from $-d_-(t^*)-D^2$ at time $t^*$, is given by
\be\label{eq:ycond}
y_t^0 = -\e{\alpha(t,t^*)/\ve}(d_-(t^*)+D^2) + \frac{\sigma}{\sqrt{\ve}}\int_{t^*}^t \e{\alpha(t,s)/\ve}\dd W_s
\ee
from which we deduce that
\begin{align}
 \e{-\alpha(t)/\ve}y_t^0 & = -\e{-\alpha(t^*)/\ve}(d_-(t^*)+D^2) + \frac{\sigma}{\sqrt{\ve}}\int_{t^*}^t \e{-\alpha(s)/\ve}\dd W_s \nonumber\\
 & \bydefis -\e{-\alpha(t^*)/\ve}(d_-(t^*)+D^2) + z_t^0 \label{z0}
\end{align}
For all $t \in [t^*,t^*+\Delta]$, we have
\[
\e{-\alpha(t)/\ve}(d_+(t)+D^2) \leq \e{-\alpha(t^*+\Delta)/\ve}(d_+(t^*)+D^2)
\]
Define $h(t^*,\Delta)\isdefby \e{-\alpha(t^*+\Delta)/\ve}(d_+(t^*)+D^2) +\e{-\alpha(t^*)/\ve}(d_-(t^*)+D^2) > 0$.  Then if
\[
 \sup_{t^* \leq t \leq t^*+\Delta}z_t^0 \geq h(t^*,\Delta)
\]
we must have that $z_t^0 \geq \e{-\alpha(t)/\ve}(d_+(t)+D^2) + \e{-\alpha(t^*)/\ve}(d_-(t^*)+D^2)$ for some $t \in [t^*,t^*+\Delta]$, which is equivalent to $y_t^0 \geq d_+(t)+D^2$.  By the reflection principle applied to $z_t^0$, we have
\begin{align*}
\Prob^{t^*,-d_-(t^*)-D^2}\left\{\sup_{t^* \leq t \leq t^*+\Delta}z_t^0 \geq h(t^*,\Delta)\right\} & = 2\Prob^{t^*,-d_-(t^*)-D^2}\left\{z^0_{t^*+\Delta} \geq h(t^*,\Delta)\right\}\\
& = \frac{2}{\sqrt{\pi}}\int_L^{-\infty}\e{-z^2}\dd z
\end{align*}
where $L = h(t^*,\Delta)/\sqrt{2\Var(z^0_{t^*+\Delta})}$.  If we can show that $L \to 0$ as $\sez$, then we will be done.  We have
\be\label{eq:firstpart}
0 \leq \frac{h(t^*,\Delta)}{\sqrt{2\Var(z^0_{t^*+\Delta})}} = \frac{\e{-\alpha(t^*+\Delta,t^*)/\ve}(d_+(t^*)+D^2)+d_-(t^*)+D^2}{\sqrt{2 \e{2\alpha(t^*)/\ve}\Var(z^0_{t^*+\Delta})}}
\ee
where
\[
 2 \e{2\alpha(t^*)/\ve}\Var(z^0_{t^*+\Delta}) = \frac{2\sigma^2}{\ve}\int_{t^*}^{t^*+\Delta}\e{-2\alpha(s,t^*)/\ve}\dd s \geq \frac{\sigma^2}{A_0}(\e{2A_0 \Delta/\ve}-1)
\]
which means that
\[
 \frac{\e{-\alpha(t^*+\Delta,t^*)/\ve}(d_+(t^*)+D^2)+d_-(t^*)+D^2}{\sqrt{2 \e{2\alpha(t^*)/\ve}\Var(z^0_{t^*+\Delta})}} \leq \frac{\e{A_1 \Delta/\ve}(d_+(t^*)+D^2)+d_-(t^*)+D^2}{\si \sqrt{(\e{2A_0 \Delta/\ve}-1)/A_0}}
\]
Using Taylor expansions for the exponential terms, we see from (\ref{eq:f+}) that the right-hand side tends to zero as $\sez$.
\\[2mm]
(3) Since the distribution of $y_t^0$, when started at $0$, is symmetric about $y=0$, $y_t^0$ satisfies a reflection principle about this line (see Appendix of \cite{nb}) and we have
\be\label{eq:zero}
 \Prob^{t^*,-d_-(t^*)-D^2}\left\{\inf_{t^* \leq t \leq t^* + \Delta} y_t^0 < 0\right\} = 2 \Prob^{t^*,-d_-(t^*)-D^2}\left\{y_{t^*+\Delta}^0 < 0\right\}
\ee
where we recall from (\ref{eq:ycond}) that the conditional process is given by
\[
y_t^0 = -\e{\alpha(t,t^*)/\ve}(d_-(t^*)+D^2) + \frac{\sigma}{\sqrt{\ve}}\int_{t^*}^t \e{\alpha(t,s)/\ve}\dd W_s
\]
Therefore,
\[
2 \Prob^{t^*,-d_-(t^*)-D^2}\left\{y_{t^*+\Delta}^0 \leq 0\right\} = \frac{2}{\sqrt{\pi}}\int_{-\infty}^U \e{-z^2}\dd z
\]
where
\begin{align*}
U = \frac{\e{\alpha(t^*+\Delta,t^*)/\ve}(d_-(t^*)+D^2)}{\sqrt{2\Var(y_{t^*+\Delta}^0)}} \leq \frac{d_-(t^*)-D^2}{\si \sqrt{(\e{2A_1 \Delta/\ve}-1)/A_1}}
\end{align*}
This inequality for $U$, which is negative, comes from the bound
\[
 2\e{-2\alpha(t^*+\Delta,t^*)/\ve} \Var(y_{t^*+\Delta}^0) \leq \si^2 (\e{2A_1 \Delta/\ve}-1)/A_1
\]
Again using Taylor expansions and (\ref{eq:f+}), we see that the upper bound for $U$ goes to $-\infty$ as $\sez$.
\end{proof}
\begin{proof}[Proof of Theorem \ref{main}]
First we suppose that $\si^2 |\ln \si| \ll \ve \ll \si |\ln \si|^{-1/2}$.  Then we can pick $D$ such that $\si^2 |\ln \si| \ll D^2 \ll \ve$ and it follows that
\[
 \frac{D^2}{\si^2}\exp\left\{-\frac{D^2}{\si^2}\right\} \ll \si^2 \ll \ve
\]
This means we can apply Proposition \ref{prop:yleqD} to show $|y_t|$ remains bounded by $D$ almost surely as $\sez$.  We can choose $|\ln \si|^{1/2} \ll f_+(\si) \ll \si/\ve = \min(\si/D^2,\si/\ve)$ so that
\[
 f_+(\si)^2\exp\left\{-f_+(\si)^2\right\} \ll \si^2 \ll \ve
\]
and can apply Lemmas \ref{lem:up} and \ref{lem:low} to $1/f_+$ and $f_+$, respectively, to show that we only need to consider hitting times of $-d_-(t)-D^2$ of the form $t^* = b/a-1-\si f(\si)/a$, where $1/f_+(\si) \leq f(\si) \leq f_+(\si)$.  Then we can apply Proposition \ref{prop:3} to show that the conditional probability of hitting $d_+(t)+D^2$ before $d_-(t)+D^2$ goes to one.

Now suppose that
\[
 \frac{1}{\si^{2/3}}\exp\left\{-\frac{1}{\si^{2/3}}\right\} \ll \ve \ll \si^2 |\ln \si|
\]
Pick $D$ such that $\si^2 |\ln \si| \ll D^2 \ll \si^{4/3}$ and
\[
 \frac{1}{\si^{2/3}}\exp\left\{-\frac{1}{\si^{2/3}}\right\} \ll \frac{D^2}{\si^2}\exp\left\{-\frac{D^2}{\si^2}\right\} \ll \ve
\]
We can again apply Proposition \ref{prop:yleqD} to bound $|y_t|$ by $D$.  Letting $f_+(\si) = D/\si \ll \si/D^2 = \min(\si/D^2,\si/\ve)$, we can apply Lemmas \ref{lem:up} and \ref{lem:low} to $1/f_+$ and $f_+$, respectively.  Then Proposition \ref{prop:3} holds and we are done.

Note that taking instead $\si^{4/3} \ll D^2 \ll \si$ gives the same lower bound on $\ve$ because in that case we need $\ve \gg f_+(\si)^2 \exp\{-f_+(\si)^2\}$ where $f_+(\si) \ll \si/D^2 \ll \si^{-1/3}$.
\end{proof}
We end this paper by commenting on the case of a quadratic potential $U$.  For such potentials, there is no non-linear term, $y_t^1$, and $D \equiv 0$.  We just have to show that $y_t^0$ has probability $1/2$ to hit $d_+(t)$ before $d_-(t)$.  For this we consider the conditional probability to hit $d_+(t)$ when starting from $-d_-(t)$.  If we define the analogue of $\tau_L$ as $\tilde{\tau}_L = \inf\{t \geq 0: |y_t^0| \geq -d_-(t)\}$ then when $\ve \ll \si^2$ we can show this conditional probability goes to one with only an upper bound for $\tilde{\tau}_L$.  Since we do not need to bound $|y_t^0|$, no lower bound on $\ve$ is required.  For $\si^2 \ll \ve \ll \si |\ln \si|^{-1/2}$, a lower bound on $\tilde{\tau}_L$ is needed to show the conditional probability goes to one, but this holds for such $\ve$ without additional assumptions.


text:
\begin{document}

\title[Series representations]{Series representations in spaces of vector-valued functions via Schauder decompositions}
\author[K.~Kruse]{Karsten Kruse}
\address{TU Hamburg \\ Institut f\"ur Mathematik \\
Am Schwarzenberg-Campus~3 \\
Geb\"aude E \\
21073 Hamburg \\
Germany}
\email{karsten.kruse@tuhh.de}

\subjclass[2010]{Primary 46E40, Secondary 46A32, 46E10}

\keywords{series representation, Schauder basis, Schauder decomposition, vector-valued function, injective tensor product}

\date{\today}
\begin{abstract}
It is a classical result that every $\C$-valued holomorphic function has a local power series representation.
This even remains true for holomorphic functions with values in a locally complete locally convex Hausdorff space $E$ over
$\C$. Motivated by this example we try to answer the following question. Let $E$ be a locally convex Hausdorff space
over a field $\K$, $\mathcal{F}(\Omega)$ be a locally convex Hausdorff space of $\K$-valued functions on a set $\Omega$
and $\mathcal{F}(\Omega,E)$ be an $E$-valued counterpart of $\mathcal{F}(\Omega)$
(where the term $E$-valued counterpart needs clarification itself).
For which spaces is it possible to lift series representations of elements of $\mathcal{F}(\Omega)$ to elements
of $\mathcal{F}(\Omega,E)$?
We derive sufficient conditions for the answer to be affirmative using Schauder decompositions
which are applicable for many classical spaces of functions
$\mathcal{F}(\Omega)$ having an equicontinuous Schauder basis.
\end{abstract}
\maketitle
\section{Introduction}
\input{Intro}

\section{Notation and Preliminaries}
\input{Notation1}

\section{Schauder decomposition}
\input{Schauder_decomposition}

\section{Applications}
\input{Applications_v1}

\bibliography{biblio}
\bibliographystyle{plain}
\end{document}

config: arxiv


text:
\begin{document}
\title[Short Title]{The Full Title of an AMS Journal Article}
\author{Author One}
\address[A. One and A. Two]{Author OneTwo address line 1\\
Author OneTwo address line 2}
\email[A. One]{aone@aoneinst.edu}
\urladdr{http://www.authorone.oneuniv.edu}
\author{Author Two}
\curraddr[A. Two]{Author Two current address line 1\\
Author Two current address line 2}
\email[A.~Two]{atwo@atwoinst.edu}
\urladdr{http://www.authortwo.twouniv.edu}
\author{Author Three}
\address[A. Three]{Author Three address line 1\\
Author Three address line 2}
\urladdr{http://www.authorthree.threeuniv.edu}
\thanks{Thanks for Author One.}
\thanks{Thanks for Author Two.}
\thanks{This paper is in final form and no version of it will be submitted for
publication elsewhere.}
\date{December 26, 1997}
\subjclass[2000]{Primary 05C38, 15A15; Secondary 05A15, 15A18}
\keywords{Keyword one, keyword two, keyword three}
\dedicatory{Dedicated to the memory of S. Bach.}
\begin{abstract}
Replace this text with your own abstract.

\end{abstract}
\maketitle

\section{AMS\ Journal Article}

This document illustrates the appearance of an article created with the shell
\textbf{AMS\ Journal Article}. It is intended for the preparation of articles
to appear in AMS journals. The extensive front matter includes fields for full
and short title, address, email address, URL address, thanks, dedication, and
key words.

The document class base file for this shell is \texttt{amsart.cls}. The
typesetting specification supports a number of class options. To see a list of
the available class options, choose \textsf{Typeset, }choose \textsf{Options
and Packages}, select the \textsf{Class Options} tab, and then click the
\textsf{Modify} button. This shell uses the default class options.

The typesetting specification for this shell document uses these options and
packages with the defaults indicated:

\begin{center}
\
\begin{tabular}
[c]{ll}
\textbf{Options and Packages} & \textbf{Defaults}\\\hline
Document class options & Standard\\
Packages: & \\
\quad amsfonts & None\\\hline
\end{tabular}

\end{center}

\section{Using This Shell}

The front matter of this shell has a number of sample entries that you should
replace with your own. Replace the body of this document with your own text.
You may delete all of the text in this document to start with a blank document.

Changes to the typeset format of this shell and its associated \LaTeX{}
formatting file (\texttt{amsart.cls}) are not supported by MacKichan Software,
Inc. If you want to make such changes, please consult the \LaTeX{} manuals or
a local \LaTeX{} expert. The typeset formatting file \texttt{amsart.cls} is a
required component of \LaTeX{}.

\section{Headings and Tags}

\includegraphics[scale=1]{.\1fiberswallowing.pdf}

\subsection{Subsection}

\subsubsection{Subsubsection}

This is just some harmless text under a subsubsection.

\paragraph{Subsubsubsection}

This is just some harmless text under a subsubsubsection.

\subparagraph{Subsubsubsubsection}

This is just some harmless text under a subsubsubsubsection.

\subsection{Tags}

You can apply the logical markup tag \emph{Emphasized}.

You can apply the visual markup tags \textbf{Bold}, \textit{Italics},
\textrm{Roman}, \textsf{Sans Serif}, \textsl{Slanted}, \textsc{Small Caps},
and \texttt{Typewriter}.

You can apply the special mathematics-only tags $\mathbb{BLACKBOARD}$
$\mathbb{BOLD}$, $\mathcal{CALLIGRAPHIC}$, and $\mathfrak{fraktur}$. Note that
blackboard bold and calligraphic are correct only when applied to uppercase
letters A through Z.

You can apply the size tags {\tiny tiny}, {\scriptsize scriptsize},
{\footnotesize footnotesize}, {\small small}, {\normalsize normalsize},
{\large large}, {\Large Large}, {\LARGE LARGE}, {\huge huge} and {\Huge Huge}.

This is a Body Math paragraph. Each time you press the Enter key, Scientific
WorkPlace switches to mathematics mode. This is convenient for carrying out
``scratchpad'' computations.

Following is a group of paragraphs marked as Short Quote. This environment is
appropriate for a short quotation or a sequence of short quotations.

\begin{quote}
The only thing we have to fear is fear itself. \emph{Franklin D. Roosevelt,
}Mar. 4, 1933

Ask not what your country can do for you; ask what you can do for your
country. \emph{John F. Kennedy, }Jan. 20. 1961

There is nothing wrong with America that cannot be cured by what is right with
America. \emph{William J. \textquotedblleft Bill\textquotedblright\ Clinton,
}Jan. 21, 1993
\end{quote}

The Long Quotation tag is used for quotations of more than one paragraph.
Following is the beginning of \emph{Alice's Adventures in Wonderland }by Lewis Carroll:

\begin{quotation}
Alice was beginning to get very tired of sitting by her sister on the bank,
and of having nothing to do: once or twice she had peeped into the book her
sister was reading, but it had no pictures or conversations in it, `and what
is the use of a book,' thought Alice `without pictures or conversation?'

So she was considering in her own mind (as well as she could, for the hot day
made her feel very sleepy and stupid), whether the pleasure of making a
daisy-chain would be worth the trouble of getting up and picking the daisies,
when suddenly a White Rabbit with pink eyes ran close by her.

There was nothing so very remarkable in that; nor did Alice think it so very
much out of the way to hear the Rabbit say to itself, `Oh dear! Oh dear! I
shall be late!' (when she thought it over afterwards, it occurred to her that
she ought to have wondered at this, but at the time it all seemed quite
natural); but when the Rabbit actually took a watch out of its
waistcoat-pocket, and looked at it, and then hurried on, Alice started to her
feet, for it flashed across her mind that she had never before seen a rabbit
with either a waistcoat-pocket, or a watch to take out of it, and burning with
curiosity, she ran across the field after it, and fortunately was just in time
to see it pop down a large rabbit-hole under the hedge.

In another moment down went Alice after it, never once considering how in the
world she was to get out again.
\end{quotation}

Use the Verbatim tag when you want \LaTeX{} to preserve spacing, perhaps when
including a fragment from a program such as:
\begin{verbatim}
#include <iostream>        // < > is used for standard libraries.
void main(void)            // "main" method always called first.
{
  cout << "Hello World.";  // Send to output stream.
}
\end{verbatim}

\section{Mathematics and Text}

Let $H$ be a Hilbert space, $C$ be a closed bounded convex subset of $H$, $T$
a nonexpansive self map of $C$. Suppose that as $n\rightarrow\infty$,
$a_{n,k}\rightarrow0$ for each $k$, and $\gamma_{n}=\sum_{k=0}^{\infty}\left(
a_{n,k+1}-a_{n,k}\right)  ^{+}\rightarrow0$. Then for each $x$ in $C$,
$A_{n}x=\sum_{k=0}^{\infty}a_{n,k}T^{k}x$ converges weakly to a fixed point of
$T$ .

The numbered equation
\begin{equation}
u_{tt}-\Delta u+u^{5}+u\left|  u\right|  ^{p-2}=0\text{ in }\mathbf{R}
^{3}\times\left[  0,\infty\right[ \label{eqn1}
\end{equation}
is automatically numbered as equation \ref{eqn1}.

\section{List Environments}

You can create numbered, bulleted, and description lists using the Item Tag
popup list on the Tag toolbar.

\begin{enumerate}
\item List item 1

\item List item 2

\begin{enumerate}
\item A list item under a list item.

The typeset style for this level is different than the screen style. The
screen shows a lower case alphabetic character followed by a period while the
typeset style uses a lower case alphabetic character surrounded by parentheses.

\item Just another list item under a list item.

\begin{enumerate}
\item Third level list item under a list item.

\begin{enumerate}
\item Fourth and final level of list items allowed.
\end{enumerate}
\end{enumerate}
\end{enumerate}
\end{enumerate}

\begin{itemize}
\item Bullet item 1

\item Bullet item 2

\begin{itemize}
\item Second level bullet item.

\begin{itemize}
\item Third level bullet item.

\begin{itemize}
\item Fourth (and final) level bullet item.
\end{itemize}
\end{itemize}
\end{itemize}
\end{itemize}

\begin{description}
\item[Description List] Each description list item has a term followed by the
description of that term. Double click the term box to enter the term, or to
change it.

\item[Bunyip] Mythical beast of Australian Aboriginal legends.
\end{description}

\section{Theorem-like Environments}

The following theorem-like environments (in alphabetical order) are available
in this style.

\begin{acknowledgement}
This is an acknowledgement
\end{acknowledgement}

\begin{algorithm}
This is an algorithm
\end{algorithm}

\begin{axiom}
This is an axiom
\end{axiom}

\begin{case}
This is a case
\end{case}

\begin{claim}
This is a claim
\end{claim}

\begin{conclusion}
This is a conclusion
\end{conclusion}

\begin{condition}
This is a condition
\end{condition}

\begin{conjecture}
This is a conjecture
\end{conjecture}

\begin{corollary}
This is a corollary
\end{corollary}

\begin{criterion}
This is a criterion
\end{criterion}

\begin{definition}
This is a definition
\end{definition}

\begin{example}
This is an example
\end{example}

\begin{exercise}
This is an exercise
\end{exercise}

\begin{lemma}
This is a lemma
\end{lemma}

\begin{proof}
This is the proof of the lemma.
\end{proof}

\begin{notation}
This is notation
\end{notation}

\begin{problem}
This is a problem
\end{problem}

\begin{proposition}
This is a proposition
\end{proposition}

\begin{remark}
This is a remark
\end{remark}

\begin{solution}
This is a solution
\end{solution}

\begin{summary}
This is a summary
\end{summary}

\begin{theorem}
This is a theorem
\end{theorem}

\begin{proof}
[Proof of the Main Theorem]This is the proof.
\end{proof}

Following is a short bibliography. It has no relationship to the previous
text, but can be used to show sample citations such as \cite{Hirsh64} and
\cite{Reid87}. This typesetting style places each citation inside square
brackets. If you want multiple citations to appear in a single set of square
brackets you must type all of the citation keys inside a single citation,
separating each with a comma. Here is an example: \cite{HB98a,HB98b,Hirsh64}.


TITLE: I need help to prove that $\alpha=\beta$ and $w$ bisects the angle $\theta$
QUESTION [0 upvotes]: Let $u$ and $v$ be vectors and $w =\|v\|u + \|u\|v$.
Let $\theta$ be the angle between $u$ and $v$, $\alpha$ be the angle between $u$ and $w$ and $\beta$, be the angle between $v$ and $w$. Show that $w$ bisects angle $\theta$.

REPLY [0 votes]: Here is a slightly different (and more tedious than the others) approach that utilizes the fact that a rotation in $\mathbb{R}^2$ is completely specified by its action on a non-zero vector (because rotations in $\mathbb{R}^2$ commute).
First note that since $w = \|u\| \|v\| ( \frac{u}{\|u\|} + \frac{v}{\|v\|} )$, and the angles are unchanged by positive scalar multiplication, we can assume that $u,v$ are of unit norm to start with.
If $u=v$, then $w=2u$ and we see that all angles are zero, hence the angle is trivially bisected.
If $u=-v$, then $w=0$ and the angle is undefined.
So, suppose $u,v$ are not collinear. In particular, they span a two dimensional subspace. Consider the  basis vectors $b_1 = \frac{u+v}{\|u+v\|}$, $b_2 = \frac{u-v}{\|u-v\|}$. It is straightforward to check that these are orthonormal.
We have $u = \frac{\|u+v\|}{2} b_1 + \frac{\|u-v\|}{2} b_2$, $v = \frac{\|u+v\|}{2} b_1 - \frac{\|u-v\|}{2} b_2$, $w = \|u+v\| b_1$.
Define the rotation $Q$ by $Q b_1 = v$, and the rotation $J$ by $Jb_1 = b_2, Jb_2 = -b_1$. (Define $Qx=x,Jx=x$ for $x \in \operatorname{sp} \{ b_1, b_2 \}^\bot $.) Since $Q,J$ commute, we have $Q b_2 = Q J b_1 = J Q b_1$ (so $Q$ is completely specified by its action on $b_1$).
In the following, let $z$ represent an element of $\operatorname{sp} \{ b_1, b_2 \}^\bot $.
We have
$$Q(\alpha_1 b_1 + \alpha_2 b_2 +z) = (\alpha_1 \frac{\|u+v\|}{2} + \alpha_2 \frac{\|u-v\|}{2}) b_1 + (-\alpha_1 \frac{\|u+v\|}{2} + \alpha_2 \frac{\|u-v\|}{2}) b_2 + z$$
from which it follows that
$$Q^T(\alpha_1 b_1 + \alpha_2 b_2 +z) = (\alpha_1 \frac{\|u+v\|}{2} - \alpha_2 \frac{\|u-v\|}{2}) b_1 + (\alpha_1 \frac{\|u+v\|}{2} + \alpha_2 \frac{\|u-v\|}{2}) b_2 + z$$
 and so we have $Q^T b_1 = u$.
Hence we have $Q \frac{w}{\|w\|} = v$ and $Q^T \frac{w}{\|w\|} = u$. Since $Q^T$ represents a rotation by the same angle in the opposite direction to $Q$, we have the desired result.
The angle can be computed from the unique solution (modulo $2 \pi$) to the equations $\cos (-\theta) = \frac{\|u+v\|}{2}$, $\sin(-\theta) = \frac{\|u-v\|}{2}$ (the $-\theta$ arises from my unfortunate choice of $Q b_1 = v$, and for the angle to be in the standard direction with respect to the basis $b_1,b_2$).


text:
\begin{document}

\maketitle

\clearpage

\begin{abstract}
One main obstacle for the wide use of deep learning in medical and engineering sciences is its interpretability. While neural network models are strong tools for making predictions, they often provide little information about which features play significant roles in influencing the prediction accuracy. To overcome this issue, many regularization procedures for learning with neural networks have been proposed for dropping non-significant features. Unfortunately, the lack of theoretical results casts doubt on the applicability of such pipelines.
In this work, we propose and establish a theoretical guarantee for the use of the adaptive group lasso for selecting important features of neural networks. Specifically, we show that our feature selection method is consistent for single-output feed-forward neural networks with one hidden layer and hyperbolic tangent activation function. We demonstrate its applicability using both simulation and data analysis.
\end{abstract}

\section{Introduction}
\label{intro}

One main obstacle for the wide use of deep learning in medical and engineering sciences is its interpretability and explainability.
Due to the complicated nature of deep neural networks and deep learning methods, they have been mostly treated as black-box tools.
Although the model has strong predictive power, it often provides little information about which features play significant roles in influencing prediction accuracy.
This limitation creates a bottle-neck for the applicability of deep learning in a time when the interpretability of scientific results is becoming increasingly important.
This issue is even more severe in application contexts where the most important aspect of the learning problem may not be about prediction but about

feature selection (that is, detecting which inputs control the outputs).
A prime example of such scenarios is the task of identifying genes that increase the risk of cancer.

 Recently, a wide variety of methods have been proposed to perform feature selection for neural networks.
 A popular approach is to derive a ranking of feature importance based on local perturbations.
This includes fitting a model in the local region around the input or locally perturbing the input to see how the prediction changes \cite{simonyan2013deep, ribeiro2016should, lundberg2017unified, shrikumar2017learning, ching2018opportunities}.
For high-dimensional inputs, high-level representation of the data can also be extracted, for example by an auto-encoder \cite{nezhad2016safs} or a Deep Boltzmann Machine \cite{ibrahim2014multi, taherkhani2018deep}) before a classical feature selection method can be applied.
Although these methods can provide useful insights, they often focus on specific architectures of the neural nets and can be difficult to generalize \cite{lu2018deeppink}.

In another direction, researchers employ regularization procedures to drop non-significant features out of the model.

Regularization methods have been used extensively for reducing the number of parameters in a deep neural network and

can be naturally adapted to obtain heuristic methods for feature selection by penalizing the weights of the first hidden layer.

Lasso ($\ell_1$ regularization), which penalizes the sum of absolute values of the weights, is probably the most popular regularization method.
This method, however, is not

ideal for feature selection because a feature can only be dropped if all of its connections have been shrunk to zero together, an objective that is not actively pursued in the scope of Lasso.

\citet{zhao2015heterogeneous} and \citet{scardapane2017group} address this concern by utilizing Group Lasso for selecting features of deep neural networks.
Alternatively, \citet{li2016deep} propose adding a sparse one-to-one linear layer between the input layer and the first hidden layer of a neural network and performing $\ell_1$ regularization on the weights of this extra layer.

While pipelines for feature selection for neural networks exist, the lack of theoretical results casts doubt on their applicability to real-world data.
Even worse, several works have indicated that Lasso and Group Lasso could be inconsistent for feature selection \cite{zou2006adaptive, zhang2018non}.
This is further complicated by the fact that neural network models are highly non-linear.

We tackle this issue directly by proposing and establishing a theoretical guarantee for the use of the Adaptive Group Lasso for selecting important features of neural networks.
Using the empirical risk minimizer (ERM) or the Group Lasso as an initial estimate,

the Adaptive Group Lasso constructs a data-dependent weighted regularizing function in such a way that as the sample size grows, the penalty for non-significant features get inflated (to infinity), whereas the penalty for significant features are bounded.

Under this framework, we show that for a single-output feed-forward neural network model with one hidden layer and the hyperbolic tangent activation function, the proposed feature selection method is consistent.
Additionally, we demonstrate the performance of our method in both simulation and data analysis.

\section{Mathematical framework}
\label{sec:math}

\subsection{Generating model}

For the simplicity of presentation, we consider a single-output feed-forward neural network for regression with one hidden layer and hyperbolic tangent activation function, where the corresponding numbers of nodes in each layer are $(n_I, n_H, 1)$.
We also separate the inputs into two groups $ s  \in \mathbb{R}^{n_s}$ and $ z  \in \mathbb{R}^{n_z}$ (with $n_s + n_z = n_I$) that denote the significant and non-significant variables, respectively.
An input is non-significant when the true value of all weights associated with it is zero.
We note that this separation is simply for mathematical convenience and the training algorithm is not aware of such dichotomy.

The forward model is visualized in Figure \ref{fig:network} and can be summarized as
\[
f_{u, v, w, b_1, b_2}( s,  z ) = w \cdot h + b_2
\]
where
\[
h^{[i]} = \tanh \left (\sum_{k=1}^{n_s}{u^{[i, k]} ~s^{[k]}} + \sum_{k=1}^{n_z}{v^{[i, k]} ~z^{[k]}} + b_1^{[i]} \right).
\]
Here, $u \in \mathbb{R}^{n_H \times n_s}$, $v \in \mathbb{R}^{n_H \times n_z}$, $h,w, b_1 \in \mathbb{R}^{n_H}$, $b_2 \in \mathbb{R}$; $h^{[i]}$ denotes the $i$-th component of a vector $h$, and $u^{[i, k]}$ denotes the $[i,k]$-entry of a matrix $u$.

We will study the learning problem in the model-based setting, whereas training data $\{(X_i, Y_i)\}_{i=1}^n$ are independent and identically distributed (i.i.d ) samples generated from $P^*_{X, Y}$ such that
\[
Y_i= f_{u^*, 0, w^*, b_1^*, b_2^*}(X_i) + \epsilon_i
\]
where $\epsilon_i \sim \mathcal{N}(0, \sigma_e^2)$.
Without loss of generality, we assume further that the input density $p_X$ is positive and continuous on its bounded domain and that the weight space (the set of all feasible vectors $\alpha = (u, v, w, b_1, b_2)$ for the model) is a compact set on a Euclidean space.
We note that when $v=0$, the function $f_{u, 0, w, b_1, b_2}$ does not depend on $Z$.

\begin{figure}
\centering
    \includegraphics[width=0.5\columnwidth]{network}
\caption{A single-output feed-forward neural network for regression with one hidden layer.}
\label{fig:network}
\end{figure}

\subsection{Identifiability of the feed-forward neural network model}

Before moving forward to the description of the feature selection problem, it is worth reminding that one of the distinguishing properties of neural network models is that it is inherently \emph{unidentifiable}.
A simple rearrangement of the nodes in the same hidden layer, an addition of a constant node or a sign flip of the weights may lead to a new configuration that produces the same input-output mapping.
These redundancies make it more difficult to analyze the performance of these models in both theoretical and practical perspectives.

To understand the identifiability of the model, we begin by studying the properties of weight space transformations that leave the network input-output mapping unchanged.
We will refer to the set of all vectors $\alpha = (u, v, w, b_1, b_2)$ as the weight space and denote it by $\mathcal{W}$.

\begin{Definition}
Two weight vectors $\alpha$ and $\alpha'$ are functionally equivalent (denote by $\alpha \sim_F \alpha'$) if
\[
f_{\alpha} = f_{\alpha'}.
\]
\end{Definition}

We consider two types of transformations: node interchanges and sign flips.

\begin{itemize}
\item An interchange consists of a permutation in which the weight vectors of two hidden nodes on the same hidden layer are interchanged.
A compensatory interchange of the weights of the nodes in the next layer that receive the inputs from the two interchanged nodes then removes the effect of the exchange in the previous layer.
\begin{Definition}
Two weight vectors $\alpha$ and $\alpha'$ are \emph{interchange equivalent} (denoted by $\alpha \sim_I \alpha' $)
if there exists a bijection $\pi$ on the set of hidden nodes $\mc{E}$ such that $\alpha_i = \alpha'_{\pi(i)}$ for all $i \in \mc{E}$.
Here, $\alpha_i$ is the set of weights associated with node $i$.
\end{Definition}
\item A sign flip is a transformation where the signs of all of the weights of a node on a hidden layer are multiplied by -1.
A compensatory sign flip is then carried out on all of the weights of nodes of the next layer associated with the input from the sign-flipped node.
\begin{Definition}
Two weight vectors $\alpha$ and $\alpha'$ are \emph{sign-flip equivalent} (denoted by $\alpha \sim_S \alpha' $)
if for all $i \in \mc{E}$, either $\alpha_i = \alpha'_{i}$ or $\alpha'_i = -\alpha_i$.
\end{Definition}
\end{itemize}

We have the following results.

\begin{Theorem}[\citet{kuurkova1994functionally}]
For a single-output feed-forward neural network with one hidden layer and hyperbolic tangent activation function, functionally equivalence are compositions of interchange and sign flip equivalence.
\[
\sim_F = \sim_I \circ \sim_S.
\]
\end{Theorem}

We recall the following two definitions from \citet{sussmann1992uniqueness} and \citet{fukumizu1996regularity}.

\begin{Definition}
A feed-forward model $f_{u, v, w, b_1, b_2}$ is \textbf{minimal} if no network with fewer hidden units have the same input-output map.

A feed-forward model $f_{u, v, w, b_1, b_2}$ is \textbf{irreducible} if
\begin{itemize}
\item[(i)] $(u^{[i,:]}, v^{[i,:]}) \ne 0$ and $w^{[i]} \ne 0$ for all $i$.
\item[(ii)] For any two different indices $i$ and $j$
\[
(u^{[i,:]}, v^{[i,:]},  b_1^{[i]})  \ne \pm (u^{[j,:]}, v^{[j,:]},  b_1^{[j]}).
\]

\end{itemize}
Here, $u^{[i,:]}, v^{[i,:]}$ denote the $i$-th row of the matrices $u, v$ respectively.
\end{Definition}

We note that the two condition $(i)$ and $(ii)$ guarantee that there are no non-participating or constant hidden nodes and no pair of identical nodes in the generating model.
While the two definitions are related, in general, irreducibility is an intrinsic property that can be verified based on the weights of a network alone, while minimality can only be verified in relative to other feasible networks from the model.
We have the following theorem that relates the two concepts.

\begin{Theorem}[\citet{sussmann1992uniqueness}]
For single-output feed-forward neural networks with one hidden layer and the hyperbolic tangent activation function, a network is irreducible if and only if it is minimal.
\label{unique}
\end{Theorem}

Throughout the rest of the manuscript, we make the following assumption.

\begin{Assumption}
The generating forward model $f_{\alpha^*}$, where $\alpha^*=(u^*, 0, w^*, b_1^*, b_2^*)$, is irreducible.
\label{irr}
\end{Assumption}

This assumption, along with Theorem $\ref{unique}$, provides us a way to characterize the set $\mathcal{H}^*$ of all weight vectors that produce the same input-output map as the generating model as follows.

\begin{Lemma}
Under Assumption $\ref{irr}$, we have
\begin{itemize}
\item [(i)] If $\alpha \in \mc{W}$ is a weigh vector such that
$f_{\alpha} =f_{\alpha^*}$, then $f_{\alpha}$ is also irreducible.
\item [(ii)]If $ (u, v, w, b_1, b_2 ) \sim_F  \alpha^*$ then $v =0$ and $u^{[:, k]} \ne 0$ for all $k = 1, \ldots, n_s$.

\item [(iii)] The set $ \mc{H}^* =\{\alpha \in \mc{W}: f_{\alpha} = f_{\alpha^*}\}$ is finite.
\end{itemize}
\label{lem:characterization}

\end{Lemma}
\begin{proof}
Since $f_{\alpha^*}$ is irreducible, Theorem $\ref{unique}$ implies that it is also minimal.
Let $\alpha \in \mc{W}$ is a weigh vector such that $f_{\alpha} =f_{\alpha^*}$
Since $\alpha \in \mc{W}$, the network $f_{\alpha}$ have the same number of hidden units as $f_{\alpha^*}$.
We conclude that $f_{\alpha}$ is minimal and thus is irreducible.

Finally, since we can only obtain the same input-output map through compositions of node-interchange and sign-flip transformations (for which the number of all possible transformations is finite), $\mc{H}^*$ is also finite.
Each of those transformations also leaves the zero/non-zero components unchanged, which validates $(ii)$.
\end{proof}

\section{Feature selection for neural networks}

In this section, we propose an Adaptive Group Lasso procedure for feature selection for neural networks.
The procedure consists of two steps:
\paragraph{Step 1:} An initial estimate for the weights is obtained using either the empirical risk minimizer (ERM) or Group Lasso.
\paragraph{Step 2:} Using the initial estimate, we build a weighted penalty for a new shrinkage estimator, which is utilized for deciding which features are significant.

The details of these steps are as follows:

\subsection{Empirical risk minimizer and Group Lasso}

For a set of training data $\{(X_i, Y_i)\}_{i=1}^n$,

the ERM estimate is defined by
\[
\hat \alpha_n = \argmin_{\alpha \in \mc{W}}  \frac{1}{n}\sum_{i=1}^n{\ell(\alpha, X_i, Y_i)},
\]
where $\ell(\alpha, x, y) = (y-f_{\alpha}(x))^2$, and the Group Lasso estimate is
\begin{align*}
(\bar u_n, &\bar v_n, \bar w_n, \bar b_{1,n}, \bar b_{2,n}) := \argmin_{(u,  v, w, b_1, b_2) }{ ~~\frac{1}{n}\sum_{i=1}^n{\ell(\alpha, X_i, Y_i)} + \lambda_n L(u, v) },
\end{align*}
where
\[
L(u, v) =  \sum_{k=1}^{n_s}{\|u^{[:, k]}\|} +  \sum_{k=1}^{n_z}{\|v^{[:, k]}\|},
\]
$\lambda_n$ is an appropriate regularizing constant and $\|\cdot\|$ denotes the usual Euclidean $\ell_2$-norm.

We note that in the setting of feature selection, only parameters of the first layer (namely, $u$ and $v$) are regularized.
Unlike the classical Lasso penalty (which takes the summation of the absolute values of individual parameters), parameters that associate with the same input are grouped together (through the $\ell_2$-norm) in the Group Lasso.
This grouping forces all outgoing connections from a non-significant input neuron to be simultaneously shrunk to zero.

\subsection{Adaptive Group Lasso}

\paragraph{ERM initialization:} We define the ERM + Adaptive Group Lasso estimator as
\begin{align*}
(\tilde u_n, &\tilde v_n, \tilde w_n, \tilde b_{1,n}, \tilde b_{2,n}) := \argmin_{(u,  v, w, b_1, b_2) }{ ~~\frac{1}{n}\sum_{i=1}^n{\ell(\alpha, X_i, Y_i)} + \zeta_n M_n(u, v) },
\end{align*}
where
\begin{align*}
M_n(u, v) = & \sum_{k=1}^{n_s} {\frac{1}{\| \hat u_n^{[:, k]}\|^{\gamma}}\| u^{[:, k]}\|} + \sum_{k=1}^{n_z} {\frac{1}{\| \hat v_n^{[:, k]}\|^{\gamma}}\|v^{[:, k]}\|},
\end{align*}
$\gamma$ is a positive number, $\hat \alpha_n$ is the ERM estimate, and $\zeta_n$ is an appropriate regularizing constant.
Here, we use the convention $0/0=1$.

Just as with Group Lasso, only the parameters of the first layer are regularized and they are grouped according to the inputs.
However, the Adaptive Group Lasso penalty aggressively shrinks the parameters associated with non-significant variables to zero by inflating their penalty weights.

\paragraph{Group Lasso initialization:} We also consider the following Group Lasso + Adaptive Group Lasso estimator:
\begin{align*}
(\check u_n, &\check v_n, \check w_n, \check b_{1,n}, \check b_{2,n}) := \argmin_{(u,  v, w, b_1, b_2) }{ ~~\frac{1}{n}\sum_{i=1}^n{\ell(\alpha, X_i, Y_i)} + \zeta_n G_n(u, v) },
\end{align*}
where
\begin{align*}
G_n(u, v) = & \sum_{k=1}^{n_s} {\frac{1}{\| \bar u_n^{[;, k]}\|^{\gamma}}\|u^{[:, k]}\|} + \sum_{k=1}^{n_z} {\frac{1}{\| \bar v_n^{[:, k]}\|^{\gamma}}\|v^{[:, k]}\|},
\end{align*}
$\gamma$ is a positive number, $\bar{\alpha_n}$ is the Group Lasso estimate, and $\zeta_n$ is the regularizing parameter.

\section{Consistent feature selection via Adaptive Group Lasso}

 As we see in the previous section, an Adaptive Group Lasso estimator inherently depends on the initialization.
Naturally, the performance of an Adaptive Group Lasso depends strongly on theoretical properties of the initial estimator.
In this section, we first investigate the convergence of the ERM and the Group Lasso, then use such results to establish feature selection consistency of the Adaptive Group Lasso.

\subsection{Convergence of the ERM and the Group Lasso}

First, we define the risk and empirical risk functions as follows:
\begin{align*}
R(\alpha) &= \mathbb{E}_{(X, Y)\sim P_{X,Y}^*}[(f_{\alpha}(X) - Y)^2] \\
R_n (\alpha) &=\frac{1}{n}\sum_{i=1}^n{(f_{\alpha}(X_i) - Y_i)^2}.
\end{align*}
We note that that the set of all minimizers of the function $R(\alpha)$ is $\mc{H}^*$ (the set of all weight vectors that produce the same input-output map as the generating model).
We have the following results about the risk functions, for which the proofs can be found in the Appendix.

\begin{Lemma}[Lipschitzness of the risk functions]
\textbf{}

\begin{itemize}
\item $R(\alpha)$ is a Lipschitz function with Lipschitz constant $c_0 > 0$.
\item For any $\delta > 0$, there exists $M_\delta > c_0$ such that $R_n(\alpha)$ is an $M_\delta$-Lipschitz function with probability at least $1 - \delta$.
\end{itemize}
\label{lem:almostLip}
\end{Lemma}

Next, we provide a generalization bound on the deviation of the empirical risk function from its expected value (proof is in the Appendix).
\begin{Lemma}[Generalization bound]
For any $\delta>0$, there exist $c_1(\delta)>0$ such that
\[
|R_n( \alpha) - R(\alpha) | \le c_1 \frac{\log n}{\sqrt{n}}, \h \forall \alpha \in \mc{W}.
\]
with probability at least $1-\delta$.
\label{lem:generalization}
\end{Lemma}

We also need the following information bound.
\begin{Lemma}
For any $ \alpha \in \mc{H}^*$, there exist $c_2(\alpha)>0$ and a neighborhood $\mc{U}_\alpha$ such that
\[
R(\beta) - R(\alpha)  \ge c_2 \|\beta-\alpha\|^2
\]
for all $ \beta \in \mc{U}_\alpha$.
\label{lem:information}
\end{Lemma}

\begin{proof}
We recall that the Fisher information matrix $I_{\alpha}$ for a Gaussian model is given by
\[
(I_{\alpha})_{ij} = \frac{1}{2 \sigma_e^2} E [ \nabla^2_{\alpha} \ell(\alpha, Y, X) ].
\]
Lemma \ref{lem:characterization} implies that under Assumption $\ref{irr}$, $f_\alpha$ is irreducible for all $\alpha \in \mc{H}^*$.
By Theorem 1 in \citet{fukumizu1996regularity}, the Fisher information matrix $I_{\alpha}$ is positive definite for all $\alpha \in \mc{H}^*$.
We deduce that
\begin{align*}
\ell(\beta, Y, X) - \ell(\alpha, Y, X) &= 2 (f_\alpha(X) - Y)  (\beta - \alpha) \cdot \nabla_{\alpha} f_\alpha(X) \\
&+  \frac{1}{2}(\beta - \alpha)^t \nabla^2_{\alpha} \ell(\alpha, Y, X)  (\beta - \alpha) + o(\|\beta -\alpha\|^2).
\end{align*}
This implies
\[
R(\beta) - R(\alpha) = \sigma^2_e (\beta - \alpha)^t [I_{\alpha}] (\beta - \alpha) + o(\|\beta -\alpha\|^2)
\]
which completes the proof since the Fisher information matrix is positive definite.
\end{proof}

Combining Lemma $\ref{lem:generalization}$ and $\ref{lem:information}$, we have.
\begin{Theorem}[Convergence of ERM]
For any $\delta>0$, there exist $C_\delta>0$ and $N_\delta>0$ such that for all $n \geq N_\delta$,
\[
\min_{\alpha \in \mc{H}^*} \|\hat \alpha_n -\alpha\| \le \frac{C_\delta\sqrt{\log n}}{n^{1/4} }
\]
with probability at least $1-\delta$.
\label{erm}
\end{Theorem}

\begin{proof}

We define
\[
\mc{U} = \bigcup_{\alpha \in \mc{H}^*}{\mc{U}(\alpha)}
\]
where $\mc{U}(\alpha)$ is defined in Lemma $\ref{lem:information}$.
Since $\mc{W} \setminus \mc{U}$ is compact and $R(\alpha)$ is a continuous function, there exist $C_{\mc{U}}>0$ such that
\[
R(\alpha) - R(\alpha^*)  \ge C_{\mc{U}} \h \forall \alpha \not\in \mc{U}.
\]
Thus, for all $n$ such that
\[
2  c_1 \frac{\log n}{\sqrt{n}} \le \frac{C_{\mc{U}}}{2}
\]
we have
\begin{align*}
0 \leq R(\hat \alpha_n)- R(\alpha^*) &\le R_n(\hat \alpha_n) - R_n(\alpha^*) + 2  c_1 \frac{\log n}{\sqrt{n} } \le 2  c_1 \frac{\log n}{\sqrt{n}} < C_{\mc{U}}
\end{align*}
with probability at least $1-\delta$.
We deduce that $\hat \alpha_n \in \mc{U}$ with probability at least $1-\delta$.
We have
\begin{align*}
c_2 \|\hat \alpha_n - \alpha\|^2 &\le R(\hat \alpha_n)- R( \alpha) \le 2  c_1 \frac{\log n}{\sqrt{n}}
\end{align*}
which completes the proof.
\end{proof}

Similarly, we also have the following theorem for Group Lasso.
\begin{Theorem}[Convergence of Group Lasso]
Assuming that $\lambda_n \to 0$.
For any $\delta>0$, there exist $C_\delta>0$, $N_\delta>0$ such that for all $n \ge N_\delta$,
\begin{align*}
\min_{\alpha \in \mc{H}^*} \|\bar \alpha_n -\alpha\| &\le C_\delta \left( \frac{\log n}{\sqrt{n} } + \lambda_n^2\right)^{1/2}
\end{align*}
with probability at least $1-\delta$.
\label{group}
\end{Theorem}

\begin{proof}
We have
\[
R_n( \bar \alpha_n) + \lambda_n L(\bar \alpha_n)  \le R_n( \alpha^*) + \lambda_n L(\alpha^*)
\]
which implies
\begin{align*}
R(\bar \alpha_n)- R( \alpha^*) &\le R_n(\bar \alpha_n) - R_n(\alpha^*) + 2  c_1 \frac{\log n}{\sqrt{n}} \\
&\le 2  c_1 \frac{\log n}{\sqrt{n}} + \lambda_n \left( L(\alpha^*) - L(\bar \alpha_n)\right)
\end{align*}
since $\mc{W}$ is bounded.
Using the same argument as in the proof of Theorem $\ref{erm}$, we conclude that when $n$ is large enough, $\hat \alpha_n \in \mc{U}(\alpha)$ for some $\alpha \in \mc{H}^*$.
Since $L(\alpha)$ is a Lipschitz function, we have
\begin{align*}
c_2 \|\bar  \alpha_n - \alpha\|^2  &\le 2  c_1 \frac{\log n}{\sqrt{n}} + \lambda_n \left( L(\alpha) - L(\bar \alpha_n)\right)\\
&\le 2  c_1 \frac{\log n}{\sqrt{n}} + \lambda_n C\|\bar \alpha_n - \alpha\|\\
&\le 2  c_1 \frac{\log n}{\sqrt{n}} + \frac{c_2}{2}\|\bar \alpha_n - \alpha\|^2 + \frac{C^2\lambda_n^2}{2 c_2}
\end{align*}
which completes the proof.
\end{proof}

\subsection{Feature selection consistency of the adaptive group lasso}

\begin{Theorem}[Convergence of ERM + Adaptive Group Lasso]
Assuming that $\zeta_n \to 0$, for all $\delta>0$, there exists $C_{\delta}, N_{\delta}>0$ such that for all $n \ge N_{\delta}$, we have
\[
\min_{\alpha \in \mc{H}^*} \|\tilde \alpha_n -\alpha\|  \le C_{\delta}\left( \frac{\log n}{\sqrt{n}} + \zeta_n \right)^{1/2}
\]
\label{erm-agl}
with probability at least $1-\delta$.
\end{Theorem}

\begin{proof}
By Theorem $\ref{erm}$, for $n$ large enough,
\[
\min_{\alpha \in \mc{H}^*} \|\hat \alpha_n -\alpha\| \le \frac{C_\delta\sqrt{\log n}}{n^{1/4} }
\]
with probability at least $1-\delta$.
Therefore,
\[
\min_{\alpha \in \mc{H}^*} \|\hat u_n^{[:, k]} - u^{[:, k]}\| \le \frac{C_\delta\sqrt{\log n}}{n^{1/4} }, \h \forall k =1, \ldots, n_s.
\]
By Lemma $\ref{lem:characterization}$, we conclude that $\hat u_n^{[:, k]}$ is bounded away from zero as $n \to \infty$.
We deduce that for $\alpha \in \mc{H}^*$,
\[
M_n(\alpha) = \sum_{i=1}^{n_s} \frac{1}{\| u_n^{[:, k]}\|^{\gamma}} \|u^{[:, k]}\|  < \infty.
\]
Thus,
\begin{align*}
R(\tilde\alpha_n)- R( \alpha^*) &\le R_n(\tilde \alpha_n) - R_n(\alpha^*) + 2  c_1 \frac{\log n}{\sqrt{n}} \\
&\le 2  c_1 \frac{\log n}{\sqrt{n}} + \zeta_n \left( M_n(\alpha^*) - M_n(\bar \alpha_n)\right)\\
&\le 2  c_1 \frac{\log n}{\sqrt{n}} + \zeta_n M_n(\alpha^*).
\end{align*}
Since $M_n(\alpha)$ is bounded, using the same argument as in the proof of Theorem $\ref{erm}$, we conclude that when $n$ is large enough, $\tilde \alpha_n \in \mc{U}(\alpha)$ for some $\alpha \in \mc{H}^*$ and there exists $\alpha \in \mc{H}^*$
\[
c_2 \|\bar  \alpha_n - \alpha\|^2 \le R(\tilde\alpha_n)- R( \alpha^*) \le 2  c_1 \frac{\log n}{\sqrt{n}} + \zeta_n M_n(\alpha^*)
\]
which completes the proof.
\end{proof}

We are now ready to prove the main theorem of our paper.

\begin{Theorem}[Feature selection consistency of ERM + Adaptive Group Lasso]
For $\gamma>0$, $\mu \in (0, \gamma/4)$ and  $\zeta_n =\Omega (n^{-\gamma/4 + \mu})$, then the ERM + Adaptive Group Lasso is consistent for feature selection.
That is, for any $\delta >0$, there exists $N_\delta$ such that for $n > N_\delta$, $\tilde u_n^{[:, k]} \ne 0, ~ \forall k =1, \ldots, n_s$, and $\tilde v_n^{[:, k]} = 0, ~ \forall k =1, \ldots, n_z$ with probability at least $1 -\delta$.
\end{Theorem}

\begin{proof}
Theorem $\ref{erm-agl}$ provides that
\[
\min_{\alpha \in \mc{H}^*} \|\tilde \alpha_n -\alpha\|  \le C_{\delta}\left( \frac{\log n}{\sqrt{n}} + \zeta_n \right)^{1/2}
\]
By Lemma $\ref{lem:characterization}$, we conclude that $\tilde u_n^{[:, k]}$ is different from zero for $n$ large enough.
Next, we will prove that $\tilde v_n =0$.
Note that from Theorem $\ref{erm}$
\[
\|\hat v_n^{[:, k]}\| \le  \frac{C_\delta \sqrt{\log n}}{n^{1/4} }
\]
with probability at least $1-\delta$.
Thus,
\begin{equation}
\lim_{n \to \infty}{~\zeta_n \frac{1}{\|\hat v_n^{[:, k]}\|^{\gamma}}} \ge C_\delta^{-\gamma} \lim_{n \to \infty}{~\zeta_n \frac{n^{\gamma/4}}{(\log n)^{\gamma/2}}} = \infty.
\label{bound}
\end{equation}
Now, we assume that $\tilde v_n^{[:, k]} \ne 0$ for some $k$ and define a new weight configuration $g_n$ obtained from $\tilde \alpha_n$ by setting the $v^{[:, k]}$ component to $0$.
By definition of the estimator $\tilde \alpha_n$, we have
\[
R_n( \tilde \alpha_n) +  \zeta_n   \frac{1}{\|\hat v_n^{[:, k]}\|^{\gamma}} \|\tilde v_n^{[:, k]}\|\le R_n(g_n).
\]
By Lemma $\ref{lem:almostLip}$, we have
\begin{align*}
 \zeta_n   \frac{1}{\|\hat v_n^{[:, k]}\|^{\gamma}} \|\tilde v_n^{[:, k]}| &\le  R_n(g_n) - R_n( \tilde \alpha_n)\\
 & \le c_0 \|g_n - \tilde \alpha_n\|  = M_{\delta} \|\tilde v_n^{[:, k]}\|.
\end{align*}
with probability at least $1-\delta$.
Since $\tilde v_n^{[:, k]} \ne 0$, we deduce that
\[
\zeta_n   \frac{1}{\|\hat v_n^{[:, k]}\|^{\gamma}}  \le M_{\delta},
\]
which contradicts $\eqref{bound}$.
This completes the proof.
\end{proof}

\begin{Theorem}[Feature selection consistency of Group Lasso + Adaptive Group Lasso]
For $\gamma>0$,  $\mu \in (0, \gamma/4)$, $\zeta_n =\Omega (n^{-\gamma/4 + \mu})$ and $\zeta_n = \Omega(\lambda_n^{\gamma+\mu})$ where $\lambda_n$ is the regularizing constant of the initial Group Lasso estimate.
Then, the Group Lasso + Adaptive Group Lasso is consistent for feature selection.
\end{Theorem}

\begin{proof}
The proof is similar to that of the ERM + Adaptive Group Lasso, with the only exception that
\[
\|\bar v_n^{[:, k]}  \| \le C \left( \frac{\log n}{\sqrt{n} } + \lambda_n^2 \right)^{1/2}.
\]
with probability $1-\delta$.
\end{proof}

\section{Simulation studies}

We use synthetic data to investigate the performance of our feature selection procedure.
Throughout the experiment, we consider single-output feed-forward neural networks with one hidden layer of ten hidden nodes.
Two separate sets of experiments are considered.
In the first set, one of the two input variables is non-significant while the other is significant.
In the second set of simulations, two of the five input variables are non-significant and the other three are.

In each set of experiments, we simulate datasets of size $n=1000$ from the generative model
\[
Y= f_{u^*, 0, w^*, b_1^*, b_2^*}(X) + \epsilon
\]
where the noise $\epsilon^{[i]}$ is sampled independently from $\mathcal{N}(0, \sigma^2_e)$ with various values of $\sigma_e^2$.
Each component of $X, b_1^*, b_2^*$ is sampled independently from $\mathcal{N}(0,1)$, and each component of $u^*, w^*$ is independently sampled from $ \mathcal{N}(1,1)$.

\begin{figure}[ht]
\centering
    \includegraphics[width=0.47\textwidth]{five_variables_1}
 \includegraphics[width=0.47\textwidth]{five_variables_2}
 \caption{False Discovery Rates (FDRs) of the two non-significant variables (each subfigure corresponds to a non-significant variable) in the second set of experiment with varying degrees of noise by the three feature selection algorithms:  the Group Lasso (red curves), the ERM + Adaptive Group Lasso (green), and Group Lasso + Adaptive Group Lasso (blue).}
\label{fig:5var}
\end{figure}

We are interested in the performance of our feature selection procedures with various noise levels.
For each $\sigma_e^2$ in $\{0, 0.2, 0.4, 0.6, 0.8, 1\}$, we simulate 100 datasets of size $n=1000$.
We apply three methods of feature selection (the Group Lasso, the ERM + Adaptive Group Lasso, and Group Lasso + Adaptive Group Lasso) on each simulated dataset with $\gamma = 2$.
The regularizing constant is chosen from the set $\{0.001, 0.01, 0.1, 1\}$ using three-fold cross-validation.
The algorithms are trained using Adam optimizer over 10000 epochs with batch-size of 200, of which the codes are created using Python package \emph{lasagne} \cite{lasagne} and Python library \emph{theano} \cite{team2016theano}.
Following the convention used in \citet{scardapane2017group}, we deselect a variable if the $\ell_2$-norm of the groups of parameters associated with that variable is below the cut-off value $10^{-3}$.

The result is presented in Figure \ref{fig:5var}, where the False Discovery Rate (FDR) of the two non-significant variables in the second set of experiments (with 5 input variables) for each of the algorithms are reported.
A similar result for the first set of experiments is also provided in the Appendix.
In both sets of experiments, we observe that across all algorithms, the FDR tends to increase as the variance of the error increases.
The Group Lasso + Adaptive Group Lasso perform the best, while the Group Lasso fails to recognize the non-significant variables when there are errors in the model ($\sigma^2_e >0$).
The result indicates that for highly non-linear models such as neural networks, standard Lasso-type algorithms may not be aggressive enough to enforce sparsity and data-dependent approaches such as Adaptive Lasso might be necessary for effective feature selection.
On the other hand, it is worth noting that all three methods can detect the significant variables efficiently.

\section{Data analysis}

\begin{figure*}
\centering
    \includegraphics[width=1\textwidth]{analysis}
\caption{Frequency of being selected by Group Lasso + Adaptive Group Lasso of each feature, computed out of 100 runs of the algorithm.}
\label{fig:analysis}
\end{figure*}

In addition to the simulation examples presented in the previous section, we also demonstrate the practical use of our feature selection approach to analyze the Divorce Prediction dataset \citep{yontem2019divorce} from the UCI Machine Learning Repository \cite{Dua:2019}.
The dataset consists of survey data from 170 married couples, where each participant rates 54 statements about their marriage on the scale from 0--4 (a detailed list of the statements are provided in the Appendix).
The dataset was collected from seven different regions of Turkey (but predominantly from the Black Sea region) through face-to-face interviews and via Google Drive.
Among the participants, 49\% were divorced and 51\% were married couples.

Based on the success of its performance in simulation studies, we apply the Group Lasso + Adaptive Group Lasso to analyze the data set.
In this analysis, we also use the feed-forward network structure with one hidden layer and the hyperbolic tangent activation function.
Since the data is binary, we further add a sigmoid activation function to the output layer of the network.
The model is then trained using Adam optimizer with binary cross-entropy loss over 10000 epochs with batch-size of 32.
The 3-fold cross-validation is employed to select the regularizing parameter $\lambda$ from the set $\{0.001, 0.01, 0.1, 1, 2, 4, 8, 16 \}$.
To take into account the stochasticity of the selection procedure, we run the algorithm 100 times and record the number of times each feature is chosen by the algorithm.
The result is presented in Figure \ref{fig:analysis}, which shows that the five most important variables are selected more than 75 times while all other variables appear fewer than 50 times on the selections.

To validate the result, we fit the dataset on the full set of variables and on the selected set of five variables using ERM algorithm with the same neural network structure and compare their performances.
In each iteration, the dataset is split into a training set and a test set by a ratio of 75-25.
We use ERM to train on the training set with the binary cross-entropy loss and compute the accuracy on the test set.
The process is repeated 100 times and the average testing accuracy for the full model and the selected model are 97.7\% and 98.1\%, respectively.
The result indicates that our feature selection procedure can select the most important features for prediction.

\section*{Discussions and future work}

In this work, we propose the use of the Adaptive Group Lasso for selecting important features of neural networks.
Under mild regularity conditions, we establish feature selection consistency of two versions of the Adaptive Group Lasso on three-layer neural networks with hyperbolic tangent activation function.
We have demonstrated the efficiency and effectiveness of our methods with both synthetic and real datasets.
We show that the standard Group Lasso estimator has difficulty identifying the input support while the Adaptive Group Lasso can accurately and consistently select the correct set of significant variables.

There are several avenues for future directions of this work.

Firstly, \citet{sussmann1992uniqueness} and \citet{fukumizu1996regularity} outlined conditions of the activation functions for which characterization of functional equivalent neural networks can be obtained.
This might be used to extend our results to other activation functions, including the sigmoid and the leaky-ReLU functions.
Secondly, the possibility of extending the results to deeper networks relies mainly on re-establishing Lemma $\ref{lem:characterization}$, which dictates that: \emph{there is no alternative weight vector with different input support but has the same input-output mapping as the generating model}.
We note that if this condition fails, then the feature selection problem is ill-defined and it is not possible to do feature selection in this scenario.
Further understanding about the landscape of deep neural networks will provide additional insights about when feature selection is possible and how our method can be adapted to tackle the problem.

\section*{Acknowledgement}

LSTH was supported by startup funds from Dalhousie University, the Canada Research Chairs program, the NSERC Discovery Grant RGPIN-2018-05447, and the NSERC Discovery Launch Supplement DGECR-2018-00181.

\clearpage

\bibliographystyle{chicago}
\bibliography{alg}

\clearpage

\onecolumn

\appendix
\section{Proofs}
\subsection{Lemma \ref{lem:almostLip}}

Since the hyperbolic tangent function is a bounded Lipschitz function and the weight space $\mc{W}$ is bounded, $f_{\alpha}(X)$ is also bounded.
Therefore,
\begin{align*}
|R(\alpha) - R(\alpha')| &\leq C_1 \| \alpha - \alpha' \| \cdot \mathbb{E}_{(X, Y)\sim P_{X,Y}^*}| f_{\alpha}(X) + f_{\alpha'}(X) - 2Y | \\
&\leq C_2 \| \alpha - \alpha' \| (C_2 + 2\mathbb{E}_{(X, Y)\sim P_{X,Y}^*}| Y - f_{\alpha^*}(X) | ) \\
&\leq c_0 \| \alpha - \alpha' \|.
\end{align*}
Similarly,
\begin{align*}
| R_n(\alpha) - R_n(\alpha') | & \leq C_1 \| \alpha - \alpha' \| \left ( C_2 + \frac{2}{n}\sum_{i=1}^n{| Y_i - f_{\alpha^*}(X_i) |} \right ) \\
&= C_1 \| \alpha - \alpha' \| \left ( C_2 + \frac{2}{n}\sum_{i=1}^n{| \epsilon_i |} \right ).
\end{align*}
The proof is completed by noting
\[
P \left (\frac{1}{n}\sum_{i=1}^n{| \epsilon_i |} > M \right ) \leq \frac{E | \epsilon_1 |}{M}
\]
that for any $M > 0$.

\subsection{Lemma \ref{lem:generalization}}

Note that $n R_n( \alpha)/\sigma^2_e$ follows a  non-central chi-squared distribution with $n$ degrees of freedom and $f_\alpha(X)$ is bounded.
By applying Theorem 7 in \citet{zhang2018}, we have
\begin{align*}
& \mathbb{P}\left[ | R_n( \alpha) - R(\alpha) | > t/2 \right] \\
& \leq  2\exp \left (- \frac{C_1 n^2 t^2}{n + 2 \sum_{i=1}^n{[f_\alpha(X) - f_{\alpha^*}(X)]^2} } \right ) \\
& \leq 2\exp(- C_2 n t^2),
\end{align*}
for all
\[
0 < t < \frac{n + \sum_{i=1}^n{[f_\alpha(X) - f_{\alpha^*}(X)]^2}}{n}.
\]
We define the events
\[
\mc{A}(\alpha, t) = \{|R_n( \alpha) - R(\alpha) | > t/2 \},
\]
\begin{align*}
\mc{B}(\alpha, t) = \{&\exists \alpha' \in \mc{W}~\text{such that}~\\
&\|\alpha'-\alpha\| \le \frac{t}{4M_\delta}~ \text{and}~ |R_n( \alpha') - R(\alpha') | > t \},
\end{align*}
and
\[
\mc{C} = \{ |R_n( \alpha) - R_n( \alpha')| \leq M_\delta \| \alpha - \alpha' \|, \forall \alpha, \alpha' \in \mc{W} \}.
\]
Here, $M_\delta$ is defined in Lemma \ref{lem:almostLip}.
By Lemma \ref{lem:almostLip}, $\mc{B}(\alpha, t) \cap \mc{C} \subset \mc{A}(\alpha, t)$ and $P(\mc{C}) \geq 1 - \delta$.

Let $m=dim(\mc{W})$, there exist $C_3(m) \ge 1$ and a finite set $\mathcal{H} \subset \mc{W}$ such that
\[
\mc{W} \subset \bigcup_{\alpha \in \mathcal{H}}{\mc{V}(\alpha, \epsilon)} \h \text{and}\h  |\mathcal{H}| \le C_3 /\epsilon^{m}
\]
where $\epsilon=t/(4M_\delta)$, $\mc{V}(\alpha, \epsilon)$ denotes the open ball centered at $\alpha$ with radius $\epsilon$, and $|\mathcal{H}|$ denotes the cardinality of $\mathcal{H}$.
By a union bound, we have
\[
\mathbb{P}\left[ \exists \alpha \in \mathcal{H}: \left |R_n( \alpha) - R(\alpha)\right |> t/2\right]  \le 2 \frac{C_3 (4M_\delta)^m}{t^m}e^{- C_2 n t^2}.
\]
Using the fact that $\mc{B}(\alpha, t) \cap \mc{C} \subset \mc{A}(\alpha, t), ~\forall \alpha \in \mathcal{H}$, we deduce
\[
\mathbb{P}\left[ \{ \exists \alpha  \in \mc{W}: \left |R_n( \alpha) - R(\alpha)\right |> t \right \} \cap \mc{C}]  \le C_4 t^{-m}e^{- C_2 n t^2}.
\]
Hence,
\[
\mathbb{P}\left[ \{ \exists \alpha  \in \mc{W}: \left |R_n( \alpha) - R(\alpha)\right |> t \right \}]  \le C_4 t^{-m}e^{- C_2 n t^2} + \delta.
\]

To complete the proof, we chose $t$ in such a way that $C_4 t^{-m}e^{- C_2 n t^2} ~ \le~ \delta$.
This can be done by choosing $t = \mathcal{O}(\log n /\sqrt{n})$.

\section{Simulation studies}

\begin{figure}[ht]
\centering
    \includegraphics[width=0.47\textwidth]{two_variables}
 \caption{False Discovery Rates (FDRs) of the non-significant variable in the first set of experiment with varying degrees of noise by the three feature selection algorithms:  the Group Lasso (red curves), the ERM + Adaptive Group Lasso (green), and Group Lasso + Adaptive Group Lasso (blue).}
\label{fig:2var}
\end{figure}

\section{Divorce Predictors Dataset }

This dataset is available on the UCI Machine Learning Repository (\url{https://archive.ics.uci.edu/ml/datasets/Divorce+Predictors+data+set}.) The list of $54$ statements of the data are
\begin{enumerate}
\item If one of us apologizes when our discussion deteriorates, the discussion ends.
\item I know we can ignore our differences, even if things get hard sometimes.
\item When we need it, we can take our discussions with my spouse from the beginning and correct it.
\item When I discuss with my spouse, to contact him will eventually work.
\item The time I spent with my wife is special for us.
\item We don't have time at home as partners.
\item We are like two strangers who share the same environment at home rather than family.
\item I enjoy our holidays with my wife.
\item I enjoy traveling with my wife.
\item Most of our goals are common to my spouse.
\item I think that one day in the future, when I look back, I see that my spouse and I have been in harmony with each other.
\item My spouse and I have similar values in terms of personal freedom.
\item My spouse and I have similar sense of entertainment.
\item Most of our goals for people (children, friends, etc.) are the same.
\item Our dreams with my spouse are similar and harmonious.
\item We're compatible with my spouse about what love should be.
\item We share the same views about being happy in our life with my spouse.
\item My spouse and I have similar ideas about how marriage should be.
\item My spouse and I have similar ideas about how roles should be in marriage.
\item My spouse and I have similar values in trust.
\item I know exactly what my wife likes.
\item I know how my spouse wants to be taken care of when she/he sick.
\item I know my spouse's favorite food.
\item I can tell you what kind of stress my spouse is facing in her/his life.
\item I have knowledge of my spouse's inner world.
\item I know my spouse's basic anxieties.
\item I know what my spouse's current sources of stress are.
\item I know my spouse's hopes and wishes.
\item I know my spouse very well.
\item I know my spouse's friends and their social relationships.
\item I feel aggressive when I argue with my spouse.
\item When discussing with my spouse, I usually use expressions such as `you always' or `you never'.
\item I can use negative statements about my spouse's personality during our discussions.
\item I can use offensive expressions during our discussions.
\item I can insult my spouse during our discussions.
\item I can be humiliating when we discussions.
\item My discussion with my spouse is not calm.
\item I hate my spouse's way of open a subject.
\item Our discussions often occur suddenly.
\item We're just starting a discussion before I know what's going on.
\item When I talk to my spouse about something, my calm suddenly breaks.
\item When I argue with my spouse, I only go out and I don't say a word.
\item I mostly stay silent to calm the environment a little bit.
\item Sometimes I think it's good for me to leave home for a while.
\item I'd rather stay silent than discuss with my spouse.
\item Even if I'm right in the discussion, I stay silent to hurt my spouse.
\item When I discuss with my spouse, I stay silent because I am afraid of not being able to control my anger.
\item I feel right in our discussions.
\item I have nothing to do with what I've been accused of.
\item I'm not actually the one who's guilty about what I'm accused of.
\item I'm not the one who's wrong about problems at home.
\item I wouldn't hesitate to tell my spouse about her/his inadequacy.
\item When I discuss, I remind my spouse of her/his inadequacy.
\item I'm not afraid to tell my spouse about her/his incompetence.
\end{enumerate}

\end{document}


TITLE: Could gravitons be dimensionless?
QUESTION [0 upvotes]: If the metric $g_{\mu\nu}$ is dimensionless (i.e. does not have associated dimensional units) and gravitons are quantum excitations of the metric does that mean that gravitons themselves are dimensionless?
I say this as locally the metric is just the flat metric $\eta_{\mu\nu}=\hbox{diag}(-1,1,1,1)$ with the dimensions in the co-ordinates $x^\mu$. Surely the coefficients in Pythagoras' theorem don't have units?
In terms of Newtonian gravitation we have the gravitational potential given by:
$$\Phi \sim -\frac{G M}{R}$$
In natural units, $\hbar=c=1$ (dimensionless), Newton's gravitational constant is $G=1/M_{pl}^2$ where $M_{pl}$ is the Planck mass. Therefore the dimensions of the gravitational potential field $\Phi$ is
$$[\Phi] = \frac{[M]^{-2}[M]}{[M]^{-1}}=1$$
If gravitons are excitations of $\Phi$ then they must themselves be dimensionless.
This is unlike other fields and their associated particles that have dimensions of mass/energy $[M]$.

REPLY [1 votes]: If the metric gμν is dimensionless and gravitons are quantum excitations of the metric does that mean that gravitons themselves are dimensionless?

It has dimensions , at least look here to a particular metric , the matrix elements have the dimensions of meter square.

Is graviton energy included in the stress-energy tensor Tμν?

In phsyics there are several frames where different theories describe mathematically the data and observations, and also predict new systems, but in mainstream physics they are consistent with one another in the region of overlap.
Metrics of General Relativity belong to large dimensions  large masses . GR is a classical theory, which means it is not quantized. The graviton is the hypothetical gauge boson of a quantized gravitational theory, expected in the future. At the moment only effective quantization of gravitation is used in mainstream cosmological models.
So it is mixing up apples and oranges to require GR to have gravitons included.

Actually classical gravitational waves can be detected so does that imply that gravitons can't be dimensionless?

It is not clear what you mean by dimensionless. All the particles in the standard model are point particles , thus have no dimensions. They are described when interacting with an energy and momentum four vector.
In string theories, which can quantize gravity, the graviton is part of the particle spectrum. If a definite theory is found then the graviton will be like the photon a zero dimension elementary particle.
Note that it is expected that mathematically, the quantized gravity theory will reduce to GR for large masses and dimensions. Here one can see the equivalent for electromagnetism, how classical electromagnetic fields emerge from the quantized fields of QED.
This emergent behavior happens in regions of phase space overlap between theories, for example: thermodynamics emerges from classical statistical mechanics. Otherwise physics would not be a consistent theoretical system.


TITLE: Suppose that $a \equiv b\mod m$ and that $a \equiv b\mod n $ .Assuming gcd$(m;n) = 1 $ ,prove that $a \equiv b\mod mn$
QUESTION [1 upvotes]: How should I approach this question? I was thinking of splitting it up such as as $ mt=b$ and $ax + ny=b$

REPLY [1 votes]: $ a  \equiv b \mod m $ means $m |(a-b)$
if $ m | (a-b)$ and $n | (a - b)$ then $lcm(m,n)|(a-b)$
since $lcm(m,n)gcd(m,n) = mn$ and $gcd(m,n) = 1$, we have $lcm(m,n) = mn$
so $mn | (a-b)$


TITLE: Representation of the mapping class group in terms of flips on triangulations
QUESTION [3 upvotes]: $\DeclareMathOperator{\MCG}{\operatorname{MCG}}$Consider a bordered, punctured, orientable surface $S$. Associated to it there is its mapping class group $\MCG(S)$. One way to concretely think about it is in terms of generators and relations, which can be done in many ways. Typically one chooses a set of closed simple curves on $S$, defines associated Dehn twists and half-twists and then gives set of relations among them in terms of how curves intersect on $S$.
Recently I discovered that there is a very different presentation of the mapping class group in terms of its actions on (isotopy classes of) triangulations on the surface, see for example Penner's book on decorated Teichmuller theory. Essentially, the idea is that a MCG element takes a triangulation to another one with the same combinatorics. Since triangulations are acted transitively by a set of moves (flips, quasi-flips and further generalization used in the context of cluster algebras), each mapping class group element is corresponds to a sequence of flips which connects two combinatorially equivalent triangulations. Furthermore, if one adds 2-dimensional faces corresponding to the so-called pentagon identities, the complex whose vertices are (isotopy classes of) triangulations and whose edges are flips one gets a simply connected 2-dim CW-complex.
By choosing any vertex in this complex, one can then think of the mapping class group as the set of paths that join this vertex to any other vertex labelled by a combinatorially equivalent triangulation, modulo the contractible paths (which are a combination of the pentagon identities).
My question is, is there an explicit/algorithmic way to produce a sequence-of-flips representative for a given Dehn twist?

REPLY [2 votes]: Yes there is an entirely constructive process to produce a sequence of flips (and a relabelling) from a labelled triangulation $\mathcal{T}$ to its image $D_\gamma(\mathcal{T})$ under a Dehn twist about a curve $\gamma$. You will need to specify $\gamma$ combinatorially, for example, by specifying the number of time it intersects each edge of $\mathcal{T}$. This is implemented within curver in the encode_twist method of a (multi)curve.
The trick is to first consider the case in which $\gamma$ appears very simply on $\mathcal{T}$. For example, orienting $\gamma$ and following along it you see that it enters each triangle and turns either left or right to exit. Assume that it only performs a single left turn (in which case we say that $\mathcal{T}$ is a short triangulation for $\gamma$). In this case it is easy to produce a sequence of flips: flip the edge in between $\gamma$ turning left and turning right (to get a new short triangulation), identify the new edge where $\gamma$ goes from turning left to turning right and flip that, and repeat this process (approximately) $\iota(\gamma, \mathcal{T})$ times. This forms the base case that curver implements, but it also has some more advanced patterns to allow it to compute powers of twists in log space.
If $\gamma$ does not appear simply on $\mathcal{T}$ then we can use a shortening process to find a short triangulation for $\gamma$. It turns out that there is a always a flip which reduces $\iota(\gamma, \mathcal{T})$. Again curver implements this process, again with some more advanced tricks to allow it to take advantage of accelerating twists. So by repeatedly applying using this you can produce a "conjugating" path to get to a base case triangulation, apply the base case implementation and then follow the "conjugating" path back. This is actually what is being done here.