TITLE: For a constant mass system, does breaking translation invariance imply breaking time invariance?
QUESTION [0 upvotes]: If a system broke translation invariance and hence momentum isn't conserved, would this not allow an object to increase its velocity relative to an observer that is initially stationary with respect to it, and hence its kinetic energy?
I understand that time and space are linked in general relativity, but does this not imply they are also linked in Newtonian physics (as momentum and energy conservation also arise in Newtonian physics from translation and time invariance)?
Anything that could help me understand what I'm misinterpreting would be greatly appreciated.

REPLY [2 votes]: Spatial translation invariance means that if initial conditions $(x,v)$ at $t=t_0$ give rise to a motion $x=x(t)$, then initial conditions $(x+s,v)$ at time $t_0$ give rise to a motion $x=x(t)+s$ for every spatial translation $s$.
Temporal invariance means that if initial conditions $(x,v)$ at $t=t_0$ give rise to a motion $x=x(t)$, then inital conditions $x,v$ at time $t_0+\tau$ give rise to a motion $x=x(t-\tau)$ for every temporal translation $\tau$.
There is no general relation between the two types of invariance. As an example, a mass $m>0$ attached to a spring of constant $k>0$ and length $x_0$ satisfies Newton equation $$m\frac{d^2x}{dt^2} = -k(x-x_0)\:.$$
If $k$>0 and $m>0$ are constant, this system is not invariant under spatial translations but is invariant under temporal translations in the sense pointed out above.
If we instead assume that $k=k_0t$ for some constant $k_0>0$ and $m>0$ constant, also temporal invariance is broken.
If $k=0$ and $m>0$ is constant,  both notions of invariance are valid.
If eventually we assume $k=0$ and $m=m_0t$ for some constant $m_0>0$, we finally have spatial invariance preserved and temporal invariance broken.
Alternatively, a system with spatial invariance preserved and temporal invariance broken is
$$m\frac{d^2x}{dt^2} = -k_0t\:,$$
where the mass $m$ is constant.


TITLE: Surface Tension - Lung Alveoli
QUESTION [5 upvotes]: So, the way I understand this is as follows :
The alveoli (pretend they're bubbles) have diameters of the order of microns implying a massive pressure required to inflate them by the Young-Laplace equation.
$p_{in}-p_{out}=\frac{2\gamma}{r}$
However, the presence of pulmonary surfactant molecules (lets just pretend they're like detergents molecules in washing liquid) can effectively reduce the surface tension at the unexpanded alveoli and hence allow easy inflation.
Now this bit I don't understand :
As the alveoli expand the distance between the individual surfactant molecules on the alveoli increases and hence the surface tension rises again therefore decreasing the rate of expansion.
What is the mathematical connection between surface tension and separation between surfactant molecules ? How can I rationalise the statement in bold ?

REPLY [1 votes]: Suppose the surface tension of a pure interface (without surfactants) is $\gamma_0$. Now adding the surfactants lowers the surface tension to $\gamma(a) = \gamma_0 - \Pi(a)$, where $a$ is the area per surfactant (or the inverse of the surface density; one can use either, but $a$ is more customary). The term $\Pi(a)$ is called the surface pressure precisely because it acts like a two-dimensional pressure. Let me elaborate.
Taking this view, then, the pure interface wants to contract, get smaller, to minimize the size of the interface. As the surfactants thus become ever closer to one another, they start repelling and create a force to the opposite direction. It is this force that is called the surface pressure.
In fact if you were to push "too many" surfactants to the interface very quickly (this assumption of very quicky is important, for otherwise the surfactants would jump off the interface), the surface tension would become negative and the surface would then tend to bend, or change its shape, so that the size of the surface would actually grow. This happens a lot in certain classes of systems (say, lipid bilayers, which are more or less connected to the monolayers that you are talking about when you talk about lung alveoli).
As to how to model $\Pi(a)$, the simplest models might be that of ideal gas, or van der Waals equations of state. Some more elaborate, but still quite elementary methods would be some lattice models (a la Langmuir adsorption model), and at the more advanced end you have density functional theory and fundamental measure theory and the stuff that branches off of them. The literature on the subject is huge.


TITLE: Proving that a language is regular and context-free
QUESTION [0 upvotes]: Let Σ be a finite alphabet and L ⊆ Σ be a language. Let Σ0 ⊆ Σ. For each string w = w1 · · · wn ∈ Σ , define res(w, Σ 0 ) = y1 · · · yn where yi = wi if wi ∈ Σ 0 , and yi =  if wi ∈ Σ \ Σ 0 , for each 1 ≤ i ≤ n. (For example res(abracadabra, {a, b}) = abaaaba.) Then define res(L, Σ 0 ) = {res(w, Σ 0 ) : w ∈ L}
I have a very general direction and thought process for these problems, however, i am not completely sure i could construct a viable proof for the following problems. If anyone could point me in the right direction, that would be much appreciated.
(a) Show that if L ⊆ Σ* ? is regular and Σ0 ⊆ Σ, res(L, Σ 0 ) must be regular.
i know for this one, a DFA cannot be made for this language. So i will have to figure out a different way. A subset of any regular language is not necessarily regular.
(b) Show that if L ⊆ Σ* is context-free and Σ0 ⊆ Σ, res(L, Σ 0 ) must be context-free.
I know that for this problem, one way to solve it would be to provide a context free grammar
(c) Show that if L ⊆ Σ* and res(L, Σ 0 ) is regular whenever Σ0 ⊂ Σ, L need not be regular.
For this question, we can say that even if L is not regular, that the empty language is a regular subset of this language. This would be true in all languages (whether it is regular or not)

REPLY [0 votes]: HINTS: For the first question, start with a right-regular grammar for $L$. Keep all productions of the form $X\to\epsilon$ and all productions of the form $X\to x$ and $X\to xY$ such that $x\in\Sigma_0$. If $x\in\Sigma\setminus\Sigma_0$, replace each production of the form $X\to x$ with $X\to\epsilon$; what should you do with the productions of the form $X\to xY$?
Alternatively, you could start with a DFA for $L$ and change it to an NFA for $\operatorname{res}(L,\Sigma_0)$ by replacing certain transitions with $\epsilon$-transitions; which ones?
For the second question you can adapt my first suggestion above to show how to modify a context-free grammar for $L$ to a context-free grammar for $\operatorname{res}(L,\Sigma_0)$.
For the third question try using $\Sigma=\{0,1\}$ and taking $L$ to be a very commonly used example of a context-free language that is not regular.


text:
\section{Properties of the interpolating mutual information}
We recall that $u_{y}(x) := \ln P_{\mathrm{out}}(y|x)$, and that $u'_{y}(\cdot)$ and $u''_{y}(\cdot)$ are the first and second derivatives of $u_y(\cdot)$.
We denote $P_{\mathrm{out}}'(y|x)$ and $P_{\mathrm{out}}''(y|x)$ the first and second derivatives of $x\mapsto P_{\mathrm{out}}(y|x)$.
Finally, the scalar overlap is $Q := \frac{1}{k_n}\sum_{i=1}^{n} X^*_i x_i$.
\subsection{Derivative of the interpolating mutual information}\label{appendix:derivative_interpolating_mutual_information}
\begin{proposition_derivative}

Suppose that $\Delta > 0$ and that all of \ref{hyp:bounded},~\ref{hyp:c2} and \ref{hyp:phi_gauss2} hold.
Further assume that $\E_{X \sim P_0}[X^2] = 1$.
The derivative of the interpolating mutual information \eqref{interpolating_mutual_information} with respect to $t$ satisfies for all $(t,\epsilon) \in [0,1] \times \mathcal{B}_n$:
\begin{multline}\label{eq:derivative_i_n(t)}
i_{n,\epsilon}'(t)
=  \mathcal{O}\bigg(\frac{1}{\sqrt{n \rho_n}}\bigg)
+ \mathcal{O}\bigg(\sqrt{\frac{\alpha_n}{\rho_n}\Var\,\frac{\ln {\cal Z}_{t,\epsilon}}{m_n}}\bigg)
+ \frac{\rho_n}{2\alpha_n}r_\epsilon(t) (1 - q_\epsilon(t))\\
+ \frac{1}{2} \E\,\bigg\langle \big(Q - q_\epsilon(t)\big) \bigg( \frac{1}{m_n}\sum_{\mu=1}^{m_n} u_{Y_\mu^{(t,\epsilon)}}'(S_\mu^{(t,\epsilon)}) u'_{Y_\mu^{(t,\epsilon)}}(s_\mu^{(t,\epsilon)}) - \frac{\rho_n}{\alpha_n}r_\epsilon(t)\bigg)\bigg\rangle_{\!\! n,t,\epsilon}\;,
\end{multline}
where
\begin{equation*}
\bigg\vert \mathcal{O}\bigg(\frac{1}{\sqrt{n \rho_n}}\bigg) \bigg\vert
\leq \frac{S^2 C}{\sqrt{n \rho_n}}
\quad\text{and}\quad
\bigg\vert \mathcal{O}\bigg(\sqrt{\frac{\alpha_n}{\rho_n}\Var \, \frac{\ln {\cal Z}_{t,\epsilon}}{m_n}}\bigg) \bigg\vert
\leq S^2 \sqrt{D \frac{\alpha_n}{\rho_n}\Var \, \frac{\ln {\cal Z}_{t,\epsilon}}{m_n}}\;;
\end{equation*}
with ($\partial_x \varphi$ and $\partial_{xx} \varphi$ denote the first and second partial derivatives of $\varphi$ with respect to its first argument):
\begin{align*}
C &:= \bigg\Vert \frac{\partial_x \varphi}{\sqrt{\Delta}} \bigg\Vert_\infty^2\bigg(64\bigg\Vert \frac{\varphi}{\sqrt{\Delta}} \bigg\Vert_\infty^4 + 2\bigg\Vert \frac{\varphi}{\sqrt{\Delta}} \bigg\Vert_\infty^2 + 12.5\bigg)
+ \bigg\Vert \frac{\partial_{xx} \varphi}{\sqrt{\Delta}} \bigg\Vert_\infty\bigg(8\bigg\Vert \frac{\varphi}{\sqrt{\Delta}} \bigg\Vert_\infty^3 + 2 \sqrt{\frac{2}{\pi}}\,\bigg)\;;\\
D &:= \bigg\Vert \frac{\partial_x \varphi}{\sqrt{\Delta}} \bigg\Vert_\infty^4 + \frac{1}{2}\bigg\Vert \frac{\partial_{xx} \varphi}{\sqrt{\Delta}} \bigg\Vert_\infty^2\;.
\end{align*}

In addition, if both sequences $(\alpha_n)_n$ and $(\nicefrac{\rho_n}{\alpha_n})_n$ are bounded, i.e., if there exist real positive numbers $M_\alpha, M_{\rho/\alpha}$ such that $\forall n \in \N^*: \alpha_n \leq M_{\alpha}, \nicefrac{\rho_n}{\alpha_n}  \leq  M_{\rho/\alpha}$ then for all $(t,\epsilon) \in [0,1] \times \mathcal{B}_n$:
\begin{multline}\label{eq:derivative_i_n(t)_bis}
i_{n,\epsilon}'(t)
=  \mathcal{O}\bigg(\frac{1}{\sqrt{n}\,\rho_n}\bigg)
+ \frac{\rho_n}{2\alpha_n}r_\epsilon(t) (1 - q_\epsilon(t))\\
+ \frac{1}{2} \E\,\bigg\langle \big(Q - q_\epsilon(t)\big) \bigg( \frac{1}{m_n}\sum_{\mu=1}^{m_n} u_{Y_\mu^{(t,\epsilon)}}'(S_\mu^{(t,\epsilon)}) u'_{Y_\mu^{(t,\epsilon)}}(s_\mu^{(t,\epsilon)}) - \frac{\rho_n}{\alpha_n}r_\epsilon(t)\bigg)\bigg\rangle_{\!\! n,t,\epsilon}\;,
\end{multline}
where
\begin{equation*}
\bigg\vert \mathcal{O}\bigg(\frac{1}{\sqrt{n}\,\rho_n}\bigg) \bigg\vert
\leq \frac{S^2 C + S^2 \sqrt{ D \big( \widetilde{C}_1 + M_{\rho/\alpha}\widetilde{C}_2 + M_\alpha \widetilde{C}_3\big)}}{\sqrt{n}\,\rho_n}\;.
\end{equation*}
Here $\widetilde{C}_1,\widetilde{C}_2, \widetilde{C}_3$ are the polynomials in $\big(S,\big\Vert \frac{\varphi}{\sqrt{\Delta}} \big\Vert_\infty, \big\Vert \frac{\partial_x \varphi}{\sqrt{\Delta}} \big\Vert_\infty, \big\Vert \frac{\partial_{xx} \varphi}{\sqrt{\Delta}} \big\Vert_\infty\big)$ defined in Proposition~\ref{prop:concentration_free_entropy}.
\end{proposition_derivative}

\begin{proof}
We recall that $\cZ_{t,\epsilon}$ is the normalization to the joint posterior density of $(\bX^*,\bW^*)$ given $(\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)$.
We define the average interpolating free entropy $f_{n,\epsilon}(t) := \nicefrac{\E\ln\cZ_{t,\epsilon}}{m_n}$.
Note that $i_{n,\epsilon}(t) := \nicefrac{I((\bX^*,\bW^*);(\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)})\vert\bm{\Phi},\bV)}{m_n}$ satisfies:
\begin{align*}
i_{n,\epsilon}(t)
&= -\frac{\E\ln\cZ_{t,\epsilon}}{m_n} + \frac{1}{m_n}\E\big[\ln\big(e^{-\frac{\Vert \widetilde{\bZ} \Vert^2}{2}} P_{\mathrm{out}}(Y_{\mu}^{(t,\epsilon)} \vert S_{\mu}^{(t,\epsilon)}) \big)\big]\\
&= -f_{n,\epsilon}(t) - \frac{1}{2\alpha_n}+ \E\big[\ln P_{\mathrm{out}}(Y_{1}^{(t,\epsilon)} \vert S_{1}^{(t,\epsilon)}) \big]
\end{align*}
Given $\bX^*$, $S_{1}^{(t,\epsilon)} \sim \mathcal{N}(0,V^{(t)})$ where $\rho^{(t)} := \frac{1-t}{k_n} \Vert \bX^* \Vert^2 + t + 2s_n$. Then:
\begin{equation*}
\E\,\ln P_{\mathrm{out}}(Y_{1}^{(t,\epsilon)} \vert S_{1}^{(t,\epsilon)})
= \E\big[\E[\ln P_{\mathrm{out}}(Y_{1}^{(t,\epsilon)} \vert S_{1}^{(t,\epsilon)}) \vert \bX^* ]\big]
= \E[h(\rho^{(t)})]\;,
\end{equation*}
where $h: \rho \in [0,+\infty) \mapsto \E_{V \sim \mathcal{N}(0,1)} \int u_y(\sqrt{\rho}\,V)e^{u_y(\sqrt{\rho}\,V)}dy$.
All in all, we have:
\begin{equation}\label{link_mutual_info_free_entropy}
i_{n,\epsilon}(t)
= \E[h(\rho^{(t)})] - f_{n,\epsilon}(t) - \frac{1}{2\alpha_n} \;.
\end{equation}
We directly obtain for the derivative of $i_{n,\epsilon}(\cdot)$:
\begin{equation}\label{link_derivatives_mutual_info_free_entropy}
i'_{n,\epsilon}(t)
= -\E\bigg[h'(\rho^{(t)})\bigg(\frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)\bigg] - f'_{n,\epsilon}(t)\;,
\end{equation}
where $h', f'_{n,\epsilon}$ are the derivatives of $h, f_{n,\epsilon}$.
In Lemma~\ref{lemma:property_I_Pout} of Appendix~\ref{appendix:scalar_channels}, we compute $h'$ and show:
\begin{equation*}
\forall \rho \in [0,+\infty): \vert h'(\rho) \vert \leq C := C\bigg(\bigg\Vert \frac{\varphi}{\sqrt{\Delta}} \bigg\Vert_\infty, \bigg\Vert \frac{\partial_x \varphi}{\sqrt{\Delta}} \bigg\Vert_\infty, \bigg\Vert \frac{\partial_{xx} \varphi}{\sqrt{\Delta}} \bigg\Vert_\infty\bigg)
\end{equation*}
with
$C(a,b,c) := b^2(64a^4 + 2a^2 + 12.5) + c\big(8a^3 + 2 \sqrt{\frac{2}{\pi}}\,\big)$. The first term on the right-hand side of \eqref{link_derivatives_mutual_info_free_entropy} thus satisfies:
\begin{equation}\label{upperbound_new_term_derivative}
\bigg\vert \E\bigg[h'(\rho^{(t)})\bigg(\frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)\bigg] \bigg\vert
\leq C \sqrt{\Var\bigg(\frac{\Vert \bX^* \Vert^2}{k_n}\bigg)}
= \frac{C}{k_n} \sqrt{n\Var\big((X_1^*)^2\big)}
= \frac{C S^2}{\sqrt{n \rho_n}} \;.
\end{equation}
We now turn to the computation of $f'_{n,\epsilon}$.
\paragraph{Derivative of the average interpolating free entropy}
Note that
\begin{equation}\label{formula_f_n(t)_for_derivative}
f_{n,\epsilon}(t)
= \frac{1}{m_n} \E\bigg[\int \frac{d\by d\widetilde{\by}}{\sqrt{2\pi}^n}  e^{-\cH_{t,\epsilon}(\bX^*,\bW^*;\by,\widetilde{\by},\bm{\Phi},\bV)}
\ln \int dP_{0,n}(\bx) \mathcal{D}\bw \, e^{-\cH_{t,\epsilon}(\bx,\bw;\by,\widetilde{\by},\bm{\Phi},\bV)}\bigg]
\end{equation}
where the expectation is over $\bX^*, \bm{\Phi}, \bV, \bW^*$, $\mathcal{D}\bw := \frac{d\bw e^{-\frac{\Vert \bw \Vert^2}{2}}}{\sqrt{2\pi}^{m_n}}$ and the Hamiltonian ${\cal H}_{t,\epsilon}$ is:
\begin{equation}

\cH_{t,\epsilon}(\bx,\bw;\by,\widetilde{\by},\bm{\Phi},\bV)
:=-\sum_{\mu=1}^{m_n} \ln P_{\mathrm{out}} ( y_\mu  \vert s_\mu^{(t, \epsilon)})
+ \frac{1}{2} \sum_{i=1}^{n}\big(\widetilde{y}_i  - \sqrt{R_{1}(t,\epsilon)}\, x_i\big)^2 \;.
\end{equation}
We will need its derivative ${\cal H}_{t,\epsilon}'$ with respect to $t$:
\begin{equation}\label{derivative_hamiltonian}
\cH_{t,\epsilon}'(\bx,\bw;\by,\widetilde{\by},\bm{\Phi},\bV)
:=
- \sum_{\mu=1}^{m_n}
\frac{\partial s_{\mu}^{(t,\epsilon)}}{\partial t} u'_{y_\mu}( s_\mu^{(t,\epsilon)})
- \frac{r_\epsilon(t)}{2\sqrt{R_1(t,\epsilon)}} \sum_{i=1}^{n} x_i (\widetilde{y}_{i} - \sqrt{R_1(t,\epsilon)} x_i)\:.
\end{equation}
The derivative of $f_{n,\epsilon}$ can be obtained by differentiating \eqref{formula_f_n(t)_for_derivative} under the expectation:
\begin{align}
f_{n,\epsilon}'(t)
&= -\frac{1}{m_n}  \E \big[\cH_{t,\epsilon}'(\bX^*,\bW^*;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)\ln \cZ_{t,\epsilon}
	\big]\nonumber\\
&\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad
-\frac{1}{m_n} \E \big\langle \cH_{t,\epsilon}'(\bx,\bw;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV) \big\rangle_{n,t,\epsilon}\nonumber\\
&= -\frac{1}{m_n}  \E \big[\cH_{t,\epsilon}'(\bX^*,\bW^*;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)\ln \cZ_{t,\epsilon}
\big]\nonumber\\
&\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad
-\frac{1}{m_n} \E[\cH_{t,\epsilon}'(\bX^*,\bW^*;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)] \;.\label{formula_derivative_free_entropy}
\end{align}
The last equality follows from the Nishimory identity
\begin{equation*}
\E\,\langle \cH_{t,\epsilon}'(\bx,\bw;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV) \rangle_{n,t,\epsilon} = \E[\cH_{t,\epsilon}'(\bX^*,\bW^*;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)] \;.
\end{equation*}
Evaluating \eqref{derivative_hamiltonian} at $(\bx,\bw;\by,\widetilde{\by},\bm{\Phi},\bV) = (\bX^*,\bW^*;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)$ yields:
\begin{equation}\label{derivative_hamiltonian_at_ground_truth}
\cH_{t,\epsilon}'(\bX^*,\bW^*;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)
= -\!\sum_{\mu=1}^{m_n}
\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})
- \frac{r_\epsilon(t)}{2\sqrt{R_1(t,\epsilon)}} \sum_{i=1}^{n} X_i^* \widetilde{Z}_{i} \,.
\end{equation}
The expectation of \eqref{derivative_hamiltonian_at_ground_truth} is zero:
\begin{align*}
\E\,\cH_{t,\epsilon}'(\bX^*,\bW^*;\bY^{(t,\epsilon)},\widetilde{\bY}^{(t,\epsilon)},\bm{\Phi},\bV)
&= - \sum_{\mu=1}^{m_n} \E\bigg[\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})\bigg]\\
&= - \sum_{\mu=1}^{m_n} \E\bigg[\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} \E\Big[u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)}) \Big\vert \bX^*, \bW^*, \bV, \bm{\Phi}\Big]\bigg]\\
&= - \sum_{\mu=1}^{m_n} \E\bigg[\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} \int u'_{y}(S_\mu^{(t,\epsilon)}) P_{\mathrm{out}}(y \,\vert\, S_\mu^{(t,\epsilon)})dy\bigg]\\
&= - \sum_{\mu=1}^{m_n} \E\bigg[\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} \int P'_{\mathrm{out}}(y \,\vert\, S_\mu^{(t,\epsilon)})dy\bigg]\\
&=0\;.
\end{align*}
The last equality is because for all $x$:
\begin{equation*}
\int P'_{\mathrm{out}}(y \,\vert\, x)dy = \int dP_A(\ba)\partial_x\varphi(x,\ba) \int \frac{y - \varphi(x,\ba)}{\Delta}\frac{ e^{-\frac{(y - \varphi(x,\ba))^2}{2 \Delta}} }{\sqrt{2\pi \Delta}}dy = 0 \;.
\end{equation*}
The expectation of \eqref{derivative_hamiltonian_at_ground_truth} being zero, the identity \eqref{formula_derivative_free_entropy} reads:
\begin{equation}\label{second_formula_derivative_free_entropy}
f_{n,\epsilon}'(t)
= \frac{1}{m_n} \sum_{\mu=1}^{m_n} \E\bigg[\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})\ln \cZ_{t,\epsilon}\bigg]\\
+\frac{1}{m_n}\frac{r_\epsilon(t)}{2\sqrt{R_1(t,\epsilon)}} \sum_{i=1}^{n} \E\big[X_i^* \widetilde{Z}_i\ln \cZ_{t,\epsilon}\big] \;.
\end{equation}
First, we compute the first kind of expectation on the right-hand side of \eqref{second_formula_derivative_free_entropy}. $\forall \mu \in \{1,\dots,m_n\}$:
\begin{multline}
\E\bigg[\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})\ln \cZ_{t,\epsilon}\bigg]\\
=\frac{1}{2}
	\E \Big[\bigg(
			- \frac{(\bm{\Phi} \bX^*)_{\mu}}{\sqrt{k_n (1-t)}}
			+ \frac{q_\epsilon(t)V_{\mu}}{ \sqrt{R_2(t,\epsilon)}}
			+ \frac{(1 - q_\epsilon(t)) W^*_{\mu}}{\sqrt{t + 2s_n - R_2(t,\epsilon) }}
	\bigg) u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)}) \ln \cZ_{t,\epsilon}\bigg]\,. \label{1st_expectation_derivative_f_n(t)}
\end{multline}
An integration by parts w.r.t.\ the independent standard Gaussians $(\Phi_{\mu i})_{i=1}^n$ yields:
\begin{align}
&\E\bigg[\frac{(\bm{\Phi} \bX^*)_{\mu}}{\sqrt{k_n(1-t)}}
			u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})
			\ln \cZ_{t,\epsilon}\bigg] \nonumber\\
&=\sum_{i=1}^n\E \bigg[ \frac{\Phi_{\mu i} X_i^*}{\sqrt{k_n(1-t)}} \!\int\!\! d\by d\widetilde{\by} \,
			u_{y_\mu}' (S_\mu^{(t,\epsilon)}) e^{-\cH_{t,\epsilon}(\bX^*,\bW^*;\by,\widetilde{\by},\bm{\Phi},\bV)}
			\ln\! \int\!\! dP_{0,n}(\bx) \mathcal{D}\bw \, e^{-\cH_{t,\epsilon}(\bx,\bw;\by,\widetilde{\by},\bm{\Phi},\bV)} \bigg]\nonumber\\
&= \sum_{i=1}^{n}
	\E\bigg[\frac{(X_i^*)^2}{k_n}\big(u_{Y_\mu^{(t,\epsilon)}}''(S_\mu^{(t,\epsilon)}) + u_{Y_\mu^{(t,\epsilon)}}' (S_\mu^{(t,\epsilon)})^2\big) \ln \cZ_{t,\epsilon}
		+ \frac{X_i^* u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})}{k_n}
	\big\langle x_i u'_{Y_\mu^{(t,\epsilon)}} (s_\mu^{(t,\epsilon)})\big\rangle_{n,t,\epsilon}\bigg]\nonumber\\
&= \E\bigg[ \frac{\Vert \bX^* \Vert^2}{k_n}
			\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
			\ln \cZ_{t,\epsilon}\bigg]
+ \E\,\Big\langle Q \, u_{Y_\mu^{(t,\epsilon)}}'(S_\mu^{(t,\epsilon)}) u'_{Y_\mu^{(t,\epsilon)}}(s_\mu^{(t,\epsilon)}) \Big\rangle_{n,t,\epsilon}\;,\label{eq:compA1}
\end{align}
where, in the last equality, we used the identity $u_{y}'' ( x ) + u_{y}' ( x )^2 = \frac{P_{\mathrm{out}}''(y | x)}{P_{\mathrm{out}}(y | x)}$.
Another Gaussian integration by parts, this time with respect to \ $V_\mu \sim {\cal N}(0,1)$, gives:
\begin{align}
&\E\bigg[\frac{q_\epsilon(t)V_{\mu}}{ \sqrt{R_2(t,\epsilon)}} u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})
\ln \cZ_{t,\epsilon}\bigg] \nonumber\\
&\quad
= \E \bigg[ \frac{q_\epsilon(t)V_{\mu}}{ \sqrt{R_2(t,\epsilon)}}  \int d\by d\widetilde{\by} \,
u_{y_\mu}' (S_\mu^{(t,\epsilon)}) e^{-\cH_{t,\epsilon}(\bX^*,\bW^*;\by,\widetilde{\by},\bm{\Phi},\bV)}
\ln \int dP_{0,n}(\bx) \mathcal{D}\bw \, e^{-\cH_{t,\epsilon}(\bx,\bw;\by,\widetilde{\by},\bm{\Phi},\bV)} \bigg]\nonumber\\
&\quad
= \E\bigg[q_\epsilon(t)\big(u_{Y_\mu^{(t,\epsilon)}}''(S_\mu^{(t,\epsilon)}) + u_{Y_\mu^{(t,\epsilon)}}' (S_\mu^{(t,\epsilon)})^2\big) \ln \cZ_{t,\epsilon}
+ q_\epsilon(t) u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})
\big\langle u'_{Y_\mu^{(t,\epsilon)}} (s_\mu^{(t,\epsilon)})\big\rangle_{n,t,\epsilon}\bigg]\nonumber\\
&\quad
= \E\bigg[ q_\epsilon(t)
\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\ln \cZ_{t,\epsilon}\bigg]
+ \E\,\Big\langle q_\epsilon(t)  u_{Y_\mu^{(t,\epsilon)}}'(S_\mu^{(t,\epsilon)}) u'_{Y_\mu^{(t,\epsilon)}}(s_\mu^{(t,\epsilon)}) \Big\rangle_{n,t,\epsilon}\;,\label{eq:compA2}
\end{align}
Finally, a Gaussian integration by part w.r.t.\ $W_\mu^* \sim {\cal N}(0,1)$ gives:
\begin{equation}\label{eq:compA3}
\E\bigg[\frac{(1 - q_\epsilon(t)) W^*_{\mu}}{\sqrt{t + 2s_n - R_2(t,\epsilon) }} u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})
\ln \cZ_{t,\epsilon}\bigg]
= \E\bigg[ (1 - q_\epsilon(t))
\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\ln \cZ_{t,\epsilon}\bigg]\;.
\end{equation}
Plugging \eqref{eq:compA1}, \eqref{eq:compA2} and \eqref{eq:compA3} back in \eqref{1st_expectation_derivative_f_n(t)}, we obtain:
\begin{multline}\label{1st_expectation_derivative_f_n(t)_simplified}
\E\bigg[\frac{\partial S_{\mu}^{(t,\epsilon)}}{\partial t} u'_{Y_\mu^{(t,\epsilon)}}(S_\mu^{(t,\epsilon)})\ln \cZ_{t,\epsilon}\bigg]
= -\frac{1}{2}\E\bigg[
\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)\ln \cZ_{t,\epsilon}\bigg]\\
-\frac{1}{2} \E\,\Big\langle \big(Q - q_\epsilon(t)\big)  u_{Y_\mu^{(t,\epsilon)}}'(S_\mu^{(t,\epsilon)}) u'_{Y_\mu^{(t,\epsilon)}}(s_\mu^{(t,\epsilon)}) \Big\rangle_{\! n,t,\epsilon}\;.
\end{multline}
It remains to compute the first kind of expectation on the right-hand side of \eqref{second_formula_derivative_free_entropy}, i.e.,
\begin{align}
\E\big[X_i^* \widetilde{Z}_i \ln \cZ_{t,\epsilon}\big]
&= \E \Big[X_i^* \widetilde{Z}_i  \ln \int dP_{0,n}(\bx) {\cal D}\bw\,
P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} | s_\mu^{(t,\epsilon)}) e^{-\sum_{i=1}^{n}\frac{(\sqrt{R_1(t,\epsilon)}(X^*_{i}-x_i) + \widetilde{Z}_i )^2}{2} }
\Big]\nonumber\\
&= -\E \big[X_i^* \big\langle \sqrt{R_1(t,\epsilon)}(X_i^* - x_i) + \widetilde{Z}_i \big\rangle_{n,t,\epsilon} \big]\nonumber\\
&= -\sqrt{R_1(t,\epsilon)} \E\,\big\langle (\rho_n - X_i^* x_i) \big\rangle_{n,t,\epsilon}\;.\label{2nd_expectation_derivative_f_n(t)_simplified}
\end{align}
The second equality follows from a Gaussian integration by parts w.r.t.\ $\widetilde{Z}_i \sim \mathcal{N}(0,1)$.
Plugging the two simplified expectations \eqref{1st_expectation_derivative_f_n(t)_simplified} and \eqref{2nd_expectation_derivative_f_n(t)_simplified} back in \eqref{second_formula_derivative_free_entropy} yields:
\begin{multline}\label{final_formula_f'_n(t)}
f_{n,\epsilon}'(t)
= -\frac{\rho_n}{2\alpha_n}r_\epsilon(t) (1 - q_\epsilon(t))
-\frac{1}{2}\E\bigg[
\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)\frac{\ln \cZ_{t,\epsilon}}{m_n} \bigg]\\
-\frac{1}{2} \E\,\bigg\langle \big(Q - q_\epsilon(t)\big) \bigg( \frac{1}{m_n}\sum_{\mu=1}^{m_n} u_{Y_\mu^{(t,\epsilon)}}'(S_\mu^{(t,\epsilon)}) u'_{Y_\mu^{(t,\epsilon)}}(s_\mu^{(t,\epsilon)}) - \frac{\rho_n}{\alpha_n}r_\epsilon(t)\bigg)\bigg\rangle_{\!\! n,t,\epsilon}\;.
\end{multline}
The last step to end the proof of the proposition is to upper bound
\begin{equation}\label{def:A_n}
A_n^{(t,\epsilon)}
:= \E\bigg[
\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)\frac{\ln \cZ_{t,\epsilon}}{m_n} \bigg]
\end{equation}
which appears on the right-hand side of \eqref{final_formula_f'_n(t)}.
\paragraph{Upper bouding the quantity \eqref{def:A_n}}
Remember that $u_{y}'' ( x ) + u_{y}' ( x )^2 = \frac{P_{\mathrm{out}}''(y | x)}{P_{\mathrm{out}}(y | x)}$ and $P_{\mathrm{out}}(y | x) = e^{u_y(x)}$.
Therefore, $\forall x$:
\begin{equation*}
\int_{-\infty}^{+\infty} P_{\mathrm{out}}''(y|x)dy = \int_{-\infty}^{+\infty} (u_{y}^{\prime\prime}(x) + u_{y}^{\prime}(x)^2) e^{u_y(x)}dy = 0 \;,
\end{equation*}
where the second equality follows from the direct computation \eqref{int_u''+u'^2=0} in Lemma~\ref{lemma:property_I_Pout} of Appendix~\ref{appendix:scalar_channels}.
Consequently, using the tower property of the conditionnal expectation, for all $\mu \in \{1 ,\dots, m \}$:

\begin{align}
\E\bigg[
\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\bigg(\!\frac{\Vert \bX^* \Vert^2}{k_n} - 1 \!\bigg) \bigg]
&=
\E\bigg[\bigg(\!\frac{\Vert \bX^* \Vert^2}{k_n} - 1\!\bigg)
\sum_{\mu=1}^{m_n} \E\bigg[\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})} \, \bigg\vert \bX^*, \bS^{(t,\epsilon)} \bigg] \bigg]\nonumber\\
&=
\E\bigg[\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)
\sum_{\mu=1}^{m_n} \int_{-\infty}^{+\infty} P_{\mathrm{out}}''(y | S_\mu^{(t,\epsilon)}) dy \bigg]
=0 \;.\label{A_n_witout_free_entropy}
\end{align}

Making use of \eqref{A_n_witout_free_entropy} and Cauchy-Schwarz inequality, we have:
\begin{align}
\vert A_n^{(t,\epsilon)} \vert
&= \bigg\vert \E\bigg[
\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)\bigg(\frac{\ln \cZ_{t,\epsilon}}{m_n}-f_{n,\epsilon}(t)\bigg) \bigg] \bigg\vert \nonumber\\
&\leq
\E\bigg[
\bigg(\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}\bigg)^{\!\! 2}
\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)^{\!\! 2}\,\bigg]^{\frac{1}{2}}\,
\sqrt{\Var\,\frac{\ln \cZ_{t,\epsilon}}{m_n} }\;.\label{1st_upperbound_An}
\end{align}
Using again the tower property of the conditional expectation gives:
\begin{multline}
\E\bigg[
\bigg(\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}\bigg)^{\!\! 2}
\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)^{\!\! 2}\,\bigg]\\
= \E\Bigg[\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)^{\!\! 2}
\E\bigg[
\bigg(\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}\bigg)^{\!\! 2}
\, \bigg\vert \, \bX^*, \bS^{(t,\epsilon)} \bigg]\Bigg]\;. \label{eq:tower_property}
\end{multline}
Note that conditionally on $\bS^{(t,\epsilon)}$ the random variables $\big(\nicefrac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}
\big)_{\mu =1}^{m_n}$ are i.i.d.\ and centered.
Therefore:
\begin{align}
\E\bigg[
\bigg(\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}\bigg)^{\!\! 2}
\, \bigg\vert \bX^*, \bS^{(t,\epsilon)} \bigg]
&=	\E\bigg[
\bigg(\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}\bigg)^{\!\! 2}
\, \bigg\vert  \bS^{(t,\epsilon)} \bigg]\nonumber\\
&= m_n \E\bigg[
\bigg(\frac{P_{\mathrm{out}}''(Y_1^{(t,\epsilon)} | S_1^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_1^{(t,\epsilon)} \vert S_1^{(t,\epsilon)})}\bigg)^{\!\! 2}
\, \bigg\vert  \bS^{(t,\epsilon)} \bigg]\nonumber\\
&= m_n \E\bigg[\int_{-\infty}^{+\infty} \frac{P_{\mathrm{out}}''(y| S_1^{(t,\epsilon)})^2}{P_{\mathrm{out}}(y \vert S_1^{(t,\epsilon)})} dy\bigg] \,.\label{eq:var_simp}
\end{align}
We now use the formula \eqref{formula_u''+u'^2} for $u_{y}'' ( x ) + u_{y}' ( x )^2 = \nicefrac{P_{\mathrm{out}}''(y | x)}{P_{\mathrm{out}}(y | x)}$ (obtained in Lemma~\ref{lemma:property_I_Pout} of Appendix~\ref{appendix:scalar_channels}) with Jensen's equality to show that for all $x$:
\begin{align*}
\bigg(\frac{P_{\mathrm{out}}''(y | x)}{P_{\mathrm{out}}(y | x)}\bigg)^{\!\! 2}
&\leq \frac{\int \!\big(\frac{(y- \varphi(x,\ba))^2\partial_x\varphi(x,\ba)^2
		-\Delta \partial_x\varphi(x,\ba)^2
		+ \Delta \partial_{xx}\varphi(x,\ba)(y - \varphi(x,\ba))}{\Delta^2}\big)^2
	\frac{dP_A(\ba)}{\sqrt{2\pi \Delta}} e^{-\frac{(y - \varphi(x,\ba))^2}{2 \Delta}}}{\int \frac{dP_A(\ba)}{\sqrt{2\pi \Delta}} e^{-\frac{(y - \varphi(x,\ba))^2}{2 \Delta}}}\\
&= \frac{\int \!\big(\frac{(y- \varphi(x,\ba))^2\partial_x\varphi(x,\ba)^2
		-\Delta \partial_x\varphi(x,\ba)^2
		+ \Delta \partial_{xx}\varphi(x,\ba)(y - \varphi(x,\ba))}{\Delta^2}\big)^2
	\frac{dP_A(\ba)}{\sqrt{2\pi \Delta}} e^{-\frac{(y - \varphi(x,\ba))^2}{2 \Delta}}}{P_{\mathrm{out}}(y | x)}\;.
\end{align*}
It follows that for all $x$:
\begin{align*}
\int_{-\infty}^{+\infty} \frac{P_{\mathrm{out}}''(y | x)^2}{P_{\mathrm{out}}(y | x)} dy
&= \int dP_A(\ba) \int_{-\infty}^{+\infty} \bigg(\frac{(u^2 - 1)\partial_x\varphi(x,\ba)^2
		+ \sqrt{\Delta} \partial_{xx}\varphi(x,\ba)u}{\Delta}\bigg)^{\!\! 2}
	\frac{du}{\sqrt{2\pi}} e^{-\frac{u^2}{2}}\\
&\leq  4\bigg\Vert \frac{\partial_x\varphi}{\sqrt{\Delta}} \bigg\Vert_{\infty}^4
		+ 2\bigg\Vert \frac{\partial_{xx}\varphi}{\sqrt{\Delta}} \bigg\Vert_{\infty}^2 \;.
\end{align*}
Let $D := \big\Vert \frac{\partial_x\varphi}{\sqrt{\Delta}} \big\Vert_{\infty}^4
+ \frac{1}{2}\big\Vert \frac{\partial_{xx}\varphi}{\sqrt{\Delta}} \big\Vert_{\infty}^2$. Combining this last upper bound with \eqref{eq:var_simp} and \eqref{eq:tower_property} yields:
\begin{align}
\E\bigg[
\bigg(\sum_{\mu=1}^{m_n}\frac{P_{\mathrm{out}}''(Y_\mu^{(t,\epsilon)} | S_\mu^{(t,\epsilon)})}{P_{\mathrm{out}}(Y_\mu^{(t,\epsilon)} \vert S_\mu^{(t,\epsilon)})}\bigg)^{\!\! 2}
\bigg( \frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)^{\!\! 2}\,\bigg]
\leq 4D\,m_n \Var\bigg(\frac{\Vert \bX^* \Vert^2}{k_n} \bigg)
= \frac{4D \alpha_n S^4}{\rho_n}
\label{eq:borne_ddp}
\end{align}
Going back to \eqref{1st_upperbound_An}, we have $\forall (t,\epsilon) \in [0,1] \times \mathcal{B}_n$:
\begin{equation}\label{final_upperbound_A_n}
\vert A_n^{(t,\epsilon)} \vert
\leq
2S^2
\sqrt{D \frac{\alpha_n}{\rho_n} \Var\,\frac{\ln \cZ_{t,\epsilon}}{m_n} }\;.
\end{equation}

\paragraph{Putting everything together: proofs of \eqref{eq:derivative_i_n(t)} and \eqref{eq:derivative_i_n(t)_bis}}
Combining \eqref{link_derivatives_mutual_info_free_entropy} and \eqref{final_formula_f'_n(t)} yields the following formula for the derivative of $i_{n,\epsilon}$ (remember the definition \eqref{def:A_n} of $A_n^{(t,\epsilon)}$):
\begin{multline}\label{formula_derivative_i_{n,epsilon}}
i'_{n,\epsilon}(t)
= \frac{A_n^{(t,\epsilon)}}{2}-\E\bigg[h'(\rho^{(t)})\bigg(\frac{\Vert \bX^* \Vert^2}{k_n} - 1\bigg)\bigg]
+\frac{\rho_n}{2\alpha_n}r_\epsilon(t) (1 - q_\epsilon(t))\\
+\frac{1}{2} \E\,\bigg\langle \big(Q - q_\epsilon(t)\big) \bigg( \frac{1}{m_n}\sum_{\mu=1}^{m_n} u_{Y_\mu^{(t,\epsilon)}}'(S_\mu^{(t,\epsilon)}) u'_{Y_\mu^{(t,\epsilon)}}(s_\mu^{(t,\epsilon)}) - \frac{\rho_n}{\alpha_n}r_\epsilon(t)\bigg)\bigg\rangle_{\!\! n,t,\epsilon}\;.
\end{multline}
Combining the identity \eqref{formula_derivative_i_{n,epsilon}} with the upper bounds \eqref{upperbound_new_term_derivative} and \eqref{final_upperbound_A_n} yields \eqref{eq:derivative_i_n(t)}.

It remains to prove the identity \eqref{eq:derivative_i_n(t)_bis} that holds under the additional assumption that $\forall n: \alpha_n \leq M_{\alpha}, \nicefrac{\rho_n}{\alpha_n}  \leq  M_{\rho/\alpha}$.
Combining \eqref{final_upperbound_A_n} with the upper bound \eqref{eq:concentration_free_entropy_bis} on the variance of $\Var(\nicefrac{\ln \cZ_{t,\epsilon}}{m_n})$
(see Proposition~\ref{prop:concentration_free_entropy} of Appendix~\ref{appendix:concentration_free_entropy}) gives:
\begin{equation*}
\bigg\vert \frac{A_n^{(t,\epsilon)}}{2}\bigg\vert
\leq
\frac{S^2\sqrt{D(\widetilde{C}_1 + M_{\rho/\alpha}\widetilde{C}_2 + M_{\alpha}\widetilde{C}_3)}}{\sqrt{n}\rho_n} \;.
\end{equation*}
The constants $\widetilde{C}_1,\widetilde{C}_2,\widetilde{C}_3$ are defined in Proposition~\ref{prop:concentration_free_entropy} while $D$ has been defined earlier in the proof.

Besides, as $\rho_n \leq 1$, we have $\frac{1}{\sqrt{n \rho_n}} \leq \frac{1}{\sqrt{n} \rho_n}$ and we can loosen the upper bound \eqref{upperbound_new_term_derivative}:
$\Big\vert \E\Big[h'(\rho^{(t)})\big(\frac{\Vert \bX^* \Vert^2}{k_n} - 1\big)\Big] \Big\vert
\leq \frac{C S^2}{\sqrt{n} \rho_n}$.
Then, the term $\nicefrac{A_n^{(t,\epsilon)}}{2}-\E\big[h'(\rho^{(t)})(\nicefrac{\Vert \bX^* \Vert^2}{k_n} - 1)\big]$ on the right-hand side of \eqref{formula_derivative_i_{n,epsilon}} is in $\mathcal{O}(\nicefrac{1}{\sqrt{n}\rho_n})$
and this proves the identity \eqref{eq:derivative_i_n(t)_bis}.
\end{proof}

config: arxiv