text:
\begin{document}

\date{}

\maketitle

\begin{abstract}
In many applications, a state-space model depends on a parameter which needs to be inferred from
data in an online manner. In the maximum likelihood approach, this can be achieved using stochastic gradient search,
where the underlying gradient estimation is based on the optimal filter and the optimal filter derivative.
However, the optimal filter and its derivative are not analytically tractable
for a non-linear state-space model and need to be approximated numerically.
In \cite{poyiadjis&doucet&singh}, a particle approximation to this derivative
has been proposed, while the corresponding central limit theorem and $L_{p}$ error bounds
have been established in \cite{delmoral&doucet&singh2}.
We derive here bounds on the bias of this particle approximation.
Under mixing conditions, these bounds are uniform in time and inversely proportional to the number of particles.
The obtained results apply to a relatively broad class of state-space models.
\end{abstract}

\begin{keywords}
Particle Methods, Bias,
Optimal Filter, Optimal Filter Derivative,
Non-Linear State-Space Models.
\end{keywords}

\begin{AMS}
Primary 93E11; Secondary 62M20, 65C05.
\end{AMS}

\section{Introduction}\label{section0}

State-space models, also known as continuous-state hidden Markov models,
are a class of stochastic processes used to model complex time-series data
and stochastic dynamical systems.
A state-space model can be described as a latent discrete-time Markov process
observed only through noisy measurements of its states.
In this context, one of the most important problems is the optimal estimation
of the present state given the noisy observations of the present and past states.
This problem is known as optimal filtering.
For non-linear state-space models, the optimal filter does not typically admit a closed-form expression
and needs to be approximated numerically.
Numerous numerical methods for optimal filtering have been proposed and studied
in the literature --- see e.g., \cite{crisan&rozovskii} for a recent overview.
Among them, particle methods (also known as sequential Monte Carlo methods)
have gained significant attention.
Their convergence properties have been thoroughly investigated in a number of papers and books
--- see, e.g.,
\cite{cappe&moulines&ryden}, \cite{crisan&rozovskii}, \cite{delmoral1},
\cite{delmoral2}, \cite{douc&moulines&stoffer},
\cite{doucet&defreitas&gordon}.

In many scenarios of practical interest, a state-space model depends on a parameter
whose value needs to be estimated given a set of observations.
When the number of observations is very large,
it is desirable, for the sake of computational efficiency, to perform parameter estimation online.
In the maximum likelihood approach, this can be achieved using stochastic gradient search,
where the corresponding gradient estimator is a non-linear functional of the optimal filter and its derivative
--- see, e.g., \cite{kantas&doucet&singh&maciejowski&chopin},
\cite{legland&mevel2}, \cite{olsson&westerborn}, \cite{poyiadjis&doucet&singh}.
Since the optimal filter derivative is analytically intractable for non-linear
state-space models, it needs to be approximated numerically.
To the best of our knowledge, only the particle approximations to the optimal filter derivative
proposed in \cite{olsson&westerborn} and \cite{poyiadjis&doucet&singh} are numerically stable.
In the particle estimator proposed in \cite{olsson&westerborn},
the average iteration complexity is linear in the number of particles,
while the iteration running times are random.
In \cite{olsson&westerborn}, concentration inequalities and a central limit theorem have been shown
for this scheme.
The particle estimator proposed in \cite{poyiadjis&doucet&singh} has quadratic iteration complexity,
deterministic iteration running times and lower variance than the estimator in \cite{olsson&westerborn}.
In \cite{delmoral&doucet&singh2},
$L_{p}$ error bounds and a central limit theorem have been established for the scheme proposed
in \cite{delmoral&doucet&singh2}.

In this paper, we analyze the bias of the particle approximation to the optimal filter derivative
proposed in \cite{poyiadjis&doucet&singh}. Using the stability properties of the optimal filter and its derivative,
we derive bounds on this bias in terms of the number of particles.
These bounds cover several classes of state-space models met in practice.
Moreover, under mixing conditions, these bounds are uniform in time and
inversely proportional to the number of particles.
To the best of our knowledge, the results presented here are the first results on
the bias of the particle approximation to the optimal filter derivative proposed in \cite{poyiadjis&doucet&singh}.
They are also one of the first and most important stepping stones to analyze the asymptotic properties of online maximum likelihood
estimation in non-linear state-space models --- see \cite{tadic&doucet2}.

The rest of this paper is organized as follows.
In Section \ref{section1}, we define the optimal filter derivative and its particle approximation.
In the same section, we present the main results of the paper.
These results are proved in Sections \ref{section2} -- \ref{section4}.

\section{Main Results}\label{section1}

\subsection{State-Space Models, Optimal Filter and Optimal Filter Derivative}\label{ssection1.1}

To define state-space models and state the optimal filtering problem, we use the following notation.
For a set ${\cal Z}$ in a finite dimensional space,
${\cal B}({\cal Z} )$ denotes the collection of Borel subsets of ${\cal Z}$.
$d_{x}\geq 1$ and $d_{y}\geq 1$ are integers,
while ${\cal X}\in{\cal B}(\mathbb{R}^{d_{x}} )$ and
${\cal Y}\in{\cal B}(\mathbb{R}^{d_{y}} )$.
Let $(\Omega,{\cal F},P)$ be a probability space.
A state-space model can be described as
an ${\cal X}\times{\cal Y}$-valued stochastic process
$\left\{ (X_{n},Y_{n}) \right\}_{n\geq 0}$ defined on $(\Omega,{\cal F},P)$,
where the process $\{X_{n} \}_{n\geq 0}$ is unobservable
and any information on $\{X_{n} \}_{n\geq 0}$ is only available through
the observation process $\{Y_{n} \}_{n\geq 0}$.
In this context, random variables $X_{n}$ and $Y_{n}$ are (respectively) called the state
and observation at discrete-time $n$,
while sets ${\cal X}$ and ${\cal Y}$ are (respectively) referred to as the state and observation spaces.
One of the most important problems related to state-space models is the estimation of the states
$X_{n}$ and $X_{n+1}$ given observations $Y_{0:n}:=(Y_0,...,Y_n)$.
This problem is known as filtering.

In the Bayesian approach,
the estimation of states $X_{n}$ and $X_{n+1}$ given $Y_{0:n}$
is based on the optimal filtering distributions
$P(X_{n}\in dx_{n}|Y_{0:n} )$ and $P(X_{n+1}\in dx_{n+1}|Y_{0:n})$.
In practice, the filtering distributions are usually evaluated using
some approximate models.
In this paper, we assume that the model
$\left\{ (X_{n},Y_{n}) \right\}_{n\geq 0}$
can be accurately approximated by a parametric family of non-linear state-space models.
To specify such a family, we rely on the following notation:
$d\geq 1$ is an integer, while $\Theta\subseteq\mathbb{R}^{d}$ is an open set.
$\mu(dx)$ and $\nu(dy)$ are positive measures on ${\cal X}$ and ${\cal Y}$ (respectively).
$p_{\theta}(x'|x)$ and $q_{\theta}(y|x)$ are Borel-measurable functions which map
$\theta\in\Theta$, $x,x'\in{\cal X}$, $y\in{\cal Y}$ to $[0,\infty )$
and satisfy
\begin{align*}
	\int p_{\theta}(x'|x) \mu(dx') = 1,
	\;\;\;\;\;
	\int q_{\theta}(y|x) \nu(dy) = 1
\end{align*}
for any $\theta\in\Theta$, $x\in{\cal X}$.
$\xi_{\theta}(dx)$ is a parameterized probability measure on ${\cal X}$,
i.e., $\xi_{\theta}(B)$ is a mapping of $\theta\in\Theta$, $B\in{\cal B}({\cal X} )$
to $[0,1]$ which is a probability measure in $B$ for each $\theta\in\Theta$
and
Borel-measurable in $\theta$ for any $B\in{\cal B}({\cal X} )$.
With this notation, we can define a parametric family of state-space models
as an ${\cal X}\times{\cal Y}$-valued stochastic process
$\left\{ \big( X_{n}^{\theta}, Y_{n}^{\theta} \big) \right\}_{n\geq 0}$
which is defined on $(\Omega,{\cal F},P)$,
parameterized by $\theta\in\Theta$,
and satisfies
\begin{align*}
	&
	P\left( \big( X_{0}^{\theta}, Y_{0}^{\theta} \big) \in B \right)
	=
	\int\int I_{B}(x,y) q_{\theta}(y|x) \nu(dy)\xi_{\theta}(dx),
	\\
	&
	P\left(\left. \big( X_{n+1}^{\theta}, Y_{n+1}^{\theta} \big) \in B
	\right| X_{0:n}^{\theta}, Y_{0:n}^{\theta}\right)
	=
	\int\int I_{B}(x,y) q_{\theta}(y|x) p_{\theta}\big(x|X_{n}^{\theta} \big)
	\nu(dy)\mu(dx)
\end{align*}
almost surely for each $\theta\in\Theta$, $B\in{\cal B}({\cal X}\times{\cal Y})$,
$n\geq 0$.\footnote
{To evaluate the values of $\theta$ for which
$\big\{ (X_{n}^{\theta}, Y_{n}^{\theta} ) \big\}_{n\geq 0}$
provides the best approximation to
$\left\{ (X_{n}, Y_{n} ) \right\}_{n\geq 0}$,
we usually rely on the maximum likelihood principle.
For further details on maximum likelihood estimation in state-space and hidden Markov models,
see, e.g., \cite{cappe&moulines&ryden}, \cite{douc&moulines&stoffer}.  }

Throughout the paper,
we assume that $p_{\theta}(x'|x)$ and $q_{\theta}(y|x)$ are differentiable in $\theta$
for each $\theta\in\Theta$, $x,x'\in{\cal X}$, $y\in{\cal Y}$.
To show how the filtering distribution and its derivative are computed using
the approximate model
$\big\{ \big( X_{n}^{\theta}, Y_{n}^{\theta} \big) \big\}_{n\geq 0}$,
we use the following notation:
$w_{\theta}(x)$ a Borel-measurable function mapping $\theta\in\Theta$, $x\in{\cal X}$
to $\mathbb{R}^{d}$.
$r_{\theta,\boldsymbol y}^{n}(x'|x)$ and $t_{\theta,\boldsymbol y}^{n}(x'|x)$
are the functions defined by
\begin{align}\label{1.1}
	r_{\theta,\boldsymbol y}^{n}(x'|x)
	=
	p_{\theta}(x'|x) q_{\theta}(y_{n-1}|x),
	\;\;\;\;\;
	t_{\theta,\boldsymbol y}^{n}(x'|x)
	=
	\nabla_{\theta}\log\left( r_{\theta,\boldsymbol y}^{n}(x'|x) \right)
\end{align}
for $\theta\in\Theta$, $x,x'\in{\cal X}$, $n\geq 1$
and a sequence $\boldsymbol y = \{y_{n}\}_{n\geq 0}$ in ${\cal Y}$.
$r_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )$ and $t_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )$,
are the functions defined by
\begin{align}
	\label{1.3}
	r_{\theta,\boldsymbol y}^{m:m}(x_{m:m} )
	=
	1,
	&
	\;\;\;\;\;
	r_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )
	=
	\prod_{k=m+1}^{n} r_{\theta,\boldsymbol y}^{k}(x_{k}|x_{k-1} ),
	\\
	t_{\theta,\boldsymbol y}^{m:m}(x_{m:m} )
	=
	w_{\theta}(x_{m} ),
	&\;\;\;\;\;
	t_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )
	=
	w_{\theta}(x_{m} )
	+
	\sum_{k=m+1}^{n} t_{\theta,\boldsymbol y}^{k}(x_{k}|x_{k-1} )
	\nonumber
\end{align}
for $x_{m},\dots,x_{n}\in{\cal X}$, $n>m\geq 0$.
$\mathbb{R}_{\theta,\boldsymbol y}^{n}(dx_{0:n} )$ and
$\mathbb{T}_{\theta,\boldsymbol y}^{n}(dx_{0:n} )$ are the measures defined for $A\in{\cal B}({\cal X}^{n+1} )$, $n\geq 1$ by
\begin{align*}
	&
	\mathbb{R}_{\theta,\boldsymbol y}^{n}(A)
	=
	\int\cdots\int\int
	I_{A}(x_{0:n} ) r_{\theta,\boldsymbol y}^{0:n}(x_{0:n} )
	\mu(dx_{n} )\cdots\mu(dx_{1} )\xi_{\theta}(dx_{0} ),
	\\
	&
	\mathbb{T}_{\theta,\boldsymbol y}^{n}(A)
	=
	\int\cdots\int\int
	I_{A}(x_{0:n} ) t_{\theta,\boldsymbol y}^{0:n}(x_{0:n} ) r_{\theta,\boldsymbol y}^{0:n}(x_{0:n} )
	\mu(dx_{n} )\cdots\mu(dx_{1} )\xi_{\theta}(dx_{0} ).
\end{align*}
$\bar{\mathbb{R} }_{\theta,\boldsymbol y}^{n}$ and
$\bar{\mathbb{T} }_{\theta,\boldsymbol y}^{n}$ are the functions defined by
\begin{align*}
	\bar{\mathbb{R} }_{\theta,\boldsymbol y}^{n}
	=
	\mathbb{R}_{\theta,\boldsymbol y}^{n}({\cal X}^{n+1} ),
	\;\;\;\;\;
	\bar{\mathbb{T} }_{\theta,\boldsymbol y}^{n}
	=
	\mathbb{T}_{\theta,\boldsymbol y}^{n}({\cal X}^{n+1} ).
\end{align*}
$\mathbb{P}_{\theta,\boldsymbol y}^{n}(dx_{0:n} )$ and
$P_{\theta,\boldsymbol y}^{n}(dx)$ are the measures defined for $B\in{\cal B}({\cal X} )$ by
\begin{align*}
	\mathbb{P}_{\theta,\boldsymbol y}^{n}(A)
	=
	\frac{\mathbb{R}_{\theta,\boldsymbol y}^{n}(A) }{\bar{\mathbb{R} }_{\theta,\boldsymbol y}^{n} },
	&\;\;\;\;\;
	P_{\theta,\boldsymbol y}^{n}(B)
	=
	\mathbb{P}_{\theta,\boldsymbol y}^{n}({\cal X}^{n}\times B ).
\end{align*}
$\mathbb{Q}_{\theta,\boldsymbol y}^{n}(dx_{0:n} )$ and
$Q_{\theta,\boldsymbol y}^{n}(dx)$ are the measures defined by
\begin{align*}
	\mathbb{Q}_{\theta,\boldsymbol y}^{n}(A)
	=
	\frac{\mathbb{T}_{\theta,\boldsymbol y}^{n}(A) }{\bar{\mathbb{R} }_{\theta,\boldsymbol y}^{n} }
	-
	\mathbb{P}_{\theta,\boldsymbol y}^{n}(A)
	\frac{\bar{\mathbb{T} }_{\theta,\boldsymbol y}^{n} }{\bar{\mathbb{R} }_{\theta,\boldsymbol y}^{n} },
	&\;\;\;\;\;
	Q_{\theta,\boldsymbol y}^{n}(B)
	=
	\mathbb{Q}_{\theta,\boldsymbol y}^{n}({\cal X}^{n}\times B ).
\end{align*}

All results presented in this paper are based on Assumptions \ref{a1} -- \ref{a3}
(see Subsection \ref{ssection1.3}, below).
Using elementary calculus, it can be shown that the functions and measure defined above
are well-defined under these assumptions.
It is also easy to show
\begin{align}\label{1.301}
	P_{\theta,\boldsymbol y}^{n}(B)
	=
	P\left(X_{n}^{\theta}\in B| Y_{0:n-1}^{\theta}=y_{0:n-1} \right)
\end{align}
for all $\theta\in\Theta$, $B\in{\cal B}({\cal X} )$,
$n\geq 1$ and any sequence in $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in
${\cal Y}$.
Hence, $P_{\theta,\boldsymbol y}^{n}(dx)$ is the optimal filter (i.e., one-step predictor) for the model
$\left\{ (X_{n}^{\theta}, Y_{n}^{\theta} ) \right\}_{n\geq 0}$.

Let $\lambda(dx)$ be a finite positive measure on ${\cal X}$.
Moreover, let $l_{\theta}(x)$ be a Borel-measurable function
which maps $\theta\in\Theta$, $x\in{\cal X}$ to $(0,\infty )$
and satisfies the following conditions:
(i) $l_{\theta}(x)$ is differentiable in $\theta$
for each $\theta\in\Theta$, $x\in{\cal X}$, and
(ii) $\|\nabla_{\theta}l_{\theta}(x) \|$ is uniformly bounded in
$(\theta,x)$ on $\Theta\times{\cal X}$.
Suppose that Assumptions \ref{a1} -- \ref{a3} hold and
that $\xi_{\theta}(dx)$, $w_{\theta}(x)$ are of the form
\begin{align}\label{1.305}
	\xi_{\theta}(B) = \int_{B} l_{\theta}(x')\lambda(dx'),
	\;\;\;\;\;
	w_{\theta}(x) = \nabla_{\theta}\log\left(l_{\theta}(x) \right)
\end{align}
for $\theta\in\Theta$, $x\in{\cal X}$, $B\in{\cal B}({\cal X} )$.
Then, it straightforward to verify
\begin{align}\label{1.303}
	Q_{\theta,\boldsymbol y}^{n}(B)
	=
	\nabla_{\theta} P\left(X_{n}^{\theta}\in B| Y_{0:n-1}^{\theta}=y_{0:n-1} \right)
\end{align}
for the same $\theta$, $B$, $n$, $\boldsymbol y$ as in (\ref{1.301}) --- see
\cite[Section 2]{delmoral&doucet&singh2} for further details.
Given (\ref{1.301}), (\ref{1.303}),
$Q_{\theta,\boldsymbol y}^{n}(B)$ is thus the optimal filter derivative.

\subsection{Particle Approximation to Optimal Filter Derivative}\label{ssection1.2}

Unless the model
$\big\{ \big( X_{n}^{\theta}, Y_{n}^{\theta} \big) \big\}_{n\geq 0}$
is linear-Gaussian (or the state-space ${\cal X}$ has finitely many elements),
$P_{\theta,\boldsymbol y}^{n}(dx)$
and its gradient $Q_{\theta,\boldsymbol y}^{n}(dx)$ do not admit
closed-form expressions and need to be approximated numerically.

Let $\theta$ be any element in $\Theta$.
The particle method proposed in \cite{poyiadjis&doucet&singh} approximates
$P_{\theta,\boldsymbol y}^{n}(dx)$ and
$Q_{\theta,\boldsymbol y}^{n}(dx)$
respectively by the empirical distributions
\begin{align}\label{1.31}
	\hat{\xi}_{n}^{\theta}(dx)
	=
	\frac{1}{N} \sum_{i=1}^{N} \delta_{\hat{X}_{n,i}^{\theta} }(dx),
	\;\;\;
	\hat{\zeta}_{n}^{\theta}(dx)
	=
	\frac{1}{N} \sum_{i=1}^{N}
	\left( W_{n,i}^{\theta} - \frac{1}{N} \sum_{j=1}^{N} W_{n,j}^{\theta} \right)
	\delta_{\hat{X}_{n,i}^{\theta} }(dx)
\end{align}
for $n\geq 0$.
Here, $N\geq 2$ is a fixed integer and $\left\{ W_{n,i}^{\theta}: n\geq 0, 1\leq i\leq N \right\}$ are random vectors
generated through the recursion
\begin{align}\label{1.33}
	W_{n+1,i}^{\theta}
	=&
	\frac{\sum_{j=1}^{N} \left(
	p_{\theta}\big(\hat{X}_{n+1,j}^{\theta} |\hat{X}_{n,j}^{\theta} \big)
	\nabla_{\theta} q_{\theta}\big(Y_{n}|\hat{X}_{n,j}^{\theta} \big)
	+
	\nabla_{\theta} p_{\theta}\big(\hat{X}_{n+1,j}^{\theta} |\hat{X}_{n,j}^{\theta} \big)
	q_{\theta}\big(Y_{n}|\hat{X}_{n,j}^{\theta} \big)
	\right) }
	{\sum_{j=1}^{N} p_{\theta}\big(\hat{X}_{n+1,j}^{\theta} |\hat{X}_{n,j}^{\theta} \big)
	q_{\theta}\big(Y_{n}|\hat{X}_{n,j}^{\theta} \big) }
	\\ \nonumber
	&+
	\frac{\sum_{j=1}^{N} p_{\theta}\big(\hat{X}_{n+1,j}^{\theta} |\hat{X}_{n,j}^{\theta} \big)
	q_{\theta}\big(Y_{n}|\hat{X}_{n,j}^{\theta} \big) W_{n,j}^{\theta} }
	{\sum_{j=1}^{N} p_{\theta}\big(\hat{X}_{n+1,j}^{\theta} |\hat{X}_{n,j}^{\theta} \big)
	q_{\theta}\big(Y_{n}|\hat{X}_{n,j}^{\theta} \big) },
\end{align}
where $\big\{ \hat{X}_{n,i}^{\theta}: n\geq 0, 1\leq i\leq N \big\}$ are random vectors
generated through the sequential Monte Carlo scheme
\begin{align}\label{1.35}
	\hat{X}_{n+1,i}^{\theta}
	\sim
	\frac{\sum_{j=1}^{N} p_{\theta}\big(x|\hat{X}_{n,j}^{\theta} \big)
	q_{\theta}\big(Y_{n}|\hat{X}_{n,j}^{\theta} \big) \mu(dx) }
	{\sum_{j=1}^{N} q_{\theta}\big(Y_{n}|\hat{X}_{n,j}^{\theta} \big) }.
\end{align}
In recursion (\ref{1.33}), $\big\{ W_{0,i}^{\theta}: 1\leq i\leq N \big\}$
are selected as $W_{0,i}^{\theta} = w_{\theta}\big( \hat{X}_{0,i}^{\theta} \big)$.
In recursion (\ref{1.35}),
$\big\{ \hat{X}_{n+1,i}^{\theta}: 1\leq i\leq N \big\}$ are sampled independently
from one another. In the same recursion, $\big\{ \hat{X}_{0,i}^{\theta}: 1\leq i\leq N \big\}$
are sampled from $\xi_{\theta}(dx)$ independently one from another and independently from $Y_{0}$.
The random samples $\big\{ \hat{X}_{n,i}^{\theta}: n\geq 0, 1\leq i\leq N \big\}$
are commonly referred to as particles.

\begin{vremark}
Let $w_{\theta,\boldsymbol y}^{n}(x)$ be the function defined by
\begin{align*}
	w_{\theta,\boldsymbol y}^{n}(x)
	=
	E\left(
	t_{\theta,\boldsymbol y}^{0:n}(X_{0:n}^{\theta} )
	|X_{n}^{\theta}=x,Y_{0:n-1}^{\theta}=y_{0:n-1}
	\right)
\end{align*}
for $\theta\in\Theta$, $x\in{\cal X}$, $n\geq 1$ and a sequence
$\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$.
Then, we have $W_{n,i}^{\theta} = w_{\theta,\boldsymbol Y}^{n}(\hat{X}_{n,i}^{\theta} )$
for each $1\leq i\leq N$, $n\geq 1$,
where $\boldsymbol Y = \{Y_{n} \}_{n\geq 0}$.
The equivalence between this representation of random vector $W_{n,i}^{\theta}$
and recursion (\ref{1.33}) is shown and discussed
in \cite[Section 3]{delmoral&doucet&singh2}.
Recursion (\ref{1.33}) is derived in \cite[Section 2.2]{poyiadjis&doucet&singh}.
\end{vremark}

\subsection{Bias of Particle Approximation to Optimal Filter Derivative}\label{ssection1.3}

We analyze here the bias of the particle approximations (\ref{1.31}).
The analysis relies on the following notation.
For $z\in\mathbb{R}^{k}$, $k\geq 1$,
$\|z\|$ denotes the $l_{\infty}$ norm of $z$.
If $\psi_{\theta}(x)$ is a Borel-measurable function mapping
$\theta\in\Theta$, $x\in{\cal X}$ to $\mathbb{R}^{k}$,
then $\|\psi_{\theta}\|$ denotes the $L_{\infty}$ norm of $\psi_{\theta}(x)$
in $x$, i.e., $\|\psi_{\theta} \| = \sup_{x\in{\cal X} } \|\psi_{\theta}(x) \|$.
If $\varphi:{\cal X}\rightarrow\mathbb{R}$ is a Borel-measurable function
and $\eta(dx)$ is a $k$-dimensional signed vector measure on ${\cal X}$,
then $\eta(\varphi)$ denotes the integral $\eta(\varphi) = \int \varphi(x)\eta(dx)$.

The analysis carried out in this paper relies on the following assumptions.

\begin{assumption}\label{a1}
There exists a real number $\varepsilon\in(0,1)$ such that
\begin{align*}
	\varepsilon \leq p_{\theta}(x'|x) \leq \frac{1}{\varepsilon},
	\;\;\;\;\;
	\varepsilon \leq q_{\theta}(y|x) \leq \frac{1}{\varepsilon}
\end{align*}
for all $\theta\in\Theta$, $x,x'\in{\cal X}$, $y\in{\cal Y}$.
\end{assumption}

\begin{assumption}\label{a2}
There exists a real number $K\in[1,\infty)$ such that
\begin{align*}
	\max\{\|\nabla_{\theta} p_{\theta}(x'|x) \|,
	\|\nabla_{\theta} q_{\theta}(y|x) \| \}
	\leq K
\end{align*}
for all $\theta\in\Theta$, $x,x'\in{\cal X}$, $y\in{\cal Y}$.
\end{assumption}

\begin{assumption}\label{a3}
$\|w_{\theta} \|=\sup_{x\in{\cal X} } \|w_{\theta}(x) \| < \infty$
for all $\theta\in\Theta$.
\end{assumption}

Assumption \ref{a1} is a standard strong mixing condition
and is a crucial ingredient of many results on
optimal filtering and statistical inference in state-space and hidden Markov models
--- see e.g., \cite{cappe&moulines&ryden}, \cite{delmoral&guionnet}, \cite{delmoral&doucet&singh2},
\cite{douc&moulines&ryden}, \cite{legland&mevel}, \cite{legland&oudjane},
\cite{oudjane&rubenthaler}
\cite{tadic&doucet1}.
Together with Assumption \ref{a2}, it ensures that
the optimal filter and its gradient forget initial conditions exponentially fast
--- see Proposition \ref{proposition3.2}, Section \ref{section3}.
This assumption, together with Assumptions \ref{a2} and \ref{a3}, also ensures
the stability of particle approximations $(\ref{1.31})$
--- see Proposition \ref{proposition4.1}, Section \ref{section4}.
Assumption \ref{a1} is restrictive as it implicitly requires the state and observation spaces ${\cal X}$ and ${\cal Y}$
to be bounded.

Let $\boldsymbol Y$ denote stochastic process $\{Y_{n} \}_{n\geq 0}$,
i.e., $\boldsymbol Y = \{Y_{n} \}_{n\geq 0}$.
The main results of our paper are stated in the next theorem.

\begin{theorem}\label{theorem1.1}
(i) Let Assumption \ref{a1} hold.
Then, there exists a real number $L\in[1,\infty)$
(independent of $N$ and depending only on $\varepsilon$) such that
\begin{align}\label{t1.1.1*}
	\left|
	E\left(\left.
	\hat{\xi}_{n}^{\theta}(\varphi)
	-
	P_{\theta,\boldsymbol Y}^{n}(\varphi)
	\right|
	\boldsymbol Y = \boldsymbol y
	\right)
	\right|
	\leq
	\frac{L}{N}
\end{align}
for all $\theta\in\Theta$, $n\geq 1$,
any sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$
and any Borel-measurable function $\varphi:{\cal X}\rightarrow[-1,1]$.

(ii) Let Assumptions \ref{a1} -- \ref{a3} hold.
Then, there exist real numbers $\rho\in(0,1)$, $M\in[1,\infty)$
(independent of $N$ and depending only on $\varepsilon$, $d$, $K$) such that
\begin{align}\label{t1.1.3*}
	\left\|
	E\left(\left.
	\hat{\zeta}_{n}^{\theta}(\varphi)
	-
	Q_{\theta,\boldsymbol Y}^{n}
	(\varphi)
	\right|
	\boldsymbol Y = \boldsymbol y
	\right)
	\right\|
	\leq
	\frac{M(1 + \rho^{n} \|w_{\theta}\| ) }{N}
\end{align}
for all $\theta\in\Theta$, $n\geq 1$,
any sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$
and any Borel-measurable function $\varphi:{\cal X}\rightarrow[-1,1]$.
\end{theorem}

The proof of Theorem \ref{theorem1.1} is provided in Section \ref{section5}
--- see Proposition \ref{proposition5.1}.

The empirical measures $\hat{\xi}_{n}^{\theta}(dx)$ and $\hat{\zeta}_{n}^{\theta}(dx)$
are estimators of the optimal predictor
$P_{\theta, \boldsymbol Y}^{n}(dx)$
and its gradient
$Q_{\theta, \boldsymbol Y}^{n}(dx)$.
Hence, the conditional expectations in (\ref{t1.1.1*}), (\ref{t1.1.3*})
can be viewed as the bias of particle approximations (\ref{1.31}).
Theorem \ref{theorem1.1} provides bounds on this bias.
The bounds are inversely proportional to $N$ and uniform in discrete-time $n$
as $\rho^{n}\leq 1$.
The bounds depend on $\left\{ \big( X_{n}^{\theta}, Y_{n}^{\theta} \big) \right\}_{n\geq 0}$
through constants $\rho$, $L$, $M$ and the initial conditions in recursion (\ref{1.33})
(through $\|w_{\theta}\|$).

Due to their practical and theoretical importance,
particle methods have extensively been studied
in a number of papers and books
--- see e.g., \cite{andrieu&doucet&singh&tadic},
\cite{cappe&moulines&ryden}, \cite{crisan&rozovskii},
\cite{delmoral1}, \cite{delmoral2},
\cite{douc&moulines&stoffer},
\cite{doucet&defreitas&gordon},
\cite{douc&moulines&stoffer} -- \cite{kantas&doucet&singh&maciejowski&chopin}.
Within a broader analysis of the propagation of chaos in Feynman-Kac models,
the bias of particle approximations to the optimal filter has been addressed in
\cite{delmoral1} -- \cite{delmoral&jacob&lee&murray&peters},
\cite{delmoral&jasra}.
Under conditions similar or identical to Assumption \ref{a1},
the results of
\cite{delmoral1} -- \cite{delmoral&jacob&lee&murray&peters},
\cite{delmoral&jasra}
lead to Part (i) of Theorem \ref{theorem1.1}.\footnote
{Although Part (i) of Theorem \ref{theorem1.1} is
a particular case in the analysis carried out in
\cite{delmoral1}, \cite{delmoral2}, \cite{delmoral&doucet&peters},
\cite{delmoral&doucet&singh1}, \cite{delmoral&jacob&lee&murray&peters},
\cite{delmoral&jasra},
we include it in the main results for the following reasons:
(i) $\xi_{n}^{\theta}(dx)$ is an integral part
of the particle approximation (\ref{1.31}) -- (\ref{1.35}),
(ii) the bound (\ref{t1.1.1*}) is an essential prerequisite for
Part (ii) of Theorem \ref{theorem1.1},
and (iii) the proof of Part (i) of Theorem \ref{theorem1.1} presented here seems
more direct than the analysis carried out in
\cite{delmoral1}, \cite{delmoral2}, \cite{delmoral&doucet&peters},
\cite{delmoral&doucet&singh1}, \cite{delmoral&jacob&lee&murray&peters},
\cite{delmoral&jasra}. }
As opposed to particle approximations to the optimal filter,
the optimal filter derivative and its particle approximations have attracted
much less attention.
Part (ii) of Theorem \ref{theorem1.1} fills this gap in the literature
on optimal filtering and particle methods.
To the best of our knowledge, Part (ii) of Theorem \ref{theorem1.1} is the first result on
the bias of the particle approximation (\ref{1.31}) -- (\ref{1.35}).
In \cite{tadic&doucet2}, we use this result, together with the results of \cite{tadic&doucet4},
to analyze the asymptotic behavior of
recursive maximum likelihood estimation in non-linear state-space models.

\section{Results Related to Empirical Measures}\label{section2}

In this section, we present an auxiliary result on
the ratio of integrals approximated using empirical measures.
This result is a crucial ingredient in the proof of Lemma \ref{lemma5.2}
which itself is a corner-stone of the main results --- see Proposition \ref{proposition5.1}.
This result has already appeared in
     \cite[Lemma A.1]{delmoral&doucet&singh2} and \cite[Lemmas B.3, B.4]{delmoral&jasra}. For completeness, a proof is provided in the supplementary material (Section SM1).

We use the following additional notation: $d_{z}\geq 1$ is an integer,
while ${\cal Z}\in{\cal B}(\mathbb{R}^{d_{z}} )$.
$\xi(dz)$ is a probability measure on ${\cal Z}$.
$\{Z_{k} \}_{k\geq 1}$ are independent ${\cal Z}$-valued random variables
which are defined on a probability space $(\Omega, {\cal F}, P)$ and distributed according to $\xi(dz)$
(i.e., $P(Z_{k}\in B ) = \xi(B)$
for each $B\in{\cal B}({\cal Z} )$).
$\xi_{k}(dz)$ is the empirical measure defined for $k\geq 1$ by
\begin{align*}
	\xi_{k}(B)
	=
	\frac{1}{k} \sum_{i=1}^{k} \delta_{Z_{i} }(B).
\end{align*}
\begin{proposition}\label{proposition2.1}
Let $f:{\cal Z}\rightarrow \mathbb{R}$ and $g:{\cal Z}\rightarrow (0,\infty )$
be Borel-measurable functions such that
\begin{align*}
	\sup_{z\in{\cal Z} } |f(z)| < \infty,
	\;\;\;\;\;
	\sup_{z\in{\cal Z} } g(z) < \infty,
	\;\;\;\;\;
	\inf_{z\in{\cal Z} } g(z) > 0.
\end{align*}
Then, we have
\begin{align}\label{p2.1.3*}
	\left|
	E\left(
	\frac{\xi_{k}(f) }{\xi_{k}(g) } \right)
	-
	\frac{\xi(f) }{\xi(g) }
	\right|
	\leq
	\frac{2\alpha\beta^{2} }{k},
	\;\;\;\;\;
	\left(
	E\left(
	\left|
	\frac{\xi_{k}(f) }{\xi_{k}(g) }
	-
	\frac{\xi(f) }{\xi(g) }
	\right|^{2}
	\right)
	\right)^{1/2}
	\leq
	\frac{2\alpha\beta}{\sqrt{k} }
\end{align}
for any $k\geq 1$,
where $\alpha$, $\beta$ are defined by
\begin{align}\label{p2.1.1*}
	\alpha
	=
	\sup_{z',z''\in{\cal Z} }
	\left|
	\frac{f(z')}{g(z')} - \frac{f(z'')}{g(z'')}
	\right|,
	\;\;\;\;\;
	\beta
	=
	\sup_{z',z''\in{\cal Z} }
	\frac{g(z')}{g(z'')}.
\end{align}
\end{proposition}

\section{Results Related to Stability of Optimal Filter and Its Derivative}\label{section3}

In this section, we present results on the stability properties of the optimal predictor
$P_{\theta,\boldsymbol y}^{n}(dx)$ and its gradient
$Q_{\theta,\boldsymbol y}^{m:n}(dx)$.
These results are prerequisites for the proof
of the main results --- see Lemmas \ref{lemma5.1}, \ref{lemma5.2} and
Proposition \ref{proposition5.1}.

The following additional notation is used here.
${\cal P}({\cal X} )$ is the collection of probability measures on ${\cal X}$,
while the set of positive measures on ${\cal X}$ is denoted by ${\cal M}_{p}({\cal X} )$.
The collection of signed measures on ${\cal X}$ is denoted by ${\cal M}_{s}({\cal X} )$,
while ${\cal M}_{s}^{d}({\cal X} )$ is the set of $d$-dimensional signed vector measures on ${\cal X}$.
For $\xi\in{\cal M}_{s}({\cal X})$,
$|\xi|(dx)$ and $\|\xi\|$ denote (respectively) the total variation
and the total variation norm of $\xi(dx)$.
For $\zeta\in{\cal M}_{s}^{d}({\cal X} )$,
$|\zeta|(dx)$ and $\|\zeta\|$ denote (respectively) the total variation
and the total variation norm of $\zeta(dx)$ induced by $l_{1}$ vector norm.\footnote
{If $\zeta\in{\cal M}_{s}^{d}({\cal X})$,
then $|\zeta|(dx) = \sum_{i=1}^{d} |e_{i}^{T}\zeta|(dx)$
and $\|\zeta\| = \sum_{i=1}^{d} \|e_{i}^{T}\zeta\|$,
where $e_{i}$ is the $i$-th standard unit vector in $\mathbb{R}^{d}$. }
$s_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )$ is the function defined by
\begin{align}\label{1.5}
	s_{\theta,\boldsymbol y}^{m:m}(x_{m:m} )=0,
	\;\;\;\;\;
	s_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )
	=
	\nabla_{\theta}r_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )
\end{align}
for $\theta\in\Theta$, $x_{m},\dots,x_{n}\in{\cal X}$,
$n>m\geq 0$ and a sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$
($r_{\theta,\boldsymbol y}^{m:n}(x_{m:n} )$ is specified in (\ref{1.3})).
$R_{\theta,\boldsymbol y}^{m:n}(dx'|x)$ and
$S_{\theta,\boldsymbol y}^{m:n}(dx'|x)$
are the elements of ${\cal M}_{p}({\cal X})$ and ${\cal M}_{s}^{d}({\cal X})$ (respectively)
defined by
\begin{align}
	\label{1.7}
	R_{\theta,\boldsymbol y}^{m:m}(B|x)
	=
	\delta_{x}(B),
	\;\;\;\;\;
	&
	\begin{aligned}[t]
	R_{\theta,\boldsymbol y}^{m:n}(B|x)
	=
	&\int\cdots\int\int I_{B}(x_{n} ) r_{\theta,\boldsymbol y}^{m:n}(x_{m:n} ) \\
	&\cdot \mu(dx_{n} )\cdots\mu(dx_{m+1} )\delta_{x}(dx_{m} ),
	\end{aligned}
	\\
	\label{1.9}
	S_{\theta,\boldsymbol y}^{m:m}(B|x)
	=
	0,
	\;\;\;\;\;
	&
	\begin{aligned}[t]
	S_{\theta,\boldsymbol y}^{m:n}(B|x)
	=
	&\int\cdots\int\int I_{B}(x_{n} ) s_{\theta,\boldsymbol y}^{m:n}(x_{m:n} ) \\
	&\cdot\mu(dx_{n} )\cdots\mu(dx_{m+1} )\delta_{x}(dx_{m} )
	\end{aligned}
\end{align}
for $B\in{\cal B}({\cal X})$.
$F_{\theta,\boldsymbol y}^{m:n}(dx|\xi)$,
$G_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$ and
$H_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$
are the elements of ${\cal P}({\cal X})$, ${\cal M}_{s}^{d}({\cal X})$
and ${\cal M}_{s}^{d}({\cal X})$ (respectively)
defined by
\begin{align}
	\label{1.21}
	F_{\theta,\boldsymbol y}^{m:m}(B|\xi)
	=
	\xi(B),
	&\;\;\;
	F_{\theta,\boldsymbol y}^{m:n}(B|\xi)
	=
	\frac{\int R_{\theta,\boldsymbol y}^{m:n}(B|x) \xi(dx) }
	{\int R_{\theta,\boldsymbol y}^{m:n}({\cal X}|x) \xi(dx) },
	\\
	\label{1.23}
	H_{\theta,\boldsymbol y}^{m:m}(B|\xi,\zeta)
	\!=\!
	\zeta(B),
	&\;\;\;
	H_{\theta,\boldsymbol y}^{m:n}(B|\xi,\zeta)
	\!=\!
	\frac{\int\! R_{\theta,\boldsymbol y}^{m:n}(B|x) \zeta(dx)
	\!+\! \int\! S_{\theta,\boldsymbol y}^{m:n}(B|x) \xi(dx)}
	{\int\! R_{\theta,\boldsymbol y}^{m:n}({\cal X}|x) \xi(dx) }
	\\
	\label{1.25}
	G_{\theta,\boldsymbol y}^{m:m}(B|\xi,\zeta)
	\!=\!
	\zeta(B),
	&\;\;\;
	G_{\theta,\boldsymbol y}^{m:n}(B|\xi,\zeta)
	\!=\!
	H_{\theta,\boldsymbol y}^{m:n}(B|\xi,\zeta)
	\!-\!
	F_{\theta,\boldsymbol y}^{m:n}(B|\xi)
	H_{\theta,\boldsymbol y}^{m:n}({\cal X}|\xi,\zeta)
\end{align}
for $\xi\in{\cal P}({\cal X})$, $\zeta\in{\cal M}_{s}^{d}({\cal X})$.
$F_{\theta,\boldsymbol y}^{m:n}(\xi)$, $G_{\theta,\boldsymbol y}^{m:n}(\xi,\zeta)$
and $H_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$
are a `short-hand' notation for $F_{\theta,\boldsymbol y}^{m:n}(dx|\xi)$, $G_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$
and $H_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$ (respectively).

\begin{vremark}
For $\theta\in\Theta$, let $\zeta_{\theta}(dx)$ be the element of ${\cal M}_{s}^{d}({\cal X} )$
specified in (\ref{5.1}) (below).
Suppose that Assumptions \ref{a1} -- \ref{a3} hold.
Then, it easy to show
\begin{align}\label{3.301}
	P_{\theta,\boldsymbol y}^{n}(B)
	=
	F_{\theta,\boldsymbol y}^{0:n}(B|\xi_{\theta} ),
	\;\;\;\;\;
	Q_{\theta,\boldsymbol y}^{n}(B)
	=
	G_{\theta,\boldsymbol y}^{0:n}(B|\xi_{\theta},\zeta_{\theta} )
\end{align}
for all $\theta\in\Theta$, $B\in{\cal B}({\cal X} )$, $n\geq 1$
and any sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$.
Given (\ref{1.301}), (\ref{1.303}), (\ref{3.301}),
$F_{\theta,\boldsymbol y}^{m:n}(dx|\xi)$ and
$G_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$
can be considered as a generalization of the optimal filter and its gradient.
In this context, $\xi$, $\zeta$ can be viewed as initial conditions
in the recursion generating the optimal filter and its gradient
(for further details, see \cite{legland&mevel}, \cite{tadic&doucet1}).
\end{vremark}

\begin{proposition}\label{proposition3.2}
(i) Let Assumption \ref{a1} hold.
Then, there exist real numbers $\rho_{1}\in(0,1)$, $C_{1}\in[1,\infty)$
(depending only on $\varepsilon$) such that
\begin{align}
	&\label{p3.2.3*}
	\left\| F_{\theta,\boldsymbol y}^{m:n}(\xi')
	-
	F_{\theta,\boldsymbol y}^{m:n}(\xi'') \right\|
	\leq
	C_{1}\rho_{1}^{n-m},
	\;\;\;\;\;
	\frac{\int R_{\theta,\boldsymbol y}^{m:n}({\cal X}|x) \xi'(dx) }
	{\int R_{\theta,\boldsymbol y}^{m:n}({\cal X}|x) \xi''(dx) }
	\leq
	C_{1}
\end{align}
for all $\theta\in\Theta$, $\xi',\xi''\in{\cal P}({\cal X})$,
$n\geq m\geq 0$ and any sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$.

(ii) Let Assumptions \ref{a1} and \ref{a2} hold.
Then, there exist real numbers $\rho_{2}\in(0,1)$, $C_{2}\in[1,\infty)$
(depending only on $\varepsilon$, $d$, $K$) such that
\begin{align}
	&\label{p3.2.5*}
	\begin{aligned}[t]
	\left\| G_{\theta,\boldsymbol y}^{m:n}(\xi',\zeta')
	-
	G_{\theta,\boldsymbol y}^{m:n}(\xi'',\zeta'') \right\|
	\leq &
	C_{2}\rho_{2}^{n-m} (1 + \|\zeta'\| + \|\zeta''\| ),
	\end{aligned}
	\\
	&
	\label{p3.2.7*}
	\left\| H_{\theta,\boldsymbol y}^{m:n}(\xi,\zeta)\right\|
	\leq
	C_{2} (n-m+\|\zeta\| )
\end{align}
for all $\theta\in\Theta$, $\xi,\xi',\xi''\in{\cal P}({\cal X})$,
$\zeta,\zeta',\zeta''\in{\cal M}_{s}^{d}({\cal X})$,
$n\geq m\geq 0$ and any sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$.
\end{proposition}

Proposition \ref{proposition3.2} is a relatively straightforward extension
of the results of \cite{legland&mevel}, \cite{legland&oudjane}, \cite{tadic&doucet1} to the optimal predictor and its gradient.
A detailed proof of the proposition is provided in the supplementary material
(Section SM2).

\section{Results Related to Stability of Particle Approximations}\label{section4}

In this section, we consider the particle approximation $\hat{\zeta}_{n}^{\theta}(dx)$
and its stability.
Using results on the (Dobrushin) ergodicity coefficient,
we show that the sequence $\big\{ \big\|\hat{\zeta}_{n}^{\theta} \big\| \big\}_{n\geq 0}$
is bounded uniformly in $\theta$.
The results presented here are prerequisites for the proof of
the main results --- see Lemma \ref{lemma5.2} and Proposition \ref{proposition5.1}.

To state the results of this section, additional notation needs to be introduced.
${\cal P}^{N}$ is the set of $N$-dimensional probability vectors.
${\cal P}^{N\times N}$ is the set of $N\times N$ (column) stochastic matrices
(i.e., $A\in{\cal P}^{N\times N}$ if and only if the columns of $A$ are elements of
${\cal P}^{N}$).
$e$ is the element of $\mathbb{R}^{N}$ whose all elements are one.
For $1\leq i\leq N$,
$e_{i}$ is the $i$-th standard unit vector in $\mathbb{R}^{N}$
(i.e., $e_{i}$ is the element of ${\cal P}^{N}$ whose $i$-th element is one).
For $z\in\mathbb{R}^{N}$,
$\|z\|_{1}$ and $\|z\|$ are (respectively) the $l_{1}$ and $l_{\infty}$ norm of $z$.
For $B\in\mathbb{R}^{d\times N}$,
$\|B\|$ is the $l_{\infty}$ norm of $B$
(i.e., $\|B\|$ is the maximum absolute value of the entries of $B$).
For $A\in{\cal P}^{N\times N}$,
$\tau(A)$ is the ergodicity coefficient of $A$, i.e.,
\begin{align}\label{4.7}
	\tau(A)
	=
	\frac{1}{2}
	\max_{1\leq j',j''\leq N} \sum_{i=1}^{N} |A_{i,j'} - A_{i,j''} |
	=
	1
	-
	\min_{1\leq j',j''\leq N}
	\sum_{i=1}^{N} \min\{A_{i,j'},A_{i,j''} \},
\end{align}
where $A_{i,j}$ is the $(i,j)$ entry of $A$
(see \cite[Section 15.2.1]{bremaud} for more details on the ergodicity coefficient and its equivalent forms).
$A_{n}^{\theta}$ and $B_{n}^{\theta}$ are (respectively)
the $N\times N$ and $d\times N$ random matrices defined by
\begin{align}\label{4.5}
	A_{n,i,j}^{\theta}
	=
	\frac{r_{\theta,\boldsymbol Y}^{n}\big(\hat{X}_{n,j}^{\theta} |\hat{X}_{n-1,i}^{\theta} \big) }
	{\sum_{k=1}^{N} r_{\theta,\boldsymbol Y}^{n}
	\big(\hat{X}_{n,j}^{\theta} |\hat{X}_{n-1,k}^{\theta} \big) },
	\;\;\;\;\;
	B_{n,j}^{\theta}
	=
	\frac{\sum_{k=1}^{N} \nabla_{\theta} r_{\theta,\boldsymbol Y}^{n}\big(\hat{X}_{n,j}^{\theta} |\hat{X}_{n-1,k}^{\theta} \big) }
	{\sum_{k=1}^{N} r_{\theta,\boldsymbol Y}^{n}
	\big(\hat{X}_{n,j}^{\theta} |\hat{X}_{n-1,k}^{\theta} \big) },
\end{align}
where $A_{n,i,j}^{\theta}$ is the $(i,j)$ entry of $A_{n}^{\theta}$
and $B_{n,j}^{\theta}$ is the $j$-th column of $B_{n}^{\theta}$.
$r_{\theta,\boldsymbol y}^{n}(x'|x)$ and $\boldsymbol Y$
are specified in (\ref{1.1}) and Subsection \ref{ssection1.3} (respectively).
$V_{n}^{\theta}$, $W_{n}^{\theta}$ and $V_{n,i}^{\theta}$
are the $d\times N$ random matrices and the $d$-dimensional random vector
defined by
\begin{align}\label{4.3}
	V_{n,i}^{\theta}
	=
	W_{n,i}^{\theta}
	-
	\frac{1}{N} \sum_{j=1}^{N} W_{n,j}^{\theta},
	\;\;\;
	V_{n}^{\theta}
	=
	\big( V_{n,1}^{\theta}, \dots, V_{n,N}^{\theta} \big),
	\;\;\;
	W_{n}^{\theta}
	=
	\big( W_{n,1}^{\theta}, \dots, W_{n,N}^{\theta} \big)
\end{align}
for $n\geq 0$.
Notice here that $V_{n,i}^{\theta}$ and $W_{n,i}^{\theta}$ are the $i$-th columns of
$V_{n}^{\theta}$ and $W_{n}^{\theta}$.
Then, it is easy to show
$A_{n}^{\theta}\in{\cal P}^{N\times N}$ and
\begin{align}\label{4.1}
	V_{n}^{\theta}
	=
	W_{n}^{\theta}
	\left( I - \frac{ee^{T} }{N} \right),
	\;\;\;\;\;
	W_{n+1}^{\theta} = W_{n}^{\theta} A_{n+1}^{\theta} + B_{n+1}^{\theta},
\end{align}
where  $I$ is the $N\times N$ unit matrix.

\begin{vremark}
Throughout this and subsequent sections, the following convention is applied.
Diacritic $\tilde{}$ is used to denote a locally defined quantity,
i.e., a quantity whose definition holds only within the proof where
the quantity appears.
\end{vremark}

\begin{proposition}\label{proposition4.1}
Let Assumptions \ref{a1} -- \ref{a3} hold.
Then, there exist real numbers $\rho_{3}\in(0,1)$, $C_{3}\in[1,\infty)$
(independent of $N$ and depending only on $\varepsilon$, $d$, $K$) such that
\begin{align}\label{l4.1.1*}
	\big\| \hat{\zeta}_{n}^{\theta} \big\|
	\leq
	C_{3} \left(1 + \rho_{3}^{n} \|w_{\theta} \| \right)
\end{align}
for all $\theta\in\Theta$, $n\geq 0$.
\end{proposition}

\begin{proof}
Throughout the proof, the following notation is used.
$\rho_{3}$, $C_{3}$ are the real numbers defined by
$\rho_{3}=1-\varepsilon^{4}$, $C_{3}=8Kd\varepsilon^{-7}$
($\varepsilon$, $K$ are specified in Assumptions \ref{a1} and \ref{a2}).
$\tilde{A}_{k,l}^{\theta}$ is the matrix defined by
\begin{align}
	\tilde{A}_{k,k}^{\theta} = I,
	\;\;\;\;\;
	\tilde{A}_{k,l}^{\theta} = A_{k+1}^{\theta} \cdots A_{l}^{\theta}
\end{align}\label{l4.1.301}
for $l > k\geq 0$.

Iterating the second part of (\ref{4.1}), we get
\begin{align}\label{l4.1.1}
	W_{n}^{\theta}
	=
	W_{0}^{\theta} \tilde{A}_{0,n}^{\theta}
	+
	\sum_{k=1}^{n} B_{k}^{\theta} \tilde{A}_{k,n}^{\theta}
\end{align}
for $n\geq 1$.
Since $\tilde{A}_{0,n}^{\theta}\in{\cal P}^{N\times N}$,
we also have $e^{T}\tilde{A}_{0,n}^{\theta}=e^{T}$.
Consequently, the first part of (\ref{4.1}) implies
\begin{align*}
	V_{0}^{\theta} \tilde{A}_{0,n}^{\theta} \left(I - \frac{ee^{T} }{N} \right)
	=&
	W_{0}^{\theta} \tilde{A}_{0,n}^{\theta} \left(I - \frac{ee^{T} }{N} \right)
	-
	\frac{W_{0}^{\theta} e}{N}
	e^{T} \tilde{A}_{0,n}^{\theta} \left(I - \frac{ee^{T} }{N} \right)
	\\
	=&
	W_{0}^{\theta} \tilde{A}_{0,n}^{\theta} \left(I - \frac{ee^{T} }{N} \right).
\end{align*}
Combining this with the first part of (\ref{4.1}) and (\ref{l4.1.1}), we get
\begin{align}\label{l4.1.3}
	V_{n}^{\theta}
	=&
	W_{0}^{\theta} \tilde{A}_{0,n}^{\theta} \left(I - \frac{ee^{T} }{N} \right)
	+
	\sum_{k=1}^{n} B_{k}^{\theta} \tilde{A}_{k,n}^{\theta}
	\left(I - \frac{ee^{T} }{N} \right)
	\\ \nonumber
	=&
	V_{0}^{\theta} \tilde{A}_{0,n}^{\theta} \left(I - \frac{ee^{T} }{N} \right)
	+
	\sum_{k=1}^{n} B_{k}^{\theta} \tilde{A}_{k,n}^{\theta}
	\left(I - \frac{ee^{T} }{N} \right).
\end{align}

Owing to Assumptions \ref{a1}, \ref{a2}, we have
$\varepsilon^{2} \leq r_{\theta,\boldsymbol Y}^{k}\big(\hat{X}_{k,j}^{\theta} |\hat{X}_{k-1,i}^{\theta} \big)
\leq 1/\varepsilon^{2}$
and
\begin{align}\label{l4.1.501}
	\|\nabla_{\theta} r_{\theta,\boldsymbol Y}^{k}\big(\hat{X}_{k,j}^{\theta} |\hat{X}_{k-1,i}^{\theta} \big)
	\|
	\leq &
	q_{\theta}(Y_{k-1}|\hat{X}_{k-1,i} )
	\|\nabla_{\theta} p_{\theta}\big(\hat{X}_{k,j}^{\theta} |\hat{X}_{k-1,i}^{\theta} \big) \|
	\\ \nonumber
	&+
	p_{\theta}\big(\hat{X}_{k,j}^{\theta} |\hat{X}_{k-1,i}^{\theta} \big)
	\|\nabla_{\theta} q_{\theta}(Y_{k-1}|\hat{X}_{k-1,i} ) \|
	\leq
	\frac{2K}{\varepsilon}
\end{align}
for $1\leq i,j\leq N$, $k\geq 1$.
Therefore, we get
\begin{align*}
	&
	N\varepsilon^{2}
	\leq
	\sum_{i=1}^{N}
	r_{\theta,\boldsymbol Y}^{k}\big(\hat{X}_{k,j}^{\theta} |\hat{X}_{k-1,i}^{\theta} \big)
	\leq
	\frac{N}{\varepsilon^{2} },
	\;\;\;\;\;
	\sum_{i=1}^{N}
	\left\|
	\nabla_{\theta}
	r_{\theta,\boldsymbol Y}^{k}\big(\hat{X}_{k,j}^{\theta} |\hat{X}_{k-1,i}^{\theta} \big)
	\right\|
	\leq
	\frac{2KN}{\varepsilon}.
\end{align*}
Consequently, (\ref{4.5}) implies
\begin{align}\label{l4.1.5}
	A_{k,i,j}^{\theta}
	\geq
	\frac{\varepsilon^{4} }{N},
	\;\;\;\;\;
	\left\| B_{k,j}^{\theta} \right\|
	\leq
	\frac{2K}{\varepsilon^{3} },
	\;\;\;\;\;
	\left\| B_{k}^{\theta} \right\|
	=
	\max_{1\leq j\leq N} \|B_{k,j}^{\theta} \|
	\leq
	\frac{2K}{\varepsilon^{3} }.
\end{align}
Hence, (\ref{4.7}) yields
$\tau(A_{k}^{\theta} )\leq 1 - \varepsilon^{4} = \rho_{3}$.

Due to the well-known results in Markov chain theory (e.g. \cite[Theorems 15.2.4, 15.2.5]{bremaud}), we have
\begin{align}
	\tau(A'A'')\leq\tau(A')\tau(A''),
	\;\;\;\;\;
	\|A(z'-z'')\|_{1}\leq\tau(A)\|z'-z''\|_{1}
\end{align}
for any $A,A',A''\in{\cal P}^{N\times N}$, $z',z''\in{\cal P}^{N}$.
Then, using (\ref{l4.1.301}), we get
$\tau(\tilde{A}_{k,k}^{\theta} )=1$ and
\begin{align*}
	\tau(\tilde{A}_{k,l}^{\theta} )
	\leq
	\tau(A_{k+1}^{\theta} ) \cdots \tau(A_{l}^{\theta} )
	\leq
	\rho_{3}^{l-k}
\end{align*}
for $l>k\geq 0$.
Since $e_{i},\frac{e}{N}\in{\cal P}^{N}$, we deduce
\begin{align}\label{l4.1.23}
	\left\|
	\tilde{A}_{k,l}^{\theta} \left( e_{i} - \frac{e}{N} \right)
	\right\|_{1}
	\leq
	\tau(\tilde{A}_{k,l}^{\theta} )
	\left\|
	e_{i} - \frac{e}{N}
	\right\|_{1}
	\leq
	2\rho_{3}^{l-k}
\end{align}
for $1\leq i\leq N$, $l\geq k\geq 0$.
Consequently, (\ref{l4.1.5}) yields
\begin{align*}
	&
	\left\|
	V_{0}^{\theta} \tilde{A}_{0,l}^{\theta} \left( e_{i} - \frac{e}{N} \right)
	\right\|
	\leq
	\left\|V_{0}^{\theta}  \right\|
	\left\|
	\tilde{A}_{0,l}^{\theta} \left( e_{i} - \frac{e}{N} \right)
	\right\|_{1}
	\leq
	2\rho_{3}^{l} \left\|V_{0}^{\theta}  \right\|,
	\\
	&
	\left\|
	B_{k}^{\theta} \tilde{A}_{k,l}^{\theta} \left( e_{i} - \frac{e}{N} \right)
	\right\|
	\leq
	\left\|B_{k}^{\theta}  \right\|
	\left\|
	\tilde{A}_{k,l}^{\theta} \left( e_{i} - \frac{e}{N} \right)
	\right\|_{1}
	\leq
	\frac{4K\rho_{3}^{l-k} }{\varepsilon^{3} }.
\end{align*}
As vectors
$B_{k}^{\theta} \tilde{A}_{k,l}^{\theta} \left(e_{i} - \frac{e}{N} \right)$,
$V_{0}^{\theta} \tilde{A}_{0,l}^{\theta} \left(e_{i} - \frac{e}{N} \right)$
are the $i$-th columns of matrices
\linebreak
$B_{k}^{\theta} \tilde{A}_{k,l}^{\theta} \left(I - \frac{ee^{T} }{N} \right)$,
$V_{0}^{\theta} \tilde{A}_{0,l}^{\theta} \left(I - \frac{ee^{T} }{N} \right)$
(respectively), we conclude
\begin{align*}
	&
	\left\|
	V_{0}^{\theta} \tilde{A}_{0,l}^{\theta} \left( I - \frac{ee^{T} }{N} \right)
	\right\|
	=
	\max_{1\leq i\leq N}
	\left\|
	V_{0}^{\theta} \tilde{A}_{0,l}^{\theta} \left( e_{i} - \frac{e}{N} \right)
	\right\|
	\leq
	2\rho_{3}^{l} \left\|V_{0}^{\theta}  \right\|,
	\\
	&
	\left\|
	B_{k}^{\theta} \tilde{A}_{k,l}^{\theta} \left( I - \frac{ee^{T} }{N} \right)
	\right\|
	=
	\max_{1\leq i\leq N}
	\left\|
	B_{k}^{\theta} \tilde{A}_{k,l}^{\theta} \left( e_{i} - \frac{e}{N} \right)
	\right\|
	\leq
	\frac{4K\rho_{3}^{l-k} }{\varepsilon^{3} }.
\end{align*}
Hence, (\ref{l4.1.3}) implies
\begin{align}\label{l4.1.25}
	\left\| V_{n}^{\theta} \right\|
	\leq &
	\left\|
	V_{0}^{\theta} \tilde{A}_{0,n}^{\theta} \left( I - \frac{ee^{T} }{N} \right)
	\right\|
	+
	\sum_{k=1}^{n}
	\left\|
	B_{k}^{\theta} \tilde{A}_{k,n}^{\theta} \left( I - \frac{ee^{T} }{N} \right)
	\right\|
	\\ \nonumber
	\leq &
	2\rho_{3}^{n} \left\| V_{0}^{\theta} \right\|
	+
	\frac{4K}{\varepsilon^{3} } \sum_{k=1}^{n} \rho_{3}^{n-k}
	\leq
	\frac{4K}{\varepsilon^{7} }
	\left(
	1
	+
	\rho_{3}^{n} \left\| V_{0}^{\theta} \right\|
	\right)
\end{align}
for $n\geq 1$.

Since $W_{0,i}^{\theta} = w_{\theta}\big(\hat{X}_{0,i}^{\theta} \big)$,
Assumption \ref{a3} and (\ref{4.3}) yield
\begin{align*}
	\left\|V_{0,i}^{\theta} \right\|
	\leq
	\left\|W_{0,i}^{\theta} \right\|
	+
	\frac{1}{N} \sum_{j=1}^{N} \left\|W_{0,j}^{\theta} \right\|
	\leq
	2\|w_{\theta} \|
\end{align*}
for $1\leq i\leq N$.
Thus, we have $\|V_{0}^{\theta}\|\leq\|w_{\theta}\|$.
Consequently, (\ref{l4.1.25}) implies
\begin{align*}
	\left\|V_{n,i}^{\theta} \right\|
	\leq
	\left\|V_{n}^{\theta} \right\|
	\leq
	\frac{8K}{\varepsilon^{7} }
	\left(1 + \rho_{3}^{n} \|w_{\theta} \| \right)
\end{align*}
for $n\geq 0$.
As $\hat{\zeta}_{n}^{\theta}(dx)=\frac{1}{N}\sum_{j=1}^{N}V_{n,j}^{\theta}\delta_{\hat{X}_{n,j}^{\theta} }(dx)$
(due to (\ref{1.31}), (\ref{4.3})),
we get for $B\in{\cal B}({\cal X} )$
\begin{align*}
	\big\| \hat{\zeta}_{n}^{\theta}(B) \big\|
	\leq
	\frac{1}{N} \sum_{i=1}^{N} \big\| V_{n,i}^{\theta} \big\|
	\leq
	\frac{8K}{\varepsilon^{7} }
	\left(1 + \rho_{3}^{n} \|w_{\theta} \| \right).
\end{align*}
Hence, we have
\begin{align*}
	\big\| \hat{\zeta}_{n} \big\|
	\leq
	\frac{8dK}{\varepsilon^{7} }
	\left(1 + \rho_{3}^{n} \|w_{\theta} \| \right)
	=
	C_{3} \left(1 + \rho_{3}^{n} \|w_{\theta} \| \right).
\end{align*}
\end{proof}

\section{Proof of Main Results}\label{section5}

In this section, Proposition \ref{proposition5.1} is proved, while
Theorem \ref{theorem1.1} directly follows from it.
Lemma \ref{lemma5.2} and decompositions (\ref{l5.3.25*}), (\ref{p5.1.1}), (\ref{p5.1.29})
can be considered as the corner-stones in the proof of Proposition \ref{proposition5.1}
--- see inequalities
(\ref{p5.1.3}) -- (\ref{p5.1.25}), (\ref{p5.1.31}),
(\ref{p5.1.33}).
Proposition \ref{proposition2.1},
conditional distributions (\ref{l5.2.7}), (\ref{l5.2.9}) and
identities (\ref{l5.2.3}), (\ref{l5.2.53}), (\ref{l5.2.49})
are the main ingredients in the proof of Lemma \ref{lemma5.2}
--- see inequalities (\ref{l5.2.25}), (\ref{l5.2.27}), (\ref{l5.2.59}), (\ref{l5.2.61}),
(\ref{l5.2.67}).
Propositions \ref{proposition3.2}, \ref{proposition4.1} and Lemma \ref{lemma5.3} are important ingredients
of the proof of Lemma \ref{lemma5.2}, too
--- see inequalities (\ref{l5.2.21}), (\ref{l5.2.23}), (\ref{l5.2.57}),
(\ref{l5.2.201}), (\ref{l5.2.63}).
Proposition \ref{proposition3.2} plays an important role in the proof of Lemma \ref{lemma5.2}, either
--- see inequalities (\ref{l5.1.5}) -- (\ref{l5.1.9}).

Throughout this section, the following notation is used: $u_{\theta}(x)$, $\bar{w}_{\theta}$ and $\zeta_{\theta}(dx)$ are the functions
and the element of ${\cal M}_{s}^{d}({\cal X} )$ defined by
\begin{align}\label{5.1}
	\bar{w}_{\theta} = \int w_{\theta}(x') \xi_{\theta}(dx'),
	\;\;\;\;\;
	u_{\theta}(x) = w_{\theta}(x) - \bar{w}_{\theta},
	\;\;\;\;\;
	\zeta_{\theta}(B) = \int_{B} u_{\theta}(x')\xi_{\theta}(dx')
\end{align}
for $\theta\in\Theta$, $x\in{\cal X}$, $B\in{\cal B}({\cal X} )$.
$\hat{\xi}_{-1}^{\theta}(dx)$ and $\hat{\zeta}_{-1}^{\theta}(dx)$
are the elements of
${\cal P}({\cal X})$ and ${\cal M}_{s}^{d}({\cal X})$ (respectively)
defined by
\begin{align}\label{5.5}
	\hat{\xi}_{-1}^{\theta}(B) = \xi_{\theta}(B),
	\;\;\;\;\;
	\hat{\zeta}_{-1}^{\theta}(B) = \zeta_{\theta}(B).
\end{align}
$\hat{v}_{\theta,\boldsymbol Y}^{n}(x)$
is the (random) function defined by
\begin{align}\label{5.3}
	\hat{v}_{\theta,\boldsymbol Y}^{0}(x)
	\!=\!
	u_{\theta}(x),
	\;\;\;
	\hat{v}_{\theta,\boldsymbol Y}^{n}(x)
	\!=\!
	\frac{\int r_{\theta,\boldsymbol Y}^{n}(x|x') \hat{\zeta}_{n-1}^{\theta}(dx')
	+ \int \nabla_{\theta} r_{\theta,\boldsymbol Y}^{n}(x|x') \hat{\xi}_{n-1}^{\theta}(dx') }
	{\int r_{\theta,\boldsymbol Y}^{n}(x|x') \hat{\xi}_{n-1}^{\theta}(dx') }
\end{align}
for $n\geq 1$ ($r_{\theta,\boldsymbol Y}^{n}(x|x')$ and $\boldsymbol Y$ are defined in
(\ref{1.1}) and Subsection \ref{ssection1.3}, respectively).
$\hat{\alpha}_{\theta,\boldsymbol Y}^{n}(dx|\xi)$ and
$\hat{\beta}_{\theta,\boldsymbol Y}^{n}(dx)$
are the (random) elements of ${\cal M}_{s}^{d}({\cal X})$ defined by
\begin{align}\label{5.9}
	\hat{\alpha}_{\theta,\boldsymbol Y}^{n}(B|\xi)
	=
	\int_{B} \hat{v}_{\theta,\boldsymbol Y}^{n}(x) \xi(dx),
	\;\;\;\;\;
	\hat{\beta}_{\theta,\boldsymbol Y}^{n}(B)
	=
	\hat{\alpha}_{\theta,\boldsymbol Y}^{n}(B|\hat{\xi}_{n}^{\theta} )
\end{align}
for $\xi\in{\cal P}({\cal X})$, $n\geq 0$.
$\hat{\alpha}_{\theta,\boldsymbol Y}^{n}(\xi)$ and $\hat{\beta}_{\theta,\boldsymbol Y}^{n}$ are a `short-hand' notation for
$\hat{\alpha}_{\theta,\boldsymbol Y}^{n}(dx|\xi)$ and $\hat{\beta}_{\theta,\boldsymbol Y}^{n}(dx)$
(respectively).
$\hat{F}_{\theta,\boldsymbol Y}^{m:n}(dx)$,
$\hat{G}_{\theta,\boldsymbol Y}^{m:n}(dx)$ and
$\hat{H}_{\theta,\boldsymbol Y}^{m:n}(dx)$
are the (random) elements of ${\cal P}({\cal X})$, ${\cal M}_{s}^{d}({\cal X})$ and
${\cal M}_{s}^{d}({\cal X})$ (respectively) defined by
\begin{align}
	\label{5.7.1}
	\hat{F}_{\theta,\boldsymbol Y}^{-1:n}(B)
	=
	F_{\theta,\boldsymbol Y}^{0:n}(B|\hat{\xi}_{-1}^{\theta} ),
	&\;\;\;\;\;
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(B)
	=
	F_{\theta,\boldsymbol Y}^{m:n}(B|\hat{\xi}_{m}^{\theta} ),
	\\
	\label{5.7.3}
	\hat{G}_{\theta,\boldsymbol Y}^{-1:n}(B)
	=
	G_{\theta,\boldsymbol Y}^{0:n}(B|\hat{\xi}_{-1}^{\theta}, \hat{\zeta}_{-1}^{\theta} ),
	&	\;\;\;\;\;
	\hat{G}_{\theta,\boldsymbol Y}^{m:n}(B)
	=
	G_{\theta,\boldsymbol Y}^{m:n}(B|\hat{\xi}_{m}^{\theta}, \hat{\zeta}_{m}^{\theta} ),
	\\
	\label{5.7.5}
	\hat{H}_{\theta,\boldsymbol Y}^{-1:n}(B)
	=
	H_{\theta,\boldsymbol Y}^{0:n}(B|\hat{\xi}_{-1}^{\theta}, \hat{\zeta}_{-1}^{\theta} ),
	&	\;\;\;\;\;
	\hat{H}_{\theta,\boldsymbol Y}^{m:n}(B)
	=
	H_{\theta,\boldsymbol Y}^{m:n}(B|\hat{\xi}_{m}^{\theta}, \hat{\zeta}_{m}^{\theta} )
\end{align}
for $n\geq m\geq 0$
($F_{\theta,\boldsymbol y}^{m:n}(dx|\xi)$,
$G_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$, $H_{\theta,\boldsymbol y}^{m:n}(dx|\xi,\zeta)$
are specified in (\ref{1.21}) -- (\ref{1.25})).
$\hat{F}_{\theta,\boldsymbol Y}^{m:n}$, $\hat{G}_{\theta,\boldsymbol Y}^{m:n}$ and
$\hat{H}_{\theta,\boldsymbol Y}^{m:n}$ are a `short-hand' notation for
$\hat{F}_{\theta,\boldsymbol Y}^{m:n}(dx)$, $\hat{G}_{\theta,\boldsymbol Y}^{m:n}(dx)$ and
$\hat{H}_{\theta,\boldsymbol Y}^{m:n}(dx)$ (respectively).
$\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(dx'|x)$ and
$\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(dx'|x)$
are the (random) elements of ${\cal M}_{s}^{d}({\cal X})$ defined by
\begin{align}
	&\label{5.21}
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(B|x)
	=
	R_{\theta,\boldsymbol Y}^{m:n}(B|x) \hat{v}_{\theta,\boldsymbol Y}^{m}(x)
	+
	S_{\theta,\boldsymbol Y}^{m:n}(B|x),
	\\
	&\label{5.23}
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(B|x)
	=
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(B|x)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(B)
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}({\cal X}|x).
\end{align}
$\hat{C}_{\theta,\boldsymbol Y}^{m:n}(dx|\xi)$,
$\hat{B}_{\theta,\boldsymbol Y}^{m:n}(dx|\xi)$ and
$\hat{A}_{\theta,\boldsymbol Y}^{m:n}(dx|\xi)$
are the (random) elements of ${\cal M}_{s}^{d}({\cal X})$ defined by
\begin{align}
	&\label{5.25}
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	=
	\frac{\int \hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(B|x) \xi(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}({\cal X}|x) \xi(dx) },
	\\
	&\label{5.27}
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	=
	-
	\big(
	F_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(B)
	\big)
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}({\cal X}|\xi),
	\\
	&\label{5.29}
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	=
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(B)
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}({\cal X}|\xi).
\end{align}
$\hat{C}_{\theta,\boldsymbol Y}^{m:n}(\xi)$, $\hat{B}_{\theta,\boldsymbol Y}^{m:n}(\xi)$
and $\hat{A}_{\theta,\boldsymbol Y}^{m:n}(\xi)$ are a `short-hand' notation for
$\hat{C}_{\theta,\boldsymbol Y}^{m:n}(dx|\xi)$, $\hat{B}_{\theta,\boldsymbol Y}^{m:n}(dx|\xi)$
and $\hat{A}_{\theta,\boldsymbol Y}^{m:n}(dx|\xi)$ (respectively).

\begin{lemma}\label{lemma5.3}
(i) Let Assumption \ref{a1} hold. Then, we have
\begin{align}\label{l5.3.1*}
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(B)
	=
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(B|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}({\cal X}|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
\end{align}
for all $\theta\in\Theta$, $B\in{\cal B}({\cal X})$,
$n\geq m\geq 0$.

(ii) Let Assumptions \ref{a1} -- \ref{a3} hold. Then, we have
\begin{align}
	&\label{l5.3.21*}
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	=
	H_{\theta,\boldsymbol Y}^{m:n}
	\big(B|\xi, \hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi) \big),
	\\
	&\label{l5.3.25*}
	\hat{G}_{\theta,\boldsymbol Y}^{m:n}(B)
	=
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(B|\hat{\xi}_{m}^{\theta} )
	+
	\hat{B}_{\theta,\boldsymbol Y}^{m:m}(B|\hat{\xi}_{m}^{\theta} ),
	\\
	&\label{l5.3.23*}
	\begin{aligned}[t]
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	=&
	\big(
	F_{\theta,\boldsymbol Y}^{m:n}(B|\xi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(B)
	\big)
	H_{\theta,\boldsymbol Y}^{m:n}
	\big({\cal X}|\xi, \hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi) \big)
	\\
	&
	+
	G_{\theta,\boldsymbol Y}^{m:n}
	\big(B|\xi, \hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi) \big)
	\end{aligned}
\end{align}
for all $\theta\in\Theta$, $B\in{\cal B}({\cal X})$, $\xi\in{\cal P}({\cal X})$,
$n\geq m\geq 0$.
We also have
\begin{align}
	\label{l5.3.3*}
	\hat{H}_{\theta,\boldsymbol Y}^{m\!-\!1:n}(B)
	\!=\!
	\frac{\int\! \hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(B|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) }
	{\int\! R_{\theta,\boldsymbol Y}^{m:n}({\cal X}|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) },
	\;\:
	\hat{G}_{\theta,\boldsymbol Y}^{m\!-\!1:n}(B)
	\!=\!
	\frac{\int\! \hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(B|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) }
	{\int\! R_{\theta,\boldsymbol Y}^{m:n}({\cal X}|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) }
\end{align}
for the same $\theta$, $B$, $n$, $m$.
\end{lemma}

Lemma \ref{lemma5.3} summarizes relatively straightforward relationships between
the measures defined in (\ref{5.7.1}) -- (\ref{5.7.5}) and (\ref{5.25}) -- (\ref{5.29}).
A detailed proof of the lemma is provided in the supplementary material
(Section SM3).

\begin{lemma}\label{lemma5.1}
Let Assumptions \ref{a1} and \ref{a2} hold.
Then, there exist real numbers $\rho_{4}\in(0,1)$, $C_{4}\in[1,\infty)$
(depending only on $\varepsilon$, $d$, $K$) such that
\begin{align}
	&\label{l5.1.1*}
	\max\left\{
	\big\|
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(\hat{\xi}_{m}^{\theta} )
	\big\|,
	\big\| \hat{H}_{\theta, \boldsymbol Y}^{m-1:n} \big\|
	\right\}
	\leq
	C_{4} \left(1 + n - m + \rho_{4}^{m} \|w_{\theta} \| \right),
	\\
	&\label{l5.1.3*}
	\big\|
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(\xi')
	-
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(\xi'')
	\big\|
	\leq
	C_{4} \rho_{4}^{n-m} \left(1 + n - m + \rho_{4}^{m} \|w_{\theta} \| \right)
\end{align}
for all $\theta\in\Theta$, $\xi',\xi''\in{\cal P}({\cal X})$,
$n\geq m\geq 0$.
\end{lemma}

\begin{proof}
Throughout the proof, the following notation is used.
$\theta$ is any element of $\Theta$,
while $x$, $x'$ are any elements of ${\cal X}$.
$B$ is any element of ${\cal B}({\cal X} )$, while
$\xi$, $\xi'$, $\xi''$ are any elements of ${\cal P}({\cal X} )$.
$\rho_{4}$ is the real number defined by
$\rho_{4}=\max\big\{\rho_{1}, \rho_{2}, \rho_{3} \big\}$,
while $\tilde{C}_{1}$, $\tilde{C}_{2}$, $\tilde{C}_{3}$, $\tilde{C}_{4}$, $C_{4}$
are the real numbers defined as
$\tilde{C}_{1}=2dC_{3}$,
$\tilde{C}_{2}=4\tilde{C}_{1}K\varepsilon^{-4}\rho_{4}^{-1}$,
$\tilde{C}_{3}=d\tilde{C}_{2}$, $\tilde{C}_{4}=3C_{2}\tilde{C}_{1}\tilde{C}_{3}$,
$C_{4}=3C_{1}\tilde{C}_{4}$
($\varepsilon$, $\rho_{1}$, $\rho_{2}$, $K$, $C_{1}$, $C_{2}$ are specified in Assumptions \ref{a1}, \ref{a2}
and Proposition \ref{proposition3.2}).
$n$, $m$ are any integers satisfying $n\geq m\geq 0$.

Relying on Assumption \ref{a3} and (\ref{5.1}), (\ref{5.3}), we conclude
\begin{align}\label{l5.1.301}
	\|\hat{v}_{\theta,\boldsymbol Y}^{0}(x) \|
	=
	\|u_{\theta}(x) \|
	\leq
	\|w_{\theta}(x) \|
	+
	\int \|w_{\theta}(x') \|\xi_{\theta}(dx')
	\leq
	2\|w_{\theta} \|.
\end{align}
Consequently, (\ref{5.1}), (\ref{5.5}) imply
\begin{align*}
	\|\hat{\zeta}_{-1}^{\theta}(B) \|
	=
	\|\zeta_{\theta}(B) \|
	\leq
	\int_{B} \|u_{\theta}(x) \|\xi_{\theta}(dx)
	\leq
	2\|w_{\theta}\|.
\end{align*}
Hence, we have
$\|\hat{\zeta}_{-1}^{\theta}\|=\|\zeta_{\theta}\|\leq 2d\|w_{\theta}\|$.
Combining this with Proposition \ref{proposition4.1}, we get
\begin{align}\label{l5.1.303}
	\|\hat{\zeta}_{k}^{\theta} \|
	\leq
	2dC_{3}(1 + \rho_{3}^{k}\|w_{\theta}\| )
	\leq
	\tilde{C}_{1}(1 + \rho_{4}^{k}\|w_{\theta}\| )
\end{align}
for $k\geq -1$.

Using Assumptions \ref{a1}, \ref{a2} and the same arguments as in Proposition \ref{proposition4.1}
(see (\ref{l4.1.501})), we deduce
\begin{align*}
	\varepsilon^{2}
	\leq
	r_{\theta,\boldsymbol Y}^{k}(x'|x)
	\leq
	\frac{1}{\varepsilon^{2} },
	\;\;\;\;\;
	\big\|
	\nabla_{\theta} r_{\theta,\boldsymbol Y}^{k}(x'|x)
	\big\|
	\leq
	\frac{2K}{\varepsilon}
\end{align*}
for $k\geq 1$.
Then, (\ref{5.3}), (\ref{l5.1.303}) yield
\begin{align*}
	\left\|\hat{v}_{\theta,\boldsymbol Y}^{k}(x) \right\|
	\leq &
	\frac{
	\int r_{\theta,\boldsymbol Y}^{k}(x|x') \: |\hat{\zeta}_{k-1}^{\theta} |(dx')
	+
	\int \|\nabla_{\theta} r_{\theta,\boldsymbol Y}^{k}(x|x') \| \: \hat{\xi}_{k-1}^{\theta}(dx') }
	{\int r_{\theta,\boldsymbol Y}^{k}(x|x') \: \hat{\xi}_{k-1}^{\theta}(dx') }
	\\
	\leq &
	\frac{2K}{\varepsilon^{3} }
	+
	\frac{\|\hat{\zeta}_{k-1}^{\theta}\|}{\varepsilon^{4} }
	\leq
	\frac{4\tilde{C}_{1}K (1+\rho_{3}^{k-1}\|w_{\theta} \| ) }{\varepsilon^{4} }
	\leq
	\tilde{C}_{2} (1+\rho_{4}^{k}\|w_{\theta} \|  ).
\end{align*}
Combining this with (\ref{5.9}), (\ref{l5.1.301}), we get
\begin{align}\label{l5.1.1}
	\left\|\hat{\alpha}_{\theta,\boldsymbol Y}^{k}(B|\xi) \right\|
	\leq
	\int_{B} \left\|\hat{v}_{\theta,\boldsymbol Y}^{k}(x) \right\| \xi(dx)
	\leq
	\tilde{C}_{2} (1+\rho_{4}^{k}\|w_{\theta} \|  )
\end{align}
for $k\geq 0$.
Thus, we have
\begin{align}\label{l5.1.3}
	\left\|\hat{\alpha}_{\theta,\boldsymbol y}^{k}(\xi) \right\|
	\leq
	d\tilde{C}_{2}(1 + \rho_{4}^{k} \|w_{\theta} \| )
	=
	\tilde{C}_{3}(1 + \rho_{4}^{k} \|w_{\theta} \| ).
\end{align}
Consequently, Proposition \ref{proposition3.2} and Lemma \ref{lemma5.3} imply
\begin{align}\label{l5.1.5}
	\left\| \hat{C}_{\theta,\boldsymbol Y}^{m:n}(\xi) \right\|
	=
	\left\| H_{\theta,\boldsymbol Y}^{m:n}(\xi,\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi) ) \right\|
	\leq &
	C_{2}
	\left(
	n-m
	+
	\left\|\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi) \right\|
	\right)
	\\ \nonumber
	\leq &
	C_{2}\tilde{C}_{3}
	\left(
	1 + n - m + \rho_{4}^{m}\|w_{\theta}\|
	\right)
	\\ \nonumber
	\leq&
	\tilde{C}_{4}
	\left(
	1 + n - m + \rho_{4}^{m}\|w_{\theta}\|
	\right).
\end{align}
Moreover, if $m\geq 1$, Proposition \ref{proposition3.2}
and (\ref{5.7.5}), (\ref{l5.1.303}) yield
\begin{align}\label{l5.1.5'}
	\left\| \hat{H}_{\theta,\boldsymbol Y}^{m-1:n} \right\|
	=
	\left\| H_{\theta,\boldsymbol Y}^{m-1:n}(\hat{\xi}_{m-1}^{\theta}, \hat{\zeta}_{m-1}^{\theta} ) \right\|
	\leq &
	C_{2} \big(n - m + \|\hat{\zeta}_{m-1}^{\theta} \| \big)
	\\ \nonumber
	\leq &
	C_{2}\tilde{C}_{1}(1 + n - m + \rho_{4}^{m-1} \|w_{\theta} \| )
	\\ \nonumber
	\leq &
	\tilde{C}_{4}(1 + n - m + \rho_{4}^{m} \|w_{\theta} \| ).
\end{align}
The same arguments also imply
\begin{align}\label{l5.1.305}
	\left\| \hat{H}_{\theta,\boldsymbol Y}^{-1:n} \right\|
	=
	\left\| H_{\theta,\boldsymbol Y}^{0:n}(\hat{\xi}_{-1}^{\theta}, \hat{\zeta}_{-1}^{\theta} ) \right\|
	\leq
	C_{2} \big(n + \|\hat{\zeta}_{-1}^{\theta} \| \big)
	\leq &
	C_{2}\tilde{C}_{1}(1 + n + \rho_{4}^{-1} \|w_{\theta} \| )
	\\ \nonumber
	\leq &
	\tilde{C}_{4}(1 + n + \rho_{4}^{-1} \|w_{\theta} \| ).
\end{align}
Using (\ref{l5.1.5}) -- (\ref{l5.1.305}), we conclude that (\ref{l5.1.1*}) holds.

Owing to Proposition \ref{proposition3.2} and Lemma \ref{lemma5.3}, we have
\begin{align}\label{l5.1.7}
	\left\|
	F_{\theta,\boldsymbol Y}^{m:n}(\xi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}
	\right\|
	=
	\left\|
	F_{\theta,\boldsymbol Y}^{m:n}(\xi)
	-
	F_{\theta,\boldsymbol Y}^{m:n}
	\big( \hat{F}_{\theta,\boldsymbol Y}^{m-1:m} \big)
	\right\|
	\leq
	C_{1} \rho_{1}^{n-m}.
\end{align}
Due to the same lemma and (\ref{l5.1.3}), we also have
\begin{align}\label{l5.1.9}
	\left\|
	G_{\theta,\boldsymbol Y}^{m:n}(\xi',\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi') )
	-
	G_{\theta,\boldsymbol Y}^{m:n}(\xi'',\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi'') )
	\right\|
	\leq &
	C_{2}\rho_{2}^{n-m}
	\!\left(
	1
	+\!
	\left\|\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi') \right\|
	+\!
	\left\|\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi'') \right\|
	\right)
	\\ \nonumber
	\leq &
	3C_{2}\tilde{C}_{3}\rho_{2}^{n-m}
	\left( 1 + \rho_{4}^{m} \|w_{\theta}\| \right)
	\\ \nonumber
	\leq &
	\tilde{C}_{4}\rho_{2}^{n-m}
	\left( 1 + \rho_{4}^{m} \|w_{\theta}\| \right).
\end{align}
Combining Lemma \ref{lemma5.3} and (\ref{l5.1.5}), (\ref{l5.1.7}),  (\ref{l5.1.9}), we get
\begin{align}\label{l5.1.21}
	\left\|
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(\xi')
	-
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(\xi'')
	\right\|
	\leq &
	\left\|
	G_{\theta,\boldsymbol Y}^{m:n}(\xi',\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi') )
	-
	G_{\theta,\boldsymbol Y}^{m:n}(\xi'',\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi'') )
	\right\|
	\\ \nonumber
	&+
	\left\|
	F_{\theta,\boldsymbol Y}^{m:n}(\xi')
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}
	\right\|
	\left\| H_{\theta,\boldsymbol Y}^{m:n}(\xi',\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi') ) \right\|
	\\ \nonumber
	&+
	\left\|
	F_{\theta,\boldsymbol Y}^{m:n}(\xi'')
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}
	\right\|
	\left\| H_{\theta,\boldsymbol Y}^{m:n}(\xi'',\hat{\alpha}_{\theta,\boldsymbol Y}^{m}(\xi'') ) \right\|
	\\ \nonumber
	\leq &
	\tilde{C}_{4}\rho_{2}^{n-m}
	\!\left( 1 + \!\rho_{4}^{m} \|w_{\theta}\| \right)
	+
	2C_{1}\tilde{C}_{4}\rho_{1}^{n-m}
	\!\left(
	1 + n - m + \!\rho_{4}^{m} \|w_{\theta}\|
	\right)
	\\ \nonumber
	\leq &
	3C_{1}\tilde{C}_{4}\rho_{4}^{n-m}
	\left( 1 + n - m + \!\rho_{4}^{m} \|w_{\theta}\| \right).
\end{align}
Using (\ref{l5.1.21}),
we deduce that (\ref{l5.1.3*}) holds.
\end{proof}

\begin{lemma}\label{lemma5.2}
(i) Let Assumption \ref{a1} hold.
Then, there exists a real number $C_{5}\in[1,\infty)$
(independent of $N$ and depending only on $\varepsilon$) such that
\begin{align}
	&\label{l5.2.21*}
	\left|
	E\left(\left.
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|
	\boldsymbol Y
	\right)
	\right|
	\leq
	\frac{C_{5}\rho_{1}^{n-m} }{N},
	\\
	&\label{l5.2.23*}
	\left(
	E\left(\left.
	\left|
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|^{2}
	\right|
	\boldsymbol Y
	\right)
	\right)^{1/2}
	\leq
	\frac{C_{5}\rho_{1}^{n-m} }{\sqrt{N} }
\end{align}
almost surely for all $\theta\in\Theta$, $n\geq m\geq 0$
and any Borel-measurable function $\varphi:{\cal X}\rightarrow[-1,1]$
($\rho_{1}$ is specified in Proposition \ref{proposition3.2}).

(ii) Let Assumptions \ref{a1} -- \ref{a3} hold.
Then, there exist real numbers $\rho_{5}\in(0,1)$, $C_{6}\in[1,\infty)$
(independent of $N$ and depending only on $\varepsilon$, $d$, $K$) such that
\begin{align}
	&\label{l5.2.1*}
	\left\|
	E\left(\left.
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|
	\boldsymbol Y
	\right)
	\right\|
	\leq
	\frac{C_{6}(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| ) }{N},
	\\
	&\label{l5.2.3*}
	\left(
	E\left(\left.
	\left\|
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right\|^{2}
	\right|
	\boldsymbol Y
	\right)
	\right)^{1/2}
	\leq
	\frac{C_{6}(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| ) }{\sqrt{N} },
	\\
	&\label{l5.2.5*}
	\left\|
	E\left(\left.
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	\right|
	\boldsymbol Y
	\right)
	\right\|
	\leq
	\frac{C_{6}(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| ) }{N},
	\\
	&\label{l5.2.7*}
	\left(
	E\left(\left.
	\left\|
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	\right\|^{2}
	\right|
	\boldsymbol Y
	\right)
	\right)^{1/2}
	\leq
	\frac{C_{6}(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| ) }{\sqrt{N} }
\end{align}
almost surely for all $\theta\in\Theta$, $n\geq m\geq 0$
and any Borel-measurable function $\varphi:{\cal X}\rightarrow[-1,1]$.
\end{lemma}

\begin{proof}
(i) Throughout this part of the proof,
the following notation is used.
$\theta$ is any element of $\Theta$,
while $\xi'$, $\xi''$ are any elements of ${\cal P}({\cal X} )$.
$\varphi(x)$ is any Borel-measurable function mapping $x\in{\cal X}$ to $[-1,1]$.
$n,m$ are any integers satisfying $n\geq m\geq 0$.

Relying on (\ref{1.1}), (\ref{1.3}), (\ref{1.35}),
(\ref{1.7}), (\ref{1.21}), (\ref{5.7.1}), we conclude
\begin{align}\label{l5.2.7}
	&
	P\!\left(\!\left.
	\hat{X}_{k,1}^{\theta}\in B_{1}, \dots, \hat{X}_{k,N}^{\theta}\in B_{N}
	\right| \!\boldsymbol Y, \hat{\xi}_{k-1}^{\theta}
	\!\right)
	\\ \nonumber
	&=
	\!\prod_{i=1}^{N}
	\!\left(\!
	\frac{\int\!\int I_{B_{i} }(x') r_{\theta,\boldsymbol Y}^{k}(x'|x)
	\mu(dx') \hat{\xi}_{k-1}^{\theta}(dx) }
	{\int\!\int r_{\theta,\boldsymbol Y}^{k}(x'|x)
	\mu(dx') \hat{\xi}_{k-1}^{\theta}(dx) }
	\right)
	=
	\prod_{i=1}^{N}
	\hat{F}_{\theta,\boldsymbol Y}^{n-1:n} (B_{i} )
\end{align}
almost surely for any $B_{1},\dots,B_{N}\in{\cal B}({\cal X})$, $k\geq 1$.
Similarly, using (\ref{1.21}), (\ref{5.5}), (\ref{5.7.1}), we deduce
\begin{align}\label{l5.2.9}
	P\left(\left.
	\hat{X}_{0,1}^{\theta}\in B_{1}, \dots, \hat{X}_{0,N}^{\theta}\in B_{N}
	\right| \boldsymbol Y, \hat{\xi}_{-1}^{\theta}
	\right)
	=
	\prod_{i=1}^{N}
	\hat{\xi}_{-1}^{\theta}(B_{i} )
	=
	\prod_{i=1}^{N}
	\hat{F}_{\theta,\boldsymbol Y}^{-1:0} (B_{i} )
\end{align}
almost surely for the same $B_{1},\dots,B_{N}$.
Moreover, Lemma \ref{lemma5.3} and (\ref{1.21}), (\ref{5.7.1}) imply
\begin{align}
	&\label{l5.2.3}
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	=
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m} (dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) },
	\;\;\;\;\;
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	=
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{\xi}_{m}^{\theta}(dx) }.
\end{align}
Here and throughout the proof, $\boldsymbol 1(x)$ denotes the function which maps $x\in{\cal X}$ to one.

Let $C_{5}=2C_{1}^{3}$
($C_{1}$ is specified in Proposition \ref{proposition3.2}).
Owing to Proposition \ref{proposition3.2}, we have
\begin{align}\label{l5.2.21}
	\left|
	\frac{\int\! R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x) \xi'(dx) }
	{\int\! R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x) \xi'(dx) }
	\!-\!
	\frac{\int\! R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x) \xi''(dx) }
	{\int\! R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x) \xi''(dx) }
	\right|
	\!=\!
	\left|
	F_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi')
	\!-\!
	F_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi'')
	\right|
	\!\leq\!
	C_{1} \rho_{1}^{n\!-\!m}.
\end{align}
Due to Lemma \ref{lemma5.3}, we also have
\begin{align}\label{l5.2.23}
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x) \xi'(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x) \xi''(dx) }
	\leq
	C_{1}.
\end{align}
Using Proposition \ref{proposition2.1}
and (\ref{l5.2.7}) -- (\ref{l5.2.23}), we conclude
\begin{align}\label{l5.2.25}
	&
	\left|
	E\left(\left.
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|
	\boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right|
	\\ \nonumber
	&=
	\left|
	E\left(\left.
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x) \hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x) \hat{\xi}_{m}^{\theta}(dx) }
	-
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	\right|
	\boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right|
	\\ \nonumber
	&
	\leq
	\frac{2C_{1}^{3}\rho_{1}^{n-m} }{N}
	=
	\frac{C_{5}\rho_{1}^{n-m} }{N}
\end{align}
almost surely.\footnote
{To get (\ref{l5.2.25}), (\ref{l5.2.27}), the following should be done:
In Proposition \ref{proposition2.1}, set $z=x$, $k=N$ and replace
$f(z)$, $g(z)$, $\xi_{k}(dz)$, $\xi(dz)$ with
$R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)$, $R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)$,
$\hat{\xi}_{m}^{\theta}(dx)$, $\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx)$. }
Relying on the same arguments, we deduce
\begin{align}\label{l5.2.27}
	&
	E\left(\left.
	\left|
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|^{2}
	\right|
	\boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\!\right)
	\\ \nonumber
	&=
	E\left(\left.
	\left|
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x) \hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x) \hat{\xi}_{m}^{\theta}(dx) }
	-
	\frac{\int R_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	\right|^{2}
	\right|
	\boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\!\right)
	\\ \nonumber
	&
	\leq
	\left( \frac{2C_{1}^{2}\rho_{1}^{n-m} }{\sqrt{N} } \right)^{2}
	\leq
	\left( \frac{C_{5}\rho_{1}^{n-m} }{\sqrt{N} } \right)^{2}
\end{align}
almost surely.
Combining (\ref{l5.2.25}), (\ref{l5.2.27}) with the tower property of conditional expectations,
we conclude that (\ref{l5.2.21*}), (\ref{l5.2.23*}) hold almost surely.

(ii) Let $\theta$, $\xi$, $\xi'$, $\xi''$, $\varphi(x)$, $n$, $m$ have the same meaning as in (i).
Using (\ref{5.23}), (\ref{5.25}), (\ref{5.29}), it is straightforward to verify
\begin{align}\label{l5.2.301}
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|\xi)
	=
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|x)
	\xi(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\xi(dx) }.
\end{align}
Hence, (\ref{5.25}) implies
\begin{align}
	&\label{l5.2.53}
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	=
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{\xi}_{m}^{\theta}(dx) },
	\;\;\;\;\;
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	=
	\frac{\int
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{\xi}_{m}^{\theta}(dx) }.
\end{align}
Similarly, Lemma \ref{lemma5.3} yields
\begin{align}\label{l5.2.49}
	&
	\hat{G}_{\theta,\boldsymbol Y}^{m\!-\!1:n}
	(\varphi)
	\!=\!
	\frac{\int\!
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) }
	{\int\! R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) },
	\;\;\;
	\hat{H}_{\theta,\boldsymbol Y}^{m\!-\!1:n}
	(\varphi)
	\!=\!
	\frac{\int\!
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) }
	{\int\! R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m\!-\!1:m}(dx) }.
\end{align}

Let $\rho_{5}=\max\{\sqrt{\rho_{1} },\sqrt{\rho_{4} } \}$,
$\tilde{C}_{1}=\max_{n\geq 1} n\rho_{5}^{n}$,
$\tilde{C}_{2}=2C_{4}\tilde{C}_{1}$,
$\tilde{C}_{3}=2C_{1}^{2}\tilde{C}_{2}$
($\rho_{1}$, $\rho_{4}$, $C_{1}$, $C_{4}$ are specified in
Proposition \ref{proposition3.2} and  Lemma \ref{lemma5.1}).
Since $\rho_{4}\leq\rho_{5}$, $\rho_{4}^{n-m}(n-m)\leq\tilde{C}_{1}\rho_{5}^{n-m}$,
Lemma \ref{lemma5.1} and (\ref{l5.2.301}) imply
\begin{align}\label{l5.2.57}
	\left\|
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|x)
	\xi'(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\xi'(dx) }
	\!-\!
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}
	(\varphi|x)
	\xi''(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\xi''(dx) }
	\right\|
	&\!=\!
	\left\|
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi')
	-
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi'')
	\right\|
	\\ \nonumber
	&\!\leq\!
	C_{4} \rho_{4}^{n-m}
	\left(1 + n - m + \rho_{4}^{m} \|w_{\theta} \| \right)
	\\ \nonumber
	&\!\leq\!
	2C_{4}\tilde{C}_{1}
	\left(\rho_{5}^{n-m} + \rho_{5}^{m} \|w_{\theta} \| \right)
	\\ \nonumber
	&\!\leq\!
	\tilde{C}_{2} \left(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta} \| \right).
\end{align}
Using Proposition \ref{proposition2.1}
and (\ref{l5.2.7}), (\ref{l5.2.9}), (\ref{l5.2.23}),
(\ref{l5.2.53}) -- (\ref{l5.2.57}), we conclude
\begin{align}\label{l5.2.59}
	&
	\left\|
	E\left(\left.
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right\|
	\\ \nonumber
	&=
	\left\|
	E\left(\left.
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	-
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right\|
	\\ \nonumber
	&\leq
	\frac{2C_{1}^{2}\tilde{C}_{2} \left(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta} \| \right) }
	{N}
	=
	\frac{\tilde{C}_{3} \left(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta} \| \right)}
	{N}
\end{align}
almost surely.\footnote
{To get (\ref{l5.2.59}), (\ref{l5.2.61}), the following should be done:
In Proposition \ref{proposition2.1}, set $z=x$, $k=N$ and replace
$f(z)$, $g(z)$, $\xi_{k}(dz)$, $\xi(dz)$ with
$\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)$,
$R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)$,
$\hat{\xi}_{m}^{\theta}(dx)$, $\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx)$. }
Relying on the same arguments, we deduce
\begin{align}\label{l5.2.61}
	&
	E\left(\left.
	\left\|
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right\|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\\ \nonumber
	&=
	E\left(\left.
	\left\|
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	-
	\frac{\int
	\hat{\Phi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	\right\|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\\ \nonumber
	&\leq
	\left(
	\frac{2C_{1}\tilde{C}_{2} \left(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta} \| \right)}
	{\sqrt{N} }
	\right)^{2}
	\leq
	\left(
	\frac{\tilde{C}_{3}\left(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta} \| \right)}
	{\sqrt{N} }
	\right)^{2}
\end{align}
almost surely.

Let $\tilde{C}_{4}=C_{5}\tilde{C}_{2}$
($C_{5}$ is defined in (i)).
Since $\tilde{C}_{1}\rho_{5}^{m-n}\geq n-m$,
Lemma \ref{lemma5.1} implies
\begin{align}\label{l5.2.201}
	\left\|
	\hat{H}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right\|
	\leq
	C_{4} \left(1 + n - m + \rho_{4}^{m} \|w_{\theta} \| \right)
	\leq &
	2C_{4} \tilde{C}_{1} \left(\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta} \| \right)
	\\ \nonumber
	\leq &
	\tilde{C}_{2} \left(\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta} \| \right).
\end{align}
As $\hat{H}_{\theta,\boldsymbol Y}^{m-1:n}(\boldsymbol 1)$
is measurable with respect to $\boldsymbol Y$, $\hat{\xi}_{m-1}^{\theta}$
and
$\rho_{1}^{n-m}\leq \rho_{5}^{2(n-m)}$,
(\ref{l5.2.25}), (\ref{l5.2.201}) yield
\begin{align}\label{l5.2.73}
	&
	\left\|
	E\left(\left(\left.
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right)
	\hat{H}_{\theta,\boldsymbol Y}^{m-1:n}(\boldsymbol 1)
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right\|
	\\ \nonumber
	&\leq
	\left|
	E\left(\left.
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right|
	\left\|
	\hat{H}_{\theta,\boldsymbol Y}^{m-1:n}(\boldsymbol 1)
	\right\|
	\\ \nonumber
	&
	\leq
	\frac{C_{5}\tilde{C}_{2} \rho_{1}^{n-m}
	(\rho_{5}^{m-n} + \rho_{5}^{m}\|w_{\theta}\| ) }
	{N}
	\leq
	\frac{\tilde{C}_{4}
	(\rho_{5}^{n-m} + \rho_{5}^{n}\|w_{\theta}\| ) }
	{N}
\end{align}
almost surely.

Let $\tilde{C}_{5}=C_{5}\tilde{C}_{3}$
($C_{5}$ is defined in (i)).
Since $\tilde{C}_{1}\rho_{5}^{m-n}\geq n-m$,
Lemma \ref{lemma5.1} implies
\begin{align}\label{l5.2.63}
	\left\|
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	\right\|
	\leq
	C_{4}
	\left(1 + n - m + \rho_{4}^{m} \|w_{\theta} \| \right)
	\leq &
	2C_{4}\tilde{C}_{1}
	\left(\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta} \| \right)
	\\ \nonumber
	\leq &
	\tilde{C}_{2} \left(\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta} \| \right).
\end{align}
Consequently, (\ref{5.25}) yields
\begin{align}\label{l5.2.65}
	\left\|
	\frac{\int
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\xi'(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\xi'(dx) }
	\!-\!
	\frac{\int
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\xi''(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\xi''(dx) }
	\right\|
	&
	\!=\!
	\left\|
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi')
	-
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi'')
	\right\|
	\\ \nonumber
	&\!\leq\!
	\left\|
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi')
	\right\|
	+
	\left\|
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}(\varphi|\xi'')
	\right\|
	\\ \nonumber
	&\!\leq\!
	2\tilde{C}_{2} \left(\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta} \| \right).
\end{align}
Using Proposition \ref{proposition2.1}
and (\ref{l5.2.7}), (\ref{l5.2.9}), (\ref{l5.2.23}),
(\ref{l5.2.49}), (\ref{l5.2.53}), (\ref{l5.2.65}),
we conclude
\begin{align}\label{l5.2.67}
	&
	E\left(\left.
	\left\|
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{H}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right\|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\\ \nonumber
	&=
	E\left(\left.
	\left\|
	\frac{\int
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{\xi}_{m}^{\theta}(dx) }
	-
	\frac{\int
	\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	{\int R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx) }
	\right\|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\\ \nonumber
	&\leq
	\left(
	\frac{2C_{1}\tilde{C}_{2} \left(\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta} \| \right)}
	{\sqrt{N} }
	\right)^{2}
	\leq
	\left(
	\frac{\tilde{C}_{3} \left(\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta} \| \right)}
	{\sqrt{N} }
	\right)^{2}
\end{align}
almost surely.\footnote
{To get (\ref{l5.2.67}), the following should be done:
In Proposition \ref{proposition2.1}, set $z=x$, $k=N$ and replace
$f(z)$, $g(z)$, $\xi_{k}(dz)$, $\xi(dz)$ with
$\hat{\Psi}_{\theta,\boldsymbol Y}^{m:n}(\varphi|x)$,
$R_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1|x)$,
$\hat{\xi}_{m}^{\theta}(dx)$, $\hat{F}_{\theta,\boldsymbol Y}^{m-1:m}(dx)$. }
As $\rho_{1}^{n-m}\leq \rho_{5}^{2(n-m)}$,
H\"{o}lder inequality and (\ref{l5.2.27}), (\ref{l5.2.67}) imply
\begin{align}\label{l5.2.71}
	&
	\left\|
	E\left(\left.
	\left(
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right)
	\left(
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\boldsymbol 1|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{H}_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1)
	\right)
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right\|
	\\ \nonumber
	&
	\begin{aligned}[b]
	\leq &
	\left(
	E\left(\left.
	\left|
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right)^{1/2}
	\\
	&\cdot
	\left(
	E\left(\left.
	\left\|
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\boldsymbol 1|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{H}_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1)
	\right\|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right)^{1/2}
	\end{aligned}
	\\ \nonumber
	&
	\leq
	\frac{C_{5}\tilde{C}_{3}\rho_{1}^{n-m}
	(\rho_{5}^{m-n} + \rho_{5}^{m}\|w_{\theta}\| ) }
	{N}
	\leq
	\frac{\tilde{C}_{5}
	(\rho_{5}^{n-m} + \rho_{5}^{n}\|w_{\theta}\| ) }
	{N}
\end{align}
almost surely.
Similarly, (\ref{l5.2.27}), (\ref{l5.2.63}) yield
\begin{align}\label{l5.2.75}
	&
	E\left(\left.
	\left\|
	\left(
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right)
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\boldsymbol 1|\hat{\xi}_{m}^{\theta} \big)
	\right\|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\\ \nonumber
	&\leq
	\tilde{C}_{2}^{2} (\rho_{5}^{m-n} + \rho_{5}^{m} \|w_{\theta}\| )^{2}
	E\left(\left.
	\left|
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\\ \nonumber
	&
	\leq
	\left(
	\frac{C_{5}\tilde{C}_{2} \rho_{1}^{n-m}
	(\rho_{5}^{m-n} + \rho_{5}^{m}\|w_{\theta}\| ) }
	{\sqrt{N} }
	\right)^{2}
	\leq
	\left(
	\frac{\tilde{C}_{4}
	(\rho_{5}^{n-m} + \rho_{5}^{n}\|w_{\theta}\| ) }
	{\sqrt{N} }
	\right)^{2}
\end{align}
almost surely.

Let $\tilde{C}_{6}=\tilde{C}_{4}+\tilde{C}_{5}$.
Due to (\ref{5.27}), we have
\begin{align}\label{l5.2.77}
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	=&
	-\left(
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right)
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\boldsymbol 1|\hat{\xi}_{m}^{\theta} \big)
	\\ \nonumber
	=&
	-\left(
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right)
	\left(
	\hat{C}_{\theta,\boldsymbol Y}^{m:n}
	\big(\boldsymbol 1|\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{H}_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1)
	\right)
	\\ \nonumber
	&
	-
	\left(
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right)
	\hat{H}_{\theta,\boldsymbol Y}^{m:n}(\boldsymbol 1).
\end{align}
Consequently,  (\ref{l5.2.73}), (\ref{l5.2.71}) and the second part of (\ref{l5.2.77}) imply
\begin{align}\label{l5.2.501}
	\left\|
	E\left(\left.
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}
	\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\right)
	\right\|
	\leq
	\frac{(\tilde{C}_{4}+\tilde{C}_{5} )
	(\rho_{5}^{n-m} + \rho_{5}^{n}\|w_{\theta}\| ) }
	{N}
	=
	\frac{\tilde{C}_{6}	(\rho_{5}^{n-m} + \rho_{5}^{n}\|w_{\theta}\| ) }
	{N}
\end{align}
almost surely.
Moreover, the first part of (\ref{l5.2.77}) and (\ref{l5.2.75}) yield
\begin{align}\label{l5.2.503}
	E\left(\left.
	\!\left\|
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}\big(\varphi|\hat{\xi}_{m}^{\theta} \big)
	\right\|^{2}
	\right| \boldsymbol Y, \hat{\xi}_{m-1}^{\theta}
	\!\right)
	\leq
	\!\left(
	\frac{\tilde{C}_{4}
	(\rho_{5}^{n-m} + \rho_{5}^{n}\|w_{\theta}\| ) }
	{\sqrt{N} }
	\right)^{\!\!2}
	\!\leq
	\!\left(
	\frac{\tilde{C}_{6}
	(\rho_{5}^{n-m} + \rho_{5}^{n}\|w_{\theta}\| ) }
	{\sqrt{N} }
	\right)^{\!\!2}
\end{align}
almost surely.

Let $C_{6}=\tilde{C}_{3}+\tilde{C}_{6}$.
Then, combining (\ref{l5.2.59}), (\ref{l5.2.61}), (\ref{l5.2.501}), (\ref{l5.2.503})
with the tower property of conditional expectations,
we conclude that (\ref{l5.2.1*}) -- (\ref{l5.2.7*}) hold almost surely.
\end{proof}

\begin{proposition}\label{proposition5.1}
(i) Let Assumption \ref{a1} hold.
Then, there exists a real number $L\in[1,\infty)$
(independent of $N$ and depending only on $\varepsilon$) such that
\begin{align}
	&\label{p5.1.1*}
	\left|
	E\left(\left.
	\hat{\xi}_{n}^{\theta}(\varphi)
	-
	F_{\theta,\boldsymbol Y}^{0:n}\big(\varphi|\xi_{\theta} \big)
	\right|
	\boldsymbol Y =\boldsymbol y
	\right)
	\right|
	\leq
	\frac{L}{N},
	\\
	&\label{p5.1.3*}
	\left(
	E\left(\left.
	\left|
	\hat{\xi}_{n}^{\theta}(\varphi)
	-
	F_{\theta,\boldsymbol Y}^{0:n}\big(\varphi|\xi_{\theta} \big)
	\right|^{2}
	\right|
	\boldsymbol Y = \boldsymbol y
	\right)
	\right)^{1/2}
	\leq
	\frac{L}{\sqrt{N} }
\end{align}
for all $\theta\in\Theta$, $n\geq 0$,
any sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$
and any Borel-measurable function $\varphi:{\cal X}\rightarrow[-1,1]$.

(ii) Let Assumptions \ref{a1} -- \ref{a3} hold.
Then, there exist real numbers $\rho\in(0,1)$, $M\in[1,\infty)$
(independent of $N$ and depending only on $\varepsilon$, $d$, $K$) such that
\begin{align}
	&\label{p5.1.5*}
	\left\|
	E\left(\left.
	\hat{\zeta}_{n}^{\theta}(\varphi)
	-
	G_{\theta,\boldsymbol Y}^{0:n}
	\big(\varphi|\xi_{\theta}, \zeta_{\theta} \big)
	\right|
	\boldsymbol Y = \boldsymbol y
	\right)
	\right\|
	\leq
	\frac{M(1 + \rho^{n} \|w_{\theta}\| ) }{N},
	\\
	&\label{p5.1.7*}
	\left(
	E\left(\left.
	\left\|
	\hat{\zeta}_{n}^{\theta}(\varphi)
	-
	G_{\theta,\boldsymbol Y}^{0:n}
	\big(\varphi|\xi_{\theta}, \zeta_{\theta} \big)
	\right\|^{2}
	\right|
	\boldsymbol Y = \boldsymbol y
	\right)
	\right)^{1/2}
	\leq
	\frac{M(1 + \rho^{n} \|w_{\theta}\| ) }{\sqrt{N} }
\end{align}
for all $\theta\in\Theta$, $n\geq 0$,
any sequence $\boldsymbol y = \{y_{n} \}_{n\geq 0}$ in ${\cal Y}$
and any Borel-measurable function $\varphi:{\cal X}\rightarrow[-1,1]$.
\end{proposition}

\begin{vremark}
Proposition \ref{proposition5.1} can be considered as an extended version of Theorem \ref{theorem1.1},
while the bounds in (\ref{p5.1.3*}), (\ref{p5.1.7*}) can be viewed as by-products of Theorem \ref{theorem1.1}.
Under the same conditions as in Proposition \ref{proposition5.1},
bounds similar to (\ref{p5.1.3*}), (\ref{p5.1.7*}) have been derived in \cite{delmoral&doucet&singh2}.
\end{vremark}

\begin{proof}
(i) Throughout this part of the proof,
the following notation is used.
$\theta$ is any element of $\Theta$.
$\boldsymbol y = \{y_{n} \}_{n\geq 0}$ is any sequence in ${\cal Y}$,
while $\varphi(x)$ is any Borel-measurable function mapping $x\in{\cal X}$ to $[-1,1]$.
$L$ is the real number defined by $L=C_{5}(1-\rho_{1} )^{-1}$
($\rho_{1}$ is specified in Proposition \ref{proposition3.2}),
while $n$ is any non-negative integer.

It is straightforward to verify
\begin{align*}
	\hat{\xi}_{n}^{\theta}(\varphi)
	=\hat{F}_{\theta,\boldsymbol Y}^{n:n}(\varphi),
	\;\;\;\;\;
	F_{\theta,\boldsymbol Y}^{0:n}\big(\varphi|\xi_{\theta} \big)
	=F_{\theta,\boldsymbol Y}^{-1:n}\big(\varphi|\hat{\xi}_{-1}^{\theta} \big)
	=\hat{F}_{\theta,\boldsymbol Y}^{-1:n}(\varphi).
\end{align*}
Therefore, we get
\begin{align}\label{p5.1.1}
	\hat{\xi}_{n}^{\theta}(\varphi)
	-
	F_{\theta,\boldsymbol Y}^{0:n}\big(\varphi|\xi_{\theta} \big)
	=
	\hat{F}_{\theta,\boldsymbol Y}^{n:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{-1:n}(\varphi)
	=
	\sum_{m=0}^{n}
	\left(
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right).
\end{align}
Then, Lemma \ref{lemma5.2} implies
\begin{align}\label{p5.1.3}
	\left|
	E\left(\left.
	\hat{\xi}_{n}^{\theta}(\varphi)
	-
	F_{\theta,\boldsymbol Y}^{0:n}\big(\varphi|\xi_{\theta} \big)
	\right|
	\boldsymbol Y
	\right)
	\right|
	\leq &
	\sum_{m=0}^{n}
	\left|
	E\left(\left.
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|
	\boldsymbol Y
	\right)
	\right|
	\\ \nonumber
	\leq &
	\frac{C_{5} }{N} \sum_{m=0}^{n} \rho_{1}^{n-m}
	\leq
	\frac{L}{N}
\end{align}
almost surely.
Moreover, Minkowski inequality, Lemma \ref{lemma5.2} and (\ref{p5.1.1}) yield
\begin{align}\label{p5.1.5}
	\left(
	E\left(\left.
	\left|
	\hat{\xi}_{n}^{\theta}(\varphi)
	-
	F_{\theta,\boldsymbol Y}^{0:n}\big(\varphi|\xi_{\theta} \big)
	\right|^{2}
	\right|
	\boldsymbol Y
	\right)
	\right)^{\!1/2}
	\leq &
	\sum_{m=0}^{n}
	\left(
	E\left(\left.
	\left|
	\hat{F}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{F}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right|^{2}
	\right|
	\boldsymbol Y
	\right)
	\right)^{\!1/2}
	\\ \nonumber
	\leq &
	\frac{C_{5}}{\sqrt{N} } \sum_{m=0}^{n} \rho_{1}^{n-m}
	\leq
	\frac{L}{\sqrt{N} }
\end{align}
almost surely.
Using (\ref{p5.1.3}), (\ref{p5.1.5}),
we conclude that (\ref{p5.1.1*}), (\ref{p5.1.3*}) hold.

(ii) Let $\theta$, $\boldsymbol y$, $\!\varphi(x)$ have the same meaning as in (i),
while $n,m$ are any integers satisfying $n\geq m\geq 0$.
Moreover, let $\rho\!=\!\sqrt{\rho_{5} }$,
$\tilde{C}=\max_{n\geq 1} n\rho^{n}$, while
$M=4C_{6}\tilde{C}(1-\rho )^{-1}$
($\rho_{5}$, $C_{6}$ are specified in
Lemma \ref{lemma5.2}).
Owing to Lemmas \ref{lemma5.3}, \ref{lemma5.2}, we have
\begin{align}\label{p5.1.23}
	\left\|
	E\left(\left.
	\hat{G}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right| \boldsymbol Y
	\right)
	\right\|
	\leq &
	\left\|
	E\left(\left.
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}\big(\varphi |\hat{\xi}_{m}^{\theta} \big)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right| \boldsymbol Y
	\right)
	\right\|
	\\ \nonumber
	&+
	\left\|
	E\left(\left.
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}\big(\varphi |\hat{\xi}_{m}^{\theta} \big)
	\right| \boldsymbol Y
	\right)
	\right\|
	\\ \nonumber
	\leq &
	\frac{2C_{6}(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| ) }{N}
\end{align}
almost surely.
Similarly, due to Minkowski inequality and Lemmas \ref{lemma5.3}, \ref{lemma5.2},
we have
\begin{align}\label{p5.1.25}
	\left(
	E\!\left(\left.
	\left\|
	\hat{G}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	\!-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right\|^{2}
	\right| \!\boldsymbol Y
	\!\right)
	\right)^{\!\!1/2}
	\leq &
	\left(
	E\!\left(\left.
	\left\|
	\hat{A}_{\theta,\boldsymbol Y}^{m:n}\big(\varphi |\hat{\xi}_{m}^{\theta} \big)
	\!-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right\|^{2}
	\right| \!\boldsymbol Y
	\!\right)
	\right)^{\!\!1/2}
	\\ \nonumber
	&+
	\left(
	E\!\left(\left.
	\left\|
	\hat{B}_{\theta,\boldsymbol Y}^{m:n}\big(\varphi |\hat{\xi}_{m}^{\theta} \big)
	\right\|^{2}
	\right| \!\boldsymbol Y
	\!\right)
	\right)^{\!\!1/2}
	\\ \nonumber
	\leq &
	\frac{2C_{6}(\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| ) }{\sqrt{N} }
\end{align}
almost surely.

It is straightforward to verify
\begin{align*}
	\hat{\zeta}_{n}^{\theta}(\varphi)
	=\hat{G}_{\theta,\boldsymbol Y}^{n:n}(\varphi),
	\;\;\;\;\;
	G_{\theta,\boldsymbol Y}^{0:n}\big(\varphi|\xi_{\theta},\zeta_{\theta} \big)
	=G_{\theta,\boldsymbol Y}^{-1:n}\big(\varphi|\hat{\xi}_{-1}^{\theta}, \hat{\zeta}_{-1}^{\theta} \big)
	=\hat{G}_{\theta,\boldsymbol Y}^{-1:n}(\varphi).
\end{align*}
Therefore, we get
\begin{align}\label{p5.1.29}
	\hat{\zeta}_{n}^{\theta}(\varphi)
	\!-
	G_{\theta,\boldsymbol Y}^{0:n}
	\big(\varphi|\xi_{\theta}, \zeta_{\theta} \big)
	\!=&
	\hat{G}_{\theta,\boldsymbol Y}^{n:n}(\varphi)
	\!-
	G_{\theta,\boldsymbol Y}^{0:n}
	\big(\varphi|\xi_{\theta}, \zeta_{\theta} \big)
	\!=
	\sum_{m=0}^{n}
	\left(
	\hat{G}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	\!-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right).
\end{align}
Then, (\ref{p5.1.23}) implies
\begin{align}\label{p5.1.31}
	&
	\left\|
	E\left(\left.
	\hat{\zeta}_{n}^{\theta}(\varphi)
	-
	G_{\theta,\boldsymbol Y}^{0:n}
	\big(\varphi|\xi_{\theta}, \zeta_{\theta} \big)
	\right| \boldsymbol Y
	\right)
	\right\|
	\\ \nonumber
	&\leq
	\sum_{m=0}^{n}
	\left\|
	E\left(\left.
	\hat{G}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right| \boldsymbol Y
	\right)
	\right\|
	\\ \nonumber
	&\leq
	\frac{2C_{6} }{N}
	\sum_{m=0}^{n} (\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| )
	\leq
	\frac{2C_{6} }{(1-\rho_{5} ) N}
	+
	\frac{2C_{6} (n+1) \rho_{5}^{n} \|w_{\theta} \| }{N}
	\leq
	\frac{M (1 + \rho^{n} \|w_{\theta} \| ) }{N}
\end{align}
almost surely.
Moreover, Minkowski inequality
and (\ref{p5.1.25}) -- (\ref{p5.1.29}) yield
\begin{align}\label{p5.1.33}
	&
	\left(
	E\left(\left.
	\left\|
	\hat{\zeta}_{n}^{\theta}(\varphi)
	-
	G_{\theta,\boldsymbol Y}^{0:n}
	\big(\varphi|\xi_{\theta}, \zeta_{\theta} \big)
	\right\|^{2}
	\right| \boldsymbol Y
	\right)
	\right)^{1/2}
	\\ \nonumber
	&\leq
	\sum_{m=0}^{n}
	\left(
	E\left(\left.
	\left\|
	\hat{G}_{\theta,\boldsymbol Y}^{m:n}(\varphi)
	-
	\hat{G}_{\theta,\boldsymbol Y}^{m-1:n}(\varphi)
	\right\|^{2}
	\right| \boldsymbol Y
	\right)
	\right)^{1/2}
	\\ \nonumber
	&\leq
	\frac{2C_{6} }{\sqrt{N} }
	\sum_{m=0}^{n} (\rho_{5}^{n-m} + \rho_{5}^{n} \|w_{\theta}\| )
	\!\leq\!
	\frac{2C_{6} }{(1-\rho_{5} ) \sqrt{N} }
	+
	\frac{2C_{6} (n+1) \rho_{5}^{n} \|w_{\theta} \| }
	{\sqrt{N} }
	\!\leq\!
	\frac{M(1 + \rho^{n} \|w_{\theta} \| ) }{\sqrt{N} }
	\\ \nonumber
\end{align}
almost surely.
Using (\ref{p5.1.31}), (\ref{p5.1.33}),
we conclude that (\ref{p5.1.5*}), (\ref{p5.1.7*}) hold.
\end{proof}


TITLE: Orthogonal Complements property
QUESTION [1 upvotes]: I have a question about how to prove a certain property of orthogonal complements of vector subspaces. Given $\mathrm E$, a vector space over a commutative field k, define: $$\phi\text{ : } \mathrm E \times \mathrm E^* \to k\text{ , where } \mathrm E^* \text{ is the dual space of E, and } \\ \phi(x,\omega) = \omega(x)$$
we can now define $\mathrm F^ \bot  = \{\omega \in \mathrm E^* \mid \phi(x,\omega)=0  , \forall x\in \mathrm F \}$, where $\mathrm F $ is a vector subspace of $\mathrm E $
The first couple of properties such as proving that $\phi$ is in fact a mixed tensor, that $\mathrm F^ \bot $ is also a subspace and $ \dim F + \dim F^\bot = \dim E$ are simple enough, but I've gotten to one that I don't know how to go about, which is: $$ {(F+G)}^\bot = F^\bot\cap G^\bot $$
Where, of course, $F$ and $G$ are vector subspaces.
Also for anyone wondering this is in the context of duality in projective spaces, because proving these properties is fundamental to the principle of duality, or at least the way I'm learning it. Thanks in advance.

REPLY [2 votes]: Try this: $\omega \in F^\bot \cap G^\bot $ implies $\omega \in F^\bot $ and $\omega \in G^\bot$. That is, $\omega (x) = 0$ for every $x \in F$ and $\omega (y) = 0$ for every $y\in G$. Hence, $\omega (x+y) = 0$ for all $x+y \in F + G$. So $\omega \in (F+G)^\bot$.
So far we have proved one inclusion: $F^\bot \cap G^\bot \subset (F+G)^\bot$. For the other inclusion, namely $(F+G)^\bot \subset F^\bot \cap G^\bot$, just do the same: start with the definition of $\omega$ being in $(F+G)^\bot$ and see what happens. (If you get stuck, ask me again.)


text:
{\bf Problem.} Let $z$ be a complex number with $|z| = \sqrt{2}.$  Find the maximum value of
\[|(z - 1)^2 (z + 1)|.\]
{\bf Level.} Level 5
{\bf Type.} Intermediate Algebra
{\bf Solution.} Let $z = x + yi,$ where $x$ and $y$ are real numbers.  Since $|z| = \sqrt{2},$ $x^2 + y^2 = 2.$  Then
\begin{align*}
|z - 1| &= |x + yi - 1| \\
&= \sqrt{(x - 1)^2 + y^2} \\
&= \sqrt{x^2 - 2x + 1 + 2 - x^2} \\
&= \sqrt{3 - 2x},
\end{align*}and
\begin{align*}
|z + 1| &= |x + yi + 1| \\
&= \sqrt{(x + 1)^2 + y^2} \\
&= \sqrt{x^2 + 2x + 1 + 2 - x^2} \\
&= \sqrt{2x + 3},
\end{align*}so
\[|(z - 1)^2 (z + 1)| = \sqrt{(3 - 2x)^2 (2x + 3)}.\]Thus, we want to maximize $(3 - 2x)^2 (2x + 3),$ subject to $-\sqrt{2} \le x \le \sqrt{2}.$

We claim the maximum occurs at $x = -\frac{1}{2}.$  At $x = -\frac{1}{2},$ $(3 - 2x)^2 (2x + 3) = 32.$  Note that
\[32 - (3 - 2x)^2 (2x + 3) = -8x^3 + 12x^2 + 18x + 5 = (2x + 1)^2 (5 - 2x) \ge 0,\]so $(3 - 2x)^2 (2x + 3) \le 32$ for $-\sqrt{2} \le x \le \sqrt{2},$ with equality if and only if $x = -\frac{1}{2}.$

Therefore, the maximum value of $|(z - 1)^2 (z + 1)| = \sqrt{(3 - 2x)^2 (2x + 3)}$ is $\sqrt{32} = \boxed{4 \sqrt{2}}.$

set_name: MATH


TITLE: Circle containing a point of square and touching two sides
QUESTION [0 upvotes]: So the problem goes,circle contains one vertex of a square and touches two sides. Length of side is 1cm. What is circumference of a circle?
My attempt was trying to find a connection between radius of circle, and side of square, but I'm not sure if Imagined and drew it properly.

REPLY [1 votes]: Extend $AB$ one unit to the right to get point $E$,
$AD$ unit upwards to get point $F$
and connect these points.
Triangle $AEF$ is a right triangle
with the sides $a=b=2$, $c=2\sqrt2$.
The circle is
the inscribed circle for this triangle.
Recall that for any right triangle with sides $a,b$
and hypotenuse $c$, the radius
of the inscribed circle is just
\begin{align}
r&=\tfrac12(a+b-c)
,\\
\text{so }\quad
r&=2-\sqrt2
.
\end{align}


text:
\begin{definition}[Definition:Idempotence/Relation]
Let $\RR \subseteq S \times S$ be a [[Definition:Endorelation|relation]] on $S$.
Then $\RR$ is '''idempotent''' {{iff}}:
:$\RR \circ \RR = \RR$
where $\circ$ denotes [[Definition:Composition of Relations|composition of relations]].
\end{definition}

config: wiki