text:
\begin{document}

\title{A useful technique for piecewise deterministic Markov decision processes}
\author{Xin Guo \thanks{Department of Mathematical Sciences, University of
Liverpool, Liverpool, L69 7ZL, U.K.. E-mail:  X.Guo21@liv.ac.uk.}~ and Yi
Zhang \thanks{Corresponding author. Department of Mathematical Sciences, University of
Liverpool, Liverpool, L69 7ZL, U.K.. E-mail: yi.zhang@liv.ac.uk.}}
\date{}
\maketitle

\par\noindent{\bf Abstract:}
This paper presents with justifications a technique that is useful for the study of piecewise deterministic Markov decision processes (PDMDPs) with general policies and unbounded transition intensities. This technique produces an auxiliary PDMDP from the original one. As to be discussed and claified, the auxiliary PDMDP possesses certain desired properties, which may not be possessed by the original PDMDP. Moreover, the performance measure of any policy in the original PDMDP can be replicated by the auxiliary PDMDP for a large class of performance criteria. As an application, we apply this technique to risk-sensitive PDMDPs with total cost criteria.
\bigskip

\par\noindent {\bf Keywords:} Continuous-time Markov decision processes. General nonstationary policies. Piecewise deterministic Markov decision processes. Unbounded transition intensities.
\bigskip

\par\noindent
{\bf AMS 2000 subject classification:} Primary 90C40,  Secondary
60J75

\section{Introduction}
This paper concerns the optimal control of piecewise deterministic Markov processes, where the state evolves according to a deterministic and uncontrolled flow between two consecutive jumps, and the transition intensities and post-jump distributions are controlled. Henceforth it will be termed as a piecewise deterministic Markov decision process (PDMDP).

A powerful method of studying PDMDPs is to reduce it to an equivalent discrete-time Markov decision process (DTMDP) by inspecting the PDMDP at each of its jump moments and regarding the (possibly relaxed) control function used during a sojourn time as an action in the DTMDP,  see e.g., \cite{BauerleRieder:2011,Costa:2013,Davis:1993,GuoZhangAMO,Yushkevich:1980,Zhang:2017}. Consequently, the action space in the induced DTMDP, as a set of measurable mappings, is in general a more complicated object than the action space in the original PDMDP. The reason for applying this reduction is to gain access to the rich toolbox of known results on DTMDPs that have been studied since 1950s.

It is well appreciated that the theory of DTMDPs is better established when the underlying DTMDP model satisfies some compactness-continuity conditions, see \cite{Hernandez-Lerma:1996,Hernandez-Lerma:1999,Jaskiewicz:2008}. One example of such compactness-continuity conditions is that the action space is a compact Borel space, the loss function is lower semicontinuous in the action, and the transition kernel possesses a strong Feller property with respect to the action, i.e., it maps each bounded measurable function on the state space to a function, which is jointly measurable in the state and action, and also continuous in the action. (A precise formulation of this is in the appendix.)

However, even if the original PDMDP satisfies a natural set of compactness-continuity conditions, see Condition \ref{GZyExponentialCondition02} below, it can happen that the transition kernel in the induced DTMDP fails to satisfy the desired continuity condition. We demonstrate this in Example \ref{JapanExample01} below. On the other hand, it turns out that this inconvenience does not appear if the transition intensities of the PDMDP are strongly positive, i.e., bounded away from zero by a constant, see Proposition \ref{09June2020PropositionA01}.

In the more specific context of continuous-time Markov decision processes (CTMDPs), where the state does not change between two consecutive jumps, another difficulty associated with nonstationary policies and transition intensities not strongly positive, which is different from the aforementioned one, was documented in \cite{Feinberg:2004,GuoXPZhang:2017,Piunovskiy:2015}.

The contribution of this paper is that we present a technique, which produces an auxiliary PDMDP model satisfying the following: a) the original PDMDP can be thinned out of the auxiliary PDMDP, and the performance of any policy in the original PDMDP is replicated by a corresponding policy in the auxiliary PDMDP; b) the transition intensities in the auxiliary PDMDP are strongly positive, and its induced DTMDP satisfies the desired compactness-continuity conditions\footnote{See Proposition \ref{09June2020PropositionA01} for the precise definition of the compactness-continuity condition for the induced DTMDP.} if so does the original PDMDP. Then as an application, we extend some optimality results for risk-sensitive PDMDPs with total cost criteria, which were previously obtained in \cite{GuoZhangAMO} under the extra requirement on the transition intensities being strongly positive, see the footnote in Proposition \ref{09June2020PropositionA01}. This requirement is omitted here with the help of the proposed technique.

In the context of CTMDPs, there have been known techniques, which produce new CTMDP models, in which the performance measure in the original model can be replicated. One example is the uniformization in \cite{puterman,Serfozo:1979} requiring bounded transition intensities and justified under stationary policies. Its more recent variants that deal with models with unbounded transition intensities can be found in \cite{Cekyay:2018,Piunovskiy:2015}, which are also similar in nature: the former one considers CTMDPs with a denumerable state space and deterministic stationary policies, whereas in the latter one, models with more general policies were considered. Our technique can be viewed as their extension from CTMDPs to PDMDPs, but there are some notable differences.

In greater detail, our technique is closer to \cite{Piunovskiy:2015} as we also consider general policies and unbounded transition intensities. Nevertheless, apart from a more general process under control being dealt with here, let us mention another more important difference from \cite{Piunovskiy:2015} as follows. The justification in \cite{Piunovskiy:2015} only serves the risk-neutral problem with total cost criteria, as it is based on the comparison of (total) occupation measures in the original model and the new model. This is not suitable for risk-sensitive problems, where in general the performance measures cannot be readily written as integrals of the cost rate with respect to the occupation measures. In contrast, here our justification applies to both risk-neutral and risk-sensitive problems with both total cost and average cost criteria, see Remark \ref{09June2020Remark01}(c).

The rest of this paper is organized as follows. In Section \ref{NoteGZSec01} we describe the PDMDP model. In Section \ref{NoteGZSec01} we present and prove the main result with an application to risk-sensitive PDMDPs. The appendix presents the induced DTMDP. While the reference to the appendix can be avoided in the main text, it clarifies and demonstrates the issues mentioned in the beginning of this introduction.

\section{Description of PDMDP model}\label{NoteGZSec01}

Let $S$ be a nonempty Borel state space endowed with the Borel $\sigma$-algebra ${\cal B}(S)$, $A$ be
a nonempty Borel action space, and $q$
stand for a signed kernel $q(dy|x,a)$ on ${\cal{B}}(S)$ given
$(x,a)\in S\times A$ such that
\begin{eqnarray*}
\tilde{q}(\Gamma_S|x,a):=q(\Gamma_S\setminus\{x\}|x,a)\ge 0
\end{eqnarray*}
for all $\Gamma_S\in{\cal{B}}(S)$, and
\begin{eqnarray*}
q(S|x,a)=0,~\bar{q}_x=\sup_{a\in A}q_x(a)<\infty,
\end{eqnarray*}
where $q_x(a):=-q(\{x\}|x,a)$ is the transition intensity. The signed kernel $q$ is also called the
transition rate. Between two consecutive jumps, the state of the process evolves according to a measurable mapping $\phi$ from $S\times [0,\infty)$ to $S$, see (\ref{GGGAMO01}) below. It is assumed that for each $x\in S$
\begin{eqnarray}\label{09June2020Eqn08}
\phi(x,t+s)=\phi(\phi(x,t),s),~\forall~s,t\ge 0;~\phi(x,0)=x,
\end{eqnarray}
and $t\rightarrow \phi(x,t)$ is continuous.  Unless stated otherwise, the term of
measurability is always understood in the Borel sense.
Finally let the cost rate $c$ be a $[0,\infty)$-valued measurable function on $S\times A$.

For the rest of this paper, it is convenient to introduce the
following notations. Let $\mathbb{P}(A)$ be the space of
probability measures on ${\cal B}(A)$, endowed with the
standard weak topology. For each $\mu\in \mathbb{P}(A)$,
\begin{eqnarray*}
&& q_x(\mu):=\int_A q_x(a)\mu(da),~\tilde{q}(dy|x,\mu):= \int_A \tilde{q}(dy|x,a)\mu(da),~c(x,\mu):=\int_A c(x,a)\mu(da).
\end{eqnarray*}

\begin{condition}\label{GGZyExponentialConditionExtra}
For each $x\in S$, $\int_{0}^t\overline{q}_{\phi(x,s)}ds<\infty$, and $\int_{0}^t \sup_{a\in A} c(\phi(x,s),a)ds<\infty$, for each $t\in[0,\infty).$
\end{condition}
Condition \ref{GGZyExponentialConditionExtra} is assumed to hold throughout this paper. The integrals in Condition \ref{GGZyExponentialConditionExtra} are well defined because the integrands are universally measurable.

Now we briefly describe the PDMDP with the system primitives $\{S,A,q,\phi,c\}$. Let us take the sample space $\Omega$ by adjoining to the
countable product space $S\times((0,\infty)\times S)^\infty$ the
sequences of the form
$(x_0,\theta_1,\dots,\theta_n,x_n,\infty,x_\infty,\infty,x_\infty,\dots),$
where $x_0,x_1,\dots,x_n$ belong to $S$,
$\theta_1,\dots,\theta_n$ belong to $(0,\infty),$ and
$x_{\infty}\notin S$ is the isolated point. We equip $\Omega$ with
its Borel $\sigma$-algebra $\cal F$.

Let $t_0:=0=:\theta_0,$ and for each $n\geq 0$, and each
element $\omega:=(x_0,\theta_1,x_1,\theta_2,\dots)\in \Omega$, let \begin{eqnarray*}
h_n:=(x_0,\theta_1,\dots,\theta_n,x_n);~
t_n:=t_{n-1}+\theta_n;~ t_\infty(\omega):=\lim_{n\rightarrow\infty}t_n.
\end{eqnarray*}
Then, $(\Omega,{\cal F})$ is the canonical sample space of the marked point process $(t_n,x_n)$ with the mark space $S$, and $\theta_{n}=t_n-t_{n-1}$ is the sojourn time, where the convention of $\infty-\infty:=\infty$ is in use.
Define the process, which evolves according to the flow $\phi$ during a sojourn time: \begin{eqnarray}\label{GGGAMO01}
\xi_t=\left\{\begin{array}{ll} \phi(x_n,t-t_n), & \mbox{ if } t_n\le t<t_{n+1};\\
x_\infty, & \mbox{ if } t_\infty\le
t, \end{array}\right.
\end{eqnarray}
where $x_\infty\notin S$ is an isolated cemetery point. The process is controlled through its local characteristics as follows.

A policy $\pi$ is given by a
sequence $(\pi_n)$ such that, for each $n=0,1,2,\dots,$
$\pi_n(da|h_n,s)$ is a stochastic kernel on
$A$ given $h_n,s$ with $s>0$, and for each $\omega=(x_0,\theta_1,x_1,\theta_2,\dots)\in
\Omega$, $t> 0,$
\begin{eqnarray}\label{09June2020Eqn12}
\pi(da|\omega,t)&:=&I\{t\ge t_\infty\}\delta_{a_\infty}(da)+
\sum_{n=0}^\infty I\{t_n< t\le
t_{n+1}\}\pi_{n}(da|h_n, t-t_n),
\end{eqnarray}
defines a $\mathbb{P}(A\cup\{a_\infty\})$-valued (relaxed) control process,
where $a_\infty\notin A$ is some isolated point. If for some measurable mapping $\varphi$ from $S$ to $A$ such that $\pi_n(da|x_0,\theta_1,\dots,\theta_n,x,t)\equiv\delta_{\varphi(x)}(da)$, then the policy $\pi=(\pi_n)$ is called deterministic stationary and is identified with the mapping $\varphi.$

A policy $\pi$ and an initial state $x$ define a probability measure $P_x^\gamma$ on the canonical sample space, under which $P_x^\pi(x_0=x)=1$, and the conditional distribution of $(\theta_{n+1},x_{n+1})$ given $h_n$ satisfies
\begin{eqnarray}\label{09June2020Eqn01}
&&P_\gamma^\pi(\theta_{n+1}\in \Gamma_1,~x_{n+1}\in \Gamma_2|h_n)\nonumber\\
&=&\int_{\Gamma_1}e^{-\int_0^t \int_A q_{\phi(x_n,s)}(a)\pi_n(da|h_n,s)ds}\int_{A}\tilde{q}(\Gamma_2|\phi(x_n,t),a)\pi_n(da|h_n,t)dt,\nonumber\\
&&~\forall~\Gamma_1\in{\cal B}((0,\infty)),~\Gamma_2\in{\cal B}(S); \\
&&P_\gamma^\pi(\theta_{n+1}=\infty,~x_{n+1}=x_\infty|h_n)=e^{-\int_0^\infty  \int_A q_{\phi(x_n,s)}(a)\pi_n(da|h_n,s)ds}\nonumber
\end{eqnarray}
on $\{x_n\in S\}$.

The proposed technique in this paper will be applied to the risk-sensitive optimal control problem for the PDMDP with a total cost criterion, which reads
\begin{eqnarray*}
&\mbox{Minimize over all policies $\pi$:~}& E_x^\pi\left[e^{\int_0^\infty \int_A
c(\xi_t,a)\pi(da|\omega,t)dt}\right]\\
&&=E_x^\pi\left[e^{\sum_{n=0}^\infty \int_{t_n}^{t_{n+1}} \int_A
c(\phi(x_n,s),a)\pi_{n}(da|h_n,s-t_{n})ds}\right]=:V(x,\pi)
\end{eqnarray*}
Here $\int_{t_n}^{t_{n+1}}$ is understood as $\int_{(t_n,t_n]\bigcap \mathbb{R}}$, and we put $c(x_\infty,a)\equiv 0$.
The value function is defined by $V^\ast(x)=\inf_{\pi}V(x,\pi)$ for all $x\in S.$
We shall call the above system primitives $\{S,A,q,\phi,c\}$ and the corresponding optimal control problem the ``original model'', to distinguish it from the auxiliary model that will appear later.

However, our technique is also applicable to PDMDPs with other performance measure. For instance, one may consider the expected long run average cost defined by
\begin{eqnarray}\label{09June2020Eqn15}
&&\overline{V}(x,\pi):=\limsup_{T\rightarrow \infty} E_x^\pi\left[ \frac{\int_0^T  \int_A
c(\xi_t,a)\pi(da|\omega,t)dt }{T}\right]\nonumber\\
&=&\limsup_{T\rightarrow \infty}  E_x^\pi\left[\frac{\sum_{n=0}^\infty \int_{t_n\wedge T}^{t_{n+1}\wedge T}  \int_A
c(\phi(x_n,s),a)\pi_{n}(da|h_n,s-t_{n})ds }{T}\right],
\end{eqnarray}
where $t_n\wedge T:=\min\{t_n,T\}$. See Remark \ref{09June2020Remark01}(c).

\section{Main result}

Fix $\lambda>0$ in what follows. We introduce an auxiliary model  $\{\breve{S},A,\breve{q},\breve{\phi},\breve{c}\}$ defined in terms of the system primitives of the original model. When there is a danger of confusion, we shall primarily use breves to signify the auxiliary model. Without special explanations, all the objects signified with breves are understood similarly to their counterparts without breves.

Roughly speaking, the auxiliary model arises from inserting additional inspections of the state process during each sojourn time in the original model (up to the moment of explosion) taking place in an independent Poisson process with rate $\lambda$. The changes in the second coordinate of the state in the auxiliary model take place at and only at each of such inspection epochs, which will be recorded as ``fictitious'' jumps and generate strongly positive transition intensities.

The state space is $\breve{S}=S\times \{-1,1\}$, endowed with the product topology, where $\{-1,1\}$ is with the discrete topology. The action space is $A$. The transition rate $\breve{q}$ on ${\cal B}(\breve{S})$ given $\breve{S}\times A$ is defined as follows:
\begin{eqnarray*}
\breve{q}(dy\times\{-i\}|(x,i),a)=\lambda \delta_x(dy);~\breve{q}(dy\times\{i\}|(x,i),a)=q(dy|x,a)-\lambda\delta_x(dy),
\end{eqnarray*}
with $\delta_{x}(dy)$
being the Dirac measure concentrated on the singleton $\{x\},$
so that
\begin{eqnarray*}
&&\breve{q}_{(x,i)}(a)=\breve{q}(\breve{S}\setminus \{(x,i)\}|(x,i),a)=\breve{q}(S\setminus\{x\}\times\{-1,1\}|(x,i),a)+\breve{q}(S\times\{-i\}|(x,i),a)\\
&=&q_x(a)+\lambda~\forall~(x,i)\in \breve{S},~a\in A.
\end{eqnarray*}
In other words, the auxiliary model has strongly positive transition intensities. The flow is defined by \begin{eqnarray*}
\breve{\phi}((x,i),t)=(\phi(x,t),i).
\end{eqnarray*} The cost rate is \begin{eqnarray*}
\breve{c}((x,i),a)=c(x,a)~\forall~(x,i)\in \breve{S},~a\in A.
\end{eqnarray*}

Let
 \begin{eqnarray*}
 \breve{V}((x,i),\breve{\pi})=\breve{E}^{\breve{\pi}}_{(x,i)}[e^{\int_0^\infty \breve{c}(\breve{\xi}_t,a)\breve{\pi}(da|\breve{\omega},t)dt}].
 \end{eqnarray*}

\begin{definition}\label{9thJuneDefinition01}
Consider the canonical sample space of the marked point process $(\breve{t}_n,x_n,i_n)$, and a sample path
\begin{eqnarray*}
\breve{\omega}=((x_0,i_0),\breve{\theta}_1,(x_1,i_1),\breve{\theta}_2,\dots, (x_{n-1},i_{n-1}),\breve{\theta}_n,(x_n,i_n),\dots).
\end{eqnarray*}
We say a mark $(x_l,i_l)$ ($l\ge 1$) is immediately after a fictitious jump if $i_l=-i_{l-1}$, or equivalently, $x_l=\phi(x_{l-1},\breve{\theta}_l)$, where $\breve{\theta}_l$ is the sojourn time before the mark $(x_l,i_l)$. A mark that is not immediately after a fictitious jump is called immediately after an honest jump. We regard $(x_0,i_0)$ as a mark immediately after an honest jump.
\end{definition}
Using the notation in the above definition, we may consider out of $(\breve{t}_n,x_n,i_n)$ another marked point process $(\tau_{(m)},x_{(m)},i_{(m)})$ with $\tau_{(0)}:=0$ by counting only the points with marks immediately after honest jumps. Since $(x_0,i_0)$ is regarded as a mark immediately after an honest jump, $x_0=x_{(0)}$ and $i_0=i_{(0)}$. Since $i_{(m)}=i_{(0)}$ for all $m\ge 0$ almost surely in $(\tau_{(m)},x_{(m)},i_{(m)})$, with $i_{(0)}$ being fixed we may simply consider the marked point process $(\tau_{(m)},x_{(m)})$ instead of $(\tau_{(m)},x_{(m)},i_{(m)})$.

\begin{theorem}\label{09June2020Thm09}
Suppose Condition \ref{GGZyExponentialConditionExtra} is satisfied. For each policy $\pi=(\pi_n)$ in the original PDMDP model, there is a policy $\breve{\pi}=(\breve{\pi}_n)$ in the auxiliary PDMDP model such that for all $x\in S$ and $i\in\{-1,1\}$:
\begin{itemize}
\item[(a)]
The distribution of the marked point process  $(\tau_{(m)},x_{(m)})$ under $\breve{P}_{(i,x)}^{\breve{\pi}}$ coincides with the distribution of the marked point process $(t_m,x_m)$ under $P_{x}^\pi$. In other words, the marked point process in the original model (under $P_x^\pi$) may be thinned out of $(\breve{t}_n,x_n,i_n)$ in the auxiliary model (under $\breve{P}_{(x,i_0)}^{\breve{\pi}}$) by counting only the points with marks immediately after honest jumps.
\item[(b)]
 $V(x,\pi)=\breve{V}((x,i),\tilde{\pi})$.
\end{itemize}
\end{theorem}

\par\noindent\textit{Proof.} We will make use of the notation in Definition \ref{9thJuneDefinition01} freely.

(a) Let a policy $\pi=(\pi_n)$ for the original model be fixed. Consider the corresponding policy $\breve{\pi}=(\breve{\pi}_n)$ in the auxiliary model defined as follows.  For the $n$-history \begin{eqnarray*}
\breve{h}_n=((x_0,i_0),\breve{\theta}_1,(x_1,i_1),\breve{\theta}_2,\dots, (x_{n-1},i_{n-1}),\breve{\theta}_n,(x_n,i_n))
\end{eqnarray*}
in the auxiliary model, let $m=m(\breve{h}_n)$ be the number of honest jumps over $(0,t_n]$ within $\breve{h}_n$, so that if we count the initial mark $(i_0,x_0)$ as immediately after an honest jump, then there are $m+1$ marks immediately after honest jumps within $\breve{h}_n$. Then we define
\begin{eqnarray}\label{09June2020Eqn10}
&&\breve{\pi}_n(da|\breve{h}_n,t)=\pi_m(da|x_0,\tau_{(1)},x_{(1)},\tau_{(2)}-\tau_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)},t+\breve{t}_n-\tau_{(m)})
\end{eqnarray}
for $t>0.$

Consequently, for each $n,m\ge 0$ and for each $t\in (0,\infty)$ satisfying $t\in (\breve{t}_n,\breve{t}_{n+1}] \subseteq (\tau_{(m)},\tau_{(m+1)}]$, we have
\begin{eqnarray}\label{Correction05}
\breve{\pi}(da|\breve{\omega},t)&=&\breve{\pi}_n(da|\breve{h}_n,t-\breve{t}_n)\nonumber\\
&=&\pi_m(da|x_0,\tau_{(1)},x_{(1)},\tau_{(2)}-\tau_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)},t-\tau_{(m)}),
\end{eqnarray}
where the first equality is by (\ref{09June2020Eqn12}) applied to $\breve{\pi}$.

For brevity, below we put
\begin{eqnarray}\label{09June2020Eqn05}
&&\tilde{q}(dy|\phi(x_{(m)},t),\pi_m):=
\int_A\tilde{q}(dy|\phi(x_{(m)},t),a)\pi_m(da|x_{(0)},\tau_{(1)},x_{(1)},\tau_{(2)}-\tau_{(1)},\dots,x_{(m)},t); \nonumber\\
&& {q}(dy|\phi(x_{(m)},t),\pi_m):=
\int_A {q}(dy|\phi(x_{(m)},t),a)\pi_m(da|x_{(0)},\tau_{(1)},x_{(1)},\tau_{(2)}-\tau_{(1)},\dots,x_{(m)},t);\nonumber\\
&& q_{\phi(x_{(m)},t)}(\pi_m):=\tilde{q}(S|\phi(x_{(m)},t), \pi_m).
\end{eqnarray}

Now let us show that the distribution of the marked point process $(\tau_m,x_m)$ under $\breve{P}_{(x,i_0)}^{\breve{\pi}}$ coincides with the distribution of the marked point process in the original model under $P_x^\pi$.
To this end, in view of (\ref{09June2020Eqn01}), $x_{(0)}=x_0$ and $\tau_{(0)}=0$, it is sufficient to show that
\begin{eqnarray}\label{09June2020Eqn02}
&&\breve{P}_{(x,i)}^{\breve{\pi}}(x_{(m+1)}\in \Gamma,~\tau_{(m+1)}-\tau_{(m)}\in[0,T]|x_{(0)},\tau_{(1)},x_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)})\nonumber\\
&=& \int_{0}^T  \tilde{q}(\Gamma|\phi(x_{(m)},t),\pi_m)  e^{-\int_0^t q_{\phi(x_{(m)},s)}(\pi_m)ds}dt
\end{eqnarray}
on $\{\tau_{(m)}<\infty\}$ for each $T>0,$ $\Gamma\in{\cal B}(S)$ and $m\ge 0.$
Note that
\begin{eqnarray*}
&&\breve{P}^{\breve{\pi}}_{(x,i)}(x_{(m+1)}\in \Gamma,~\tau_{(m+1)}-\tau_{(m)}\in[0,T]|~x_{(0)},\tau_{(1)},x_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)})\\
&=&\sum_{n=0}^\infty \breve{P}^{\breve{\pi}}_{(x,i)}(x_{(m+1)}\in \Gamma,~\tau_{(m+1)}-\tau_{(m)}\in[0,T],\\
&&~ \mbox{exactly $n$ ficticious jumps over $[\tau_{(m)},\tau_{(m+1)}]$}|~x_{(0)},\tau_{(1)},x_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)}).
\end{eqnarray*}
Since $\sum_{n\ge 0}\frac{\lambda^nt^n}{n!}=e^{\lambda t}$, equality (\ref{09June2020Eqn02}) would be justified once we show that
\begin{eqnarray}\label{09June2020Eqn07}
&&\breve{P}^{\breve{\pi}}_{(x,i)}(x_{(m+1)}\in \Gamma,~\tau_{(m+1)}-\tau_{(m)}\in[0,T],\nonumber\\
&&~ \mbox{exactly $n$ ficticious jumps over $[\tau_{(m)},\tau_{(m+1)}]$}|~x_{(0)},\tau_{(1)},x_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)})\nonumber\\
&=& \int_0^T \frac{\lambda^n t^n}{n!}\tilde{q}(\Gamma|\phi(x_{(m)},t),\pi_m)e^{-\lambda t}e^{-\int_0^t q_{\phi(x_m,s)}(\pi_m) ds}.
\end{eqnarray}
For this, let us verify for each $T>0$ and $y\in S$ that
\begin{eqnarray}\label{09June2020Eqn06}
&&f(T,y,n-1):=\int_0^T \int_0^{T-r_1}\int_{0}^{T-\sum_{i=1}^2r_i}\dots\int_{0}^{T-\sum_{i=1}^{n-1}r_i} \tilde{q}(\Gamma|\phi(y,\sum_{i=1}^{n-1}r_i+t),\pi_m)\nonumber\\
&&\times e^{-\int_0^t (q_{\phi(y,\sum_{i=1}^{n-1}r_i+s)}(\pi_m)+\lambda)ds}\lambda^{n-1} e^{-\int_0^{\sum_{i=1}^{n-1}r_i} (q_{\phi(y,s)}(\pi_m)+\lambda)ds}   dt dr_{n-1}\dots d r_1\nonumber\\
&=& \int_0^T \frac{\lambda^{n-1}t^{n-1}}{(n-1)!} \tilde{q}(\Gamma|\phi(y,t),\pi_m)e^{-\int_{0}^t (q_{\phi(y,s)}(\pi_m)+\lambda)ds} dt
\end{eqnarray}
for each $n\ge 1,$ where the notation in (\ref{09June2020Eqn05}) is in use with $x_{(m)}$ being replaced by $y.$ This would yield the desired relation (\ref{09June2020Eqn07}) because by (\ref{Correction05})
\begin{eqnarray*}
&&f(T,x_{(m)},n-1)=\breve{P}_{(x,i)}^{\breve{\pi}}(x_{(m+1)}\in \Gamma,~\tau_{(m+1)}-\tau_{(m)}\in[0,T],\\
&&~ \mbox{exactly $n-1$ ficticious jumps over $[\tau_{(m)},\tau_{(m+1)}]$}|~x_{(0)},\tau_{(1)},x_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)}).
\end{eqnarray*}

Relation (\ref{09June2020Eqn06}) holds trivially when $n=1$ because
\begin{eqnarray*}
f(T,y,0)=\int_0^T \tilde{q}(\Gamma|\phi(y,t),\pi_m)e^{-\int_0^t (q_{\phi(y,s)}(\pi_m)+\lambda)ds}dt
\end{eqnarray*}
by definition.
Suppose (\ref{09June2020Eqn06}) holds. Then
\begin{eqnarray*}
&&f(T,y,n)=\int_0^T \lambda\int_0^{T-r_1}\int_{0}^{T-\sum_{i=1}^2r_i}\dots\int_{0}^{T-\sum_{i=1}^{n-1}r_i}\int_{0}^{T-\sum_{i=1}^{n}r_i} \tilde{q}(\Gamma|\phi(y,\sum_{i=1}^{n}r_i+t),\pi_m)\nonumber\\
&&\times e^{-\int_0^t (q_{\phi(y,\sum_{i=1}^{n}r_i+s)}(\pi_m)+\lambda)ds}\lambda^{n-1} e^{-\int_0^{\sum_{i=1}^{n}r_i} (q_{\phi(y,s)}(\pi_m)+\lambda)ds}   dt dr_{n}\dots dr_2d r_1\nonumber\\
&=&\int_0^T \lambda e^{-\int_0^{r_1} (q_{\phi(y,s)}(\pi_m)+\lambda)ds} \\
&&\times \left\{\int_0^{T-r_1}\int_{0}^{T-r_1-r_2}\dots \int_{0}^{T-r_1-\sum_{i=2}^{n}r_i} \tilde{q}(\Gamma|\phi(\phi(y,r_1),\sum_{i=2}^{n}r_i+t),\pi_m)\nonumber\right.\\
&&\left.\times e^{-\int_0^t (q_{\phi(\phi(y,r_1),\sum_{i=2}^{n}r_i+s)}(\pi_m)+\lambda)ds}\lambda^{n-1} e^{-\int_{0}^{\sum_{i=2}^{n}r_i} (q_{\phi(\phi(y,r_1),s)}(\pi_m)+\lambda)ds}   dt dr_{n}\dots dr_2 \right\}dr_1
\end{eqnarray*}
where the second equality holds because of (\ref{09June2020Eqn08}) and that
\begin{eqnarray*}
&&e^{-\int_0^{\sum_{i=1}^{n}r_i} (q_{\phi(y,s)}(\pi_m)+\lambda)ds} =e^{-\int_0^{r_1} (q_{\phi(y,s)}(\pi_m)+\lambda)ds} e^{-\int_{r_1}^{\sum_{i=1}^{n}r_i} (q_{\phi(y,s)}(\pi_m)+\lambda)ds}\\
&=&e^{-\int_0^{r_1} (q_{\phi(y,s)}(\pi_m)+\lambda)ds}
e^{-\int_{0}^{\sum_{i=2}^{n}r_i} (q_{\phi(\phi(y,r_1),s)}(\pi_m)+\lambda)ds}.
\end{eqnarray*}

Now after applying the induction supposition (\ref{09June2020Eqn06}) (with $y$ being replaced by $\phi(y,r_1)$, $T$ being replaced by $T-r_1$, and $\sum_{i=2}^n r_i$ playing the same role as $\sum_{i=1}^{n-1}r_i$ therein) to the previous inner integral, we see that
\begin{eqnarray*}
&&f(T,y,n)=\int_0^T \lambda e^{-\int_0^{r_1} (q_{\phi(y,s)}(\pi_m)+\lambda)ds }\\
&&\times \left\{\int_{0}^{T-r_1} \frac{\lambda^{n-1}t^{n-1}}{(n-1)!}\tilde{q}(\Gamma|\phi(\phi(y,r_1),t),\pi_m)e^{-\int_0^t (q_{\phi(\phi(x,r_1),s)}(\pi_m)+\lambda)ds}dt\right\}dr_1\\
&=&\int_0^T   \left\{\int_{0}^{T-r_1} \frac{\lambda^{n}t^{n-1}}{(n-1)!}\tilde{q}(\Gamma|\phi(y,r_1+t),\pi_m)e^{-\int_0^{t+r_1} (q_{\phi(x,s)}(\pi_m)+\lambda)ds}dt\right\}dr_1\\
&=&\int_0^T \int_0^t \frac{\lambda^n (t-r_1)^{n-1}}{(n-1)!} \tilde{q}(\Gamma|\phi(y,t),\pi_m)e^{-\int_0^{t} (q_{\phi(x,s)}(\pi_m)+\lambda)ds}dr_1 dt\\
&=&\int_0^T   \frac{\lambda^n t^{n}}{n!} \tilde{q}(\Gamma|\phi(y,t),\pi_m)e^{-\int_0^{t} (q_{\phi(x,s)}(\pi_m)+\lambda)ds}  dt,
\end{eqnarray*}
as desired, where the second equality follows from (\ref{09June2020Eqn08}) and that
\begin{eqnarray*}
e^{-\int_0^{r_1} (q_{\phi(y,s)}(\pi_m)+\lambda)ds } e^{-\int_0^t (q_{\phi(\phi(x,r_1),s)}(\pi_m)+\lambda)ds}=e^{-\int_0^{t+r_1} (q_{\phi(y,s)}(\pi_m)+\lambda)ds },
\end{eqnarray*}
and the third equality is by a change of variable and interchanging legitimately the order of integration. Part (a) is thus proved.

(b)
It follows from part (a) that \begin{eqnarray*}
&& \breve{V}((x,i_0),\breve{\pi})=\breve{E}^{\breve{\pi}}_{(x,i_0)}[e^{\int_0^\infty \breve{c}(\breve{\xi}_t,a)\breve{\pi}(da|\breve{\omega},t)dt}]\\
 &=&\breve{E}^{\breve{\pi}}_{(x,i_0)}\left[e^{\sum_{m=0}^\infty\int_{\tau_{(m)}}^{\tau_{(m+1)}}\int_A c(\phi(x_{(m)},s),a)\pi_{m}(da|x,\tau_{(1)},x_{(2)},\tau_{(2)}-\tau_{(1)},\dots,\tau_{(m)}-\tau_{(m-1)},x_{(m)},s-\tau_{(m)})ds} \right]\\
&=& {E}^{{\pi}}_{(x,i_0)}\left[e^{\sum_{m=0}^\infty\int_{t_{m}}^{t_{m+1}}\int_A c(\phi(x_{m},s),a)\pi_{m}(da|x,t_1,x_2,t_2-t_1,\dots,t_{m}-t_{m-1},x_{m},s-t_m)ds} \right]=V(x,\pi),
 \end{eqnarray*}
 where the second equality holds by (\ref{Correction05}) and the definition of $\breve{c}.$ The proof is completed. $\hfill\Box$

\begin{remark}\label{09June2020Remark01}
\begin{itemize}
\item[(a)] By Theorem \ref{09June2020Thm09}(b),
$\breve{V}^\ast(x)\le V^\ast(x)$ for each $x\in S.$
\item[(b)]
By inspecting the proof of Theorem \ref{09June2020Thm09} (see especially (\ref{09June2020Eqn10}) and (\ref{Correction05}) therein), one can tell that for a deterministic stationary policy in the auxiliary model, which depends on $(x,i)\in S\times \{-1,1\}$ only through $x\in S$, and is identified by a measurable mapping $\varphi$ from $S$ to $A$, $\breve{V}((x,i),\varphi)=V(x,\varphi)$ for all $x\in S$ and $i\in \{-1,1\}$. Therefore, if such a deterministic stationary policy $\varphi$ is optimal in the auxiliary model, then so is it in the original model, and $ V^\ast(x)=\breve{V}^\ast(x)=\breve{V}((x,i),\varphi)=V(x,\varphi)$ for each $x\in S.$
\item[(c)] By Theorem \ref{09June2020Thm09}(a) and the second equality in (\ref{09June2020Eqn15}), we see  $\overline{V}(x,\pi)=\breve{\overline{V}}((x,i),\breve{\pi})$ for the same policies $\pi$ and $\breve{\pi}$ in Theorem \ref{09June2020Thm09}, too.
\end{itemize}
\end{remark}

Let us introduce a natural set of compactness-continuity conditions on the original PDMDP.
\begin{condition}\label{GZyExponentialCondition02}
\begin{itemize}
\item[(a)] For each bounded measurable function $f$ on $S$ and each $x\in S$, $\int_S f(y)\tilde{q}(dy|x,a)$ is continuous in $a\in A.$
\item[(b)] For each $x\in S,$ the (nonnegative) function $c(x,a)$ is lower semicontinuous in $a\in A.$
\item[(c)] The action space $A$ is a compact Borel space.
\end{itemize}
\end{condition}

The usefulness of the auxiliary PDMDP also partially lies in the next observation.
\begin{lemma}\label{09June2020Lemma01}
If the original PDMDP model satisfies Conditions \ref{GZyExponentialCondition02} and \ref{GGZyExponentialConditionExtra}, then the auxiliary model satisfies the corresponding versions of Conditions \ref{GZyExponentialCondition02} and \ref{GGZyExponentialConditionExtra}, too.
\end{lemma}
\par\noindent\textit{Proof.} We only verify the version of Condition \ref{GZyExponentialCondition02}(a). For any bounded measurable function $f$ on $\breve{S}$,
it holds that
\begin{eqnarray*}
&&\int_{\breve{S}}f(y,j)\tilde{\breve{q}}(d(y,j)|(x,i),a)= \int_{\breve{S}}f(y,j){\breve{q}}(d(y,j)|(x,i),a)+f(x,i)\breve{q}_{(x,i)}(a)\\
&=& \int_S f(y,i)(q(dy|x,a)-\lambda\delta_x(dy))+\int_S f(y,-i)\lambda \delta_x(dy)+f(x,i)(\lambda+q_x(a))\\
&=&\int_S f(y,i)\tilde{q}(dy|x,a)-q_x(a)f(x,i)-\lambda f(x,i)+\lambda f(x,-i)+f(x,i)(\lambda+q_x(a))\\
&=&\int_S f(y,i)\tilde{q}(dy|x,a) +\lambda f(x,-i),
\end{eqnarray*}
which is clearly continuous in $a\in A$ when the original model satisfies Condition \ref{GZyExponentialCondition02}. $\hfill\Box$
\bigskip

The following statement was obtained in Theorem 3.1 and Remark 3.1 of \cite{GuoZhangAMO}.
\begin{proposition}\label{CorrectionRemark01}
Suppose Conditions \ref{GGZyExponentialConditionExtra} and \ref{GZyExponentialCondition02} are satisfied. In addition\footnote{See the footnote in Proposition \ref{09June2020PropositionA01}.}, $\inf_{(x,a)\in S\times A}q_x(a)>0$. Then the following assertions hold.
\begin{itemize}
\item[(a)] The value function $V^\ast$ is the minimal $[1,\infty]$-valued measurable solution to the following optimality equation:
  \begin{eqnarray}\label{09June2020Eqn11}
&& -(V(\phi(x,t))-V(x)) \\
&=&\int_0^t \inf_{a\in A}\left\{ \int_S V(y)\tilde{q}(dy|\phi(x,\tau),a)- (q_{\phi(x,\tau)}(a)-c(\phi(x,\tau),a) )V(\phi(x,\tau))\right\}d\tau \nonumber\\
&&\forall~t\in[0,\infty),x\in S^\ast:=\{x\in S:~V^\ast(x)<\infty\};~V(x)<\infty~\forall~x\in S^\ast; ~V(x)=\infty ~\forall~x\notin S^\ast.\nonumber
 \end{eqnarray}
In particular, $V^\ast(\phi(x,t))$ is absolutely continuous in $t$ for each $x\in S^\ast.$
 \item[(b)]  Any measurable mapping $\varphi$ from $S$ to $A$ such that
 \begin{eqnarray*}
 &&\inf_{a\in A}\left\{ \int_S V^\ast(y)\tilde{q}(dy|x,a)- (q_{x}(a)-c(x,a))V^\ast(x)\right\}\\
 &=&\int_S V^\ast(y)\tilde{q}(dy|x,\varphi(x))- (q_{x}(\varphi(x))-c(x,\varphi(x)))V^\ast(x),~\forall~x\in S^\ast.
 \end{eqnarray*}
 defines a deterministic stationary optimal policy in the original model. Such measurable selectors $\varphi$ exist.
\end{itemize}
\end{proposition}

As an application of Theorem \ref{09June2020Thm09} (more precisely, Remark \ref{09June2020Remark01} drawn from it), we may remove the redundant condition on the strong positivity of the transition intensities from Proposition \ref{CorrectionRemark01}.
\begin{corollary}\label{09June2020Corollary01}
Under Conditions \ref{GGZyExponentialConditionExtra} and \ref{GZyExponentialCondition02}, the assertions stated in Proposition \ref{CorrectionRemark01} all hold without requiring $\inf_{(x,a)\in S\times A}q_x(a)>0.$
\end{corollary}

\par\noindent\textit{Proof.} The statement follows from Remark \ref{09June2020Remark01} and applying Proposition \ref{CorrectionRemark01} to the auxiliary model, which is legitimate in view of Lemma \ref{09June2020Lemma01} and that the transition intensities in the auxiliary model are strongly positive. The details are as follows.

Step 1. We show that the value function $\breve{V}^\ast((x,i))$ in the auxiliary PDMDP model depends on $(x,i)$ only through $x$, and can thus be identified as $\breve{V}^\ast(x)$.

For this, we will apply the following result from \cite{GuoZhangAMO}: under the conditions in Proposition \ref{CorrectionRemark01}, including that the transition intensities are strongly positive:
\begin{itemize}
\item The value function $V^\ast$ in the original model is the minimal $[1,\infty]$-valued measurable solution to the optimality equation $V={\cal T}\circ V$, where
\begin{eqnarray}\label{Correction01}
{\cal T}\circ V(x)&:=& \inf_{\rho\in {\cal R}}\left\{\int_0^\infty e^{-\int_0^\tau (q_{\phi(x,s)}(\rho_s)-c(\phi(x,s),\rho_s))ds} \left(\int_S V(y)\tilde{q}(dy|\phi(x,\tau),\rho_\tau)\right)d\tau\right.\nonumber\\
&&\left. +e^{-\int_0^\infty q_{\phi(x,s)}(\rho_s)ds}e^{\int_0^\infty c(\phi(x,s),\rho_s)ds} \right\},~\forall~x\in S,
\end{eqnarray}
Here and below, ${\cal R}$ is the space of $\mathbb{P}(A)$-valued measurable mappings $\rho=(\rho_t(da))$ on $(0,\infty)$\footnote{Two elements in ${\cal R}$ that coincide almost everywhere are not distinguished.},  and $e^{-\int_0^\infty q_{\phi(x,s)}(\rho_s)ds}e^{\int_0^\infty c(\phi(x,s),\rho_s)ds}:=0$
whenever
$e^{-\int_0^\infty q_{\phi(x,s)}(\rho_s)ds}=0.$
\item The value function $V^\ast$ can be obtained from the successive approximation: $V^\ast(x)=\lim_{n\rightarrow \infty}{\cal T}\circ V_0(x)$ with $V_0(x)\equiv 1$.
\end{itemize}

According to Lemma \ref{09June2020Lemma01} and that the transition intensities in the auxiliary model are strongly positive (as $\inf_{x\in S,a\in A}\breve{q}_{(x,i)}(a)\ge \lambda>0$), we may apply the result just quoted above to the auxiliary model and conclude that $\breve{V}^\ast$ is the minimal $[1,\infty]$-valued measurable function to the following equation
\begin{eqnarray*}
&&\breve{V}((x,i))\\
&=&\inf_{\rho\in{\cal R}}\left\{\int_0^\infty e^{\int_0^\theta \breve{c}(\breve{\phi}((x,i),s),\rho_s)ds} e^{-\int_0^\theta \breve{q}_{\breve{\phi}((x,i),s)}(\rho_s) ds}\left(\int_{\breve{S}} \breve{V}((y,j))\tilde{\breve{q}}(d(y,j)|\breve{\phi}((x,i),\theta),\rho_\theta) \right)d\theta\right\}\\
&=&\inf_{\rho\in{\cal R}}\left\{\int_0^\infty e^{\int_0^\theta c(\phi(x,s),\rho_s)ds} e^{-\int_0^\theta (q_{\phi(x,s)}(\rho_s)+\lambda )ds}\right.\\
&&\left.\times\left(\int_S \tilde{q}(dy|\phi(x,\theta),\rho_\theta)\breve{V}((y,i))+\lambda \breve{V}((\phi(x,\theta),-i))\right)d\theta\right\}.
\end{eqnarray*}
Moreover, $\breve{V}^\ast$ is the pointwise limit of the sequence of functions $\{\breve{V}_n\}_{n=0}^\infty$ with
\begin{eqnarray*}
&&\breve{V}_0((x,i)):\equiv 1,\\
&&\breve{V}_{n+1}((x,i)):=\inf_{\rho\in{\cal R}}\left\{\int_0^\infty e^{\int_0^\theta c(\phi(x,s),\rho_s)ds} e^{-\int_0^\theta (q_{\phi(x,s)(\rho_s)}+\lambda )ds}\right.\\
&&\left.\left(\int_S \tilde{q}(dy|\phi(x,\theta),\rho_\theta)\breve{V}_n((y,i))+\lambda \breve{V}_n((\phi(x,\theta),-i))\right)d\theta\right\}.
\end{eqnarray*}
An inductive argument reveals that $\breve{V}_{n+1}((x,i))$ does not depend on $i$ for all $n\ge 0$ and thus $\breve{V}^\ast((x,i))$ does not depend on $i.$  Below, we write $\breve{V}^\ast(x)$ for $\breve{V}^\ast((x,i))$.

Step 2. Again by Lemma \ref{09June2020Lemma01} and that the transition intensities in the auxiliary model are strongly positive,  we apply Proposition \ref{CorrectionRemark01}(b) to the auxiliary model to obtain a deterministic stationary optimal policy $\varphi$. It is possible to take $\varphi$, which only depends on $x\in S$ (independent on $i\in \{-1,1\}$) because for each $(x,i)\in \breve{S}^\ast:=\{x\in S: \breve{V}^\ast(x)<\infty\}\times \{-1,1\}$,
\begin{eqnarray*}
 &&\int_{\breve{S}} \breve{V}^\ast(y)\tilde{\breve{q}}(d(y,j)|(x,i),a)- (\breve{q}_{(x,i)}(a)-\breve{c}((x,i),a))\breve{V}^\ast(x))\\
 &=&  \int_S \breve{V}^\ast(y)\tilde{q}(dy|x,a)- (q_{x}(a)-c(x,a))\breve{V}^\ast(x))
 \end{eqnarray*}
does not involve $i\in\{-1,1\}$, where the equality holds by the definition of $\breve{q}$ and $\breve{c}$ and a similar calculation as the one in the proof of Lemma \ref{09June2020Lemma01}.

Step 3. Step 2 and Remark \ref{09June2020Remark01}(b) imply $\breve{V}^\ast(x)=V^\ast(x)$. Note that the optimality equations for both the original model and the auxiliary model are the same and given by (\ref{09June2020Eqn11}), the statement of this corollary follows from applying again Proposition \ref{CorrectionRemark01} to the auxiliary model. $\hfill\Box$

\appendix
\section{Appendix: Induced DTMDP}\label{NoteGZSec02}
We shall formulate a DTMDP model induced from the PDMDP model with total cost criteria by inspecting the PDMDP at each of its jump moments and regarding the relaxed control functions used during a sojourn time as the actions in the DTMDP. The first coordinate in the state space of the induced DTMDP records the most recent sojourn time, and the second coordinate records the state in the PDMDP immediately after the corresponding jump. This is to serve the formulation of Example \ref{JapanExample01} below.

The DTMDP induced by the PDMDP $\{S,A,q,\phi,c\}$ is specified by the following system primitives:
\begin{itemize}
\item The state space is $\textbf{X}:=((0,\infty)\times S)\bigcup\{(\infty,x_\infty)\}$. Whenever the topology is concerned, $(\infty,x_\infty)$ is regarded as an isolated point in $\textbf{X}.$
\item The action space is $\textbf{A}:={\cal R}$, where ${\cal R}$ was defined in the proof of Corollary \ref{09June2020Corollary01}. We endow ${\cal R}$ with the Young topology.\footnote{The Young topology on ${\cal R}$ is the weakest
topology with respect to which the function
$
\rho\in {\cal{R}}\rightarrow \int_0^\infty \int_A
f(t,a)\rho_t(da)dt
$
is continuous for each strongly integrable Carath\'{e}odory function
$f$ on $(0,\infty)\times A$ . Here a real-valued measurable
function $f$ on $(0,\infty)\times A$ is called a strongly
integrable Carath\'{e}odory function if for each fixed
$t\in(0,\infty)$, $f(t,a)$ is continuous in $a\in A,$ and for each
fixed $a\in A,$ $\sup_{a\in A}|f(t,a)|$ is integrable in $t$,
i.e., $\int_0^\infty \sup_{a\in A}|f(t,a)|dt<\infty.$}
\item The transition kernel $p$ on ${\cal B}(\textbf{X})$ from $\textbf{X}\times \textbf{A}$  is given for each $\rho=(\rho_t(da))_{t>0}\in \textbf{A}$ by
    \begin{eqnarray*}
    p(\Gamma_1\times\Gamma_2|(\theta,x),\rho)&:=&\int_{\Gamma_2} e^{-\int_0^t q_{\phi(x,s)}(\rho_s)ds}\tilde{q}(\Gamma_1|\phi(x,t),\rho_t)dt,\nonumber\\
    &&~\forall~\Gamma_1\in {\cal B}(S),~\Gamma_2\in {\cal B}((0,\infty)),~x\in S,~\theta\in (0,\infty),\nonumber\\
     p(\{(\infty,x_\infty)\}|(\theta,x),\rho)&:=&e^{-\int_0^\infty q_{\phi(x,s)}(\rho_s)ds},~\forall~x\in S,~\theta\in (0,\infty);\nonumber\\
     p(\{(\infty,x_\infty)\}|(\infty,x_\infty),\rho)&:=&1.\nonumber
\end{eqnarray*}
(Recall that the notation $q(dy|x,\rho_t)=\int_A q(dy|x,a)\rho_t(da)$ is in use.)
\item The cost function $l$ is a $[0,\infty]$-valued measurable function on $\textbf{X}\times\textbf{A}\times \textbf{X}$ given by
\begin{eqnarray*}\label{ZyExponential55}
l((\theta,x),\rho,(\tau,y)):=\int_0^\infty I\{s<\tau\} c(\phi(x,s),\rho_s)ds,~\forall~((\theta,x),\rho,(\tau,y))\in \textbf{X}\times\textbf{A}\times \textbf{X}.
\end{eqnarray*}
\end{itemize}

For the induced DTMDP $\{\textbf{X},\textbf{A},p,l\}$, following the reasoning in the proof of Lemma 3.2 of \cite{Costa:2013} and Chapter 4 of \cite{Davis:1993}, one can see the following statement.
\begin{proposition}\label{09June2020PropositionA01}
Under Conditions \ref{GGZyExponentialConditionExtra} and \ref{GZyExponentialCondition02}, for each $(\theta,x)\in \textbf{X}$ and $(\tau,y)\in\textbf{X}$, $\rho\in\textbf{A}\rightarrow l((\theta,x),\rho,(\tau,y))$ is lower semicontinuous, and $\textbf{A}$ is a compact Borel space. If in addition, the transition intensities are strongly positive\footnote{This requirement was unfortunately missing and overlooked in \cite{GuoZhangAMO}. Indeed, the proof of Lemma 4.1 of \cite{GuoZhangAMO} made use of the strong Feller property of the transition kernel $p$ in the induced DTMDP, which could fail to hold without this additional requirement, as demonstrated in Example \ref{JapanExample01}. Since the rest of the arguments in \cite{GuoZhangAMO} are largely based on that lemma, this missing requirement on the strong positivity of transition intensities should be added in \cite{GuoZhangAMO} wherever appropriate. We thank Dr Yonghui Huang (Sun Yat-Sen University, China) for drawing our attention on this inaccuracy.}, then for each $(\theta,x)\in \textbf{X}$, the function $\rho\in \textbf{A}\rightarrow \int_{\textbf{X}}f(z)p(dz|(\theta,x),\rho)$ is continuous for each bounded measurable function $f$ on $\textbf{X}$.
\end{proposition}

The next example shows that if the transition intensities are not strongly positive, then it can happen that $\rho\in \textbf{A}\rightarrow \int_{\textbf{X}}f(z)p(dz|(\theta,x),\rho)$ is not continuous for  some bounded measurable function $f$ on $\textbf{X}$.

\begin{example}\label{JapanExample01}
Suppose $S$ is any finite set (endowed with discrete topology), and $A=[0,1]$, which is a compact Borel space, $q_x(a)=a$ and $c(x,a)\equiv 0$, and $\phi(x,t)\equiv x$. Evidently, Conditions \ref{GGZyExponentialConditionExtra} and \ref{GZyExponentialCondition02} are satisfied by this PDMDP model. Consider $\rho\in \textbf{A}$ and $(\rho^{(n)})\subseteq \textbf{A}$ defined as follows: for each $t\ge 0,$ $\rho^{(n)}_t(da)=\delta_{\frac{1}{n}}(da)$, and $\rho_t(da)=\delta_0(da).$
Then for each strongly integrable Carath\'{e}odory function $g(t,a)$,
\begin{eqnarray*}
\int_0^\infty g(t,\rho^{(n)}_t)dt-\int_0^\infty g(t,\rho^{(0)}_t)dt=\int_0^\infty (g(t,\frac{1}{n})-g(t,0))dt\rightarrow 0
\end{eqnarray*}
as $n\rightarrow \infty,$ by using the dominated convergence theorem. Thus, $\rho^{(n)}\rightarrow \rho$ as $n\rightarrow \infty.$  (Recall that $\textbf{A}$ is endowed with the Young topology.)
Now for $f(t,x)\equiv 0$ on $(0,\infty)\times S$ and $f(\infty,x_\infty)=1$,
\begin{eqnarray*}
  \int_{\textbf{X}}f(z)p(dz|(\theta,x),\rho^{(n)})=e^{-\int_0^\infty q_x(\rho^{(n)}_s)ds}=e^{-\int_0^\infty \frac{1}{n}ds}=0<  1=e^{-\int_0^\infty 0 ds}= \int_{\textbf{X}}f(z)p(dz|(\theta,x),\rho).
\end{eqnarray*}
\end{example}


TITLE: Green's function on torus
QUESTION [8 upvotes]: I have a question about the Green's function $G(z,w)$ on torus which takes the form (for example the first equation in the paper https://annals.math.princeton.edu/wp-content/uploads/annals-v172-n2-p03-p.pdf)
\begin{equation}
-\nabla^2_z G(z,w)=\delta_w(z)-1/|T|
\end{equation}
where the first term on the RHS is a delta function, and the second term on the RHS $-1/|T|$ is a constant term. This Green's function may have some relevance in CFT on a torus. My question is why this constant term come into play? What is the meaning of the constant term?

REPLY [12 votes]: When both "space" ($x$) and "time" ($y$) directions are periodic, the Laplacian on torus with coodinate $z=x+iy$ has a normalized zero mode
$$
\varphi_0(z) = \frac 1 {\sqrt{{\rm Im}(\tau)}}
$$
(Here $\tau$ is the modular parameter defining the torus.)
As
$$
-\nabla^2 \varphi_0=0.
$$
the zero mode means that the Laplace operator is not 1-1 and so prevents  the Laplacian with periodic boundary conditions from having an  inverse. There is therefore no actual Green function. Instead we  must therefore resort to a modified Green function. We can make use of a theta function with characteristics defined by
$$
\theta\left[ \matrix{a\cr b}\right] (z|\tau)= \sum_{m=-\infty}^\infty \exp\{i\pi \tau(m+a)^2 +2\pi i (m+a)(z+b)\}, \quad {\rm Im}(\tau)>0\quad a,b \in {\mathbb R}.
$$
Observe that
$$
F(x,y) \equiv e^{-\pi y^2/ {\rm Im}(\tau)} \theta\left[ \matrix{\textstyle{\frac 12}\cr \textstyle{\frac 12}}\right] (z|\tau)
$$
obeys
$$
F(x+1,y) =-F(x,y), \quad F(x+ {\rm Re}(\tau), y+{\rm Im}(\tau)) =({\rm phase}) F(x,y)
$$
so
$$
G_0(x,y)) = -\frac{1}{2\pi} \ln |F| =  -\frac{1}{2\pi} \ln\left| \theta\left[ \matrix{\textstyle{\frac 12}\cr \textstyle{\frac 12}}\right] (z|\tau)\right|
+ \frac 12 y^2 / {\rm Im}(\tau)\\
= -\frac 1{2\pi} \ln |E(z)| + \frac 12 y^2 / {\rm Im}(\tau)+const.
$$
is both periodic on the torus and obeys
$$
-\nabla^2 G_0(x,y) = \delta^2(x,y) - 1/{\rm Im}(\tau)\\
= \sum_n \varphi_n(z) \varphi^*_n(0)- \varphi_0(z)\varphi_0(0)
$$
Here the sum is over all $n$ including $n=0$. The $\varphi_n(z)$, $n>0$  are the eigenfuctions of the Laplacian with non-zero eigenvalues.
The modified Green function can be used to solve
$$
-\nabla^2 \phi= f(x,y)
$$ as
$$
\phi(x,y) =
\int_{\rm torus} G_0(x-x', y-y'))f(x',y') dx' dy'=0.
$$
provided that $f$ is perpendicular to the the zero mode, i.e.
$$
\int_{\rm torus} f(x,y) dx dy=0.
$$
This is consistent with the Fredholm alternative for linear operators.

REPLY [7 votes]: The constant term is needed because on a compact manifold with periodic boundary conditions there is a zero mode in the spectrum of the laplacian. This is easier to see on the circle, where $\frac{d^2}{dt^2}f = \lambda f$ has the periodic constant solution $f(t) =c$. This makes the operator not-inveritble.
One should instead work in the space orthogonal to the zero modes. Let's look at the spectral decomposition of the Green function:
$$G(t, t') = \sum_\lambda u_\lambda^*(t) u_\lambda(t') \times \lambda^{-1}, $$
where $u_\lambda(t) $are the eigenfunctions of the Operator. Note that
$$\nabla^2 G = \sum_\lambda u_\lambda^*(t) u_\lambda(t') = \delta(t - t') $$
by completeness.
Now you see in the expression for the Green function why $\lambda = 0$ would be problematic. We remove this from the sum to work on the subspace orthogonal to this function, getting
$$G'(t, t') = \sum_{\lambda \neq 0} u_\lambda^*(t) u_\lambda(t') \times \lambda^{-1}$$
Following which we get a modified Green equation
$$\nabla^2 G = \sum_{\lambda \neq 0} u_\lambda^*(t) u_\lambda(t') = \delta(t - t') - u_0^*(t)u(t').$$
This last term corresponds to the constant piece you're subtracting in the case of periodic boundary conditions, because we saw that the zero eigenfunctions is constant. The factor of T is simply from some normalisation condition.
The other way of seeing it is to think in terms of electrostatics. On a compact manifold the periodicity is inconsistent with the Green function that represents the response to a point charge placed at some point:
$$\int_{M} \delta(t, t') = \int_{M} \nabla^{2} G = \int_{\partial M}\nabla G \cdot dn = 0$$
since the Green function will be periodic. Yet the left hand side does not integrate to 0. To fix this we must add a constant negative "background charge" which modifies the LHS to $\int_{M}\delta(t, t') - |c|^{2}$ with $c$ chosen so that the integral vanishes. In other words this background charge is needed to ensure compatibility with the periodic boundary conditions.


text:
\begin{document}
\title{Dynamic risk measures on variable exponent Bochner--Lebesgue spaces}

\author{Fei Sun\and Yijun Hu}
\institute{F. Sun \at
	School of Mathematics and Computational Science, Wuyi University, Jiangmen 529020, China \\
	\email{fsun.sci@outlook.com}

	\and
	Y. Hu \at
	School of Mathematics and Statistics, Wuhan University, 	Wuhan 430072,  China\\
	\email{yjhu.math@whu.edu.cn}
}

\date{}
\maketitle

\begin{abstract} In this paper, we consider dynamic risk measures on  $L^{p(\cdot)}$.
\keywords{  risk measure \and dynamic}

 \subclass{91B32}
\end{abstract}

\section{Introduction}

The remainder of this paper is organized as follows. In Section 2, we briefly review the definition and main properties of variable exponent Bochner--Lebesgue spaces. In Section 3, we consider the dual representation of convex risk measures on variable exponent Bochner--Lebesgue spaces. Section 4 discusses a class of specific risk measures known as optimized certainty equivalents. Finally, in Section 5, we study dual representation of dynamic risk measures on the variable exponent Bochner--Lebesgue spaces. The related time consistency is also studied.

\section{Preliminaries}

In this section, we briefly introduce  properties of variable exponent Bochner--Lebesgue spaces.

\section{Convex risk measures on $\mathbf{L^{p(\cdot)}}$}

The main aim of this paper is to study the dual representation of dynamic risk measures on variable exponent Bochner--Lebesgue spaces. To this end, this section first considers convex risk measures, which will be used later for the dynamic risk measures. The convex risk measure was first introduced by  F\"{o}llmer and Schied (2002) and  Frittelli and Rosazza Gianin (2002).\\

In the absence of ambiguity, we denote the variable exponent Bochner--Lebesgue space by $L^{p(\cdot)}:=L^{p(\cdot)}(\Omega, E)$. Let $T$ be a discrete time horizon which can reach infinity and consider a filtered probability space $(\Omega,\mathcal{F},(\mathcal{F}_{t})_{t=0}^{T},\mathbb{P})$ with $\{\emptyset,\Omega\}=\mathcal{F}_{0}\subset \mathcal{F}_{1}\subset\ldots \subset\mathcal{F}_{T}=\mathcal{F}$. Let $L^{p(\cdot)}(\mathcal{F}_{t})$ be the space of all strongly $\mathcal{F}_{t}$-measurable functions $f_{t}$ which satisfy Definition~\ref{D22}. Note that $L^{p(\cdot)}=L^{p(\cdot)}(\mathcal{F}_{T})$. We denote $L^{p(\cdot)}(K):=\{f\in L^{p(\cdot)}: f:\Omega\rightarrow K\}$ and $L^{p'(\cdot)}(K_{0}):=\{g\in L^{p'(\cdot)}: g:\Omega\rightarrow K_{0}\}$. We denote the space of all essentially bounded $\mathcal{F}_{t}$-measurable random variables by $L_{t}^{\infty}:=L^{\infty}(\Omega,\mathcal{F}_{t},\mu)$.

\begin{Remark}\label{R23}\textnormal
	{
		As the Banach space $E$ is partially ordered by $K$, the definition of $L^{p(\cdot)}$ implies that each $f\in L^{p(\cdot)}$ is an $E$-valued measurable function. Thus, in the absence of ambiguity, we also consider $L^{p(\cdot)}$ to be partially ordered by $K$. Next, the definition of a convex risk measure on $L^{p(\cdot)}$ is introduced using an axiomatic approach.
	}
\end{Remark}

\begin{Definition}\label{D31}\textnormal
	{
		Let $E$ be a Banach space ordered by the partial ordering relation induced by a cone $K$ with interior point $z$, and $L^{p(\cdot)}$ be a variable exponent Bochner--Lebesgue space. A function $\varrho:L^{p(\cdot)}\rightarrow \mathbb{R}$ is said to be a $p(\cdot)$-convex risk measure if it satisfies the following:
		\begin{description}
			\item[A1] Monotonicity: for any $f_{1},f_{2}\in L^{p(\cdot)}$, $f_{1}\leq_{K}f_{2} \textrm{ implies } \varrho(f_{1})\geq \varrho(f_{2})$;
			\item[A2] Translation invariance: for any $m\in \mathbb{R}$ and $f\in L^{p(\cdot)}$, $\varrho(f+mz)=\varrho(f)-m$;
			\item[A3] Convexity: for any $f_{1},f_{2}\in L^{p(\cdot)}$ and $\lambda\in(0,1)$, $\varrho(\lambda f_{1}+(1-\lambda)f_{2})\leq \lambda\varrho(f_{1})+(1-\lambda)\varrho(f_{2})$.
		\end{description}
	}
\end{Definition}

\begin{Remark}\label{R32}\textnormal
	{
		The order in $\mathbf{A1}$ is the partial order under a cone $K$ which is defined by Remark~\ref{R21}.
		The interior point $z$ of $K$ in $\mathbf{A2}$ is considered to be the numeraire asset, which means that $mz\in E$ for any $m\in \mathbb{R}$. Before we study the dual representation of the $p(\cdot)$-convex risk measures, the acceptance sets should be defined.
	}
\end{Remark}

\begin{Definition}\label{D32}\textnormal
	{
		The acceptance set of the $p(\cdot)$-risk measure $\varrho$ is defined as
		\begin{displaymath}
		\mathcal{A}_{\varrho}:=\big\{f\in L^{p(\cdot)}: \varrho(f)\leq0\big\}
		\end{displaymath}
		and we denote $\mathcal{A}_{\varrho}^{0}$ by
		\begin{displaymath}
		\mathcal{A}^{0}_{\varrho}:=\Big\{g\in \big(L^{p(\cdot)}\big)^{\ast}: \langle g,f\rangle\geq 0 \textrm{ for any } f\in \mathcal{A}_{\varrho}\Big\}.
		\end{displaymath}
	}
\end{Definition}

\begin{Remark}\label{R33}\textnormal
	{
		It is relatively easy to check that $\mathcal{A}_{\varrho}$ is a convex set if $\varrho$ satisfies the convexity property. $\mathcal{A}^{0}_{\varrho}$ can be considered as the positive polar cone of $\mathcal{A}_{\varrho}$.
	}
\end{Remark}

Now, we provide the dual representation of $p(\cdot)$-convex risk measures which will be used in the proof of
$p(\cdot)$-dynamic risk measures in Section 5.

\begin{Theorem}\label{T32}\textnormal
	{
		If $\varrho:L^{p(\cdot)}\rightarrow \mathbb{R}$ is a $p(\cdot)$-convex risk measure, then for any $f\in L^{p(\cdot)}$,
		\begin{displaymath}
		\varrho(f)=\sup_{g\in Q_{p(\cdot)}}\{ \langle g,-f\rangle-\alpha(g)\}
		\end{displaymath}
		where
		\begin{displaymath}
		Q_{p(\cdot)}:=\Big\{g\in \big(L^{p(\cdot)}\big)^{\ast}:\langle \frac{dg}{d\mu}, z\rangle=1, \frac{dg}{d\mu}\in L^{p'(\cdot)}(K_{0})\Big\}
		\end{displaymath}
		and the minimal penalty function $\alpha_{\min}$ is denoted by
		\begin{displaymath}
		\alpha_{\min}(g):=\sup_{f\in L^{p(\cdot)}}\big\{\langle g,-f\rangle-\varrho(f)\big\}=\sup_{f\in\mathcal{A}_{\varrho}}\big\{\langle g,-f\rangle\big\}.
		\end{displaymath}
	}
\end{Theorem}

\noindent \textbf{Proof.}
For any $g\in Q_{p(\cdot)}$, we denote
\begin{displaymath}
\alpha(g)=\sup_{f\in L^{p(\cdot)}}\big\{\langle g,-f\rangle-\varrho(f)\big\}
\end{displaymath}
and
\begin{displaymath}
\alpha_{\min}(g)=\sup_{f\in\mathcal{A}_{\varrho}}\big\{\langle g,-f\rangle\big\}.
\end{displaymath}
We now show that $\alpha(g)=\alpha_{\min}(g)$ for any $g\in Q_{p(\cdot)}$. Note that $\alpha(g)\geq \alpha_{\min}(g)$. Indeed, for any $f\in\mathcal{A}_{\varrho}$, $\langle g,-f\rangle-\varrho(f)\geq\langle g,-f\rangle$. Hence,
\begin{displaymath}
\sup_{f\in L^{p(\cdot)}}\big\{\langle g,-f\rangle-\varrho(f)\big\}\geq \sup_{f\in\mathcal{A}_{\varrho}}\big\{\langle g,-f\rangle-\varrho(f)\big\}\geq \sup_{f\in\mathcal{A}_{\varrho}}\big\{\langle g,-f\rangle\big\}.
\end{displaymath}
We now prove that $\alpha(g)\leq \alpha_{\min}(g)$. For any $f\in L^{p(\cdot)}$, consider $f_{1}=f+\varrho(f)z\in \mathcal{A}_{\varrho}$. Thus,
\begin{eqnarray*}
	\alpha_{\min}(g)\geq \langle g,-f_{1}\rangle &=& \langle g,-f\rangle-\varrho(f)\langle g,z\rangle\\
	&=& \langle g,-f\rangle-\varrho(f)\int_{\Omega}\langle \frac{dg}{d\mu}, z\rangle d\mu\\
	&=& \langle g,-f\rangle-\varrho(f).
\end{eqnarray*}
Hence, we have $\alpha(g)=\alpha_{\min}(g)$, and it is easy to check that
\begin{displaymath}
\varrho(f)\geq\sup_{g\in Q_{p(\cdot)}}\big\{ \langle g,-f\rangle-\alpha(g)\big\}.
\end{displaymath}
Next, we show that the above inequality only holds in the case of equality. Suppose there is some $f_{0}\in  L^{p(\cdot)}$ such that
\begin{displaymath}
\varrho(f_{0})>\sup_{g\in Q_{p(\cdot)}}\big\{ \langle g,-f_{0}\rangle-\alpha(g)\big\}.
\end{displaymath}
Hence, there exists some $m\in\mathbb{R}$ such that
\begin{displaymath}
\varrho(f_{0})>m>\sup_{g\in Q_{p(\cdot)}}\big\{ \langle g,-f_{0}\rangle-\alpha(g)\big\}.
\end{displaymath}
Thus, we have
\begin{displaymath}
\varrho(f_{0}+mz)=\varrho(f_{0})-m>0,
\end{displaymath}
which means that $f_{0}+mz\notin \mathcal{A}_{\varrho}$. As $\{f_{0}+mz\}$ is a singleton set, it is also a convex set. On the contrary, $\mathcal{A}_{\varrho}$ is also a closed convex set because $\varrho$ is a $p(\cdot)$-convex risk measure. Then, by the Strong Separation Theorem for convex sets, there exists some $\pi\in (L^{p(\cdot)})^{\ast}$ such that
\begin{equation}\label{32}
\langle \pi,f_{0}+mz\rangle > \sup_{f\in\mathcal{A}_{\varrho}}\langle \pi,f\rangle.
\end{equation}
By Remark~\ref{R24}, $\langle \pi,f\rangle=\int_{\Omega}\langle h, f\rangle d\mu$, where $h\in L^{p'(\cdot)}(\Omega, E^{\ast})$. It is easy to check that $h$ takes negative values on $L^{p(\cdot)}(K)$. Then, we have that $-h\in L^{p'(\cdot)}(K_{0})$. For any $-\pi\in Q_{p(\cdot)}$, $\langle h, z\rangle=-1$. Thus, by (\ref{32}), we can conclude that
\begin{eqnarray*}
	\langle \pi,f_{0}+mz\rangle > \sup_{f\in\mathcal{A}_{\varrho}}\langle \pi,f\rangle &\Rightarrow& \int_{\Omega}\langle h, f_{0}+mz\rangle d\mu > \sup_{f\in\mathcal{A}_{\varrho}}\int_{\Omega}\langle h, f\rangle d\mu\\
	&\Rightarrow&\int_{\Omega}\big(\langle h, f_{0}\rangle-m\big) d\mu > \sup_{f\in\mathcal{A}_{\varrho}}\int_{\Omega}\langle h, f\rangle d\mu\\
	&\Rightarrow&\int_{\Omega}\langle h, f_{0}\rangle d\mu -m> \sup_{f\in\mathcal{A}_{\varrho}}\int_{\Omega}\langle h, f\rangle d\mu\\
	&\Rightarrow& \langle \pi, f_{0}\rangle-\sup_{f\in\mathcal{A}_{\varrho}}\langle \pi,f\rangle>m\\
	&\Rightarrow& \langle \pi, f_{0}\rangle-\alpha(-\pi)>m.
\end{eqnarray*}
Replacing $-\pi$ by $g_{0}$, we have
\begin{displaymath}
\langle g_{0}, -f_{0}\rangle-\alpha(g_{0})>m.
\end{displaymath}
This is a contradiction, because in this case
\begin{displaymath}
m>\sup_{g\in Q_{p(\cdot)}}\big\{ \langle g,-f_{0}\rangle-\alpha(g)\big\}\geq\langle g_{0}, -f_{0}\rangle-\alpha(g_{0})>m.
\end{displaymath}
The contradiction arises from the assumption that some $f_{0}\in  L^{p(\cdot)}$ exists such that
\begin{displaymath}
\varrho(f_{0})>\sup_{g\in Q_{p(\cdot)}}\big\{ \langle g,-f_{0}\rangle-\alpha(g)\big\}.
\end{displaymath}
Hence, we have
\begin{displaymath}
\varrho(f)=\sup_{g\in Q_{p(\cdot)}}\big\{ \langle g,-f\rangle-\alpha(g)\big\}.
\end{displaymath}
For the opposite direction, it is relatively simple to check that $\varrho$ satisfies the properties of a $p(\cdot)$-convex risk measure.
This completes the proof of Theorem~\ref{T32}.\\

\indent The $p(\cdot)$-convex risk measures can  be considered as extensions of the convex risk measures studied by Frittelli and Rosazza Gianin (2002).  A special example of $p(\cdot)$-convex risk measures, the so-called OCE, is discussed in the next section. Finally, in Sections 5, the $p(\cdot)$-convex risk measures are used to study the dual representation of the $p(\cdot)$-dynamic  risk measures.

\section{Optimized Certainty Equivalent on $\mathbf{L^{p(\cdot)}}$}

In this section, we study a special class of $p(\cdot)$-convex risk measures: the  Optimized Certainty Equivalent (OCE), which will be used as an example of dynamic risk measures in Section 5.
The OCE was first introduced by Ben-Tal and Teboulle (1986) and later developed by the same researchers (Ben-Tal and Teboulle 2007). In this section, we define the OCE on variable exponent Bochner--Lebesgue spaces  $L^{p(\cdot)}$.  Further, we establish its main properties, and show how it can be used to generate $p(\cdot)$-convex risk measures.
Note that the OCE can be used as an application of convex risk measures.

\begin{Definition}\label{D41}\textnormal
	{
		Let $u:E\rightarrow [-\infty,+\infty]$ be a closed, concave, and non-decreasing (partially ordered by $K$) function. Suppose that $u(\theta)=0$, where $\theta$ is the zero element of $E$. We denote the set of such $u$ by $U$.
	}
\end{Definition}

\begin{Remark}\label{R41}\textnormal
	{
		For any $u\in U$ and $f\in L^{p(\cdot)}$, we denote by $u(f):\Omega\rightarrow\mathbb{R}$ and $\mathbb{E}u(f)$ the expectation of $u(f)$ with respect to a probability measure $\mu$.
	}
\end{Remark}

\begin{Definition}\label{D42}\textnormal
	{
		For any $u\in U$ and $f\in L^{p(\cdot)}$, the OCE of some uncertain outcome $f$ is defined by the map $S_{u}:L^{p(\cdot)}\rightarrow\mathbb{R}$,
		\begin{displaymath}
		S_{u}(f)=\sup_{\eta\in \mathbb{R}}\{\eta+\mathbb{E}u(f-\eta z)\}
		\end{displaymath}
		where the domain of $S_{u}$ is defined as dom$S_{u}=\{f\in L^{p(\cdot)}| S_{u}(f)>-\infty\}\neq \emptyset$ and $S_{u}$ is finite on dom$S_{u}$.
	}
\end{Definition}

\begin{Theorem}\label{T41}\textnormal
	{
		For any $u\in U$, the following properties hold for $S_{u}$:
		\begin{description}
			\item[(a)] For any $f\in L^{p(\cdot)}$ and $m\in \mathbb{R}$, $S_{u}(f+mz)=S_{u}(f)+m$;
			\item[(b)] For any $f_{1},f_{2}\in L^{p(\cdot)}$, $f_{1}\leq_{K} f_{2}$ implies that $S_{u}(f_{1})\leq S_{u}(f_{2})$;
			\item[(c)] For any $f_{1},f_{2}\in L^{p(\cdot)}$ and $\lambda\in (0,1)$, $S_{u}(\lambda f_{1}+(1-\lambda)f_{2})\geq \lambda S_{u}(f_{1})+(1-\lambda)S_{u}(f_{2})$.
		\end{description}
	}
\end{Theorem}

\noindent \textbf{Proof.}
\begin{description}
	\item[(a)] For any $f\in L^{p(\cdot)}$, $m\in \mathbb{R}$,
	\begin{eqnarray*}
		S_{u}(f+mz)&=&\sup_{\eta\in \mathbb{R}}\{\eta+\mathbb{E}u(f+mz-\eta z)\}\\
		&=&m+\sup_{\eta\in \mathbb{R}}\{\eta-m+\mathbb{E}u(f-(\eta-m)z)\}\\
		&=&m+S_{u}(f).
	\end{eqnarray*}
	\item[(b)] For any $f_{1},f_{2}\in L^{p(\cdot)}$ with $f_{1}\leq_{K} f_{2}$, we have $f_{1}-\eta z\leq_{K}f_{2}-\eta z$. As $u$ is non-decreasing, we have
	\begin{displaymath}
	S_{u}(f_{1})=\sup_{\eta\in \mathbb{R}}\{\eta+\mathbb{E}u(f_{1}-\eta z)\}\leq \sup_{\eta\in \mathbb{R}}\{\eta+\mathbb{E}u(f_{2}-\eta z)\}=S_{u}(f_{2}).
	\end{displaymath}
	\item[(c)] For any $f_{1},f_{2}\in L^{p(\cdot)}$ and $\lambda\in (0,1)$,
	\begin{displaymath}
	S_{u}(\lambda f_{1}+(1-\lambda)f_{2})=\sup_{\eta\in \mathbb{R}}\Big\{\eta+\mathbb{E}u\big(\lambda f_{1}+(1-\lambda)f_{2}-\eta z\big)\Big\}.
	\end{displaymath}
	We take $\eta=\lambda\eta_{1}+(1-\lambda)\eta_{2}$. Then,
	\begin{eqnarray*}
		&&S_{u}(\lambda f_{1}+(1-\lambda)f_{2})\\
		&=&\sup_{\eta_{1},\eta_{2}\in \mathbb{R}}\Big\{\lambda\eta_{1}+(1-\lambda)\eta_{2}+\mathbb{E}u\big(\lambda (f_{1}-\eta_{1}z)+(1-\lambda)(f_{2}-\eta_{2}z)\big)\Big\}\\
		&\geq&\sup_{\eta_{1},\eta_{2}\in \mathbb{R}}\Big\{\lambda\eta_{1}+(1-\lambda)\eta_{2}+\lambda \mathbb{E}u (f_{1}-\eta_{1}z)+(1-\lambda)\mathbb{E}u(f_{2}-\eta_{2}z)\Big\}\\
		&=&\sup_{\eta_{1},\eta_{2}\in \mathbb{R}}\Big\{\lambda\big(\eta_{1}+\mathbb{E}u (f_{1}-\eta_{1}z)\big)+(1-\lambda)\big(\eta_{2}+\mathbb{E}u (f_{2}-\eta_{2}z)\big)\Big\}\\
		&=&\lambda S_{u}(f_{1})+(1-\lambda)S_{u}(f_{2}).
	\end{eqnarray*}
\end{description}
This completes the proof of Theorem~\ref{T41}.\\

\begin{Theorem}\label{T42}\textnormal
	{
		The function $\varrho$, defined as $\varrho(f):=-S_{u}(f)$ for any $f\in L^{p(\cdot)}$, is a $p(\cdot)$-convex risk measure.
	}
\end{Theorem}

\noindent \textbf{Proof:}
The proof of Theorem~\ref{T42} is straightforward from Theorem~\ref{T41}.\\

\begin{Proposition}\label{P41}\textnormal
	{
		For any $u\in U$, $\alpha\in \mathbb{R}^{+}$, and $f\in L^{p(\cdot)}$, the OCE $S_{u}(f)$ is sub-homogeneous, i.e.
		\begin{description}
			\item[(a)] $S_{u}(\alpha f)\leq \alpha S_{u}(f), \qquad \forall\alpha>1$;
			\item[(b)] $S_{u}(\alpha f)\geq \alpha S_{u}(f), \qquad \forall0\leq\alpha\leq1$.
		\end{description}
	}
\end{Proposition}

\noindent \textbf{Proof.}
Denote $S(\alpha):=\frac{1}{\alpha}S_{u}(\alpha f)$. Then,
\begin{equation}\label{41}
S(\alpha)=\frac{1}{\alpha}S_{u}(\alpha f)=\sup_{\eta\in \mathbb{R}}\Big\{\eta+\mathbb{E}\frac{1}{\alpha}u\big(\alpha(f-\eta z)\big)\Big\}.
\end{equation}
Next, we show that $S(\alpha)$ is non-increasing in $\alpha>0$ for any $f\in L^{p(\cdot)}$. For $\alpha_{2}\geq\alpha_{1}\geq0$, we have
\begin{displaymath}
\frac{u(\alpha_{2}t)-u(\alpha_{1}t)}{\alpha_{2}-\alpha_{1}}\leq \frac{u(\alpha_{1}t)-u(\theta)}{\alpha_{1}-0},\qquad\textrm{for any $t\in E$}
\end{displaymath}
by the concavity of $u$. As $u(\theta)=0$, we have
\begin{displaymath}
\frac{1}{\alpha_{2}}u(\alpha_{2}t)\leq \frac{1}{\alpha_{1}}u(\alpha_{1}t).
\end{displaymath}
Then, from (\ref{41}), we have $S(\alpha_{1})\geq S(\alpha_{2})$, which clearly implies $\mathbf{(a)}$ and $\mathbf{(b)}$.\\

\begin{Proposition}\label{P42}\textnormal
	{
		(Second-order stochastic dominance)
		We denote $C_{u}(f):=u^{-1}\mathbb{E}u(f)$ for any $u\in U$ and $f\in L^{p(\cdot)}$. We also assume that the supremum in the definition of $S_{u}$ is attained. Then, for any $f_{1}, f_{2}\in L^{p(\cdot)}$,
		\begin{displaymath}
		S_{u}(f_{1})\geq S_{u}(f_{2})\qquad \textrm{if and only if}\qquad C_{u}(f_{1})\geq C_{u}(f_{2}).
		\end{displaymath}
	}
\end{Proposition}

\noindent \textbf{Proof.}
We first show the ``if'' part. If $C_{u}(f_{1})\geq C_{u}(f_{2})$, we have $\mathbb{E}{u}(f_{1})\geq \mathbb{E}{u}(f_{2})$ by the fact that $u$ is non-decreasing.
Then, from the definition of $S_{u}$, it follows that $S_{u}(f_{1})\geq S_{u}(f_{2})$. We now show the ``only if'' part. Let $\ell_{f_{1}},\ell_{f_{1}}$ be the points where the suprema of $S_{u}(f_{1})$ and
$S_{u}(f_{2})$ are attained, respectively. Then, for any $u\in U$,
\begin{displaymath}
S_{u}(f_{1})=\ell_{f_{1}}+\mathbb{E}u(f_{1}-\ell_{f_{1}} z)\geq \ell_{f_{2}}+\mathbb{E}u(f_{2}-\ell_{f_{2}} z)\geq \ell_{f_{1}}+\mathbb{E}u(f_{2}-\ell_{f_{1}} z),
\end{displaymath}
where the first inequality comes from $S_{u}(f_{1})\geq S_{u}(f_{2})$. Therefore, for any $u\in U$,\\ $\mathbb{E}u(f_{1}-\ell_{f_{1}} z)\geq \mathbb{E}u(f_{2}-\ell_{f_{1}} z)$, which implies $\mathbb{E}u(f_{1})\geq \mathbb{E}u(f_{2})$. Then, $C_{u}(f_{1})\geq C_{u}(f_{2})$.\\

\section{Dynamic risk measures on $\mathbf{L^{p(\cdot)}}$}

A person's risk assessments may change over time.
This observation motivated us to study the dynamic $p(\cdot)$-convex risk measures on the variable exponent Bochner--Lebesgue spaces.

In fact, in dynamic cases, the risk measures can be regarded as both the minimum capital requirement of some real number and the hedging of some financial positions denoted by bounded random variables. \\

Conditional $p(\cdot)$-convex risk measures are now introduced using an axiomatic approach.

\begin{Definition}\label{D51}\textnormal
	{
		A map $\varrho_{t}:L^{p(\cdot)}$ $\rightarrow$ $ L_{t}^{\infty}$ is called a conditional $p(\cdot)$-convex risk measure if it satisfies the following properties for all $f, f_{1}, f_{2}\in L^{p(\cdot)}$:
		\begin{description}
			\item[i.] Monotonicity: $f_{1}\leq_{K}f_{2} \textrm{ implies } \varrho_{t}(f_{1})\geq \varrho_{t}(f_{2})$;
			\item[ii.] Conditional cash invariance: for any $m_{t}\in L_{t}^{\infty}$, $\varrho_{t}(f+m_{t}z)=\varrho_{t}(f)-m_{t}$;
			\item[iii.] Conditional convexity: for any $\lambda\in L_{t}^{\infty}$ with $\lambda\in[0,1]$, $\varrho_{t}(\lambda f_{1}+(1-\lambda)f_{2})\leq \lambda\varrho_{t}(f_{1})+(1-\lambda)\varrho_{t}(f_{2})$;
			\item[iv.] Normalization: $\varrho_{t}(\theta)=0$, $\varrho_{t}(f)<\infty$.
		\end{description}
		Additionally, a conditional $p(\cdot)$-convex risk measure is coherent if it satisfies the following property:
		\begin{description}
			\item[v.] Conditional positive homogeneity: for any $\lambda\in L_{t}^{\infty}$ with $\lambda>0$, $\varrho_{t}(\lambda f) =\lambda\varrho_{t}(f)$.
		\end{description}
	}
\end{Definition}

\begin{Remark}\label{R51}\textnormal
	{
		Note that any element in $L_{t}^{\infty}:=L^{\infty}(\Omega,\mathcal{F}_{t},\mu)$ is a random variable, where $\mathcal{F}_{t}$ is a sub-$\sigma$-algebra of $\mathcal{F}$. As stated by Detlefsen and Scandolo (2005), if the additional information is described by a sub-$\sigma$-algebra $\mathcal{F}_{t}$ of the total information $\mathcal{F}_{T}$, then a conditional risk measure is a map assigning an $\mathcal{F}_{t}$-measurable random variable $\varrho_{t}(f)$, representing the conditional riskiness of $f$, to every $\mathcal{F}_{T}$-measurable function $f$, representing a final payoff.
	}
\end{Remark}

The acceptance set of a conditional $p(\cdot)$-convex risk measure $\varrho_{t}$ is defined as
\begin{equation}\label{51}
\mathcal{A}_{t}:=\big\{f\in L^{p(\cdot)}: \varrho_{t}(f)\leq0\big\} \textrm{ for any } 0\leq t\leq T.
\end{equation}
The corresponding stepped acceptance set is defined as
\begin{equation}\label{52}
\mathcal{A}_{t,t+s}:=\big\{f\in L^{p(\cdot)}(\mathcal{F}_{t+s}): \varrho_{t}(f)\leq0\big\} \textrm{ for any } 0\leq t<t+s\leq T.
\end{equation}

\begin{Proposition}\label{P51}\textnormal
	{
		The acceptance set $\mathcal{A}_{t}$ of a conditional $p(\cdot)$-convex risk measure $\varrho_{t}$ has the following properties:
		\begin{description}
			\item[1.] Conditional convexity: for any $f_{1}, f_{2}\in \mathcal{A}_{t}$, and an $\mathcal{F}_{t}$-measurable function $\alpha$ with $0\leq\alpha\leq1$, we have $\alpha f_{1}+(1-\alpha)f_{2}\in \mathcal{A}_{t}$;
			\item[2.] Solidity: for any $f_{1}\in \mathcal{A}_{t}$ with $f_{1}\leq_{K}f_{2}$, we have $f_{2}\in \mathcal{A}_{t}$;
			\item[3.] Normalization: $0\in \mathcal{A}_{t}$.
		\end{description}
	}
\end{Proposition}
\noindent \textbf{Proof.}
It is easy to check properties 1--3 using Definition~\ref{D51}.\\

\begin{Definition}\label{D52}\textnormal
	{
		A sequence $(\varrho_{t})_{t=0}^{T}$ is called a dynamic $p(\cdot)$-convex risk measure if each $\varrho_{t}$ is a conditional $p(\cdot)$-convex risk measure for any $0\leq t\leq T$.
	}
\end{Definition}

We now study the dual representation of a conditional $p(\cdot)$-convex risk measure. First, the notion of the $\mathcal{F}_{t}$-conditional inner product related to $L^{p(\cdot)}$ should be defined.

\begin{Definition}\label{D53}\textnormal
	{
		For any $f\in L^{p(\cdot)}$ and $g\in (L^{p(\cdot)})^{\ast}$, we define the $\mathcal{F}_{t}$-conditional inner product $\langle g,-f\rangle_{t}$ by
		\begin{equation}\label{53}
		\int_{A}\langle g,-f\rangle_{t} d\mu=\langle g,-f\rangle \quad\textrm{for any}\quad A\subseteq \mathcal{F}_{t}.
		\end{equation}
		We also define the  minimal penalty function $\alpha_{t}^{\min}$  as
		\begin{equation}\label{54}
		\alpha_{t}^{\min}(g):=\textrm{ess}\sup_{f\in\mathcal{A}_{t}}\langle g,-f\rangle_{t}.
		\end{equation}
	}
\end{Definition}

\begin{Lemma}\label{L51}\textnormal
	{
		For any $g\in Q_{p(\cdot)}$, $0\leq t\leq T$, and $A\subseteq \mathcal{F}_{t}$,
		\begin{equation}\label{55}
		\int_{A}\alpha_{t}^{\min}(g) d\mu=\sup_{f\in\mathcal{A}_{t}}\langle g,-f\rangle.
		\end{equation}
	}
\end{Lemma}
\noindent \textbf{Proof.}
We first show that there exists a sequence $(f_{n})_{n\in \mathbb{N}}$ in $\mathcal{A}_{t}$ such that
\begin{equation}\label{56}
\textrm{ess}\sup_{f\in\mathcal{A}_{t}}\langle g,-f\rangle_{t}=\lim_{n\rightarrow\infty}\langle g,-f_{n}\rangle_{t}.
\end{equation}
Indeed, for any $f_{1}, f_{2}\in \mathcal{A}_{t}$, we define $\widehat{f}:=f_{1}I_{B}+f_{2}I_{B^{c}}$ where $B:=\{\langle g,-f_{1}\rangle_{t}\geq\langle g,-f_{2}\rangle_{t}\}$. By property 1 of Proposition~\ref{P51}, we know that $\widehat{f}\in\mathcal{A}_{t}$. Hence, by the definition of $\widehat{f}$,
\begin{displaymath}
\langle g,-\widehat{f}\rangle_{t}=\max\{\langle g,-f_{1}\rangle_{t}, \langle g,-f_{2}\rangle_{t}\}.
\end{displaymath}
Thus, (\ref{56}) holds. We now have
\begin{eqnarray*}
	\int_{A}\alpha_{t}^{\min}(g) d\mu&=&\int_{A}\textrm{ess}\sup_{f\in\mathcal{A}_{t}}\langle g,-f\rangle_{t} d\mu\\
	&=&\int_{A}\lim_{n\rightarrow\infty}\langle g,-f_{n}\rangle_{t} d\mu\\&=&\lim_{n\rightarrow\infty}\int_{A}\langle g,-f_{n}\rangle_{t} d\mu\\&=&\lim_{n\rightarrow\infty}\langle g,-f_{n}\rangle\\&\leq&\sup_{f\in\mathcal{A}_{t}}\langle g,-f\rangle.
\end{eqnarray*}
The converse inequality is easy to check.\\

The following theorem gives the dual representation of conditional $p(\cdot)$-convex risk measures.

\begin{Theorem}\label{T51}\textnormal
	{
		Suppose $\varrho_{t}$ is a conditional $p(\cdot)$-convex risk measure. Then, the following statements are equivalent.
		\begin{description}
			\item[(1)] $\varrho_{t}$ has the robust representation
			\begin{equation}\label{57}
			\varrho_{t}(f)=\textrm{ess}\sup_{g\in Q_{p(\cdot)}}\big\{\langle g,-f\rangle_{t}-\alpha_{t}(g)\big\}\quad \textrm{for any}\quad f\in L^{p(\cdot)},
			\end{equation}
			where
			\begin{displaymath}
			Q_{p(\cdot)}:=\Big\{g\in \big(L^{p(\cdot)}\big)^{\ast}:\langle \frac{dg}{d\mu}, z\rangle=1, \frac{dg}{d\mu}\in L^{p'(\cdot)}(K_{0})\Big\},
			\end{displaymath}
			and $\alpha_{t}$ is a map from $Q_{p(\cdot)}$ to the set of $\mathcal{F}_{t}$-measurable random variables such that $\textrm{ess}\sup_{g\in Q_{p(\cdot)}}\{-\alpha_{t}(g)\}=0$;
			\item[(2)] $\varrho_{t}$ has a robust representation in terms of the minimal function, i.e.
			\begin{equation}\label{58}
			\varrho_{t}(f)=\textrm{ess}\sup_{g\in Q_{p(\cdot)}}\big\{\langle g,-f\rangle_{t}-\alpha_{t}^{\min}(g)\big\}\quad \textrm{for any}\quad f\in L^{p(\cdot)};
			\end{equation}
			\item[(3)] $\varrho_{t}$ is continuous from above under $K$, i.e.
			\begin{equation}\label{59}
			f_{n}\searrow f\Rightarrow \varrho_{t}(f_{n})\nearrow \varrho_{t}(f).
			\end{equation}
		\end{description}
	}
\end{Theorem}
\noindent \textbf{Proof.}
$(2)\Rightarrow (1)$ is obvious. We first prove $(1)\Rightarrow(3)$. Using Lemma 5 of Cheng and Xu (2013), suppose that $f_{n}\searrow f$. Then, by the monotonicity of $\varrho_{t}$, we have $\varrho_{t}(f_{n})\nearrow \varrho_{t}(f)$.\\
Next, we show $(3)\Rightarrow (2)$. The inequality
\begin{displaymath}
\varrho_{t}(f)\geq\textrm{ess}\sup_{g\in Q_{p(\cdot)}}\big\{\langle g,-f\rangle_{t}-\alpha_{t}^{min}(g)\big\}
\end{displaymath}
is a direct consequence of the definition of $\alpha_{t}^{\min}$. Now, we need only show the inverse inequality. To this end, we define a map $\widetilde{\varrho}: L^{p(\cdot)}\rightarrow \mathbb{R}$ as $\widetilde{\varrho}(f)=\int_{A}\varrho_{t}(f)d\mu$. It is easy to check that $\widetilde{\varrho}$ is a $p(\cdot)$-convex risk measure as defined in Section 3 which is continuous from above. Hence, by Theorem~\ref{T32}, we know that $\widetilde{\varrho}$ has the dual representation
\begin{displaymath}
\widetilde{\varrho}(f)=\sup_{g\in Q_{p(\cdot)}}\big\{\langle g,-f\rangle-\alpha(g)\big\},\quad f\in L^{p(\cdot)},
\end{displaymath}
where the minimum penalty function $\alpha_{\min}$ is given by $\alpha_{\min}(g):=\sup_{f\in \mathcal{A}_{\widetilde{\varrho}}}\big\{\langle g,-f\rangle\big\}$. By Lemma~\ref{L51}, we have
\begin{displaymath}
\int_{A}\alpha_{t}^{\min}(g) d\mu=\sup_{f\in\mathcal{A}_{t}}\langle g,-f\rangle
\end{displaymath}
for any $g\in Q_{p(\cdot)}$. As $\widetilde{\varrho}(f)\leq0$ for all $f\in\mathcal{A}_{t}$,
\begin{displaymath}
\int_{A}\alpha_{t}^{\min}(g) d\mu=\sup_{f\in\mathcal{A}_{t}}\langle g,-f\rangle\leq\alpha(g)
\end{displaymath}
for any $g\in Q_{p(\cdot)}$. Thus, we have
\begin{eqnarray*}
	\int_{A} \varrho_{t}(f) d\mu&=&\widetilde{\varrho}(f)\\
	&=&\sup_{g\in Q_{p(\cdot)}}\big\{\langle g,-f\rangle-\alpha(g)\big\}\\&\leq&\sup_{g\in Q_{p(\cdot)}}\Big\{\int_{A}\langle g,-f\rangle_{t} d\mu-\int_{A}\alpha_{t}^{\min}(g)d\mu\Big\}\\&=&\sup_{g\in Q_{p(\cdot)}}\Big\{\int_{A}\big(\langle g,-f\rangle_{t} -\alpha_{t}^{\min}(g)\big)d\mu\Big\}\\&\leq&\int_{A}\textrm{ess}\sup_{g\in Q_{p(\cdot)}}\big\{\langle g,-f\rangle_{t}-\alpha_{t}^{\min}(g)\big\}d\mu.
\end{eqnarray*}
Thus, (\ref{58}) holds.\\

Now, with the definition and dual representation, we consider  the time consistency of dynamic $p(\cdot)$-convex risk measures.

\begin{Definition}\label{D54}\textnormal
	{
		A dynamic $p(\cdot)$-convex risk measure $(\varrho_{t})_{t=0}^{T}$ is said to be time consistent if, for all $f_{1}, f_{2}\in L^{p(\cdot)}$ and $0\leq t<t+s\leq T$,
		\begin{equation}\label{510}
		\varrho_{t+s}(f_{1})\leq\varrho_{t+s}(f_{2})\Rightarrow \varrho_{t}(f_{1})\leq\varrho_{t}(f_{2}).
		\end{equation}
	}
\end{Definition}

\begin{Remark}\label{R53}\textnormal
	{
		Time consistency means that if two payoffs will have the same riskiness tomorrow in every state of nature, then the same conclusion should be drawn today.
	}
\end{Remark}

\begin{Theorem}\label{T52}\textnormal
	{
		Let $(\varrho_{t})_{t=0}^{T}$ be a dynamic $p(\cdot)$-convex risk measure such that each $\varrho_{t}$ is continuous from above. Then, the following conditions are equivalent for any $0\leq t<t+s\leq T$:
		\begin{description}
			\item[1).] $(\varrho_{t})_{t=0}^{T}$ is time consistent;
			\item[2).] $\mathcal{A}_{t}=\mathcal{A}_{t,t+s}+\mathcal{A}_{t+s}$;
			\item[3).] $\varrho_{t}\big(-\varrho_{t+s}(f)z\big)=\varrho_{t}(f)$ for any $f\in L^{p(\cdot)}$.
		\end{description}
	}
\end{Theorem}
\noindent \textbf{Proof.}
We first show the equivalence between 1) and 3). Suppose that 3) holds and $\varrho_{t+s}(f_{1})\leq\varrho_{t+s}(f_{2})$ for any $f_{1}, f_{2}\in L^{p(\cdot)}$. Then, by the monotonicity of $\varrho_{t}$,
\begin{displaymath}
\varrho_{t}(f_{1})=\varrho_{t}\big(-\varrho_{t+s}(f_{1})z\big)\leq \varrho_{t}\big(-\varrho_{t+s}(f_{1})z\big)=\varrho_{t}(f_{2}).
\end{displaymath}
Next, suppose that $(\varrho_{t})_{t=0}^{T}$ is time consistent, and set $f_{2}:=-\varrho_{t+s}(f_{1})z=-\varrho_{t+s}(f_{2})z$ for any $f_{1}\in L^{p(\cdot)}$. Thus,
\begin{displaymath}
\varrho_{t}(f_{1})=\varrho_{t}(f_{2})=\varrho_{t}\big(-\varrho_{t+s}(f_{1})z\big)=\varrho_{t}\big(-\varrho_{t+s}(f_{2})z\big).
\end{displaymath}
We now show the equivalence between 2) and 3). To this end, suppose that 3) holds and let $f_{1}\in \mathcal{A}_{t,t+s}$, $f_{2}\in \mathcal{A}_{t+s}$. Then, setting $f:=f_{1}+f_{2}$, we have
\begin{displaymath}
\varrho_{t+s}(f)=\varrho_{t+s}(f_{1}+f_{2})=\varrho_{t+s}(f_{2})-\frac{f_{1}}{z}\leq -\frac{f_{1}}{z} .
\end{displaymath}
Thus, by the monotonicity of $ \varrho_{t}$, we know that
\begin{displaymath}
\varrho_{t}(f)=\varrho_{t}\big(-\varrho_{t+s}(f)z\big)\leq \varrho_{t}(f_{1})\leq0,
\end{displaymath}
which implies
\begin{displaymath}
\mathcal{A}_{t}\supseteq\mathcal{A}_{t,t+s}+\mathcal{A}_{t+s}.
\end{displaymath}
For the inverse relation, let $f\in \mathcal{A}_{t}$ and define $f_{2}:=f+\varrho_{t+s}(f)z$, $f_{1}:=f-f_{2}=-\varrho_{t+s}(f)z$. Then, by the conditional cash invariance of $\varrho_{t}$, it is easy to check that $f_{1}\in \mathcal{A}_{t,t+s}$, $f_{2}\in \mathcal{A}_{t+s}$, which implies \begin{displaymath}
\mathcal{A}_{t}\subseteq\mathcal{A}_{t,t+s}+\mathcal{A}_{t+s}.
\end{displaymath}
Let us now suppose that 2) holds and $f\in \mathcal{A}_{t}$. It is easy to check that $f+\varrho_{t+s}(f)z\in \mathcal{A}_{t+s}$. Then, with $\mathcal{A}_{t}\subseteq\mathcal{A}_{t,t+s}+\mathcal{A}_{t+s}$, we have $-\varrho_{t+s}(f)z\in \mathcal{A}_{t,t+s}$. Hence, we know that  $\varrho_{t}\big(-\varrho_{t+s}(f)z\big)\leq0$, which implies
\begin{displaymath}
\varrho_{t}\big(-\varrho_{t+s}(f)z\big)\leq\varrho_{t}(f).
\end{displaymath}
Now, we need only show the inverse inequality. Indeed, for any $f\in L^{p(\cdot)}$ such that $-\varrho_{t+s}(f)z\in \mathcal{A}_{t,t+s}$, we have $\varrho_{t}\big(-\varrho_{t+s}(f)z\big)\leq0$. It is easy to check that $f+\varrho_{t+s}(f)z\in \mathcal{A}_{t+s}$. Thus, by $\mathcal{A}_{t}\supseteq\mathcal{A}_{t,t+s}+\mathcal{A}_{t+s}$, we have $f\in \mathcal{A}_{t}$, which implies
\begin{displaymath}
\varrho_{t}\big(-\varrho_{t+s}(f)z\big)\geq\varrho_{t}(f).
\end{displaymath}
\\

The case for the recursive property strongly relies on the validity of conditional cash invariance for $\varrho_{t}$, and hence on the interpretation as conditional capital requirements. In fact, if $\varrho_{t+s}(f)$ is the conditional capital requirement that has to be set
aside at date $t+s$ in view of the final payoff $f$, then the risky position is equivalently described, at date $t$, by the payoff $\varrho_{t}\big(-\varrho_{t+s}(f)z\big)$ occurring in $t+s$.\\

We end this section with  a special example of  conditional $p(\cdot)$-convex risk measures.

\begin{Example}\label{E51}\textnormal
	{
		(Conditional OCE) Let $u:E\rightarrow \mathbb{R}$ be a closed, concave, and non-decreasing (partially ordered by $K$) function and suppose that $u(\theta)=0$, where $\theta$ is the zero element of $E$.
		Then, for any $f\in L^{p(\cdot)}$, the conditional OCE of some uncertain outcome $f$ is defined by the map $S_{u}:L^{p(\cdot)}\rightarrow L_{t}^{\infty}$:
		\begin{displaymath}
		S_{u}(f)=\textrm{ess}\sup_{\eta\in L_{t}^{\infty}}\Big\{\eta+\mathbb{E}\big[\big(u(f-\eta z)\big)|\mathcal{F}_{t}\big]\Big\}.
		\end{displaymath}
		Thus, by Definition~\ref{D51}, it is easy to check that the function $\varrho_{t}$ defined as $\varrho_{t}(f):=-S_{u}(f)$ for any $f\in L^{p(\cdot)}$ is a conditional $p(\cdot)$-convex risk measure.
	}
\end{Example}

config: arxiv


TITLE: Estimate a complex modulus
QUESTION [0 upvotes]: I have to estimate the following quantity
$$\vert e^{iz\vert x\vert}-e^{i\lambda\vert x\vert}\vert^2$$
where $x\in\mathbb{R}^3$, $\Im z>0$ and $\lambda>0$.
So I write
$$\vert e^{i\Re z\vert x\vert}e^{-\Im z\vert x\vert}-\cos(\lambda\vert x\vert)-i\sin(\lambda\vert x\vert)\vert^2$$
Putting $A=\Re z$ and $B=\Im z$ I have
$$\vert (e^{-B\vert x\vert}\cos(A\vert x\vert)-\cos(\lambda\vert x\vert))+i(e^{-B\vert x\vert}\sin(A\vert x\vert)-\sin(\lambda\vert x\vert))\vert^2=$$
$$=(e^{-B\vert x\vert}\cos(A\vert x\vert)-\cos(\lambda\vert x\vert))^2+(e^{-B\vert x\vert}\sin(A\vert x\vert)-\sin(\lambda\vert x\vert))^2$$
Making calculations I obtain
$$\vert e^{iz\vert x\vert}-e^{i\lambda\vert x\vert}\vert^2\leq 4$$
Is something wrong?

REPLY [0 votes]: For $\;\;z=a+ib\;,\;\;x\in\mathbb{R}^3\;,\;\;b:=\operatorname{Im} z>0\;,\;\;\lambda>0\,$ :
$$\left|\, e^{iz|x|}-e^{i\lambda| x|}\,\right|=|\,e^{i|x|a-b|x|}-e^{i\lambda |x|}\,|\le e^{-b|x|}+1\le 2$$
Note I dropped the square...and thus you're right!


text:
\setcounter{figure}{0}
\renewcommand{\thefigure}{E\arabic{figure}}

\section{Analyzing the similarities using the left basis} \label{sec:appx_similarities}

Here, we take a more unsupervised approach to analyze the similarities of images and to identify isolated images present in a dataset.

Using HO-GSVD, we explained how the vectors in $V$ represent the orthogonal patterns extracted from the entire dataset. We also know that each image in the dataset can be reconstructed by a specific combination of patterns in $V$, defined by the singular values and the left basis vectors. In fact, $V$ is common among all the classes in the dataset $V$ and what distinguishes each image from other images, is its corresponding vector in $U_i \Sigma_i$. The number of rows in $U \Sigma$ correspond to the number of images in the class $i$. Let's consider the $j^{th}$ image in class $i$. We can reconstruct this image using the coefficients in the $j^{th}$ row of $U_i \Sigma_i$ to combine the vectors of $V$.\footnote{Clearly, all of these computations will be in  the wavelet space, and at the end we have to reconstruct them back in the pixel space.} Mathematically, if we have two identical images in the dataset, we will get identical coefficients for them.

Overall, we can say that the HO-GSVD extracts a set of common patterns in the entire dataset, and gives us a set of coefficients in $U_i \Sigma_i$, determining how these patterns should be combined in order to obtain the images. To analyze the similarity of images, we can analyze the similarity of their coefficients, because patterns are common for all of them. In fact, patterns can be considered images in the pixel space, but coefficients are just a vector of real numbers. This way, analyzing the similarity of two images is reduced to basically comparing two vectors.

Figure~\ref{fig:cifar_cat_similarity} shows the similarity matrix of all images in the Cat class. To derive this similarity matrix, we followed these steps:

\begin{enumerate}
    \item Compute the matrix $U_i \Sigma_i$ which has 5,000 rows and 3,000 columns for the Cat class.
    \item Compute the 2-norm distance between each row in this matrix to obtain a distance matrix of 5,000 rows and columns.
    \item Convolve the distance matrix with a Gaussian kernel to obtain the similarity matrix.
\end{enumerate}

\begin{figure}[H]
  \centering
   \includegraphics[width=0.4\linewidth]{fig_cat_similarity.jpg}
  \caption{Similarity matrix of images in the Cat class based on the coefficients in $U \Sigma$.}
  \label{fig:cifar_cat_similarity}
\end{figure}

Once we obtain a similarity matrix, a wide array of analysis can be performed. For example, one can perform clustering to identify communities of similar images. By analyzing the eigenvalues of a graph Laplacian built from the similarity matrix, we identified that there are not any large communities inside this dataset, as previously reported by \cite{yousefzadeh2020using}. However, coarse graining (i.e., organizing the data in large number of clusters) can easily identify small groups of images (usually 2 to 5) that are very similar, and sometimes nearly identical.

Figure~\ref{fig:cifar_dog_anomalies} shows dog images that are most isolated within the dog class. Clearly, we may consider them anomalies or out-of distribution images in the context of this dataset, because picture of a dog with a red bucket on its head, or picture of a dog with red flowers around the perimeter of image are not common. This relates to the broad literature on detecting out-of-distribution images and anomalies in image classification datasets, e.g., \cite{liang2018enhancing,huang2019out,ren2019likelihood}. This is another possible extension of our method.

\begin{figure}[H]
  \centering
   \includegraphics[width=0.1\linewidth]{cifar_dog_anomaly_1.png}
   \includegraphics[width=0.1\linewidth]{cifar_dog_anomaly_2.png}
   \includegraphics[width=0.1\linewidth]{cifar_dog_anomaly_3.png}
   \includegraphics[width=0.1\linewidth]{cifar_dog_anomaly_4.png}
   \includegraphics[width=0.1\linewidth]{cifar_dog_anomaly_5.png}
   \includegraphics[width=0.1\linewidth]{cifar_dog_anomaly_6.png}
   \includegraphics[width=0.1\linewidth]{cifar_dog_anomaly_9.png}
  \caption{Dog images that are most isolated within the dog class, identified by the similarity matrix. We can consider these out-of-distribution images in the context of CIFAR-10 training set.}
  \label{fig:cifar_dog_anomalies}
\end{figure}

We can also use the coefficients in $U_i \Sigma_i$ to visualize the dataset in form of an embedding. Figure~\ref{fig:cifar_cat_tsne} shows the embedding obtained by the t-SNE (t-Distributed Stochastic Neighbor Embedding) algorithm \citep{maaten2008visualizing}. t-SNE is a dimensionality reduction algorithm, specifically designed for visualizing high dimensional data.

This approach of visualizing a set of images in form of an embedding relates to a separate line of research in the literature. Those methods usually suffer from very high computational costs, because they rely on pairwise comparison of images in the pixel space, e.g., \cite{vo2019unsupervised}. We also note that t-SNE is used before for visualizing image datasets, for example by \cite{parde2017face}, but such approaches use a pre-trained VGG model for comparison of images. Our approach does not require a pre-trained model, however, we note that its usefulness requires further experiments and comparison with other methods.

\begin{figure}[H]
  \centering
  \includegraphics[width=0.9\linewidth]{fig_cat_tsne.jpg}
  \caption{t-SNE plot obtained from the $U_i \Sigma_i$ matrix of cat images.}
  \label{fig:cifar_cat_tsne}
\end{figure}


TITLE: Trying to show, in $\mathbb{R}^2$, that a linear extension $g$ of a linear functional $f$ is in fact linear?
QUESTION [0 upvotes]: How do we obtain a linear extension of a linear functional $f$ where $f$ is defined on some subspace $Z$ of $X$? I always see something like the following phrase, 'let $E$ be the set of all linear extensions $g$ of $f$...but how do we construct such a $g$?
Here is what I tried - To make things simple, consider $X = \mathbb{R}^2 = \{(x_1,x_2)\ | x_1,x_2 \in \mathbb{R}\}$ and $Z = \{(x_1,0)\}$. Suppose $f$ is a linear functional on $Z$. Define $g$ as a extension of $f$ such that $g:X\to \mathbb{R}$ and $g|_Z = f$.
Now I want to show $g$ is in fact linear. Let $a = (a_1,0) + (0,a_2)$ and $b = (b_1,0) + (0,b_2)$ for $a,b\in \mathbb{R}^2$. Then we have
\begin{align}
a + b & = (a_1,0) + (0,a_2) + (b_1,0) + (0,b_2) \\
& = (a_1+b_1,0) + (0,a_2+b_2).
\end{align}
Now I want to show linearity of the extension $g$ so I apply it to $a+b$:
\begin{align}
g(a+b) & = g((a_1+b_1,0) + (0,a_2+b_2)) \\
& = \dots
\end{align}
Here is where I am stuck. If I could say that $f = 0$ on $X\setminus Z$, then I could replace $g$ with $f$ and using the linearity of $f$ I could show $g$ is linear. But all I know is that $f$ is defined on $Z$ so it seems I can't do this.
So how do I should $g$ is linear?

REPLY [1 votes]: g is not zero on the complement! that is a big mistake a lot of people make! In fact, your definition actually involves PICKING a basis of your subspace and then extending to all of $V$, if $V$ is your vector space! After that you define $g(b_i):= f(b_i)$ if $b_i$ basivector of $Z$ and zero else. Hence $g$ is not $0$ on $V\setminus Z$ but only on the span of the additional basis vectors.
This is one of the many cases where people forget where and how important choices can be!
if you in fact want to set $g$ zero on the set theoretic complement for a non zero $f$, it will not be linear! let $v \in V\setminus Z $ and $w \in  V\setminus Z$ s.t. $0 \neq v+w \in Z$  and $f(v+w)\neq 0$ (those exist by taking an element $v\in V \setminus Z$ and setting $w=v-z$ for $0\neq z \in Z\setminus \ker(f)$)  then $$g(v-w)=g(z)=f(z) \neq 0=0+0  =g(v) + g(w) $$ (assuming that $f$ is not the zero map)


TITLE: How do I measure the spin along an arbitrary direction?
QUESTION [5 upvotes]: I need to measure the atoms spin along a direction perpendicular to the z-axis but at an angle
$$
\phi
$$
with the x-axis.
They then get the output of the spin measured is + with probability
$$
\cos^2(\phi/2)
$$
Is there a general formula I can use to compute the + spin state and then form that determine this probability?

REPLY [6 votes]: Take a spin $1/2$ particle with its spin pointing along $\hat{n}$ defined by
$$\hat{n}=(\sin{\phi}\cos{\theta},\sin{\phi}\sin{\theta},\cos{\phi})$$
We are measuring the spin along $\hat{n}$ and the operator corresponding to this observable is $\vec{S}\cdot\hat{n}$.
$$\vec{S}\cdot\hat{n}=\frac{\hbar}{2}\begin{pmatrix} \cos{\phi} & \sin{\phi}e^{-i\theta} \\ \sin{\phi}e^{i\theta} & -\cos{\phi} \end{pmatrix}$$
Since in the mean time this was label as homework, I will not provide a full answer. From this point, all you have to do is find the eigenvalues and eigenvectors of $\vec{S}\cdot\hat{n}$. After you find the eigenvectors, you should be able to arrive quickly at the desired probability of $\cos^{2}(\phi/2)$.


TITLE: A confusion about an exercise about an affine subspace that contains a hyperplane
QUESTION [1 upvotes]: I am solving this problem from Brezis's book of Functional Analysis.

Let $E$ be a normed vector spacr and $H$ a hyperplane, i.e., $H=\{x\in E: f(x)= \alpha\}$ for some linear functional $f$ and some real number $\alpha$. Let $V$ be an affine subspace (i.e., $V=U+a$ where $a \in E$ and $U$ is a vector subspace of $E$) that contains $H$.  Prove that either $V=H$ or $V=E$.

I found a proof here but could not understand it. On the other hand, It seems I have found a counter example.

Let $E$ be infinite dimensional, $f$ injective continuous, and  $\alpha=0$. This means $H = \{0\}$. Fix $0 \neq y \in E$ and let $V := \operatorname{span} \{0, y\}$. Then $H \subsetneq V \subsetneq E$.

Could you confirm if I understand the exercise properly?

REPLY [0 votes]: First, $V = H$ is equivalent to $U = H':= \{x \in E \mid f(x) = \alpha- f(a)\}$. Second, $V = E$ is equivalent to $U = E$. Third, $V$ contains $H$ is equivalent to $U$ contains $H'$. Then our problem reduces to

Let $E$ be a normed vector space, $f$ a linear functional, and $\alpha \in \mathbb R$. Let $V$ be a vector subspace that contains $H:=\{x \in E \mid f(x) = \alpha\}$. Prove that either $V = H$ or $V=E$.

Assume the contrary that $H \subsetneq V \subsetneq E$. Then there are $x_0 \in V \setminus H$ and $x_1 \in E \setminus V$. If $f(x_1) =0$ for all $x_1 \in E \setminus V$, then $H + E \setminus V \subseteq H \subsetneq V$ which is a contradiction. WLOG, we assume $f(x_1) \neq 0$. Notice that $f (x_0) \neq \alpha$. We find $\lambda \neq 0$ such that $f (x_0 + \lambda  x_1) = \alpha$. This is possible with $\lambda = (\alpha-f(x_0)) / f(x_1)$. It follows that $x_0 + \lambda  x_1 \in H \subsetneq V$. On the other hand, $x_0 \in V$. This implies $x_1 \in V$ which is a contradiction. This completes the proof.


TITLE: Are there some functions that cannot be optimized using calculus?
QUESTION [4 upvotes]: I've been working on a project to maximize a functions output using a genetic algorithm. However, from the limited calculus I know I thought there were methods to find the maximum of a mathematical function using calculus? I'd assume the reason genetic algorithms are sometimes used to maximize functions is because there are functions where the mathematical methods don't work. If I'm correct, what are those conditions? I'd guess that maybe it's because the function is not continuous or differentiable?

REPLY [1 votes]: Calculus methods are useful for functions which are differentiable. For functions which are not differentiable calculus won't help much. For instance the data set could be discrete. Some examples of this include:

A set of binary variables like a set of yes/no decisions. Which set of filters and parameters shall we use for this image to get best image quality?
Which order to visit cities will minimize cost? Or which vehicles to transport which goods to minimize time and/or required resources?
Which set of coins will minimize time to pay a beer in cash?


text:
\begin{definition}[Definition:Euclid's Definitions - Book XI/21 - Cylinder]
When, one side of those about the [[Definition:Right Angle|right angle]] in a [[Definition:Rectangle|rectangular parallelogram]] remaining fixed, the [[Definition:Rectangle|parallelogram]] is carried round and restored again to the same position from which it began to be moved, the figure so comprehended is a '''[[Definition:Cylinder|cylinder]]'''.
\end{definition}

config: wiki