TITLE: Prove that "f cross g" is a function from "A cross B" and "C cross D"
QUESTION [0 upvotes]: f is a function from A to B and g is a function from C to D. "f cross g" = {(a,b), (c,d) such that (a,b) $\in$ f and (c,d) $\in$ g}
pf. Let (a,c) $\in$ A x C and (b,d) $\in$ B x D. Then a $\in$ A and c $\in$ C. Then b $\in$ B and d $\in$ D.
I feel like I am missing one more step that will help me finish this proof, but I am not seeing it from a definition. I was wondering if I could get some help. Thanks.

REPLY [2 votes]: If $p \in A\times C $ then $p = (a, c), a\in A, c\in C $.
We define $(f \times g)(p)$ as $(f \times g)(p) = (f \times g)(a, c) = (f(a), g(c)) = (b, d) $ but $b \in B$ and $d \in D $ and $(b, d) \in B\times D $.
This shows that $f \times g$ is a function from $A \times C$ to $B \times D $.
$$(f \times g): A \times C \longrightarrow B \times D $$
$$(a, b) \longrightarrow (f(a), g(c)) $$


TITLE: Schwarz derivative and Möbius transformation.
QUESTION [1 upvotes]: In https://en.wikipedia.org/wiki/Schwarzian_derivative#Properties ,
some properties about Schwarz derivative
\begin{align}
(Sf)(z)&=
\left(\dfrac{f''(z)}{f'(z)}\right)'
-\dfrac{1}{2} \left( \dfrac{f''(z)}{f'(z)} \right)^2 \\
&=
\dfrac{f'''(z)}{f'(z)}-\dfrac{3}{2}\left(\dfrac{f''(z)}{f'(z)}\right)^2
\cdots \ast
\end{align}
are described.
And it is said that "The Schwarzian derivative of any Möbius transformation
$g(z)=\dfrac{az+b}{cz+d}$ is zero. Conversely,
the Möbius transformations are the only functions with this property."
I could understand the fact that the Schwarzian derivative of any Mobius transformation is zero.
But I cannot understand the fact that the Möbius trasformations are the only function with this property.
If $z \in \mathbb{R}$, I could understand this fact, owing to the $2$nd answer of this site, using "log". https://mathoverflow.net/questions/173182/schwarzian-derivative-of-a-diffeomorphism-is-zero-iff-linear-fractionals
However, "log" in $\mathbb{R}$ is not equal to "log" in $\mathbb{C}$.
So if $z \in \mathbb{C}$, I think I cannot use the approach using "log".
Summarizing the above, I wonder why Möbius transformations are the only solutions of $\ \ast$.
I want you give me some ideas.

REPLY [1 votes]: Per comment I will present a simple (though using a trick) proof of the fact that $S(f)=0$ implies $f$ Mobius. We work on a domain where $f'$ doesn't vanish so $S$ is analytic and we show that $f=g_1/g_2$ where $g_{1,2}''=0$ and that implies $f$ is Mobius on the given domain.
Consider a function $g$ for which $2f'g'+f''g=0$ on a small disc of center $w$ within the given domain - for example we can take $k(z)=\int_w^z \frac{f''(y)dy}{2f'(y)}$ which is analytic on a small disc centered at $w$ since $f'(z) \ne 0$ there; then take $g(z)=e^{-k(z)}$ and an easy check shows that $g'=-k'g=-f''g/(2f')$
But now differentiating the relation above we get:
$2f'g''+2f''g'+f'''g+f''g'=0$ and dividing by $f'$ and substituting $g'=-f''g/(2f')$ we get
$2g''+(f'''/f'-3(f'')^2/(2f'))g=0$ and obviously the $g$ coefficient is $S(f)=0$ so we indeed get $2g''=0$ hence $g$ is linear on the given small disc.
But now if $h=fg$ one has $h''=f''g+fg''+2f'g'=0$ since $g''=0$ and $2f'g'+f''g=0$, so $h$ is linear too and then $f=h/g$ is Mobius on the small disc, but by analytic continuation it is so on the given domain and of course we can extend it to the complex plane meromorphically with at most one pole!
Note that this proof shows how to solve $S(f)=p$ for any analytic $p$ as we get by the same trick and same computations that $f=h/g$ where now $2g''+pg=0$ and $2h''+ph=0$, so $f$ is a ratio of linearly independent solutions of a fairly simple differential equation (and conversely it is fairly straightforward to show that any such ratio gives an $f$ with $S(f)=p$)


TITLE: Obligated Bibliography for Quantum Gravity
QUESTION [2 upvotes]: I am a graduate student who wants to begin to get into Quantum Gravity. I've looked on internet for bibliography and I've found a lot of great books, but I would like to start from the beginning (of topics related to quantum gravity, the general relativity and quantum mechanics basis I already have). I'm not looking bibliography of any particular quantum gravity theory, but something more global, like an historical overview, the current state or, if I may ask for too much, any book describing how such theory should be. Also, I'm looking for formal bibliography, not for-all-ages books.
If anyone who's adept into this topic could help me, I would appreciate it so much.

REPLY [0 votes]: There's a text by Carlo Rovelli (worldly known for his contributions to Loop Quantum Gravity) telling a brief history of Quantum Gravity from its early stages (back in the 1930s) up to 2000. It is available at arXiv: gr-qc/0006061.
There's also the book Approaches to Quantum Gravity, edited by Daniele Oriti, which includes expositions of many of the main current approaches (including, but not restricted to, strings, loops, asymptotic safety, emergent relativity, discrete quantum gravity, and others). Might be a good start to get an overview of many different things.
R Percacci's An Introduction to Covariant Quantum Gravity and Asymptotic Safety focuses on Asymptotically Safe Quantum Gravity, but it also deals with many other interesting things, such as a historical overview, linearizations, the fact that quantized GR is not perturbatively renormalizable, and so on. Most of its calculations could be a bit more open, but it is a really nice presentation.


TITLE: Do we need to consider "direction" while discussing regions of integration?
QUESTION [2 upvotes]: Consider the following double integral from Thomas' Calculus
$$\int_0^3 \int_1^{4-2u} \frac{4-2u}{v^2} \ dv\ du;$$
sketch the region of integration.
If we represent $u$ on the horizontal axis and $v$ on the vertical axis and sketch the limits, we get

For $0 \leq u \leq \frac{3}{2}$, we have $1 \leq 4-2u$ (blue region); for $\frac{3}{2} \leq u \leq 3$, we have $1 \geq 4-2u$ (orange region).
Is this sketch correct?
Do we have to consider the direction when discussing the region of integration?
I think this is a higher dimensional parallel to the convention that
$$\int_a^b f\left( x \right) \ dx = -\int_b^a f\left( x \right) \ dx$$
Can someone point me to a reference with more details on this?
If you find the presentation of my question confusing, I am very sorry.

REPLY [1 votes]: Your question is very clear.
This problem from the book is not well-formulated and it causes confusion.
My suggestion to you is just forget about the problem.
If you really want to work on the problem, then I have to say that the region of integration is ``directed'' and you are right on this.
Sorry for using comment as an answer. I am new here and still can not make a comment.


TITLE: What is the probability of getting exactly two tails after tossing $x$ number of fair coins ($x\ge2$)?
QUESTION [0 upvotes]: What is the probability of getting exactly two tails after tossing $x\ge2$ number of fair coins?
I am not sure about my answer, but please allow me to share it here.
Let $A$ be the number of times we get a tail after randomly tossing an x number of different coins, where x is greater than or equal to 2. The probability of getting a tail after randomly tossing a single coin is $\frac{1}{2}$ (H / H, T). The probability of getting exactly two tails by tossing $x$ coins is
$P(A = 2) =  {x \choose 2} (\frac{1}{2})^x$, $x\ge2$
Any comments and suggestions will be much appreciated.
Thank you in advance.

REPLY [1 votes]: Your answer is correct.
A valid outcome corresponds to choosing which two of the $x$ coins you want to be tails, this can be done in $\binom x2$ ways.
The total number of possible outcomes is $2^x$.
Thus the probability is $\binom x2/2^x$.

To feel more confident about my answers of problems like this, I always like to think of them in terms of sets. We have the possibility space $\Omega = \{H,T\}^x$, i.e., the set of all possible outcomes is the set of tuples of length $x$ containing either an $H$ or a $T$. An example of a member of this set is $(H,H,T,H,T,T,H)$, if $x=7$ (for instance). We have $\#\Omega = \#(\{H,T\}^x) = (\#\{H,T\})^x = 2^x$.
Now we want the subset $A\subseteq\Omega$ containing all tuples with precisely two $T$'s. To populate this set, we pick the two out of $x$ positions to place the $T$'s, and fill the rest with $H$'s. This can be done in $\binom x2$ ways, so $\#A=\binom x2$.
Thus our probability is $$\mathsf P(A)=\frac{\#A}{\#\Omega}=\frac{\binom x2}{2^x} = \frac{x(x-1)}{2^{x+1}}.$$


TITLE: Problem on parallel transport
QUESTION [1 upvotes]: I've been trying to go through an example for parallel transport but I cannot quite follow the solution.
A surface (paraboloid) is given by the parametric equation $r(ρ, φ)$ = $ρ \cos(φ)\hat{i}$ +
$ρ \sin(φ)\hat{j}$ + $aρ^2\hat{k}$ where a is a constant. Evaluate the metric and Christoffel symbols
in the (ρ, φ) coordinates. A unit vector X initially pointing along $e_φ$ is parallel
transported along the curve given by ρ = R, φ = 2πt for 0 ≤ t ≤ 1. What is the angle
between the initial X(t = 0) and final X(t = 1) vectors?
I've found the metric:
$g_{ρρ} = 1+4a^2ρ^2$, $g_{φφ} = ρ^2$, $g_{φρ} = 0$ and the Christoffel symbols
$\Gamma^ρ_{ρρ}=\frac{4a^2ρ}{1+4a^2ρ^2}$,
$\Gamma^ρ_{φφ}=\frac{-ρ}{1+4a^2ρ^2}$,
$\Gamma^φ_{φρ}=\Gamma^φ_{ρφ}=\frac{1}{ρ}$.
Now using the parallel transport equation, $\dot{X^a} + \Gamma^a_{bc}X^b\dot{x^c} = 0$, I get
$\dot{X^ρ} - \frac{R}{1+4a^2R^2}X^φ\dot{φ}+ \frac{4a^2R}{1+4a^2R^2}X^ρ\dot{ρ} = 0$ and
$\dot{X^φ} + \frac{1}{R}X^ρ\dot{φ} + \frac{1}{R}X^φ\dot{ρ}= 0$
Then I am not sure how to proceed - any help would be much appreciated!

REPLY [1 votes]: This should be a differential geometry question, though you find this in general relativity.
Here the key is to use what you have: $\rho=R$ and $\phi=2\pi t$, i.e. $\dot{\rho}=0$ and $\dot{\phi}=2\pi$. Then you should be able to simplify and obtain two equations:
\begin{equation}\dot{X}^{\rho}-\frac{2\pi R}{1+4a^{2}R^{2}}X^{\varphi}=0\end{equation}
\begin{equation}\dot{X}^{\varphi}+\frac{2\pi}{R}X^{\rho}=0\end{equation}
From here, you can differentiate the first equation and substitute the second one to obtain a differential equation for $X^{\rho}$, and differentiate the second one and substitute into the first one to obtain the other differential equation for $X^{\varphi}$. Solve them and you are done. Here I am assuming your metric and Christoffel symbols are correct, but the idea remains.


TITLE: Probability Question Bus Frequency
QUESTION [1 upvotes]: i need a bus to run so it gives me a 1/2 chance of getting to point B faster than walking. The distance from point A to B is 1.5 miles. the speed of the bus is 25mph and the walking speed is 4mph
Distance = speed x time
Speed by walking 4 miles per hour
Speed by bus 25 miles per hour
Total distance= 1.5 miles
Time it takes to go by walking (distance divided by speed) or
2 miles / 4 mph = 1/2 an hour or 30 mins.
Time it takes to go by bus 2 miles / 25 miles per hour or 1/12.5th of an hour
or 60/12.5= 4.8 mins.
The answer is 1.59busperhour but I am not sure how to get to this answer.....any help is appreciated!

REPLY [2 votes]: Assume that the interval between buses is a constant $h$ hours.
If we arrive at the bus stop at  a "random" time, the probability that our wait will be $\le \frac{h}{2}$ is $\frac{1}{2}$.
So if $\frac{h}{2}\lt 0.3150$, then with probability $\gt \frac{1}{2}$, we will save time by taking the bus. If $\frac{h}{2}\gt 0.3150$, then with probability $\gt \frac{1}{2}$, using the bus takes longer. The "breakeven" point is $\frac{h}{2}=0.315$.  That gives $h=0.63$.
It follows that there are $\dfrac{1}{0.63}\approx 1.5873$ buses per hour if we have probability exactly $\frac{1}{2}$ of saving time by taking the bus.


TITLE: Intersection of two vectors in two dimension
QUESTION [0 upvotes]: I have read few posts about vector intersection, but I am unable to understand them. So, wanted to ask a new question with specifically my needs.
I am trying to find the intersection of two 2 D vectors. So, my first approach was to simply change the vector into lines and find their solutions.
But my problem is I would like to find the intersection of vectors and not the lines. Below diagram shows the points ( . and ° ) as the origin of the vector.
.
 \        /
  \      /
   \    °

For example the above two vectors do not intersect, even if they are extrapolated in the direction of the vector.
However, the below one will intersect if its extrapolated in the direction of the vectors.
.          .
 \        /
  \      /
   \

Is there any way, one can find if the above vectors will intersect or not?

REPLY [1 votes]: Two examples:
$p_1=(0,0),q_1=(1,1),\ \ p_2=(2,0),q_2=(0,1)$
$p_1=(0,0),q_1=(1,1),\ \ p_2=(2,1),q_2=(0,-1)$
The first example will have positive values for both s's.  The second example will have a negative value for $s_2$.
I'll let you do the arithmetic.


TITLE: How did the expression for electric field?
QUESTION [1 upvotes]: How did the expression for electric field come to be defined as F/q?

REPLY [1 votes]: Hi and welcome to physics stackexchange. Imagine you want to quantify how much force will a (test) charge feel if you place it somewhere near another charge, without considering the first (test) charge's properties (i.e. charge magnitude). So you want to define a quantity that is independent of the test particle's charge, but it is related to the second charged particle's properties (i.e. charge magnitude again). Well, the most natural way to do that is to divide by the magnitude of the test charge. The resulting quantity is what came to be known as the electric field, caused by the (non-test) charged particle to all the points in space.
It has the properties of a field, i.e. a given value (or several values if we are talking about a vector field) for each point in space and it also inherits the properties of the (non-test) charged particle, while at the same time neglects properties a test particle might have If we were placing it anywhere in space, where it would be influenced by the field caused by the non-test charge.
I know it doesn't sound like a lot, but I think this is simply a matter of definition. And the definition makes sense, because fields are essentially space-dependent functions (or vectors, whose components are functions of space). I hope this helps...


TITLE: To show that $f$ has a fixed point, that is, there exists $x_0 \in X$ such that $f (x_0) = x_o$.
QUESTION [1 upvotes]: Let $(X, d)$ be a compact metric space. Let $f : X \to X$ be such that $d(f (x), f (y)) < d(x, y)$ for all $x, y \in X$ with $x \neq y$. To show that $f$ has a fixed point, that is, there exists $x_0 \in X$ such that $f (x_0) = x_o$.
Finding it difficult to prove the existence of a fixed point.
Is the fixed point unique?

REPLY [6 votes]: Uniqueness: if $x,y$ are distinct fixed points, then $f(x)=x$ and $f(y)=y$.
So $d(x,y)=d(f(x),f(y))$, which contradicts the hypothesis.
Existence: the map $x\mapsto d(x,f(x))$ is continuous (why?), so there exists an $x_0$
which achieves the minimum.
Now let $\epsilon:=d(x_0,f(x_0))$. If $\epsilon>0$ we have $x_0\neq f(x_0)$, so $d(f(x_0),f^2(x_0))<\epsilon$
 by hypothesis, which contradicts the minimality of $\epsilon$. So $d(x_0,f(x_0))=0$, i.e. $f(x_0)=x_0$.


TITLE: Combinatorial Proof for a Recursive Sequence
QUESTION [4 upvotes]: For $n>3$ let $g_n = g_{n-1} + g_{n-3}$ where the recursion takes the value 1 for n = 0,1,2.
Prove that $g_{2n} = (g_n)^2 + 2g_{n-2}g_{n-1}$ combinatorially, $n > 1$.
For the time being I am looking for a hint on how to get started with this kind of question.  My idea is that structurally we should find a combinatorial interpretation for the first recurrence and then extend that to the identity below.  The first few terms along with initial conditions suggest a 'cluster' effect in so far as a larger index of the previous terms results in a greater increase with each step.
This is similar to the interpretation for Fibonacci numbers that describe tiling an n-board.  One site describing this interpretation is
http://www.cut-the-knot.org/arithmetic/combinatorics/FibonacciTilings.shtml
Does anyone have any ideas on how to think about these problems?

REPLY [4 votes]: Consider an $1 \times n$ board, which we want to tile with pieces of size $1 \times 1$ and $1 \times 3$. We can use any type as often as we want, but we have to tile the complete board with no overlapping pieces.
Now, to tile an $1 \times n$ board, we can either put a piece of size $1 \times 1$ at the left end (leaving us with a $1 \times (n-1)$ board), or put a large piece of size $1 \times 3$ at the far left end (leaving a $1 \times (n-3)$ board to be tiled). So if $g_n$ counts the number of ways to tile a $1 \times n$ board with these pieces, then the above recursion gives us
$$g_n = g_{n-1} + g_{n-3}.$$
Note that the "base cases" $g_0 = 1$, $g_1 = 1$ and $g_2 = 1$ are also satisfied.
To get to the recursion you want, consider a $1 \times (2n)$ board, and see this as two boards of size $1 \times n$, glued together on the short ends. To tile the complete board, we have two options: (i) tile each board separately, or (ii) tile the board in a way that uses the fact that the boards are glued together.
(i) If we just tile each board separately, then there are $g_n$ ways to tile each board. So in this case, there are $g_n^2$ ways to tile both boards.
(ii) If we do not tile each board separately, then we have to put a $1 \times 3$ piece across the middle, e.g., with one block on the left half of the $1 \times (2n)$ board, and two blocks on the right half. This leaves a $1 \times (n-1)$ board on the left side, and a $1 \times (n-2)$ board on the right side to be filled with other pieces. So there are $g_{n-1} \cdot g_{n-2}$ ways to tile the rest of the board. Since we can also let a piece cover two squares of the left half of the board and one square of the right half of the board, we get an extra factor $2$.
Combining (i) and (ii), we therefore get the final result $$g_{2n} = g_n^2 + 2 g_{n-1} g_{n-2}.$$

A generalization of this idea shows that e.g. a Fibonacci-like recursion
$$f_n = f_{n-1} + f_{n-2}, \quad (f_0 = f_1 = 1)$$
satisfies
$$f_{2n} = f_n^2 + f_{n-1}^2,$$
by applying the same argument to tiles of sizes $1 \times 1$ and $1 \times 2$. Except for the starting values, this is exactly the Fibonacci sequence. More precisely, $f_n = F_{n+1}$, where $F_n$ is the normal Fibonacci sequence starting at $f_0 = 0$, and the above identity then translates to
$$F_{2n+1} = F_{n+1}^2 + F_n^2.$$
With pieces of size $1 \times 1$ and $1 \times 4$, the same reasoning leads to a sequence
$$h_n = h_{n-1} + h_{n-4}, \qquad (h_0 = h_1 = h_2 = h_3 = 1)$$
satisfying
$$h_{2n} = h_n^2 + h_{n-2}^2 + 2 h_{n-1} h_{n-3}.$$
With pieces of size $1 \times 1$, $1 \times 2$ and $1 \times 3$ we can extend the argument even further, and show that the sequence satisfying
$$k_n = k_{n-1} + k_{n-2} + k_{n-3}, \qquad (k_0 = k_1 = 1, \ k_2 = 2)$$
also satisfies the recurrence relation
$$k_{2n} = k_n^2 + k_{n-1}^2 + 2 k_{n-1} k_{n-2}.$$


TITLE: How to show that the space 'line with two origins' is path connected?
QUESTION [4 upvotes]: Consider the equivalence relation on $\mathbb R \times \{0,1\}$ that identifies $(x,0)$ with $(x, 1)$ whenever $x \neq 0$. Let $L$ be the quotient space. This space is called line with two origins.
From the picture of it, I understand that it is path connected but analytically how can we show it?
Any help is appreciated.  Thank you.

REPLY [1 votes]: Here we make a general argument.
Recall that any topological space $X$ can be partitioned into path components and that if the path component containing a point is the whole space, then $X$ is path connected.
Proposition 1: Let $X$ be a topological space and $f: \mathbb R \to X$ and $g: \mathbb R \to X$ two continuous functions satisfying the following condition:
$\tag 1 X = f(\mathbb R) \cup g(\mathbb R) \text{ and } f(\mathbb R) \cap g(\mathbb R) \ne \emptyset$
Then $X$ is path connected.
Proof
Let $x_0$ be any point in the intersection of the two ranges and consider another arbitrary point $x_1 \in X$ that is distinct from $x_0$. Now $x_1 \in f(\mathbb R)$ or $x_1 \in g(\mathbb R)$. If we assume that $x_1 \in f(\mathbb R)$, select real numbers $a_0$ and $a_1$ such that $f(a_0) = x_0$ and $f(a_1) = x_1$. Clearly if we restrict $f$ to the closed interval defined by $a_0$ and $a_1$ we get a path connecting $x_0$ to $x_1$.
An identical argument shows that a path can be created if $x_1 \in g(\mathbb R)$.
We have shown that we can path connect $x_0$ to any other point, so $X$ is indeed path connected.$ \quad \blacksquare$
The OP can apply proposition 1 to their problem by setting up both $f$ and $g$ as the composition of two continuous functions (in a natural way).


TITLE: continuous f(x) and continuous f '(x) relation
QUESTION [1 upvotes]: IF $g(x)$ is prove to be not continuous at one point p.
$g'(x)$ is also not continuous?
**

question is asking to show that g'(x) is not continuous at 0.

**
let say we have
$$g(x) =\begin{cases}
x^2 sin(1/x) & x != 0 \\
0, x = 0 \end{cases}$$
i'm going to show that  g(x) is not continuous
if  $|x-0| < \delta \\$
, and define the  x < $\delta$  and   $\delta  <= \epsilon/2$.
h = x + $\epsilon$
|g(h) - g(0) | = $|x^2sin(1/x) - 0|  $
then   if we put x as $\epsilon/2 $
$$\epsilon<(\epsilon/2)^2 sin( 2/\epsilon) $$
then
$$ |x^2sin(1/x) - 0|   > \epsilon$$
Therefore, This function is not continuous at point 0.
can we conclude this to $g'(x) $ is not also continuous?

REPLY [1 votes]: Your function $g(x)$ is continuous everywhere. So you will not be able to show it is not continuous.
We now look at a related question that you do not explicitly raise.
If a function is differentiable at $p$, then it is automatically continuous at $p$. For if
$$\lim_{x\to p}\frac{g(x)-g(p)}{x-p}=f'(p),$$ then
$$\lim_{x\to p} (x-p)\frac{g(x)-g(p)}{x-p}=\lim_{x\to p}(x-p)f'(p)=0$$
and therefore $\lim_{x\to p}g(x)-g(p)=0$, meaning that $\lim_{x\to p}g(x)=g(p)$. This says $g(x)$ is continuous at $p$.
Equivalently, if $g(x)$ is not continuous at $p$, then $g(x)$ is not differentiable at $p$.  However, you will not be able to use this kind of argument to show that $g'(x)$ is not coninuous at $0$, since in fact $g(x)$ is continuous at $0$.
Added: The question has changed. Directly from the definition of the derivative, you can prove that $g'(0)=0$. For we are looking at the limit of $\frac{x^2\sin(1/x)-0}{x-0}$, that is, of $x\sin(1/x)$. The $\sin(1/x)$ part stays bounded, and the $x$ part approaches $0$, so the product approaches $0$.
To show $g'(x)$ is not continuous at $0$, calculate $g'(x)$ for $x\ne 0$ using the ordinary rules of differentiation. We get $-\cos(1/x)+2x\sin(1/x)$. It is not hard to show that the limit of this as $x\to 0$ does not exist. The $2x\sin(1/x)$ part behaves very nicely, but $-\cos(1/x)$ oscillates wildly as $x\to 0$.


text:
\begin{document}

\title{
Path integral control and state-dependent feedback
}

\author{
	Sep Thijssen
}
\email[]{s.thijssen@donders.ru.nl}
\author{
	H.J. Kappen
}
\email[]{b.kappen@science.ru.nl}
\homepage[]{http://www.snn.ru.nl/~bertk}
\affiliation{
Department of Neurophysics, \\
Donders Institute for Neuroscience\\
Radboud University Nijmegen, The Netherlands
}

\date{
	\today
}

\begin{abstract}
In this paper we address the problem of computing state-dependent feedback controls for path integral control
problems. To this end we generalize the path integral control formula and utilize this to construct parametrized
state-dependent feedback controllers. In addition, we show a relation between control and importance sampling:
Better control, in terms of control cost, yields more efficient importance sampling, in terms of effective sample
size. The optimal control provides a zero-variance estimate.
\end{abstract}

\pacs{02.50.Ey, 02.30.Yy, 05.10.Ln, 05.10.Gg}

\maketitle

\section{Introduction}

Control methods are used widely in many engineering
applications, such as mechanical systems, chemical plants,
finance, and robotics. Often, these methods are used to stabilize
the system around a particular set point or trajectory using state
feedback. In robotics, the problem may be to plan a sequence of
actions that yield a motor behavior such as walking or grasping
an object
\cite{rombokas, kinjo}.
In finance, the problem may be to devise a
sequence of buy and sell actions to optimize a portfolio of
assets, or to determine the optimal option price \cite{glasserman}.

Optimal control theory provides an elegant mathematical
framework for computing an optimal controller using the
Hamilton-Jacobi-Bellman (HJB) equation. In general the HJB
equation is impossible to solve analytically, and numerical
solutions are intractable due to the problem of dimensionality.
As a result, often a suboptimal linear feedback controller such
as a proportional-integral-derivative (PID) controller \cite{stengel} or
another heuristic approach is used instead. The use of suboptimal controllers may be particularly problematic for nonlinear
stochastic problems, where noise affects the optimality of the
controller.

One way to proceed is to consider the class of control
problems in which the HJB equation can be linearized. Such
problems can be divided into two closely related cases \cite{theodorou2012relative}.
The first considers infinite-time-average cost problems, while
the second considers finite-time problems. Approaches of the
first kind \cite{horowitz, kinjo} solve the control problem as an eigenvalue
problem. This class has the advantage that the solution also
computes a feedback signal, but the disadvantage that a
discrete representation of the state space is required. In the
second case the optimal control solution is given as a path
integral \cite{kappen_prl05}. This case will be the subject of this work.
Path integral approaches have led to efficient computational
methods that have been successfully applied to multiagent
systems and robot movement \cite{rombokas, broek2006, anderson, sugimoto, theu_jmlr}.

Despite its success, two key aspects have not yet been addressed.
\begin{enumerate}
\item
The issue of state feedback has been largely ignored
in path integral approaches and the resulting ``open-loop''
controllers are independent of the state; they are possibly augmented with an additional PID controller to ensure stability
\cite{rombokas}.
\item
The path integral is computed using Monte Carlo
sampling. The use of an exploring control as a type of
importance sampling has been suggested to improve the
efficiency of the sampling \cite{kappen2005path, glasserman} but there appear to be no
theoretical results to back this up.
\end{enumerate}

These two aspects are related because the exploring controls
are most effective if they are state feedback controls. In
this paper we propose solutions to these two issues. We
generalize the path integral control formula and utilize this to
construct parametrized state-dependent feedback controllers.
In Corollary~\ref{coro:feedback} we show how a feedback controller might be
obtained using path integral control computations that can
be approximated to arbitrary precision in this way if the
parametrization is correct. The parameters for all future times
can be computed using a single set of Monte Carlo samples.

We derive the key property that the path integral is
independent of the importance sampling when using infinite
samples. However, importance sampling strongly affects the
efficiency of the sampler. In Theorem~\ref{thrm:var} we derive a bound
which implies that, when the importance control approaches
the optimal control, the variance in the estimates reduces
to zero and the effective sample size becomes maximal.
This allows us to improve the estimates iteratively by using
better and better importance sampling with increasing effective
sample size.

This work is structured as follows. In Section~\ref{sect:def} we review path integral control and we extend the existing theory in Section~\ref{sect:lemma}. Using this we prove additional variance bounds in Section~\ref{sect:importance}, and generalized path integral control formulas in Section~\ref{sect:main}. In Section~\ref{sect:feedback} we construct a feedback controller, and describe how to compute it efficiently. In Section~\ref{sect:example} we show in an example how to compute several nonlinear feedback controllers for a nonlinear control problem.

\section{The Path Integral Control Problem}
\label{sect:def}

Consider the dynamical system
\begin{align}\label{eq:sdex}
dX^u(t) &= b(t, X^u(t))dt \nonumber \\
		&\phantom{=\ }+ \sigma(t, X^u(t)) \left[(u(t,X^u(t))dt + dW(t)\right],
\end{align}
for $t_0\leq t\leq t_1$ and with $X^u(t_0) = x_0$. Here $W(t)$ is $m$-dimensional standard Brownian motion, and we take $b:[t_0, t_1]\times \mathbb{R}^n \to \mathbb{R}^n$, $\sigma:[t_0, t_1]\times \mathbb{R}^n \to \mathbb{R}^{n\times m}$ and $u:[t_0, t_1]\times \mathbb{R}^n \to \mathbb{R}^m$ such that a solution of Eq.~(\ref{eq:sdex}) exists. Formulating exact conditions that guarantee existence is not the aim of this work. (See \cite{oksendal, fleming2006controlled} for details of the theory, or \cite{bierkens} for a mathematical approach to path integral control.)

Given a function $u(t, x)$ that defines the control for each state $x$ and each time $t_0\leq t\leq t_1$, we define the cost
\begin{align}
S^u(t) =
	&	\int_{t}^{t_1} V(s, X^u(s))+\frac12u(s, X^u(s))'u(s, X^u(s)) ds \nonumber \\
	&+	\int_{t}^{t_1} u(s, X^u(s))'dW(s),\label{eq:s}
\end{align}
where the prime denotes the transpose.
Note that $S$ depends on future values of $X$ and is therefore not adaptive \cite{oksendal, fleming2006controlled} with respect to the Brownian motion.

It is unusual to include a stochastic integral with respect to Brownian motion in the cost because it vanishes when taking the expectation value. However, when performing importance sampling with $u$, such a term appears naturally (see Section~\ref{sect:importance}).

The goal in stochastic optimal control is to minimize the expected cost with respect to the control.
\begin{align*}
&J(t, x) 		= \min_u\mathbb{E}\left[ S^u(t) \mid X^u(t) = x\right], \\
&u^*(\cdot, \cdot)	= \arg \min_u \mathbb{E}[ S^u(t_0) ].
\end{align*}
Here $\mathbb{E}$ denotes the expected value with respect to the stochastic process from Eq.~(\ref{eq:sdex}). The following, previously established result \cite{kappen_prl05, theu_jmlr} gives a solution of the control problem in terms of path integrals.

\begin{thrm}\label{thrm:old} The solution of the control problem is given by
\begin{align}
J(t_0, x_0)
	& = -\log \mathbb{E} \ e^{-S^u(t_0)}, 	\label{eq:pi0old}	\\
u^*(t_0, x_0) - u(t_0, x_0) \label{eq:pi1old}
	& =
		\lim_{t \to t_0} \frac
			{\mathbb{E}\left[e^{-S^u(t_0)} \int_{t_0}^tdW(s) \right]}
			{(t - t_0)\mathbb{E}\left[e^{-S^u(t_0)}\right]}.
\end{align}
\end{thrm}

\begin{proof}
Eq.~(\ref{eq:pi0old}) will be proven in Remark~\ref{rmrk:pi0} and Eq.~(\ref{eq:pi1old}) follows from the generalized Main Theorem in Section~\ref{sect:main}.
\end{proof}

Because the solution of the control problem is given in terms of a path integral Eqs.~(\ref{eq:pi0old}, \ref{eq:pi1old}), the control problem Eqs.~(\ref{eq:sdex}, \ref{eq:s}) is referred to as a path integral control problem. The formulas from Theorem~\ref{thrm:old} provide a solution at $t_0$. Of course, since $t_0$ is arbitrary, this can be utilized at any time $t$. However, for $t > t_0$, the state $X^u(t)$ is probabilistic, and consequently, the optimal control must be recomputed for each $t, x$ separately. This issue will be partly resolved in the Main Theorem, where we show that all expected optimal future controls can be expressed using a single path integral.

The optimal control solution holds for any function $u$. In particular, it holds for $u = 0$ in which case we refer to Eq.~(\ref{eq:sdex}) as the uncontrolled dynamics. Computing the optimal control in Eq.~(\ref{eq:pi1old}) with $u \neq 0$ implements a type of importance sampling, which is further discussed in Section~\ref{sect:importance}.

\begin{rmrk}
It is straightforward, but notationally tedious, to generalize the control problem to the following slightly more general form
\begin{align*}
& dX = bdt + \sigma (udt + \rho dW), \\
& S = \Phi(X^u(T)) + \int_{t_0}^{t_1} V + \frac12 u'Ru dt + \int_{t_0}^{t_1} u'R\rho dW,
\end{align*}
with $\Phi\in\mathbb{R}$, and $R,\sigma\in\mathbb{R}^{m\times m}$ with $\lambda I = R \rho \rho'$ and $\lambda\in\mathbb{R}_{>0}$. Note that we dropped dependence on $t, X^u(t)$ for brevity.
\end{rmrk}

\section{Linearizable HJB Equation and stochastic processes}
\label{sect:lemma}

In this work we use the HJB equation as a means of solving the control problem. The path integral control problem is characterized by the fact that the HJB equation can be linearized. This will be utilized in this section to obtain the Main Lemma.

\begin{deff}
Throughout the rest of this work we define
\begin{align*}
\psi(t, x)
	& = e^{- J(t, x)}, \\
\psi(t)
	& = \psi(t, X^u(t)), \\
\phi(t)
	& = e^{-S^u(t_0) + S^u(t)} .
\end{align*}
Note that $\psi(\cdot, \cdot)$ denotes a \textit{function} of time and state, while $\phi(\cdot)$ and $\psi(\cdot)$ denote \textit{stochastic processes}, the latter being equal to the function $\psi(\cdot, \cdot)$ of the stochastic process Eq.~(\ref{eq:sdex}). This convention will also be used for other functions, e.g.~$u(t) = u(t, X^u(t))$. We remark that, in contrast to $S^u(t)$, the processes $\psi(t)$ and $\phi(t)$ are adapted: They do not depend on future values of $X$.
\end{deff}

\begin{lemm} [Main Lemma] \label{lemm:main}
\begin{align}
e^{-S^u(t)} - \psi(t)
	=	\frac1{\phi(t)}\int_{t}^{t_1}\!\! \phi(s)\psi(s) \left[
			u^*(s) - u(s)
		\right]'\! dW(s).	\label{eq:main}
\end{align}
\end{lemm}

\begin{proof}
The HJB Equation \cite{fleming2006controlled} for the control problem is
\begin{align*}
-J_t
	& =	\min_u \left(
		V +
		\frac12 u'u +
		(b + \sigma u)'J_x +
		\frac12 \mathrm{Tr}\left(\sigma\sigma' J_{xx}\right)
	\right),
\end{align*}
with boundary condition $J(t_1, x) = 0$. We can solve for $u$ which gives:
\begin{align}
u^*
	& = -\sigma'J_x,	\nonumber \\
-J_t
	& = V -
		\frac12 J_x' \sigma \sigma' J_x +
		b' J_x +
		\frac12 \mathrm{Tr}\left(\sigma\sigma' J_{xx}\right). \label{eq:be}
\end{align}
This partial differential equation becomes linear in terms of $\psi$. We have
\begin{align}
&\psi_t + b'\psi_x + \frac{1}{2}\mathrm{Tr} \ \sigma \sigma' \psi_{xx}=  V\psi,\label{eq:linhjb} \\
&u^*=\frac{1}{\psi} \sigma'\psi_x,	\nonumber
\end{align}
with boundary condition $\psi(t_1,x) = e^{-J(t_1, x)} = 1$.

Using It\^o's Lemma \cite{oksendal, fleming2006controlled} we obtain a stochastic differential equation (SDE) for the process $\psi(t)$ (dropping the dependence on time for brevity)
\begin{align*}
d\psi
	& = \left( \psi_t + \psi'_x(b + \sigma u) + \frac12 \mathrm{Tr} \ \sigma\sigma' \psi_{xx}\right)dt + \psi'_x\sigma dW \\
	& = V\psi dt + \psi'_x\sigma(udt+ dW),
\end{align*}
where the last equation follows because $\psi(\cdot, \cdot)$ satisfies Eq.~(\ref{eq:linhjb}). From the definition of $\phi$ one readily verifies that it satisfies the SDE $d\phi(t) = -\phi(t)\left(V(t)dt + u(t)'dW(t)\right)$ with initial condition $\phi(t_0) = 1$. Using the product rule from stochastic calculus \cite{oksendal} we obtain
\begin{align}
d(\phi\psi)
	&=	\psi d\phi +
		\phi d\psi +
		d[\phi, \psi] \nonumber \\
	&=	-\phi\psi u'dW +
	 	\phi\psi'_x \sigma dW \nonumber \\
	&=	\phi\psi[u^*-u]'dW. \label{eq:mainsde}
\end{align}
Integrating the above from $t$ to $t_1$ gives
\begin{align*}
\phi(t_1)\psi(t_1) &- \phi(t)\psi(t)  \\
	& =	\int_t^{t_1}\phi(s)\psi(s)(u^*(s) - u(s))'dW(s).
\end{align*}
Note that $\psi(t_1) = 1$ and that $\phi(t_1) = \phi(t)e^{-S^u(t)}$. Dividing by $\phi(t)$ we obtain the statement of the lemma.
\end{proof}

\section{Optimal Importance Sampling}
\label{sect:importance}

A Monte Carlo approximation of the optimal control solution Eq.~(\ref{eq:pi1old}) is a weighted average, where the weight depends on the path cost. If the variance of the weights is high, then a lot of samples are required to obtain a good estimate. Critically, Eq.~(\ref{eq:pi1old}) holds for all $u$, so that it can be chosen to reduce the variance of the path weights. This induces a change of measure and an importance sampling scheme. By the Girsanov Theorem \cite{oksendal, fleming2006controlled}, the change in measure does not affect the weighted average (for a more detailed description in the context of path integral control, see \cite{theodorou2012relative}). The Radon-Nikodym derivative $e^{-\int (\frac12 u'u dt  + u'dW)}$ is the correction term for importance sampling with $u$, which explains why we included $\int u'dW$ in the definition of $S$.

In this section we will show that the optimal $u$ for sampling purposes turns out to be $u^*$. More generally, the variance will decrease as $u$ gets closer to $u^*$. This motivates policy iteration, in which increasingly better estimates $u$ of $u^*$ improve sampling so that even better approximations of $u^*$ might be obtained.

\begin{deff}\label{deff:is} Given the process $X^u(t)$ for $t_0 < t < t_1$:
\begin{enumerate}
\item
	The weight of a path is defined as $\alpha^u = \frac{e^{-S^u(t_0)}}{\mathbb{E}[e^{-S^u(t_0)}]}$.
\item \label{deff:fes}
	The fraction $\lambda^u$ of effective samples is $\lambda^u = \frac{1}{\mathbb{E}[(\alpha^u)^2]}$.
\end{enumerate}
\end{deff}

\begin{thrm} We have the following upper and lower bounds for the variance of the weight:
\label{thrm:var}
\begin{align}
&\operatorname{Var}(\alpha^u) \leq \int_{t_0}^{t_1}\mathbb{E}\left[(u^* - u)'(u^* - u)(\alpha^u)^2\right]  dt, \label{thrm:varupper} \\
&\operatorname{Var}(\alpha^u) \geq \int_{t_0}^{t_1}\mathbb{E}\left[(u^* - u)\alpha^u\right]'\mathbb{E}\left[(u^* - u)\alpha^u\right] dt. \label{thrm:varlower}
\end{align}
\end{thrm}

Because $\operatorname{Var}(\alpha^u) + 1 = \mathbb{E}[(\alpha^u)^2]$, the fraction of effective samples as defined in Definition~\ref{deff:is}.\ref{deff:fes} satisfies $0< \lambda^u \leq 1$. It has been suggested \cite{liubook} that this fraction can be used to determine how well one can compute a sample estimate of a weighted average. This can be connected with Theorem~\ref{thrm:var} as follows.

\begin{coro}\label{coro:fes} If $||u^* - u||^2 \leq \epsilon/(t_1 - t_0)$, then
\begin{align*}
\lambda^u \geq 1 - \epsilon.
\end{align*}
\end{coro}

\begin{proof}
This follows readily from Eq.~(\ref{thrm:varupper}).
\end{proof}

A numerical illustration of Theorem~\ref{thrm:var} can be found in Figure~\ref{fig:bound}. Before we prove Theorem~\ref{thrm:var}, we deduce a few useful facts that follow from the Main Lemma.

\begin{coro}
An optimally controlled random path is an instance of Eq.~(\ref{eq:sdex}) with $u = u^*$. Although such a path is random, its attributed cost has zero variance and is equal to the expected optimal cost to go:
\begin{align*}
S^{u^*}(t_0) = - \log \psi(t_0, x_0) = J(t_0, x_0).
\end{align*}
Furthermore we have $\alpha^{u^*} = 1$, such that the weighted average, which is independent of $u$, equals the expectation under the optimal process.
\end{coro}

\begin{proof}
Take $u = u^*$ and $t = t_0$ in Eq.~(\ref{eq:main}).
\end{proof}

\begin{coro}\label{coro:fk}
The following Feynman-Kac formula \cite{oksendal, fleming2006controlled} expresses $\psi$ as a path integral:
\begin{align}
\psi(t) = \mathbb{E} \left[e^{-S^u(t)} \mid \mathcal{F}_t \right] \label{eq:fk}.
\end{align}
Here the filtration $\mathcal{F}_t$ denotes that we are taking the expected value conditioned on events up to time $t$.
\end{coro}

\begin{proof}
Take the expected value on both sides of Eq.~(\ref{eq:main}).
\end{proof}

\begin{rmrk}\label{rmrk:pi0}
When we consider Eq.~(\ref{eq:fk}) with $t = t_0$, and take minus the logarithm on both sides, we obtain Eq.~(\ref{eq:pi0old}): a path integral formula for the optimal cost to go function.
\end{rmrk}

\begin{proof}[Proof of Theorem~\ref{thrm:var}]
Consider Eq.~(\ref{eq:main}) with $t = t_0$, and divide by $\psi(t_0, x_0)$ such that
\begin{align}
&\operatorname{Var}\left(\alpha^u\right)  \nonumber \\
	&=	\mathbb{E}\left[ \left(\int_{t_0}^{t_1}
			\frac{\phi(t)\psi(t)}{\psi(t_0)}[u^*(t) - u(t)]'dW(t)
		\right)^2\right]
\nonumber \\
	&=	\mathbb{E} \int_{t_0}^{t_1}
			\frac{\phi(t)^2\psi(t)^2}{\psi(t_0)^2}[u^*(t) - u(t)]'[u^*(t) - u(t)]dt
			\nonumber \\
	& = \mathbb{E} \int_{t_0}^{t_1}
			\!\!\left[\alpha^u\psi(t)e^{S^u(t)}\right]^2[u^*\!(t) - u(t)]'[u^*\!(t) - u(t)]dt.
		\label{eq:isom}
\end{align}
In the first line we used that $\phi(t_0) = 1$, and in the second line we applied the It\^o Isometry \cite{oksendal}. In the third line we used $\alpha^u = e^{-S^u(t_0)}/\psi(t_0)$, which follows from Eq.~(\ref{eq:fk}) with $t = t_0$.

For the upper bound we consider Eq.~(\ref{eq:fk}) and apply Jensen's inequality
\begin{align*}
\psi(t)^2
	= \mathbb{E} \left[
		e^{-S^u(t)}
	\mid \mathcal{F}_t \right]^2
	\leq \mathbb{E} \left[
		e^{-2S^u(t)}
	\mid \mathcal{F}_t \right].
\end{align*}
Substituting in Eq.~(\ref{eq:isom}) and using the Law of total expectation we obtain Ineq.~(\ref{thrm:varupper}).

For the lower bound we use Jensen's Inequality on the whole integrand of Eq.~(\ref{eq:isom}) to obtain
\begin{align*}
\operatorname{Var}\left(\alpha^u\right)
	\geq \int_{t_0}^{t_1}
			\mathbb{E}&\left\{\alpha^u\psi(t)e^{S^u(t)}[u^*(t) - u(t)]'\right\} \\
			&\mathbb{E}\left\{\alpha^u\psi(t)e^{S^u(t)}[u^*(t) - u(t)]\right\}
			dt.
\end{align*}
Using Eq.~(\ref{eq:fk}) and the Law of total expectation we obtain Ineq.~(\ref{thrm:varlower}).
\end{proof}

We conclude that the optimal control problem is equivalent to the optimal sampling problem. An important consequence, which is given in Corollary~\ref{coro:fes}, is that if the importance control is close to optimal, then so is the sampling efficiency.

\section{The Main Path Integral Theorem}
\label{sect:main}

The Main Theorem is a generalization of Theorem~\ref{thrm:old} that gives a solution of the control problem in terms of path integrals. The disadvantage of Theorem~\ref{thrm:old} is that it requires us to recompute the optimal control for each $t,x$ separately. Here, we show that we can also compute the \textit{expected} optimal future controls using a single set of trajectories with initialization $X(t_0) = x_0$. We furthermore generalize the path integral expressions by considering the product with some function $f(t, x)$. In the next section we utilize this result to construct a feedback controller. Here we proceed with the statement and the proof of the generalized path integral formula.

\begin{nota}
For any process $Y(t)$ we let $\left<Y(t)\right> = \left<Y\right>(t) = \mathbb{E}[\alpha^u Y(t)]$ denote the weighted average.
\end{nota}

\begin{thrm}[Main Theorem] \label{thrm:main}
Let $f:\mathbb{R}\times\mathbb{R}^n\to\mathbb{R}$, and consider the process $f(t) = f(t, X(t))$. Then
\begin{align}
\mathbb{E}[\psi(t)]\label{eq:pi0}
	&= \mathbb{E}\left[e^{-S^u(t)}\right], \\
\left< (u^* - u)f \right>(t) \label{eq:pi1}
	& =\lim_{r\to t}\left< \frac{\int_t^rf(s)dW(s)}{r - t}\right>(t).
\end{align}
\end{thrm}

\begin{proof}[Proof of (\ref{eq:pi0})]
Consider the Feynman-Kac Formula Eq.~(\ref{eq:fk}) and take the expectation with respect to $\mathcal{F}_{t_0}$.
\end{proof}

\begin{proof}[Proof of (\ref{eq:pi1})]
Consider Lemma~\ref{lemm:main} with $t = t_0$, multiply by $\int_t^rf(s)dW(s)$, and take the expected value:
\begin{align*}
\mathbb{E} & \left[e^{-S^u(t_0)} \int_t^rf(s)dW(s)\right] \\
 & = \mathbb{E} \int_t^r \phi(s)\psi(s)[u^*(s)  -u(s)]f(s) ds.
\end{align*}
On the left-hand side the term $\psi(t_0)\int fdW$ has disappeared because $\psi(t_0)$ is not random and the stochastic integral has zero mean. On the right-hand side we have used independent increments and the It\^o Isometry. Dividing by $r - t$ and taking the limit $r \to t$ we obtain
\begin{align*}
\lim_{r\to t}\frac{1}{r - t}\mathbb{E} &\left[e^{-S^u(t_0)}\int_t^rf(s)dW(s)\right] \\
	& = \mathbb{E} \left[\phi(t) \psi(t) (u^*(t) - u(t))f(t) \right] \\
	& = \mathbb{E} \left[e^{-S^u(t_0)} (u^*(t) - u(t))f(t) \right],
\end{align*}
where in the last line we used that $\phi(t) = e^{-S^u(t_0) + S^u(t)}$ and $\psi(t) = \mathbb{E}[e^{-S^u(t)}|\mathcal{F}_{t}]$ combined with the Law of total expectation. Dividing both sides by $\mathbb{E}[ e^{-S^u(t_0)}]$ gives Eq.~(\ref{eq:pi1}).
\end{proof}

\section{A Parametrized Feedback Controller}
\label{sect:feedback}

In this section we illustrate how Theorem~\ref{thrm:main} can be used to construct a feedback controller. To this end we will assume that $u^*$ is of the following parametrized from:
\begin{align}
u^*(t, x) = A(t)h(t, x). \label{eq:param}
\end{align}
Here $h:\mathbb{R}\times\mathbb{R}^n\to \mathbb{R}^k$ will be referred to as the $k$ `basis' functions which are assumed to be known. The ``parameters'' $A(t)\in\mathbb{R}^{m\times k}$  are assumed to be unknown. Note that the open-loop controller can be obtained by a parametrization with one basis function $h = 1$. The following corollary states that it is possible to estimate the optimal parameters from the equations in the Main Theorem.

\begin{coro}
[Path Integral Feedback]
\label{coro:feedback}
Let $f(t, x)\in\mathbb{R}^{l}$ be a function, and suppose that $u^*$ is of the form Eq.~(\ref{eq:param}), then
\begin{align}
A(t)\left< h f' \right>\!(t) = \left< u f' \right>\!(t) +
	\lim_{r\to t}\left< \frac{\int_t^rf'(s)dW(s)}{r - t}\right>.\label{eq:up}
\end{align}
\end{coro}

\begin{proof}
This follows directly from Eq.~(\ref{eq:pi1}) of the Main Theorem when the parametrized from of $u^*$ is used.
\end{proof}

Assuming that both the right-hand side and the cross correlations $\left<h f'\right>\!(t)$ can be obtained by sampling methods, Eq.~(\ref{eq:up}) gives for each time $t$ a set of $m\times k$ linear equations in the $k\times m$ unknown parameters $A(t)$. These equations can be solved uniquely if the $k\times l$ matrix $\left<hf'\right>$ is of rank $k$. Although we have to do these computations for each time $t$ separately, only one set of paths is needed to get the sampling estimates for all times.

In general it will be impossible to check whether the optimal control is of the parametrized form. However, it seems plausible that if the parametrization can represent $u^*$ quite well, it will be possible to estimate a good control function using Corollary~\ref{coro:feedback}. In the next section we perform a numerical experiment to support this statement.

Note, that we can use any importance control $u$ to estimate the optimal control $u^*$. In principle, we could use $u = 0$ and sample long enough to compute the $u^*$ sufficiently accurately. However, we find it more efficient to use an iterative method where we use the optimal control estimate $u_l$ that was computed at iteration $l$ as an importance control for the computation of the optimal control $u_{l+1}$. According to Corollary~\ref{coro:fes} we know that improved controls have a higher fraction of effective samples and thus will make more efficient use of the sampling data. In particular, if $u$ and $u^*$ are parametrized with the same basis functions and time dependent coefficients $A(t)$ and $A^*(t)$, respectively, this results in an iterative update scheme for these coefficients. We refer to this method as iterative importance sampling.

We conclude that parametrized control functions can be obtained directly from path integral estimates, where the parameters can be computed using a single set of paths. Critically, these parametrized controls can be state dependent functions. As a result, it is possible to construct (closed-loop) feedback controllers, which are more widely applicable than open-loop controllers.

\section{Example}
\label{sect:example}

We consider the following control problem, of which we know the analytical solution.

\begin{exmp}[Geometric Brownian Motion] \label{exmp:gbm}
For $t_0 \leq t \leq t_1$, the one-dimensional problem
\begin{align*}
dX^u(t)
	& =	X^u(t)\left(\frac{dt}2  + u(t, X^u(t))dt + dW(t)\right),	\\
S^u(t)
	& =	\frac{Q}2 \log(X^u(t_1))^2 +
		\frac12\int_{t}^{t_1} u(s, X^u(s))^2ds \\
	& \phantom{=\ } + \int_{t}^{t_1} u(s, X^u(s))'dW(s),
\end{align*}
has solution
\begin{align*}
&u^*(t, x)
	= \frac{-Q\log(x)}{Q(t_1 - t) + 1}.
\end{align*}
For the experiments we will take $x_0 = 1/2$, $t_0 = 0$,  $t_1 = 1$, and $Q = 10$.
\end{exmp}

In a first experiment we visualize Theorem~\ref{thrm:var}. To this end we consider a range of sub-optimal importance controls $u^{\epsilon}(t, x) = u^*(t, x) + \sqrt{\epsilon}$. Each $u^{\epsilon}$ yields a path weight $\alpha^{\epsilon} := \alpha^{u^{\epsilon}}$. Because $\left<u^*- u\right>'\left<u^*- u\right> = \epsilon$, Theorem~\ref{thrm:var} implies that
$\epsilon \leq\operatorname{Var}(\alpha^{\epsilon})\leq \frac\epsilon{1 - \epsilon}$. The results are reported in Figure~\ref{fig:bound}.

\begin{figure}
\includegraphics{fig1.pdf}
\caption{
	Estimate of $\operatorname{Var}(\alpha^{\epsilon})$, where $\alpha^{\epsilon} := e^{-S^{u^{\epsilon}}(t_0)}/\psi(t_0, x_0)$ with upper and lower bounds from Theorem~\ref{thrm:var} with respect to the control problem in Example~\ref{exmp:gbm}. Here we considered a range of sub-optimal importance controls $u^{\epsilon}(t, x) = u^*(t, x) + \sqrt{\epsilon}$. The estimate of the variance is based on $10^4$ paths that were generated with $dt = 0.001$.
}
\label{fig:bound}
\end{figure}

In a second experiment we construct feedback control functions based on various parametrizations. It is clear that a correct parametrization of the problem in Example~\ref{exmp:gbm} can be obtained with just one basis function: $\log(x)$. In the experiment we also consider three parametrizations that cannot describe $u^*$: a constant, an affine, and a quadratic function of the state. The three controllers that we obtain in this way are denoted by $u^{(0)}$, $u^{(1)}$, and $u^{(2)}$, e.g., $u^{(2)}(t, x) = a(t) + b(t)x + c(t)x^2$.

\begin{table}
\caption{\label{tbl:gbm}
	Performance estimates of various controllers based on $10^4$ sample paths. Although for numerical consistency we used $10^4$ sample paths to compute the parameters, only roughly $10^2$ samples are required to obtain well-performing controllers.
}
\begin{ruledtabular}
\begin{tabular}{l|cccccc}
				& $u = 0$	& $u^{(0)}$	&	$u^{(1)}$ &  $u^{(2)}$ &$a(t)\log(x)$ & $u^*$
\\ \hline
$\mathbb{E}[S^u(t_0)]$					& 7.526	& 5.139 & 1.507 & 1.461	& 1.422	& 1.420 \\
$\operatorname{Var}(\alpha^u)$	& 1.981	& 1.376	& 0.143	& 0.0506& 0.0085& 0.0071  \\
$\lambda^u$(\%)							& 34.3	& 42.08	& 87.5	& 95.2	& 99.1	& 99.3
\end{tabular}
\end{ruledtabular}
\end{table}

We have used iterative importance sampling with $f = h$ as described in the previous section to estimate the parameters. The performance of the resulting control functions is given in Table~\ref{tbl:gbm}. The row $\mathbb{E}[S^u(t_0)]$ gives the expected cost, which we want to minimize. The row $\operatorname{Var}(\alpha^u)$ gives the variance of the path weight, which is directly related to the FES. Clearly the open-loop controller $u^{(0)}(t, x) = a(t)$ improves upon the zero controller $u(t, x) = 0$. The control further improves when the affine and quadratic basis functions are subsequently considered. The best result is obtained, unsurprisingly, with the logarithmic parametrization.

In Figure~\ref{fig:control} we plot the state dependence of the feedback controllers at the intermediate time $t = 1/2$. Although the parametrized functions yield a control for all $x$, we are mainly interested in regions of the state space that are likely to be visited by the process $X$. This is visualized by a histogram of $10^4$ particles that are drawn from $X^{u^*}(1/2)$. We observe that the optimal logarithmic shape is fitted, and that more complex parametrizations yield a better fit.

\begin{figure}
\includegraphics{fig2.pdf}
\caption{
	The approximate controls calculated with $10^4$ sample paths in two importance sampling iterations using a time discretization of $dt = 0.001$ for numerical integration. The histogram was created with $10^4$ draws from $X^{u^*}(t)$ at $t = 1/2$.
}
\label{fig:control}
\end{figure}

\section{Discussion}

Most current feedback controllers that are used to stabilize systems are linear feedback controllers such as PID controllers. These are heuristic approaches that are optimal only if one assumes that the system dynamics is linear and the cost is quadratic. In this paper we have shown how to compute optimal feedback controllers for a class of nonlinear stochastic control problems. The optimality requires the use of the appropriate basis functions.

It should be noted that the optimal feedback is not necessarily a stabilizing term. Depending on the task it might be optimal to destabilize by amplifying the noise, for example, to create momentum efficiently.

Future work includes the development of methods for practical scenarios, based on the path integral feedback Eq.~(\ref{eq:up}). An important aspect will be the selection of basis functions. A recent related work \cite{horowitz} discusses basis functions to obtain a solution of the linearized HJB Eq.~(\ref{eq:linhjb}).

\begin{acknowledgments}
This work was funded by D-CIS Lab / Thales Research \& Technology NL, and supported by the European Community Seventh Framework Programme (FP7/2007-2013) under grant agreement 270327 (CompLACS).
\end{acknowledgments}

\bibliography{pi_feedback.bib}

\end{document}

config: arxiv


text:
\begin{document}
\maketitle
\begin{abstract} In this paper we study different aspects of the representation
of weak*-compact convex sets of the bidual $X^{**}$ of a separable
Banach space $X$ via a nested sequence of closed convex bounded
sets of $X$. \end{abstract}

\section{Introduction}

In this paper we solve several problems about nested intersections
of convex closed bounded sets in Banach spaces.

We begin with a study of different aspects of the representation
of weak*-compact convex sets of the bidual $X^{**}$ of a separable
Banach space $X$ via a nested sequence of closed convex bounded
sets of $X$. Precisely, let us say that a convex closed bounded
subset $C\subset X^{**}$ is \emph{representable} if it can be written as
the intersection
$$C = \bigcap_{n\in \N} \overline{C_n}^{w^*}$$
of a nested sequence $(C_n)$ of bounded convex closed subsets of
$X$. This topic was considered in \cite{castpapi,limit}, where the
problem of which weak*-closed convex sets of the bidual are
representable was posed. In \cite{berna}, Bernardes shows that
when $X^*$ is separable every weak*-compact convex subset of
$X^{**}$ is representable. Here we will show that compact convex
sets of $X^{**}$ are representable if and only if $X$ does not
contain $\ell_1$ and also that there are spaces without copies of
$\ell_1$ containing weak*-compact convex metrizable subsets of the bidual
not representable.

In Section 3, we solve problem (2) in \cite{limit} showing that
when the sets are viewed as the distance type (in the sense of
\cite{castpapi}) they define, i.e., as elements of $\R^X$, then
every weak*-compact convex set $C\subset X^{**}$ is represented
by a nested sequence $(C_n)$ of closed convex sets of $X$; which
means that for all $x\in X$
$$\mathrm{dist} (x, C) = \lim \mathrm{dist} (x, C_n).$$

In Section 4 we present two examples: the first one solves
Marino's question \cite{mari} about the possibility of enlarging
nested sequences of convex sets to get better intersections; the
second one solves Behrends' question about the validity of
$\varepsilon=0$ in the Helly-B\'ar\'any theorem \cite{behr}.

\section{Representation of convex sets in biduals}

\adef A Banach space is said to enjoy the Convex Representation
Property, in short CRP (resp. Compact Convex Representation
Property, in short CCRP) if every weak*-compact (resp. compact)
convex subset $C$ of $X^{**}$ can be represented as the
intersection
$$C = \bigcap_{n\in \N} \overline{C_n}^{w^*}$$
of a nested sequence $(C_n)$ of bounded convex closed subsets of
$X$. \zdef

\begin{prop} A separable Banach space has CCRP if and only if it does not contain
$\ell_1$.
\end{prop}
\begin{proof} The necessity follows from the Odell-Rosenthal characterization
\cite{odell} of separable Banach spaces containing $\ell_1$.
Indeed, if $X$ contains $\ell_1$ then there is an element $\mu\in
X^{**}$ which is not the weak*-limit of any sequence of elements
of $X$. Hence, $\{\mu\}=\bigcap_{n \in \N} \overline{C_n}^{w^*}$
is impossible: taking an element $c_n\in C_n$ one would get
$\emptyset \neq \bigcap_n \overline{\{c_k: k\geq n\}}^{w^*} \subset \bigcap_n \overline{C_n}^{w^*}  = \{\mu\}$, which means that $\mu$ is the only weak*-cluster point of the sequence $(c_n)$ and thus $\mu =
w^*-\lim c_n$.

As for the sufficiency, let $K$ be a compact convex subset of
$X^{**}$. For every $n\in \N$, let $F_n=\{z_n^k : k\in I_n \}$ be
a finite subset of $K$ for which $K\subset F_n +
n^{-1}B_{X^{**}}$. There is no loss of generality assuming that
$F_n\subset F_{n+1}$. For each $z_n^k\in F_n$, let
$(x_n^k(m))_m\subset X$ be a sequence in $X$ weak*-convergent to
$z_n^k$. Set

$$C_n = \overline{conv}\{ x_n^k(m) , k\in I_n, m\geq n \}+ n^{-1}B_X$$

It is clear that $C_n$ is a nested sequence of closed convex sets
of $X$. Moreover,

$$K\subset F_n + n^{-1}B_{X^{**}} \subset \overline{\overline{conv}\{ x_n^k(m) , k\in I_n, m\geq n\}+ n^{-1}B_X}^{w^*} =\; \overline{C_n}^{w^*},$$
and thus $K\subset \bigcap_n \overline{C_n}^{w^*}$.

Fix now $p\in \bigcap_n \overline{C_n}^{w^*}$. Since $p\in
\overline{C_n}^{w^*}$, there is a finite convex combination
$\sum_{i\in I_n} \theta_i z_n^i$ for which $\|p- \sum_{i\in I_n}
\theta_i z_n^i\|\leq n^{-1}$. This implies that
$p\in\overline{K}=K$ and thus $\bigcap_n
\overline{C_n}^{w^*}\subset K$.\end{proof}

This shows that Problem 1 in \cite{limit} has a negative answer.
On the other hand, Bernardes obtains in \cite{berna} an
affirmative answer when $X^*$ is separable, which is somehow the
best that can be expected. Let us briefly review and extend
Bernardes' result. Recall that a partially ordered set $\Gamma$ is
called filtering when for any two points $i,j\in \Gamma$ there is
$k\in \Gamma$ such that $i\leq k$ and $j\leq k$. An indexed family
of subsets $(C_\alpha)_{\alpha\in \Gamma}$ will be called
filtering when it is filtering with respect to the natural
(reverse) order; i.e., whenever $\alpha\leq \beta$ then $C_\beta
\subset C_\alpha$. One has:

\begin{proposition}\label{main} If $C$ is a convex weak*-compact set in the bidual
$X^{**}$ of a Banach space $X$ then there is a filtering family
$(C_\alpha)_{\alpha \in \Gamma}$ of  convex bounded and closed
subsets of $X$ such that
$$C = \bigcap_{\alpha\in \Gamma} \overline{C_\alpha}^{w^*}.$$
\end{proposition}
\begin{proof} There is no loss of generality assuming that $C\subset B_{X^{**}}$. Let $\Gamma$ be the partially ordered set of finite
subsets of $B_{X^*}$. For each $\alpha\in \Gamma$ we denote
$|\alpha|$ the cardinal of the set $\alpha$. Set now

$$C_\alpha = \{ x\in X: \exists z\in C : \forall y\in \alpha:
|(z-x)(y)|\leq |\alpha|^{-1}\}.$$

This family $(C_\alpha)_{\alpha \in \Gamma}$ is filtering, as well
as $(\overline{C_\alpha}^{w^*})_{\alpha \in \Gamma}$, which
ensures that $\bigcap_{\alpha\in \Gamma}
\overline{C_\alpha}^{w^*}$ is nonempty. Let us show the equality
$$C = \bigcap_{\alpha\in \Gamma} \overline{C_\alpha}^{w^*}.$$

\begin{itemize}

\item $C\subset \bigcap_{\alpha\in \Gamma}
\overline{C_\alpha}^{w^*}$: Let $z\in C \subset B_{X^{**}}$; given
$\alpha\in \Gamma$, by the Banach-Alaoglu theorem, there is $x\in
B_X$ such that $|(z-x)(y)|<|\alpha|^{-1}$ for all $y\in \alpha$.
Hence $x\in C_\alpha$ and thus $z\in \overline{C_\alpha}^{w^*}$.

\item $\bigcap_{\alpha\in \Gamma} \overline{C_\alpha}^{w^*}
\subset C$: Let $z\in \bigcap_{\alpha\in
\Gamma}\overline{C_\alpha}^{w^*}$ and let $V_{\alpha,
\varepsilon}$ be the weak*-neighborhood of $0$ determined by
$\alpha\in \Gamma$ and $\varepsilon>0$; i.e., $V_{\alpha,
\varepsilon} = \{p\in X^{**} : \forall y\in \alpha: |p(y)|\leq
\varepsilon$. Pick $\beta\in \Gamma$ with $\alpha\leq \beta$ and
$|\beta|^{-1}\leq \varepsilon$. Since $z\in
\overline{C_\beta}^{w^*}$, there is $x\in C_\beta$ such that
$|(z-x)(y)|\leq  \varepsilon$ for all $y\in \alpha$; which
moreover means that there is some $z'\in C$ such that
$|(z'-x)(y)|\leq |\beta|^{-1} \leq \varepsilon $ for all $y\in
\beta$. Putting all together one gets that for $y\in \alpha$
$$|(z-z')(y)| =|(z-x)(y) +(x-z')(y)| \leq 2\varepsilon$$
and thus $z-z'\in V_{\alpha, 2\varepsilon}$. Hence $z\in
\overline{C}^{w^*} = C$.
\end{itemize}
\end{proof}

The size of $\Gamma$ can be reduced just taking first a dense
subset $Y\subset B_{X^*}$ and then fixing as $\Gamma$ a
fundamental family of finite sets of $Y$, in the sense that every
finite subset of $Y$ is contained in some element of $\Gamma$.
Such reduction modifies the proof as follows: from the first
finite set $\alpha$ --no longer in $\Gamma$-- determining
$V_{\alpha, \varepsilon}$ one must take a set $\beta\in \Gamma$
such that for each $y\in \alpha$ there is $y'\in \beta$ so that
$\|y-y'\|\leq |\beta|^{-1} \leq \varepsilon$. Get $x$ and $z'$ as
above. Finally, for $y\in \alpha$, one gets
$$|(z-z')(y)| =|(z-z')(y-y') +(z-z')(y')| \leq \varepsilon +2\varepsilon = 3\varepsilon.$$

The consequence of such simplification is that when $X^*$ is
separable then $\Gamma$ reduces to $\N$ and thus one gets the main
result in \cite{berna}:

\begin{cor}[Bernardes] Every Banach space with separable dual has CRP.
\end{cor}

One therefore has:$$ X^* \;\; \mathrm{separable} \Longrightarrow
CRP \Longrightarrow CCRP \Longleftrightarrow \ell_1 \nsubseteq X.
$$

This suggests two questions: 1) whether CCRP implies CRP and 2)
whether CRP implies having separable dual. One has

\begin{prop} CCRP does not imply CRP.
\end{prop}

To prove this we are going to show that the James-Tree space
--perhaps the simplest space not containing $\ell_1$ but having
nonseparable dual-- fails CRP. For information about $JT$, we
refer to \cite[Chapter VIII]{vanDulst}. We begin with a
preparatory lemma that can be considered as a complement to Kalton
\cite[Lemma 5.1]{kalt}.

\begin{lemma}\label{kalty} Let $(C_n)_n$ be a nested sequence of bounded closed
convex subsets of a Banach space $X$. If  $\bigcap_n
\overline{C_n}^{w^*}$ is weak*-metrizable then:
\begin{enumerate}
\item Every $g\in \bigcap_n \overline{C_n}^{w^*}$ is the
weak*-limit of a sequence $(c_n)$ with $c_n\in C_n$. \item Every
sequence $(c_n)$ with $c_n\in C_n$ admits a weak*-convergent
subsequence.
\end{enumerate}
\end{lemma}
\begin{proof} (1) is clear: let $(V_n)_n$ be a sequence of
weak*-neighborhoods of $g$ such that $\{g\} = \cap V_n \cap
\bigcap_n \overline{C_n}^{w^*}$. Picking $c_n \in C_n \cap V_n$
one gets $\{g\}=\overline{\{c_n\}}^{w^*}$.

To prove (2), let us consider the equivalence relation on the set
$\mathcal P_\infty(\N)$ of infinite subsets of $\N$: $A\sim B$ if
and only if $A$ and $B$ coincide except for a finite set.
Moreover, $K$ will denote the set of all compact subsets of
$\bigcap_n \overline{C_n}^{w^*}$. Given a sequence $(c_n)$ with
$c_n\in C_n$ we define the following map $w: \mathcal
P_\infty(\N)/\sim \;\;\to K$:

$$w([A]) = \cap_k  \overline {\{c_n : n \in A, n > k\}}^{w^*}.$$ The set $\mathcal P_\infty(\N)/\sim$ admits a
natural order as $[A]\leq [B]$ if $A$ is eventually contained in
$B$. This order has the property that for every decreasing
sequence $([A_n])_n$ there is an element $[B]$ with $[B]\leq
[A_n]$ for all $n$. Since $\bigcap_n \overline{C_n}^{w^*}$ is
metrizable, it follows \cite[Sect. 2]{behrarch} that there is $M\in \mathcal
P_\infty (\N)$ on which $w$ is stationary; i.e., $w([C])=w([M])$
for all infinite subsets $C\subset M$. This immediately yields
that $w(\{c_n\}_{n\in M})$ has only one point, and thus
$\{c_n\}_{n\in M}$ is weak*-convergent. \end{proof}

\medskip

Let us denote by $G$ the set of all branches of the dyadic tree
$T$. For each $r\in G$, let $e_r$ denote the corresponding element
of the basis of $\ell_2(G)$ considered as a subspace of $JT^{**}$.
Let $\{e_{k,l} : k \in \N_0, 1 \leq l \leq 2^k\}$ denote the unit
vector basis of $JT$. The action of $e_r$ on $x^*\in JT^*$ is
given by
$$
\langle x^*,e_r\rangle = \lim_{\textrm{along $r$}} \langle
e_{k,l},x^*\rangle.
$$

For each $m\in \N$ we denote by $P_m$ the norm-one projection in
$JT$ defined by $P_m e_{k,l} = e_{k,l}$ if $k\geq m$, and $P_m e_{k,l} = 0$ otherwise. For each $r\in G$ we consider $f_r\in JT^*$ given
by $\langle e_{k,l},f_r\rangle$ equal to $1$ if $(k,l)\in r$, and
equal to $0$ otherwise. Observe that $\langle f_r,e_s\rangle =
\delta_{r,s}$. Let $S = \{s_n : n\in \N\}$ denote a countable
subset of $G$ such that the branches in $S$ include all the nodes
of the tree $T$.\\

\noindent \textbf{Proof of Proposition 2.} Let us show that the
closed unit ball $B$ of $\ell_2(S)$ cannot be represented. Assume
that we can write $B=\cap_{n\in\N} \overline{C_n}^{w^*}$. The set
$B$ is $w^*$-metrizable, because it is the unit ball of a
separable reflexive subspace. By Lemma \ref{kalty}, each vector in
$B$ is the $w^*$-limit of a sequence $(x_n)$ with $x_n \in C_n$.
For each $s \in S$ we select $x^s_n \in C_n$ so that $w^*$-$\lim
x^s_n = e_s$. Note that $\lim_{n}\|(I-P_k)x^s_n\| =0$ for every
$k$ and $s$.
\medskip

We take $t_1\in S$, $t_1\neq s_1$. Also we take $x_1 =
x^{t_1}_{n_1}$ with $|\langle x_1,f_{t_1}\rangle - 1|<2^{-1}$, and
select $(k_1,l_1)\in t_1\setminus s_1$ such that $\|P_{k_1}x_1\|<
2^{-1}$. Next we take $t_2\in S$ with $(k_1,l_1)\in t_2$ and
$t_2\neq s_2$. Also we take $x_2 = x^{t_2}_{n_2}$ with
$\|(I-P_{k_1})x_2\|<2^{-2}$ and $|\langle x_2,f_{t_2}\rangle -
1|<2^{-2}$, and select $(k_2,l_2)\in t_2\setminus s_2$ with $k_2>
k_1$ such that $\|P_{k_2}x_2\|< 2^{-2}$. Proceeding in this way we
obtain a sequence $(x_i)$ that is eventually contained in each
$C_n$ and an ordered sequence of different nodes $(k_i,l_i)$ that
determine a branch $r\in G\setminus S$. Since $JT$ is separable
and contains no copies of $\ell_1$, the sequence $(x_i)$ has a
subsequence that is $w^*$-convergent to some $x^{**}\in JT^{**}$
\cite[First Theorem in p. 215]{Diestel}.  Thus,
$x^{**}\in\cap_{n\in\N} \overline{C_n}^{w^*}$, but $x^{**}\notin
B$ since  $\langle f_r,x^{**}\rangle =1$. \hfill $\square$\\

Proposition 1 thus characterizes the CCRP, while Proposition 2
shows that even when compact convex sets are representable,
arbitrary weak*-metrizable convex bounded closed sets do not have
to be. The question of which convex sets are representable thus
arises. Bigger than compact spaces are the so called small sets
\cite{beka,small,arias}, but it was shown in \cite{beka} that a
closed bounded convex small set is compact.

\section{Representation of convex sets in the hyperspace}

The theory of types in Banach spaces represents the elements of a
Banach space $g\in X$ as functions $\tau_g(x)= \|x -g\|$. These
are the elementary types and the types are the closure of the set
of elementary types in $\R^X$. It can be shown that bidual types,
i.e., functions having the form $\tau_g(x)= \|x -g\|$ for $g\in
X^{**}$ are also types \cite{farm}. In close parallelism, the theory of  distance types was developed in
\cite{castpapi}: in it, the elements to be represented are the closed bounded convex
subsets $C$ of $X$ via the function $d_C(x)=\mathrm{dist}(x, C)$.
These are the elementary distance types. The $\emptyset$-distance
types are the functions of the form $d(x) = \lim d_{C_n}(x)$ where
$(C_n)$ is a nested sequence of closed bounded convex subsets of
$X$ with empty intersection. In \cite[Thm. 4.1]{castpapi} it was
shown the existence of $\emptyset$-distance types that are not
types in every nonreflexive separable Banach space. It was also
shown \cite[Thm. 5.1]{castpapi} that bidual types on separable
Banach spaces coincide with $\emptyset$-distance types defined by
"flat" (in the sense of Milman and Milman \cite{milmil}) nested
sequences of bounded convex closed sets $(C_n)$. In \cite[Thm.
1]{limit} it is shown that given a nested sequence $(C_n)$ of
bounded convex closed sets on a separable space $X$ one always has
$$\mathrm{dist}(x, \bigcap \overline{C_n}^{w^*}) = \lim \mathrm
{dist} (x, C_n).$$ While Bernardes shows in \cite[Thm. 1]{berna}
that that happens in all Banach spaces.

All this suggests the problem \cite[Problem 2]{limit} whether the
analogue of Farmaki's (bidual types are types) also holds for
distance types; i.e., if given a weak*-compact convex subset $C$ of
$X^{**}$, the \emph{bidual distance type} it defines $d_C(x)=\mathrm{dist}(x, C)$ on
$X$, is a $\emptyset$-distance type. Let us give an affirmative answer.

\begin{prop} Let $C$ be a weak*-compact convex subset of the
bidual $X^{**}$ of a separable space $X$ such that $C\cap
X=\emptyset$. There is a nested sequence $(C_n)$ of closed convex
sets of $X$ such that $C\subset \bigcap_n \overline{C_n}^{w^*}$
and for all $x\in X$
$$\mathrm{dist} (x, C) = \lim \mathrm{dist} (x, C_n).$$
\end{prop}
\begin{proof} Let $(x_n)$ be a dense sequence in $X$. Since $C$ is bounded, it is contained in
the ball $\gamma B_{X^{**}}$ for some $\gamma >0$. We proceed inductively: pick $x_1$,
let $\alpha_1=\mathrm{dist}(x_1, C)$, then set a monotone increasing
sequence $(\alpha_n^1)$ convergent to $\alpha_1$. Pick functionals
$\varphi_n^1\in B_{X^*}$ that strictly separate $C$ and
$x_1+(\alpha_n^1)B_{X^{**}}$, say $$\inf_{z\in C}z(\varphi_n^1) >
\|x_1\|+\alpha_n^1 + 2\varepsilon_n^1.$$ Set $C_{n,1} = \{ x\in X:
\exists z\in C :\; |(z - x)(\varphi_n^1)| \leq \varepsilon_n^1\}
\cap \gamma B_{X^{**}}$. The sequence of convex sets $C_{n,1}$ is
nested and every point $z\in C$ belongs to the weak*-closure of
some set $\{ x\in X:  |(z - x)(\varphi_n^1)| \leq n^{-1}\}$
which is in turn contained in $C_{n,1}$. Thus, $C \subset
\bigcap_n \overline{C_{n,1}}^{w^*}$. Moreover,
$x_1+(\alpha_1)B_{X^{**}} \cap \bigcap_n \overline{C_{n,1}}^{w^*}
=\emptyset$ because otherwise there should be elements $c_n\in
C_{n,1}$ for which $(x_1 + \alpha_1 b - c_n)(\varphi_n^1)<
\varepsilon_n^1$; since there must be $z_n\in C$ for which $|(z_n
- c_n)(\varphi_n^1)|\leq \varepsilon_n^1$, pick $z\in C$ a
weak*-accumulation point of $(z_n)$ to conclude that $(x_1 +
\alpha_1 b - z)(\varphi_n^1)= (x_1 + \alpha_1 b - c_n  + c_n -
z)(\varphi_n^1)\leq 2\varepsilon_n^1$ which immediately yields
$$z(\varphi_n^1) = (x_1 + \alpha_1 b)(\varphi_n^1) - (x_1 + \alpha_1 b - z)(\varphi_n^1) \leq \|x_1\|+\alpha_n^1 + 2\varepsilon_n^1$$
in contradiction with the separation above.

Thus, by \cite[Thm. 1]{limit} we get $\mathrm{dist} (x_1, C) =
\mathrm{dist}(x_1, \bigcap \overline{C_{n,1}}^{w^*}) = \lim
\mathrm{dist} (x_1, C_{n,1}).$

We pass to $x_2$. Everything goes as before except that all the
action is going to happen inside $\bigcap
\overline{C_{n,1}}^{w^*}$. Precisely, once $\alpha_2, \alpha_n^2,
\varphi_n^2, \varepsilon_n^2$ have been fixed by the same
procedure as above, set
$$C_{n,2} = \{ x\in X: \exists z\in C :\; \max_{i=1,2}|(z - x)(\varphi_n^i)| \leq
\varepsilon_n^i\} \cap \gamma B_{X^{**}}$$ to conclude that $C
\subset \bigcap_n \overline{C_{n,2}}^{w^*} \subset \bigcap_n
\overline{C_{n,1}}^{w^*}$ and $\mathrm{dist} (x_i, C) =
\mathrm{dist}(x_i, \bigcap \overline{C_{n,2}}^{w^*}) = \lim
\mathrm{dist} (x_i, C_{n,2})$ for $i=1,2$. Proceed inductively.
Since $C_{n,k+1}\subset C_{n,k}$, we can diagonalize the final
sequence of sequences to get the sequence $(C_{k,k})$, which
satisfies $C\subset \bigcap \overline{C_{k,k}}^{w^*}$ and,
moreover, for all $n$ one has
$$\mathrm{dist} (x_n, C) = \mathrm{dist}(x_i, \bigcap
\overline{C_{k,k}}^{w^*}) = \lim \mathrm{dist} (x_n, C_{k,k}).$$
By continuity, the equality remains valid for
 all $x\in X$.
\end{proof}

In the clasical case, as Farmaki remarks in \cite{farm}, it is not
obvious that \emph{fourth-dual} types, i.e., applications having
the form $\tau_g(x)=\|x +g\|$ for $g\in X^4$ on separable spaces
$X$ are necessarily types. One thus may ask: Let $X$ be a
separable Banach space and let $C \subset X^{2k}$ be a bounded
weak*-closed convex. Must there be a sequence $(C_n)$ of bounded
convex closed subset of $X$ such that for every $x\in X$ one has
$\mathrm{dist}(x, C) = \lim \mathrm{dist}(x, C_n)$?

\section{Further properties of nested sequences}

\subsection{Enlarging sets for better intersection: Marino's
problem} Let $A$ be a closed set. For $\varepsilon>0$ we set

$$A^\varepsilon = \{ x\in X : \mathrm{dist}(x, A)\leq
\varepsilon\}.$$

An extremely nice result of Marino \cite{mari} establishes that
given any family $(G_\gamma)$ of convex sets with nonempty
intersection then either $\bigcap_\gamma G_\gamma^\varepsilon$ is
bounded for every $\varepsilon>0$ or is unbounded for every
$\varepsilon>0$. A question left open in \cite[p.583]{limit} is
whether it is possible to have $\bigcap A_n = \emptyset$, some
intersections $\bigcap A_n^{\varepsilon}$ nonempty and bounded and
others unbounded. The next example shows it can be so:

\noindent \textbf{Example 2.} Consider in $\ell_1$ the sequence
$A_{2k} = \{x\in \ell_1: \;\; x_{k+1}\leq
-\frac{2^{k+1}-1}{2^{k+1}}\}$ and $A_{2k-1} = \{x\in \ell_1: \;\;
x_{k}\geq 1\}$. Then $\bigcap A_n =\emptyset = \bigcap
A_n^\varepsilon =\emptyset $ for all $\varepsilon<1$, while

$$\bigcap A_n^1 = \{x\in \ell_1:\;\; \forall k\;\; 0\leq x_k\leq \frac{1}{2^k}
\}$$

and $\bigcap A_n^{1+\varepsilon}$ is unbounded for all
$\varepsilon>0$ since all $x\in \ell_1$ with $-\varepsilon\leq
x_i\leq 0$ for every $i$ belong to that set.\\

The choice of $\ell_1$ for the example is not at random: during
the proof of \cite[Prop. 9]{limit} it is shown that in reflexive
spaces, $\bigcap A_n = \emptyset$ implies $\bigcap A_n^\varepsilon
= \emptyset$ for all $\varepsilon>0$. Marino's theorem in
combination with \cite[Prop. 9]{limit} yields that in a
non-reflexive space if $\alpha= \inf \{\varepsilon>0: \bigcap
A_n^\varepsilon \neq \emptyset \}$ then either $\bigcap
A_n^\varepsilon$ is bounded for all $\varepsilon>\alpha$ or
unbounded for all $\varepsilon>\alpha$.  Let us show now that
Marino's theorem remains ``almost" valid for nested sequences with
empty intersection in a finite dimensional space. In this case,
the boundedness of some $A_n$ immediately implies, by compactness,
that $\bigcap_{n\in \N} A_n\neq \emptyset$. Assume thus one has a
nested sequence of unbounded convex sets. Let $T_k =\{x\in X:
k\leq \|x\|\leq k+1\}$. One has

\begin{lemma} Let $(A_n)$ be a sequence of unbounded connected
sets in a finite dimensional space $X$. Then either $\bigcap_{n\in
\N} A_n=\emptyset$ or for all but finitely many $k\in \N$ and
every $\varepsilon>0$ there is an infinite subset $N_k\subset \N$
such that $T_k \cap \bigcap_{n\in N_k} A_n^\varepsilon\neq
\emptyset$.
\end{lemma}
\begin{proof} If for every $k\in \N$ the ball $kB$ of radius $k$
does not intersect $\bigcap_{n\in \N} A_n$ then $\bigcap_{n\in \N}
A_n=\emptyset$. Otherwise, let $x_{n,k}\in A_n\cap kB$. Since
$A_n$ is unbounded, there is a point $y_{n, k+1}$ with $\|y_{n,
k+1}\|> k+1$. Since $A_n$ is connected, there is some
$x_{n,k+1}\in A_n$ with $k\leq\|x_{n,k+1}\|\leq k+1$, and thus in
$A_n\cap T_k$. The sequence $(x_{n,k+1})_n$ lies in the compact
set $T_k$ and thus for some infinite subset $N_k\subset \N$ the
subsequence $(x_{n,k+1})_{n\in N_k}$ is convergent to some point
$x_{k+1}\in T_k$. Thus, $x_{k+1} + \varepsilon B$ intersects the
sets $\{A_n: n\in N_k\}$ and thus $\bigcap_{n\in N_k}
A_n^\varepsilon \cap T_k\neq \emptyset$.
\end{proof}

Thus we get:

\begin{proposition} Let $(A_n)$ be a nested sequence of unbounded
connected sets  in a finite dimensional space $X$. Then either
$\bigcap_{n\in \N} A_n=\emptyset$ or $\bigcap_{n\in \N}
A_n^\varepsilon$ is unbounded for every $\varepsilon>0$.
\end{proposition}

The assertion obviously fails for non-connected sets and also fails in infinite dimensional spaces:\\

\noindent \textbf{Example 1.} In $\ell_2$ take $A_n=\{x\in \ell_2:
\forall k>n,\;\; 0\leq x_k\leq 1 \;\;\mathrm{and}\;\; \forall
k\leq n, \;\; x_k=0\}$. This  is a nested sequence of unbounded
convex closed sets such that $\bigcap_{n\in \N} A_n=\emptyset$
while for all $\varepsilon>0$ the set $\bigcap_{n\in \N}
A_n^\varepsilon$ is bounded: indeed, if $y\in A_n^\varepsilon$ for
all $n$ then there is $x_n\in A_n$ for which $\|y - x_n\|\leq
\varepsilon$; thus $\sum_{i=1}^n |y_i|^2 \leq \varepsilon^2$ for
all $n$, so $\|y\|\leq \varepsilon^2$.

\subsection{On the  Helly-B\'ar\'any theorem}

In one of the main theorems in \cite{behr} Behrends establishes a
Helly-B\'ar\'any theorem for separable Banach spaces \cite[Thm.
5.5]{behr}: \emph{Let $X$ be a separable Banach space and
$\mathcal C_n$ a family of nonvoid, closed and convex subsets of
the unit ball $B$ for every $n$. Suppose that there is a positive
$\varepsilon_0 \leq 1$ such that $\bigcap_{C\in \mathcal C_n}
C+\varepsilon B =\emptyset$ for every $n$ and every
$0<\varepsilon<\varepsilon_0$. Then there are $C_n\in \mathcal
C_n$ such that $\bigcap_n C_n+ \varepsilon B =\emptyset$.}
Behrends asks \cite[Remark 2, p. 17]{behr} whether one can put
$\varepsilon=\varepsilon_0$ in the previous theorem. The following
example shows that the answer is no:\\

\noindent \textbf{Example.} In $c_0$, the family $\mathcal C_n$
contains two convex sets:

$$a_n^+ = \{x\in c_0: \forall i\in \N:\; |x_i|\leq \frac{1}{2}(1 +
\frac{1}{i})\;\mathrm{and}\; |x_n| = \frac{1}{2}(1 +
\frac{1}{n})\}$$ and

$$a_n^- = \{x\in c_0: \forall i\in \N:\; |x_i|\leq \frac{1}{2}(1 +
\frac{1}{i})\;\mathrm{and}\; |x_n| = -\frac{1}{2}(1 +
\frac{1}{n})\}.$$

One has $a_n^+ \cap a_n^- =\emptyset$ for all $n\in \N$. But for
every $z\in\{-,+\}^\N$ the choice $a_n^{z(n)}\in \mathcal C_n$ has
$x\in \bigcap_n a_n^{z(n)}\neq \emptyset$ for $x_i =
z(i)\frac{1}{2i}$.


TITLE: Question About Buoyant Force and Weight of an Object
QUESTION [0 upvotes]: Looking at the relationship between buoyant force and the weight of an object, I found this explanation, which makes sense

"If the buoyant force is greater than the object's weight, the object will rise to the surface and float. If the buoyant force is less than the object's weight, the object will sink. If the buoyant force equals the object's weight, the object will remain suspended at that depth."

BUT if this is true, than why is the buoyant force = weight of an object if the object is floating? Wouldn't the buoyant force be greater than the weight of the object?

REPLY [0 votes]: First of all, let's explain how has the buoyant force is made:
As the pressure in deep parts is bigger than the higher parts, we can say that the forces in deep parts of fluid are bigger than the higher parts of it. so we have the force difference in higher parts and the deep parts of the fluid. so there will be a net force upwards.

$F_2 -F_2$=$\;$ $P_2A-P_1A$=$\;$ $\rho_2gh_2A$-$\rho_1gh_1A$= $\;$ $F_b=Ag(\rho_ {fliud}h_2-\rho_{fliud}h_1)$=  $\;$ $F_b=\rho_{fliud}V_{object}g $
we have three kinds of buoyancy:

1-Positive Buoyancy:
Positive buoyancy occurs when an object is lighter than the fluid it displaces. The object will float because the buoyant force is greater than the object’s weight.
$F_b=mg$ $\;$ so $\;$ $\rho_{fliud}V_{object}g $=$\rho_{object}V_{object}g$ $\;$ and $\;$
$\rho_{fliud}V_{objectin}$=$\rho_{object}V_{object}$ $\;$ means that $\;$ $\rho_{fluid}$>$\rho_{object}$
2-Negative Buoyancy:
Negative buoyancy occurs when an object is denser than the fluid it displaces. The object will sink because its weight is greater than the buoyant force.
$F_b<mg$ $\;$ so $\;$ $\rho_{fliud}V_{object}g $=$\rho_{object}V_{object}g$ $\;$ and $\;$
$\rho_{fliud}$<$\rho_{object}$
3-Neutral Buoyancy:
Neutral buoyancy occurs when an object’s weight is equal to the fluid it displaces.
$F_b=mg$ $\;$ so $\;$ $\rho_{fliud}V_{object}g $=$\rho_{object}V_{object}g$ $\;$ and $\;$
$\rho_{fliud}$=$\rho_{object}$
hope this help!


TITLE: Mean fo a geometric using Conditional expection
QUESTION [1 upvotes]: I am trying to understand the derivation of the mean of a geometric distribution from the example in Bertsekas' "INTRO. TO PROB.".

He uses conditional expecations, $$E(x|A_i)$$ to find the $$E(x)$$
I don't undersand the above argument. Are they using any independece/memorylessness concept? If I know that the first n tries are failures. then the remaining tries follow still follow the same geometric distribution?
$$ E[x|x>n] = n + E[x] $$

REPLY [0 votes]: Yes, they are using memorylessness property, though the exact proof can be checked as follows;
$$ \mathbb E(X|X>1) = \sum_{k=0}^\infty \mathbb P(X > k |X > 1) =2+ \sum_{k=2}^\infty \mathbb P(X > k |X > 1) = 2+\sum_{k=2}^\infty \mathbb P(X > k-1).$$
Doing the change of variables, $k-1= u$:
$$ \mathbb E(X|X>1) = 2+\sum_{u=1}^\infty \mathbb P(X > u) = 2+\sum_{u=0}^\infty \mathbb P(X > u) - \mathbb P(X\geq 0) = 1 + \mathbb E(X).$$
Where i used the fact that $\mathbb P(X \geq 0) =1$ and $\mathbb E(X) = \sum_{n=0}^\infty\mathbb P(X>n)$ for a non-negative random variable.


TITLE: Prove that operator L on $M_n(\mathbb F)$ is diagonalizable
QUESTION [1 upvotes]: I was trying to slove the following problem:
"Let $M_n(\mathbb F)$ be the vector space of $n\times n$ matrices over a filed $\mathbb{F}$ and let $A\in M_n(\mathbb{F})$ be an invertible matrix and diagonalizable over $\mathbb{F}$. Consider the linear operator $L:M_n(\mathbb F)\rightarrow M_n(\mathbb F)$ defined by $L(M)=A^{-1}MA$. Prove that the operator $L$ is diagonalizable and express its eigenvalues and their multiplicities in terms of those of A."
The facts which I know about diagonalizable matrices are:
(1) one can find a basis consisting of its eigenvectors;
(2) its minimal polynomial is in the form of $p(x)=(x-\lambda_1)(x-\lambda_2)\cdot\cdot\cdot (x-\lambda_k)$. But I don't know where to start to solve this problem. Hints are preferred.

Thanks for all the help! I'll try to complete it here:
Let's suppose $L(B)=\lambda B$, this means $\lambda B=A^{-1}BA$, or $\lambda AB=BA$. $A$ is diagonalizable so there exists an invertible matrix $P$ such that $A=P^{-1}DP$, where $D$ is diagonal. Then we have,
$$\lambda P^{-1}DPB=BP^{-1}DP \iff \lambda DPBP^{-1}=PBP^{-1}D \iff \lambda DC=CD,$$
where $C=PBP^{-1}$.
Now we aim to understand how $C$ "looks like":
Let $D_{ii}=d_i$ and $D_{ij}=0$. Then we have following,
$$(DC)_{ij}=\sum_{k=1}^n D_{ik}C_{kj} = D_{ii}C_{ij}=d_i C_{ij},$$
$$(CD)_{ij}=\sum_{k=1}^n C_{ik}D_{kj} = C_{ij}D_{jj}=d_j C_{ij}.$$
This implies we have $\lambda d_iC_{ij}= d_jC_{ij}$, or $C_{ij}(\lambda d_i - d_j)=0$.

REPLY [3 votes]: Hint: If $u$ is an eigenvector of $A$ and $v$ an eigenvector of $A^T$, then $uv^T$ is an eigen"vector" of $L$.
Alternate Hint: Consider first the case in which $A$ is diagonal. Then, try to find the new eigenvectors if $A=PDP^{-1}$.
Alternate Hint: The matrix of this transformation with respect to the lexicographical basis $\{e_i e_j^T\}$ is $A^{-1}\otimes A$, where $\otimes$ denotes the Kronecker product.