TITLE: Using Induction to Prove a Sequence is Strictly Increasing
QUESTION [4 upvotes]: Let $x_1=0 $ and for all n, $x_{n+1}$=$\sqrt{2+x_n}$
a)Prove: $x_n <2$
b)Prove: $x_{n+1} > x_n$

I made it through part a painlessly, however, b is giving me problems
Part a: If $ x_k<2 $ then $ x_{k+1}<2$
Attempt: $x_k+2<2+2$
$\sqrt{x_k+2}<\sqrt{2+2}$
$\sqrt{x_k+2}<2$
$x_{k+1}<2$ substitution from initial statement

Part b: Prove: $x_{n+1} > x_n$
Scratch work: $x_{k+1}$=$\sqrt{2+x_k}$ using algebra
$(x_{k+1})^2-2=x_k$

REPLY [1 votes]: You can get parts (a) and (b) with a single induction.
By definition $x_1=0$ and $x_2=\sqrt{2+x_1}=\sqrt2$, so $0\le x_1<x_2<2$. Now assume that $0\le x_n<x_{n+1}<2$ for some $n\in\mathbb{Z}^+$. Then $$2<2+x_n<2+x_{n+1}<4\;,$$ so $$\sqrt2<\sqrt{2+x_n}<\sqrt{2+x_{n+1}}<2\;.\tag{1}$$ But $\sqrt{2+x_n}=x_{n+1}$, and $\sqrt{2+x_{n+1}}=x_{n+2}$, so $(1)$ immediately implies that $$0\le x_{n+1}<x_{n+2}<2\;,$$ and it follows by induction that $0\le x_n<x_{n+1}<2$ for all $n\in\mathbb{Z}^+$.


TITLE: Root mean square of a translated sine wave
QUESTION [1 upvotes]: The sine wave
$${\displaystyle y=A_{1}\sin(\omega t)\,}$$
has a root mean square value of
$$rms = \frac{A_1}{\sqrt{2}}$$
Does this change if the wave is translated up? For example,
$${\displaystyle y=A_{1}\sin(\omega t)\,+20}$$
What would the $rms$ be then?

REPLY [1 votes]: When we look at a generalized problem, find the RMS of (where $\text{K}_1$, $\text{K}_2$ and $\varphi$ are constants):
$$f(t)=\text{K}_1\cdot\cos\left(\frac{2\pi t}{\text{T}}+\varphi\right)+\text{K}_2$$
We get:
$$\text{RMS}\left[f(t)\right]=\sqrt{\frac{1}{\text{T}}\int_0^\text{T}\left\{\text{K}_1\cdot\cos\left(\frac{2\pi t}{\text{T}}+\varphi\right)+\text{K}_2\right\}^2\space\text{d}t}=\sqrt{\frac{\text{K}_1^2}{2}+\text{K}_2^2}$$


TITLE: Relation between chain rule and implicit differentiation derivation in multi variable calculus
QUESTION [1 upvotes]: So my question is on the derivation of the implicit differentiation (taken from here).
The general chain rule, from here, it says that if we have a function $z$ of $n$ variables, $x_1, x_2,\ldots,x_n$ and each of these variables are in turn a function of $m$ variables, $t_1, t_2,\ldots, t_m$.  Then for any $t_i, i=1, 2, \ldots, m$ we have (1)
$$
\frac{\partial z}{\partial t_i}=\frac{\partial z}{\partial x_1}\frac{\partial x_1}{\partial t_i}+\frac{\partial z}{\partial x_2}\frac{\partial x_2}{\partial t_i}+\cdots+\frac{\partial z}{\partial x_n}\frac{\partial x_n}{\partial t_i}$$
My question is how does did the differentiation of $F$ with respect to $x$ come up as it did.  How did it come up as (2) $$\frac{\partial F}{\partial x}\frac{\partial x}{\partial x}+\frac{\partial F}{\partial y}\frac{\partial y}{\partial x}+\cdots+\frac{\partial F}{\partial z}\frac{\partial z}{\partial x}=0$$
If I match (2) with (1), then it seems that the left hand side is $\frac{\partial F}{\partial x}$ because $F=z, x=x_1, y=x_2, z=x_3$.  As far as the initial condition of $F$ having three variables $(x, y, z)$ that in turn are each supposed to be functions of more variables, can it be assumed that $x$ and $y$ are constants; this would be mean effectively that $x=g(t_1, t_2, t_3)=x$ of some function g and $y=y(t_1, t_2, t_3)=y$.  So is the left hand side of (2) $\frac{\partial F}{\partial x}$ or some different notation.  What does the differentiation of $F$ with respect to $x$ on the left hand side look like in symbols then?

REPLY [2 votes]: When god created the world  there were three independent quantities $x$, $y$, $z$ changing with time, say. Then a physicist came and said: These three quantities are related by a so-and-so equation $$F(x,y,z)=0\ .\tag{1}$$
At any rate, at some instant $t_0$ of time one indeed had $F(x_0,y_0,z_0)=0$. The equation $(1)$ connecting the three quantities $x$, $y$, $z$ reduces the degrees of freedom by $1$. Therefore we have the feeling that, given two values $x$ and $y$, the value of $z$ should be determined by $(1)$, at least in the neighborhood of the point $P_0=(x_0,y_0,z_0)$. This means that there should be a function $z=\psi(x,y)$ defined in a neighborhood $U$ of $(x_0,y_0)$ and giving $\psi(x_0,y_0)=z_0$, such that
$$F\bigl(x,y, \psi(x,y)\bigr)=0\qquad\forall (x,y)\in U\ .\tag{2}$$
Sometimes we can explicitly solve $(1)$ for $z$, and then we have this $\psi$ as a a function of the variables $x$ and $y$ "in finite terms". But in most cases we can't. That's where the implicit function theorem comes in. It says that such a function $z=\psi(x,y)$ defined in the neighborhood of $(x_0,y_0)$ indeed exists, and that this $\psi$ is even differentiable.  In particular, we are able to compute the partial derivatives $\psi_x(x_0,y_0)$ and $\psi_y(x_0,y_0)$ from the given data. The chain rule allows to argue as follows: From $(2)$ we obtain by differentiating with respect to $x$ and putting $(x,y):=(x_0,y_0)$ afterwards:
$$F_{.1}(x_0,y_0,z_0)\cdot 1+F_{.3}(x_0,y_0,z_0)\psi_x(x_0,y_0)=0\ .$$
This gives
$$\psi_x(x_0,y_0)=-{F_{.1}(x_0,y_0,z_0)\over F_{.3}(x_0,y_0,z_0)}\ ,\tag{3}$$
under the "technical assumption" $$F_{.3}(x_0,y_0,z_0)\ne0\ .\tag{4}$$ The formula $(3)$ is usually written in the shorthand version
$${\partial z\over\partial x}\biggr|_{(x_0,y_0)}=-{F_x\over F_z}\biggr|_{(x_0,y_0,z_0)}\ .$$
Similarly, one has
$${\partial z\over\partial y}\biggr|_{(x_0,y_0)}=-{F_y\over F_z}\biggr|_{(x_0,y_0,z_0)}\ .$$
Note that the assumption  $(4)$ is essential for all this to hold.


TITLE: Expectation of dropout
QUESTION [0 upvotes]: I am new to expectation and could definitely use some resources on how to use them with respect to deep learning/reinforcement learning. Upon reading .an article on the math behind dropout we come to an expectation of a derivative. I understand how we achieve the derivative but not the expectation, I am looking for some clarity on their approach and possible resources on the matter.
Given:
$$
\frac{\partial E_D}{\partial w_i} = \frac12 (t - \sum_{i=1}^n \delta_iw_iI_i)^2 = -t\delta_iI_i + w_i\delta_i^2I_i^2 + \sum_{j=1,j\ne i}^n\delta_i\delta_jI_iI_jw_i
$$
We want to take the expectation of the gradient which is described as:
$$
\Bbb{E}\left[\frac{\partial E_D}{\partial w_i}\right]
 = -tp_iI_i + w_ip^2_iI_i^2 + w_iVar(\delta_i)I_i^2+\sum_{j=1,j\ne i}^n w_jp_ip_jI_iI_j
$$
Here is the article for reference, I am talking about figure (6) https://towardsdatascience.com/simplified-math-behind-dropout-in-deep-learning-6d50f3f47275

REPLY [1 votes]: $\delta_i$ is by definition Bernoulli($p_i$) and independent across $i$. So, using linearity of expectation $(E[X+Y]=E[X]+E[Y])$:
1) $E[\delta_i]=p_i$,
2) $E[\delta_i^2] = \mbox{Var}(\delta_i^2)+E[\delta_i]^2 = p_i(1-p_i)+p_i^2,$
3) $E[\delta_i\delta_j]=E[\delta_i]E[\delta_j]=p_ip_j$.


TITLE: One confusion over conditional expectation
QUESTION [1 upvotes]: Suppose for two random variables $X_1$ and $X_2$ $\sigma(X_1) = \sigma(X_2)$. Why $E[Y | X_1] = E[Y | X_2]$ a.e. ?
the set where $X_1= X_1(\omega)$ can be different from the set $X_2= X_2(\omega) $. How can then conditional expectation conditioned on the event can be equal ?

REPLY [2 votes]: Indeed, if $\sigma(X_1) = \sigma(X_2)$ then $E[Y | X_1] = E[Y | X_2]$ almost surely, thus $E[Y | X_1] = u_1(X_1)$ and $E[Y | X_2]=u_2(X_2)$ for some measurable functions $u_1$ and $u_2$ such that $u_1(X_1)=u_2(X_2)$ almost surely.
One can then define $E[Y | X_1=x]$ as $u_1(x)$ and $E[Y | X_2=x]$ as $u_2(x)$ for every $x$ in the target set of $X_1$ and $X_2$. This does not entail that $E[Y | X_1=x]$ and $E[Y | X_2=x]$ coincide since, in general, $u_1(x)\ne u_2(x)$.
To sum up, the condition that [$u_1(X_1)=u_2(X_2)$ almost surely] does not imply that [$u_1=u_2$].
Edit: Perhaps a simple example can help. Assume that  $Y=6X_1=3X_2$, hence $X_2=2X_1$. Then $$E[Y | X_1]=E[Y | X_2]=Y=6X_1=3X_2,$$ hence, for every $x$, $E[Y | X_1=x]=6x$ and $E[Y | X_2=x]=3x$. If one selects some $\omega$ in $\Omega$ and one measures $X_1(\omega)=x_1$ and $X_2(\omega)=x_2$ then $x_2=2x_1$ hence $E[Y | X_1=x_1]=6x_1$ and $E[Y | X_2=x_2]=3x_2$, which implies that $$E[Y | X_1=x_1]=E[Y | X_2=x_2].$$
More generally, if $\sigma(X_1)=\sigma(X_2)$, there exists some invertible bimeasurable $v$ such that $X_2=v(X_1)$ almost surely hence
$$
E[Y\mid X_2]=u_2(X_2)=u_2\circ v(X_1)=E[Y\mid X_1],
$$
thus, for almost every $x_1$ (with respect to the distribution of $X_1$),
$$
E[Y\mid X_2=v(x_1)]=u_2\circ v(x_1)=E[Y\mid X_1=x_1].
$$


TITLE: Evaluation of $\lim_{n\rightarrow \infty}\sum^{n}_{r=1}\frac{r}{n^2+n+r}$
QUESTION [5 upvotes]: Evaluation of $$\lim_{n\rightarrow \infty}\sum^{n}_{r=1}\frac{r}{n^2+n+r}$$

$\bf{My\; Try::}$ Let $$L = \lim_{n\rightarrow \infty}\sum^{n}_{r=1}\frac{r}{n^2+n+r} = \lim_{n\rightarrow \infty}\sum^{n}_{r=1}\frac{\frac{r}{n}}{\frac{r^2}{n^2}+\frac{1}{n}+\frac{r}{n^2}}\cdot \frac{1}{n}$$
I want to convert into Reinmann Integral, But it is not possible here.
So How can I solve it
Help me
Thanks

REPLY [4 votes]: Besides Jack's neat answer, a different approach giving more than the desired limit.
One may rewrite your sum with the standard harmonic numbers
$$
\begin{align}
\sum^{n}_{r=1}\frac{r}{n^2+n+r}&=\sum^{n}_{r=1}\frac{n^2+n+r-(n^2+n)}{n^2+n+r}\\\\&=n-(n^2+n)\sum^{n}_{r=1}\frac1{n^2+n+r}\\\\
&=n-(n^2+n)\left(H_{n^2+2n}- H_{n^2+n+1}\right)
\end{align}
$$ then use the asymptotics of harmonic numbers, as $ N \to \infty$,
$$
H_N=\log N+\gamma+\frac1{2N}-\frac1{12N^2}+\mathcal{O}\left(\frac1{N^4} \right)
$$ leading readily to

$$
\sum^{n}_{r=1}\frac{r}{n^2+n+r}=\frac12-\frac1{3n}+\mathcal{O}\left(\frac1{n^2} \right)
$$

as $n \to \infty$.


text:
(* Title:  Weighted_Graph.thy
   Author: Lars Noschinski, TU München
*)

theory Weighted_Graph
imports
  Digraph
  Arc_Walk
  Complex_Main
begin

section \<open>Weighted Graphs\<close>

type_synonym 'b weight_fun = "'b \<Rightarrow> real"

context wf_digraph begin

definition awalk_cost :: "'b weight_fun \<Rightarrow> 'b awalk \<Rightarrow> real" where
  "awalk_cost f es = sum_list (map f es)"

lemma awalk_cost_Nil[simp]: "awalk_cost f [] = 0"
  unfolding awalk_cost_def by simp

lemma awalk_cost_Cons[simp]: "awalk_cost f (x # xs) = f x + awalk_cost  f xs"
  unfolding awalk_cost_def by simp

lemma awalk_cost_append[simp]:
  "awalk_cost f (xs @ ys) = awalk_cost f xs + awalk_cost f ys"
  unfolding awalk_cost_def by simp

end

end

subset_name: curated