A continuous real-valued function $p(x)$ is called a probability density function (pdf) if the following two conditions are satisfied:
- $p(x) \geq 0$ for all $x.$
- $\int_{-\infty}^{\infty}p(x)dx = 1.$
Thus, given such a function $p(x),$ you can model some notion of probability. For example, your model can design (if chosen at random) "the probability that a person's height $x$ is between $5$ feet and $6$ feet" as the quantity $$\int_{x=5}^{x=6} p(x) dx.$$ Of course, you might want to pick $p(x)$ well enough that your probability is not too far from reality. (This is very nontrivial in practice.)
The cumulative distribution function (cdf) of a pdf $p(x)$ is $$P(t) := \int_{x= - \infty}^{x=t}p(x)dx.$$
Exercise. If $P(t)$ is the cdf of a pdf $p(x),$ then show that $P'(t) = p(t).$
Exercise. If $P(t)$ is the cdf of a pdf $p(x),$ then show that
- $P(t)$ is non-decreasing,
- $\lim_{t \rightarrow -\infty}P(t) = 0$,
- $\lim_{t \rightarrow \infty}P(t) = 1$, and
- $\int_{a}^{b}p(x)dx = P(b) - P(a).$
Exercise. Book 8.7: #1, #5, #14, #24.
Given any p.d.f. $p,$ its mean is $$\mu(p) := \int_{-\infty}^{\infty}x p(x) dx,$$ and we will only consider the case when this is a convergent integral.
Remark. For this, we need to ensure that $p(x)$ decays fast enough. For instance, if we let $c = \int_{1}^{\infty}x^{-2} dx$ and define $p(x) = c^{-1}x^{-2}$ for $x \geq 1$ and $0$ elsewhere, then $p(x)$ is a legitimate p.d.f, while we have $$\mu(p) = c^{-1}\int_{1}^{\infty} \frac{1}{x} dx = \infty,$$ which we cannot use in practice.
The standard deviation of $p$ is defined by $$\sigma(p) := \left(\int_{-\infty}^{\infty}(x - \mu(p))^{2}f(x)dx\right)^{1/2}.$$ Again we need to assume that $p(x)$ decays quite fast as it goes away from $0$ to $\infty$ or $-\infty.$
Given any p.d.f. $p,$ its mean is $$\mu(p) := \int_{-\infty}^{\infty}x p(x) dx,$$ and we will only consider the case when this is a convergent integral.
Remark. For this, we need to ensure that $p(x)$ decays fast enough. For instance, if we let $c = \int_{1}^{\infty}x^{-2} dx$ and define $p(x) = c^{-1}x^{-2}$ for $x \geq 1$ and $0$ elsewhere, then $p(x)$ is a legitimate p.d.f, while we have $$\mu(p) = c^{-1}\int_{1}^{\infty} \frac{1}{x} dx = \infty,$$ which we cannot use in practice.
The standard deviation of $p$ is defined by $$\sigma(p) := \left(\int_{-\infty}^{\infty}(x - \mu(p))^{2}f(x)dx\right)^{1/2}.$$ Again we need to assume that $p(x)$ decays quite fast as it goes away from $0$ to $\infty$ or $-\infty.$
Theorem 1. If $p$ is a p.d.f that decays fast enough, we have
$$\sigma(p)^{2} = \int_{-\infty}^{\infty}x^{2}p(x)dx - \mu(p)^{2}.$$ Proof. For convenience, write $\mu = \mu(p).$ By definitions of the standard deviation and the mean of $p$, we have
\begin{align*} \sigma(p)^{2} &= \int_{-\infty}^{\infty}(x - \mu(p))^{2}p(x)dx \\ &= \int_{-\infty}^{\infty}(x^{2} - 2\mu x + \mu^{2})p(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}p(x) dx - 2\mu\int_{-\infty}^{\infty}xp(x)dx + \mu^{2}\int_{-\infty}^{\infty}p(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}p(x) dx - 2\mu\mu + \mu^{2} \cdot 1 \\ &= \int_{-\infty}^{\infty}x^{2}p(x) dx - \mu^{2}. \end{align*}
Note that for the second to the last equality, we have used the condition $$\int_{-\infty}^{\infty}p(x)dx = 1,$$ which follows from the assumption that $p$ is a p.d.f. This finishes the proof.
We will compute the mean and the standard deviation of the following probability distribution function (p.d.f.): $$f_{m,s}(x) = \frac{e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2 \pi}},$$ where $m, s$ are constants and $s > 0.$ It is evident that $f_{m,s}(x) \geq 0$ for all $x$ because the exponentiation always gives us a positive value. However, to show $f_{m,s}(x)$ is a p.d.f., we also need to check that $$\int_{-\infty}^{\infty}f_{m,s}(x)dx = 1,$$ and this is not so obvious.
How to check this in your future. The easiest case is when $s = 1/\sqrt{2}$ and $m = 0$ (which can be done in Math 215 using the change of variables in polar coordinates). Then the substitution method to manipulate the integral will do the job.
Theorem 2. We have $\mu(f_{m,s}) = m.$
Proof. By definition of the mean, we have
\[\mu(f_{m,s}) = \int_{-\infty}^{\infty}xf_{m,s}(x) dx = \int_{-\infty}^{\infty} \frac{xe^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[u = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[du = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x = s\sqrt{2} \cdot u + m.$ Hence, following the computation we have made above, we get
\begin{align*}\mu(f_{m,s}) &= \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}(s\sqrt{2} \cdot u + m)e^{-u^{2}}du \\ &= \frac{m}{\sqrt{\pi}}\int_{-\infty}^{\infty}e^{-u^{2}}du \\ &= m.\end{align*}
Note that the second to the last equality uses that
\[\int_{-\infty}^{\infty}ue^{-u^{2}}du = 0,\]
which is a property for any odd function. The last equality uses the fact that $f_{0,1/\sqrt{2}}$ is a p.d.f.
Proof. By definition of the standard deviation and using Theorem 2 which says $\mu(f_{m,s}) = m$, we have
\[\sigma(f_{m,s})^{2} = \int_{-\infty}^{\infty}(x - m)^{2}f_{m,s}(x)dx = \int_{-\infty}^{\infty} \frac{(x - m)^{2}e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[t = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[dt = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x - m = st\sqrt{2}.$ Hence, following the computation we have made above, we get
\[\sigma(f_{m,s})^{2} = \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}2s^{2}t^{2}e^{-t^{2}}dt = \frac{4s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt,\]
where the last integral uses a property of even functions. Hence, we can just focus on computing the last integral. Since we only need to think about $t \geq 0,$ we can use the substitution
\[u = t^{2},\]
which comes with $du = 2tdt.$ Continuing the computation above, we have
\begin{align*}\sigma(f_{m,s})^{2} &= \frac{4s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt \\ &= \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}u^{1/2}e^{-u}du \\ &= \frac{2s^{2}}{\sqrt{\pi}}\Gamma(3/2), \end{align*}
Upshot: The distribution given by the p.d.f $f_{m,s}$ is called the normal distribution with mean $m$ and standard deviation $s.$
Median. Given a pdf $p(x),$ the median of the distribution given by $p(x)$ is the value $T$ such that $$\int_{-\infty}^{T}p(x) dx = 1/2.$$ Roughly, the median estimates what's in the middle, while the mean means the average.
Exercise. Suppose that $p(x)$ is a pdf that is symmetric to the vertical line $x = a,$ where $a$ is a fixed real number.
$$\sigma(p)^{2} = \int_{-\infty}^{\infty}x^{2}p(x)dx - \mu(p)^{2}.$$ Proof. For convenience, write $\mu = \mu(p).$ By definitions of the standard deviation and the mean of $p$, we have
\begin{align*} \sigma(p)^{2} &= \int_{-\infty}^{\infty}(x - \mu(p))^{2}p(x)dx \\ &= \int_{-\infty}^{\infty}(x^{2} - 2\mu x + \mu^{2})p(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}p(x) dx - 2\mu\int_{-\infty}^{\infty}xp(x)dx + \mu^{2}\int_{-\infty}^{\infty}p(x)dx \\ &= \int_{-\infty}^{\infty}x^{2}p(x) dx - 2\mu\mu + \mu^{2} \cdot 1 \\ &= \int_{-\infty}^{\infty}x^{2}p(x) dx - \mu^{2}. \end{align*}
Note that for the second to the last equality, we have used the condition $$\int_{-\infty}^{\infty}p(x)dx = 1,$$ which follows from the assumption that $p$ is a p.d.f. This finishes the proof.
We will compute the mean and the standard deviation of the following probability distribution function (p.d.f.): $$f_{m,s}(x) = \frac{e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2 \pi}},$$ where $m, s$ are constants and $s > 0.$ It is evident that $f_{m,s}(x) \geq 0$ for all $x$ because the exponentiation always gives us a positive value. However, to show $f_{m,s}(x)$ is a p.d.f., we also need to check that $$\int_{-\infty}^{\infty}f_{m,s}(x)dx = 1,$$ and this is not so obvious.
How to check this in your future. The easiest case is when $s = 1/\sqrt{2}$ and $m = 0$ (which can be done in Math 215 using the change of variables in polar coordinates). Then the substitution method to manipulate the integral will do the job.
Theorem 2. We have $\mu(f_{m,s}) = m.$
Proof. By definition of the mean, we have
\[\mu(f_{m,s}) = \int_{-\infty}^{\infty}xf_{m,s}(x) dx = \int_{-\infty}^{\infty} \frac{xe^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[u = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[du = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x = s\sqrt{2} \cdot u + m.$ Hence, following the computation we have made above, we get
\begin{align*}\mu(f_{m,s}) &= \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}(s\sqrt{2} \cdot u + m)e^{-u^{2}}du \\ &= \frac{m}{\sqrt{\pi}}\int_{-\infty}^{\infty}e^{-u^{2}}du \\ &= m.\end{align*}
Note that the second to the last equality uses that
\[\int_{-\infty}^{\infty}ue^{-u^{2}}du = 0,\]
which is a property for any odd function. The last equality uses the fact that $f_{0,1/\sqrt{2}}$ is a p.d.f.
The following turned out to be more difficult that I thought:
Theorem 3. We have $\sigma(f_{m,s}) = s.$
\[\sigma(f_{m,s})^{2} = \int_{-\infty}^{\infty}(x - m)^{2}f_{m,s}(x)dx = \int_{-\infty}^{\infty} \frac{(x - m)^{2}e^{-(x - m)^{2}/(2s^{2})}}{s\sqrt{2\pi}}dx.\]
Consider the substitution
\[t = \frac{x - m}{s\sqrt{2}},\]
which comes with
\[dt = \frac{dx}{s\sqrt{2}}.\]
Moreover, note that $x - m = st\sqrt{2}.$ Hence, following the computation we have made above, we get
\[\sigma(f_{m,s})^{2} = \frac{1}{\sqrt{\pi}}\int_{-\infty}^{\infty}2s^{2}t^{2}e^{-t^{2}}dt = \frac{4s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt,\]
where the last integral uses a property of even functions. Hence, we can just focus on computing the last integral. Since we only need to think about $t \geq 0,$ we can use the substitution
\[u = t^{2},\]
which comes with $du = 2tdt.$ Continuing the computation above, we have
\begin{align*}\sigma(f_{m,s})^{2} &= \frac{4s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}t^{2}e^{-t^{2}}dt \\ &= \frac{2s^{2}}{\sqrt{\pi}}\int_{0}^{\infty}u^{1/2}e^{-u}du \\ &= \frac{2s^{2}}{\sqrt{\pi}}\Gamma(3/2), \end{align*}
where
\[\Gamma(z) = \int_{0}^{\infty}u^{z-1}e^{-u}du.\]
This is called the Gamma function, which you can read about here. One of the properties in the link says that we have the duplication formula:
\[\Gamma(z)\Gamma(z + 1/2) = 2^{1 - 2z} \sqrt{\pi}\Gamma(2z).\]
It is also not difficult to figure out $\Gamma(n) = (n-1)!$ for all integer $n \geq 1.$ Hence taking $z = 1$ in the duplication formula, we get $$\Gamma(3/2) = 2^{-1}\sqrt{\pi}\Gamma(2) = 2^{-1}\sqrt{\pi}.$$ Combining with the previous computation, we have $$\sigma(f_{m,s})^{2} = \frac{2s^{2}}{\sqrt{\pi}}\Gamma(3/2) = \frac{2s^{2}}{\sqrt{\pi}}2^{-1}\sqrt{\pi} = s^{2}.$$
This implies that $\sigma(f_{m,s}) = s,$ as desired.
Upshot: The distribution given by the p.d.f $f_{m,s}$ is called the normal distribution with mean $m$ and standard deviation $s.$
Median. Given a pdf $p(x),$ the median of the distribution given by $p(x)$ is the value $T$ such that $$\int_{-\infty}^{T}p(x) dx = 1/2.$$ Roughly, the median estimates what's in the middle, while the mean means the average.
Exercise. Suppose that $p(x)$ is a pdf that is symmetric to the vertical line $x = a,$ where $a$ is a fixed real number.
- What is the median given by the distribution?
- What is the mean given by the distribution?
No comments:
Post a Comment