Ch. 3 - Continuous Distributions

Class: STAT-211

Notes:

Outline

• Probability Density Function (pdf)
• Cumulative Distribution Function (cdf)
• Expectation & Variance of Continuous RVs
• Uniform Distribution
• Normal Distribution

Probability Density Function (PDF)

From Discrete to Continuous

Recall that, if $X$ is a discrete random variable taking values 1,2,...,10:

• $P(4\le X\le 6)=f(4)+f(5)+f(6)$
• $P(4<X\le 6)=f(5)+f(6)$
• $P(4<X<6)=f(5)$

It makes sense in this case to:

• Add the values of the pmf as there are ‘only a few values’ $X$ can take within an interval.
  (That is what we mean by discrete)

• Pay attention to the end points of the intervals
  (Does the interval contain its end points?)

For a continuous random variable $X$, it takes on too many values
to ‘add’ a pmf. Besides, the probability that $X$ assumes a specific
value $c$ will be ZERO for any real number $c$.

• We cannot use the pmf here, we need some other kind of function to model a continuous probability distribution
• This is where the probability density function comes in.

Probability Density Function (1)

The probability density function (pdf), $f(·)$, of a continuous random variable $X$ is a smooth, continuous function that defines the probability distribution of $X$, such that

1. $f(x)\ge 0$, for every real number $x$, and
   • Implication: function f is non-negative everywhere.
2. ${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}f(x)dx=1$.
   • Implication: The area under the distribution curve must equal to 1.

If X is a continuous random variable with pdf $f(·)$, then for any
two real numbers a, and b, with a < b,

$P(a<X<b)$ = area under the curve of $f$ between $a$ and $b$

$P(X<a)$ = area under the curve of $f$ to the left of $a$

Pictorially represented:

• Note that by definition, $P(X<=a)$ will be to the left of the point $a$

  • The are under the curve to the left of the point $a$

• Also note that the area under the curve to the right of the point $b$ will be $P(X>b)$.

• The area of the blue region is nothing but $P(a\le X\le b)$

• By definition:

  • $P(X\le a)={\int }_{-\mathrm{\infty }}^{a}f(x)dx$
  • $P(X>b)={\int }_b^{\mathrm{\infty }}f(x)dx$

Facts:

1. $P(X=a)=0=P(X=b)$
2. $P(X\le a)=P(X<a)$
   • {X <= a} = {X < a} $\cup$
     • They are disjoint events
3. $P(X>b)=P(X\ge b)$
4. $P(a\le X\le b)=P(a<X\le b)=P(a\le X<b)=P(a<X<b)$
   • It doesn't matter if we include or not the endpoints of the boundary points
5. $P(a\le X\le b)=P(X\le b)-P(X\le a)$
   • Area under the curve over the intervale $[a,b]$
   • Isn't this are under the curve the same as the area under the curve up to the point $b$ minus the area under the curve up to the point $a$?
     • Yes! this will be used repeatedly.
6. $P(X>b)=1-P(X\le b)$
   • {X > b} =

Probability Density Function (2)

• by definition, by itself, it is not a probability

For a continuous random variable X having pdf f (·), and any pair
of real numbers a and b, with a <b:

1. P(X= a) = 0,
2. P(X ≤a) = P(X <a),
3. P(X ≥a) = P(X >a) = 1−P(X ≤a),
4. P(a ≤X ≤b) = P(a ≤X <b) = P(a <X ≤b) = P(a <X <b),
5. P(a ≤X ≤b) = P(X ≤b)−P(X ≤a).

Fact: The pdf f (·) of a continuous random variable is NOT any probability (unlike the pmf). It may take values greater than 1.

Example
Consider a continuous random variable X with pdf

Then $f(0.1)=5e^{-5(0.1)}=3.0326533>1$.

• Here the density function cannot be a probability

Example

Let the continuous random variable X have pdf

• Find the value of the (normalizing) constant c
  • Note it cannot be negative, and it cannot be 0, since then the density function will be 0 everywhere over the entire line
  • Normalizing constant has to be positive: $c>0$

• Find $P(X\le 0.4)$

• Find $P(0.1\le X<0.7)$

...

Example (2)

Let the continuous random variable X have pdf

• Can this normalizing constant be negative? NO it cannot!

• Can it be 0? NO it cannot!

• Find the value of the (normalizing) constant c.

- Observe that ${\int }_1^{\mathrm{\infty }}f(x)dx=\frac{c}{3}=1⟹c=3.$

• Find $P(X>2)$

• Find $P(1.3\le X<3.5)$

• Find $P(1.3\le X\le 3.5)$

...

Cumulative Distribution Function (CDF)

Cumulative Distribution Function

The cumulative distribution function (CDF) for a continuous random variable X , denoted F (·), is defined as

• Area under the curve up to the point $x$
• By definition, the CDF is a probability.
• Observe the CDF is not decreasing everywhere

From the Fundamental Theorem of Calculus, the PDF $f$ is the derivative of the CDF $F$:

except possibly at a "few points".

If $X$ is a continuous random variable with CDF $F(·)$, then

• $P(X\le a)=F(a)$
• $P(X>a)=1-P(X\le a)=1-F(a)$
• $P(a\le X\le b)=P(X\le b)-P(X\le a)=F(b)-F(a)$

for any two real numbers $a$ and $b$ with $a<b$

Facts:

1. $0\le F(x)\le 1,$
   • For all real no. $x$
2. $F(x)\le F(y)$ for all (x, y) with x < y
3. $\underset{x\to -\mathrm{\infty }}{lim}F(x)=0$
   $\underset{x\to \mathrm{\infty }}{lim}F(x)=1$
4. $F$: Continuous everywhere

When to use PDF and when to use CDF?

• The answer is: what are you trying to find?
• Example: from the density function you can try to evaluate the CDF, then using the CDF you can evaluate requested probabilities

Example

Suppose X is a continuous random variable having PDF

Then, its CDF:

• How did we evaluate the CDF here?

  • Case 1: $x<0⟹f(x)=0$ for all $x<0$

    • $F(x)={\int }_{-\mathrm{\infty }}^{x}f(y)dy=0$

  • Case 2: $0\le X<1$

    • $F(x)={\int }_{-\mathrm{\infty }}^{x}f(y)dy={\int }_0^{x}f(y)dy={\int }_0^{x}3y^{2}dy=x^3$

  • Case 3: $X\ge 1$

    • $F(x)=P(X\le x)={\int }_0^{1}f(y)dy=1$

Some probabilities**:

• $P(X\le 0.4)=F(0.4)=(0.4{)}^3=0.064$
• $P(X>0.6)=1-F(0.6)=1-(0.6{)}^3=0.784$
• $P(0.1\le X<0.7)=F(0.7)-F(0.1)=(0.7{)}^3-(0.1{)}^3=0.342$

Expectation & Variance of Continuous RVs

Expectation of a Continuous RV

Let X be a continuous random variable with p.d.f. f.

Then the expected value (expectation) or, population mean of X , denoted $E[X]$ or µX , is defined as

provided ${\int }_{-\mathrm{\infty }}^{\mathrm{\infty }}|x|f(x)dx$
(In this course, we shall always assume that∞ −∞|x|f (x)dx <∞unless stated otherwise.)

• Integral must be finite for $E(x)$ to exists (in this course you can always assume this)

More generally, if h(X ) is any real-valued function of X with ∞−∞|h(x)|f (x)dx <∞, the expected value of h(·) is defined as

...

(In this course, we shall always assume that∞ −∞|h(x)|f (x)dx <∞unless stated otherwise.)

Variance of a Continuous RV

Let X be a continuous random variable with p.d.f. f . Then the
variance of X , denoted σ2 X , is defined as

...

provided∞ −∞(x−µX )2f (x)dx <∞.
The standard deviation of X , denoted σX , is defined as the positive square root of its variance σ2 X , that is,

...

Note:

...

• Observe that these are population level quantities, not sample level quantities
• Without the knowledge of probability distribution theory it is not possible to get population expectation or population variance.

Fact: $\text{Var}(X)=E(X^2)-(E(X){)}^2\ge 0$

Example

Suppose X is a continuous rv having p.d.f.

Then,

...

Uniform Distribution

Uniform Distribution

• Perharps the simpliest type of a continuous probability distribution

A continuous random variable X is said to have a Uniform(a,b)
distribution over an interval $[a,b]$ if it has the following probability
density function

• Over the interval $[a,b]$, the density function is constant everywhere (doesn't change its values)
  • The density function would be a horizontal line segment

We write, $X\sim \text{Uniform}(a,b)$.

Uniform Distribution - CDF

Suppose, $X\sim \text{Uniform}(a,b)$. Then,

Uniform Distribution - Properties

If X∼Uniform(a,b), then

1. for any pair of points c and d with $a<c<d<b$,

   • $P(X\le c)=(c-a)/(b-a)$ = $F(c)$
   • $P(c\le X\le d)=(d-c)/(b-a)$ = $F(a)-F(c)$
   • $P(X>d)=(b-d)/(b-a)=1-(d-a)/(b-a)$
     • = $1-F(d)$

2. E(X) = (a + b)/2 and SD(X ) = (b−a)/√12, because

...

Uniform Distribution - Example

Let the random variable $X$ denote a number drawn at random from the interval (0,10). Then $X$ has a (continuous) uniform distribution supported over (0,10) with the following PDF:

• Find the probability that X will lie in the interval (3,6). Ans: 0.3
  • $P(3\le X\le 6)$
    • There is no harm in including both of the endpoints or you can exclude them with no harm as well.

• Find the conditional probability that X will be smaller than 4 given that X isn’t larger than 6. Ans: 2/3 ≈0.667.

...

• Find the probability that X = 2.47. Ans: 0

Normal Distribution

Normal Distribution

• The most widely used, -> has very vast applications in statistics and science

The normal or, Gaussian distribution plays a key role in probability
and statistics Write its pdf as:

where

• $-\mathrm{\infty }<x<\mathrm{\infty }$
• $-\mathrm{\infty }<\mu <\mathrm{\infty }$
• $\sigma >0$

(The parameters are $\mu$ and $\sigma$. We will return to these when we talk about expectation)

We write $X\sim N(\mu ,{\sigma }^2)$ to indicate this distribution

• (The cdf, $F(x)$, does not have a closed form expression)
• Note:
  • E(X) = $\mu$
  • Var(X) = ${\sigma }^2$
  • $\mu$: location parameter
  • $\sigma$: scale parameter
• Note:
  • If $X\sim N(\mu ,{\sigma }^2)$, then the distribution of $x-\mu /\sigma$ is independent of $(\mu ,\sigma )$

What is the area under the curve to the right of µ?

• It should be evenly distributed on both sides of µ
  • right area: $P(X>\mu )=0.5$
  • left area = $P(X\le \mu )=0.5$
• mean(X) = median = µ
  • Expected value is the same as its median (for a normal distribution)

Standard Normal Distribution

When $\mu =0$ and $\sigma =1$, we call it the standard normal distribution:

Notation:

• $Z\sim N(0,1)$
• $\varphi (x)$ is the pdf
  • Lowercase phi
  • = $\frac{1}{\sqrt{2\pi }}{e}^{-z^2/2},-\mathrm{\infty }<z<\mathrm{\infty }$
• $\Phi (x)$ is the cdf
  • Capital phi
  • = $P(Z<=z)$

Example:

• Suppose $X\sim N(\mu ,{\sigma }^2)$
• Define: $Z=\frac{X-\mu }{\sigma }$
  • and that $Z\sim N(0,1)$
  • Independent of $(\mu ,\sigma )$
  • Observe that the converse is also true
    • If $Z=\frac{X-\mu }{\sigma }\sim N(0,1)$ then,
      • $X=\mu ...$

Facts:

1. If $X\sim N(\mu ,{\sigma }^2)$ then,
   • CDF of $X$ can be expressed in terms of the cdf of a standard normal distribution
   • $F(x)=P(X\le x)$
     • This is the definition of the CDF of a continuous rv
   • We can write $X\le x⟹X-\mu \le x-\mu$
     • Is the same, you are subtracting the same quantity from both sides
     • The direction of the inequality does not change
   • So we can also write $X\le x⟹\frac{X-\mu }{\sigma }\le \frac{x-\mu }{\sigma }$
   • Which is the same as saying $Z\le \frac{x-\mu }{\sigma }$
   • So we get that:
     • $F(x)=P(Z\le \frac{x-\mu }{\sigma })$
     • $=\varphi (\frac{x-\mu }{\sigma })$
   • Suppose $X\sim N(\mu ,{\sigma }^2)$
     • Then given $0<p<1,x(p)$
     • A point $x(p)$ is called the 100pth percentile (or equivalently the p-th quantile) of $X$ if and only if the area under the curve towards the left of $x(p)$ is $p$ and to the right it is $1-p$
     • Probabilistically speaking we can write
       • Normal Dist.: $P(X\le x(p))=p=1-P(X>x(p))$
       • Standard Normal Dist.: $P(Z\le z(p))=p=1-P(Z>z(p))$
2. If $X\sim N(\mu ,{\sigma }^2)$ then,
   • $x(p)=\mu +\sigma z(p)$
   • If $P(X\le x)$ then,
     • $=P(Z\le \frac{x-\mu }{\sigma })$
     • for any x: real no. in a $X\sim N(\mu ,{\sigma }^2)$.
3. For any real no. $-\mathrm{\infty }<a<b<\mathrm{\infty }$:
   • $P(X\le a)=P(Z\le \frac{x-\mu }{\sigma })$
   • $P(X>a)=P(Z>\frac{b-\mu }{\sigma })=1-P(Z\le \frac{b-\mu }{\sigma })$
   • $P(a\le X\le b)$
     • $=P(X\le b)-P(X\le a)$
     • $=P(Z\le \frac{b-\mu }{\sigma })-P(Z\le \frac{a-\mu }{\sigma })$
     • = phi... (complete)

Example of Standard Normal Distribution

1. $P(Z>2)=P(Z\le -2)$
   • Remember the '=' in <= does not really matter.
   • The area under the curve towards the left side of -2 is the same as the area under the curve towards the right side of 2.
2. $P(Z\le 2)=P(Z>-2)$
3. $P(-2\le Z\le 2)=1-2P(Z>2)$
   • Waht is the total area under the curve? It is 1
   • What is the total area of these two shaded regions
     • It should be twice one of them (since both are the same)
   • Note that often times this probability is expressed by writing: $⟹|z|\le 2$
   • So this means:
     • $P(|z|\le 2)=1-2P(Z>2)$
4. $P(Z<-2\text{ OR }Z>2)$
   • You cannot have a variable that satisfies these two conditions, therefore these two are disjoint sets
   • So we can get it by
     • $=P(Z<-2)+P(Z>2)$
     • $=2P(Z>2)$.
       • which is basically the sum of the two shaded regions

• Note: In these facts, you can replace the real number '$2$' by any arbitrary positive real number, say $z$.

Normal Distribution PDF

• Blue distribution
  • This normal distribution is represented by the blue color solid density curve
  • Note that it is bell shaped and it has its center at 0
  • It has a small amount of variability around its center
    • ${\sigma }^2=0.2$
  • Has a very high peak at $\mu =0$
• Red distribution
  • Bell shaped and symmetric
  • Quite larger variance than the normal distribution
  • There must be some x values that are away from the center with higher probability
  • This distribution has a smaller peak
• Yellow distribution
  • Just like the first two cases, it has a bell shaped symmetric curve around $\mu =0$
  • There should be extreme values on the tails of the distribution because of the large variability
  • The peakiness of the density curve should be smaller than the other 2.
  • If variance parameter is:
    • low -> high peak along the center
    • large -> low peak and more flat along the center
• Green distribution
  • Observe that the density curve is bell shaped around the point µ = -2
  • By changing the value of µ we have shifted the location of this distribution
    • Changing µ you can shift only the center of the distribution, but not its shape.
    • Its location is determined by µ and its shaped is governed by the scale parameter σ.

Normal Distribution CDF

• $F(x)={\int }_{-\mathrm{\infty }}^{x}f(u)du=$ area under $f(x)(={\varphi }_{\mu ,{\sigma }^2}(x))$ up to $x$

• Plot of $F(x) = ${\Phi }_{\mu ,{\sigma }^2}(x)$ as the sigmoid (S - shaped) curve

Example (1)

Suppose $X\sim N(\mu =2,{\sigma }^2=8.41)$. Using the Online Normal Distribution Calculator, obtain:

• Note: σ = 2.9

• $P(X\le 5.69)$

  • $P(X\le 5.69)=P(Z\le \frac{5.69-2}{\sqrt{8.41}})=P(Z\le 1.2724)=0.89838$
  • You should select the $X<x$ option in the calculator

• $P(1.3<X<4.8)$

  • Recall that if $X\sim N(\mu ,{\sigma }^2)$ then,
    • $P(a\le X\le b)$
      • $=P(a<X<b)$
      • $=P(Z\le \frac{b-\mu }{\sigma })-P(Z\le \frac{a-\mu }{\sigma })$
  • $=P(Z\le \frac{4.8-2}{\sqrt{8.41}})-P(Z\le \frac{1.3-2}{\sqrt{8.41}})$
  • $=P(Z\le 0.96552)-P(Z\le -0.24138)$
  • $=0.83286-0.40463=0.42823$

• $P(X>4.25)$

  • Recall that for any real number $b$
    • $P(X>b)=P(Z>\frac{b-\mu }{\sigma })$
  • $P(X>4.25)=P(Z>0.77586)$
  • $=0.21892$ (using normal distribution calculator)

Example (2)

Suppose $X\sim N(\mu =2,{\sigma }^2=8.41)$. Using the Online Normal Distribution Calculator, obtain:

• the 95-th percentil of $X$

  • $=\mu +\sigma$ (95−th percentile of $Z$)

  • $=2+\sqrt{8.41}(1.64485)$

  • $\approx 6.77$

  • The area under the curve towards the left of the point $z(0.95)$ should be 0.95 by definition, and to the right it should be $0.05$.

    • If you choose the $P(X>x)$ option, what area must you specify to get the 95-th percentile of $Z$?
      • You need to insert $0.05$

  • $z(0.95)=1.64485$

  • $x(0.95)=2+2.9\times 1.64485$

• the third quartile of $X$

  • = 75−th percentile of X

  • $=\mu +\sigma$ (75−th percentile of Z )

  • $=2+\sqrt{8.41}(0.67449)$

  • $\approx 3.956$

  • $x(0.75)=\mu +\sigma \times z(0.75)$

    • = $2+2.9\times 0.67449$
    • = $3.956$

  • Note:

    • $IQR(X)=x(0.75)-x(0.25)$
    • Observe that 0.677449 = $z(0.75)$
    • The area under the curve towards the right of $z(0.75)$ is 0.25
    • The area under the curve towards the left of $z(0.25)$ is 0.25 as well
    • Therefore we observe that $z(0.25)=-z(0.75)$
      • $=\mu +\sigma z(0.75)-\mu +\sigma z(0.25)$
      • = $2\sigma z(0.75)$
        • (Does not require knowledge of µ)

Fact:

• $X\sim N(\mu ,{\sigma }^2)$
  • If and only if $a+bXN(a+b\mu ,b^2{\sigma }^2)$
  • for any real a. and any $b\ne 0$