Harvard STAT110 - Introduction to Probability: Random Variables & Distributions Flashcards

Question 1

Law of the Unconscious Statistician (LOTUS)

Accepted Answer

Allows computing the expected value of a function $g(X)$ without finding the distribution of $g(X)$:
- Discrete: $E[g(X)] = \sum_x g(x) P(X=x)$
- Continuous: $E[g(X)] = \int_{-\infty}^\infty g(x) f_X(x) dx$

Question 2

Bayes' Rule

Accepted Answer

Formula used to update conditional probabilities based on new evidence:
$$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$
Expanded via Law of Total Probability: $P(A|B) = \frac{P(B|A)P(A)}{P(B|A)P(A) + P(B|A^c)P(A^c)}$

Question 3

Probability Mass Function (PMF)

Accepted Answer

Function giving the probability that a discrete random variable $X$ equals a specific value $x$:
$$p_X(x) = P(X = x)$$
Must satisfy $p_X(x) \ge 0$ for all $x$ and $\sum_x p_X(x) = 1$.

Question 4

Probability Density Function (PDF)

Accepted Answer

Function $f_X(x)$ for continuous random variable $X$ defining probabilities over intervals:
$$P(a \le X \le b) = \int_a^b f_X(x) dx$$
Must satisfy $f_X(x) \ge 0$ and \int_{-\infty}^\infty f_X(x) dx = 1$. Note: $f_X(x)$ is density, not probability.

Question 5

Cumulative Distribution Function (CDF)

Accepted Answer

Defined for any random variable $X$ as:
$$F_X(x) = P(X \le x)$$
Properties: Non-decreasing, right-continuous, $\lim_{x 	o -\infty} F(x) = 0$, and $\lim_{x 	o \infty} F(x) = 1$.

Question 6

Expectation $E[X]$ & Linearity

Accepted Answer

The long-run weighted average of $X$.
- Discrete: $E[X] = \sum_x x P(X=x)$
- Continuous: $E[X] = \int_{-\infty}^\infty x f_X(x) dx$
**Linearity of Expectation**: $E[aX + bY + c] = aE[X] + bE[Y] + c$, which holds regardless of independence.

Question 7

Variance & Standard Deviation

Accepted Answer

Variance measures the spread of a distribution:
$$	ext{Var}(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2$$
Properties: $	ext{Var}(aX + b) = a^2 	ext{Var}(X)$. Standard Deviation is $\sigma_X = \sqrt{	ext{Var}(X)}$.

Question 8

Indicator Random Variables

Accepted Answer

A binary random variable $I_A$ where $I_A = 1$ if event $A$ occurs and $0$ otherwise.
**Fundamental Bridge**: $E[I_A] = P(A)$.
Extremely useful for breaking complex expectation problems into sums of simple indicators.

Question 9

Binomial Distribution

Accepted Answer

$X \sim 	ext{Bin}(n, p)$ represents the number of successes in $n$ independent Bernoulli trials with probability $p$.
- PMF: $P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}$
- Expectation: $E[X] = np$
- Variance: $	ext{Var}(X) = np(1-p)$

Question 10

Poisson Distribution

Accepted Answer

$X \sim 	ext{Pois}(\lambda)$ models event counts occurring at a constant average rate $\lambda > 0$.
- PMF: $P(X=k) = \frac{e^{-\lambda} \lambda^k}{k!}$ for $k \in \{0, 1, 2, \dots\}$
- Expectation & Variance: $E[X] = \lambda$, $	ext{Var}(X) = \lambda$

Question 11

Poisson Approximation (Law of Rare Events)

Accepted Answer

If $X \sim 	ext{Bin}(n, p)$ with large $n$ and small $p$ such that $n p = \lambda$ is a moderate constant, then $X$ can be closely approximated by $	ext{Pois}(\lambda)$.

Question 12

Geometric Distribution

Accepted Answer

$X \sim 	ext{Geom}(p)$ counts the number of failures before the first success in independent Bernoulli trials.
- PMF: $P(X=k) = (1-p)^k p$ for $k \in \{0, 1, 2, \dots\}$
- Expectation: $E[X] = \frac{1-p}{p}$
- Unique discrete memoryless distribution.

Question 13

Uniform Distribution (Continuous)

Accepted Answer

$X \sim 	ext{Unif}(a, b)$ has equal probability density across $[a, b]$.
- PDF: $f(x) = \frac{1}{b-a}$ for $x \in [a, b]$
- Expectation: $E[X] = \frac{a+b}{2}$
- Variance: 	ext{Var}(X) = \frac{(b-a)^2}{12}

Question 14

Universality of the Uniform

Accepted Answer

If $X$ has a continuous, strictly increasing CDF $F_X$, then $U = F_X(X) \sim 	ext{Unif}(0, 1)$. Conversely, if $U \sim 	ext{Unif}(0, 1)$, then $X = F_X^{-1}(U)$ has CDF $F_X$. Core concept for simulation.

Question 15

Normal (Gaussian) Distribution

Accepted Answer

$X \sim N(\mu, \sigma^2)$ with mean $\mu$ and variance $\sigma^2$.
- PDF: $f(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)$
- Standard Normal: $Z = \frac{X-\mu}{\sigma} \sim N(0, 1)$
- Sum of independent Normal RVs is also Normal.

Question 16

Exponential Distribution

Accepted Answer

$X \sim 	ext{Expo}(\lambda)$ models waiting time between Poisson process events.
- PDF: $f(x) = \lambda e^{-\lambda x}$ for $x \ge 0$
- CDF: $F(x) = 1 - e^{-\lambda x}$
- Mean & Variance: $E[X] = \frac{1}{\lambda}$, $	ext{Var}(X) = \frac{1}{\lambda^2}$
- Memoryless Property: $P(X > s + t \mid X > s) = P(X > t)$

Question 17

Independence of Random Variables

Accepted Answer

RVs $X$ and $Y$ are independent iff $F_{X,Y}(x,y) = F_X(x)F_Y(y)$ for all $x, y$.
If independent:
- $E[XY] = E[X]E[Y]$
- $	ext{Var}(X + Y) = 	ext{Var}(X) + 	ext{Var}(Y)$

Question 18

Markov's Inequality

Accepted Answer

For any non-negative random variable $X \ge 0$ and constant $a > 0$:
$$P(X \ge a) \le \frac{E[X]}{a}$$
Provides an upper bound on tail probabilities given only the mean.

Question 19

Chebyshev's Inequality

Accepted Answer

For any random variable $X$ with mean $\mu$ and variance $\sigma^2$, and $c > 0$:
$$P(|X - \mu| \ge c) \le \frac{	ext{Var}(X)}{c^2}$$
Bounds the probability of deviating from the mean by more than $c$ units.

Question 20

Adam's Law (Law of Total Expectation)

Accepted Answer

Relates overall expectation to conditional expectation:
$$E[X] = E[E[X|Y]]$$
Allows computing $E[X]$ by first conditioning on $Y$ and then taking the expectation over $Y$.

Question 21

Eve's Law (Law of Total Variance)

Accepted Answer

Decomposes total variance into within-group and between-group variance:
$$	ext{Var}(X) = E[	ext{Var}(X|Y)] + 	ext{Var}(E[X|Y])$$

Harvard STAT110 - Introduction to Probability: Random Variables & Distributions

Cards in this deck

Related Flashcard Sets

Harvard STAT110 - Introduction to Probability & Random Variables

HSK 1 - Basic Chinese Vocabulary, Pinyin & Character Recognition

Functional Programming - Immutability, Pure Functions & Higher-Order Functions

Neuroscience - Neuroanatomy & Brain Function