STAT110 - Probability Theory & Random Variables Flashcards

Question 1

Bayes' Theorem

Accepted Answer

Formula: $P(A|B) = \frac{P(B|A)P(A)}{P(B)}$. Updates the prior probability $P(A)$ using evidence/likelihood $P(B|A)$ to obtain the posterior probability $P(A|B)$.

Question 2

Law of Total Probability (LTP)

Accepted Answer

For a partition $A_1, A_2, \dots, A_n$ of sample space $S$: $P(B) = \sum_{i=1}^n P(B|A_i)P(A_i)$. Allows calculating an unconditional probability by conditioning on disjoint events.

Question 3

Expectation ($E[X]$) & Linearity of Expectation

Accepted Answer

Expected value: $E[X] = \sum x P(X=x)$ (discrete) or $\int_{-\infty}^{\infty} x f(x) dx$ (continuous). Linearity states $E[aX + bY + c] = aE[X] + bE[Y] + c$, which holds **regardless** of whether $X$ and $Y$ are independent.

Question 4

Law of the Unconscious Statistician (LOTUS)

Accepted Answer

Calculates the expected value of a function of $X$ without finding the PDF/PMF of $g(X)$ directly:
- Discrete: $E[g(X)] = \sum g(x) P(X=x)$
- Continuous: $E[g(X)] = \int_{-\infty}^{\infty} g(x) f_X(x) dx$

Question 5

Variance & Covariance Definitions

Accepted Answer

- Variance: $Var(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2$
- Covariance: $Cov(X, Y) = E[XY] - E[X]E[Y]$
- Property: $Var(X + Y) = Var(X) + Var(Y) + 2Cov(X, Y)$. If $X, Y$ are independent, $Cov(X,Y) = 0$.

Question 6

Binomial Distribution

Accepted Answer

$X \sim 	ext{Bin}(n, p)$ models the number of successes in $n$ independent Bernoulli trials with success probability $p$.
- PMF: $P(X=k) = \binom{n}{k} p^k (1-p)^{n-k}$
- Expectation: $E[X] = np$
- Variance: $Var(X) = np(1-p)$

Question 7

Poisson Distribution

Accepted Answer

$X \sim 	ext{Pois}(\lambda)$ models event counts in a fixed interval with average rate $\lambda$.
- PMF: $P(X=k) = \frac{e^{-\lambda} \lambda^k}{k!}$ for $k \in \{0, 1, 2, \dots\}$
- Expectation: $E[X] = \lambda$
- Variance: $Var(X) = \lambda$

Question 8

Geometric Distribution & Memorylessness

Accepted Answer

$X \sim 	ext{Geom}(p)$ models the number of failures before the 1st success (or total trials $k$).
- PMF (for trial $k$): $P(X=k) = (1-p)^{k-1}p$
- Expectation: $E[X] = \frac{1}{p}$
- Memoryless Property: $P(X > s + t \mid X > s) = P(X > t)$

Question 9

Hypergeometric Distribution

Accepted Answer

$X \sim 	ext{HGeom}(w, b, n)$ counts successes in $n$ draws **without replacement** from $w$ success and $b$ failure items.
- PMF: $P(X=k) = \frac{\binom{w}{k}\binom{b}{n-k}}{\binom{w+b}{n}}$
- Expectation: $E[X] = n \cdot \frac{w}{w+b}$

Question 10

Exponential Distribution

Accepted Answer

Continuous analog of the Geometric distribution: $X \sim 	ext{Expo}(\lambda)$.
- PDF: $f(x) = \lambda e^{-\lambda x}$ for $x \ge 0$
- Expectation: $E[X] = \frac{1}{\lambda}$
- Variance: $Var(X) = \frac{1}{\lambda^2}$
- Memoryless property: $P(X > s + t \mid X > s) = P(X > t)$

Question 11

Gaussian (Normal) Distribution

Accepted Answer

$X \sim N(\mu, \sigma^2)$ continuous distribution.
- PDF: $f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x-\mu)^2}{2\sigma^2}}$
- Expectation: $E[X] = \mu$
- Variance: $Var(X) = \sigma^2$
- Standard Normal $Z \sim N(0,1)$ has PDF $\phi(z) = \frac{1}{\sqrt{2\pi}} e^{-z^2/2}$

Question 12

Cumulative Distribution Function (CDF)

Accepted Answer

$F_X(x) = P(X \le x)$. Properties:
1. Non-decreasing and right-continuous
2. $\lim_{x 	o -\infty} F(x) = 0$ and $\lim_{x 	o \infty} F(x) = 1$
3. Derivative relation: $f_X(x) = \frac{d}{dx} F_X(x)$ (for continuous $X$)

Question 13

Moment Generating Function (MGF)

Accepted Answer

Defined as $M_X(t) = E[e^{tX}]$.
- Generates raw moments via derivatives: $E[X^k] = M_X^{(k)}(0)$
- MGF of sum of independent variables: $M_{X+Y}(t) = M_X(t) M_Y(t)$
- Uniquely determines the probability distribution.

Question 14

Joint, Marginal, and Conditional Distributions

Accepted Answer

- Joint PDF: $f_{X,Y}(x,y)$
- Marginal PDF: $f_X(x) = \int_{-\infty}^{\infty} f_{X,Y}(x,y) dy$
- Conditional PDF: $f_{Y|X}(y|x) = \frac{f_{X,Y}(x,y)}{f_X(x)}$
- Independence: $X \perp Y \iff f_{X,Y}(x,y) = f_X(x) f_Y(y)$

Question 15

Markov's Inequality

Accepted Answer

For any non-negative random variable $X \ge 0$ and constant $a > 0$:
$$P(X \ge a) \le \frac{E[X]}{a}$$
Provides an upper bound on tail probabilities using only the expectation.

Question 16

Chebyshev's Inequality

Accepted Answer

For any random variable $X$ with mean $\mu$ and variance $\sigma^2$, and $k > 0$:
$$P(|X - \mu| \ge k\sigma) \le \frac{1}{k^2}$$
Bounds probability that $X$ deviates from its mean by $k$ standard deviations.

Question 17

Law of Large Numbers (LLN)

Accepted Answer

For i.i.d. random variables $X_1, X_2, \dots, X_n$ with mean $\mu$:
Sample mean $\bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i 	o \mu$ as $n 	o \infty$.
- **Weak LLN**: Converges in probability.
- **Strong LLN**: Converges almost surely.

Question 18

Central Limit Theorem (CLT)

Accepted Answer

For i.i.d. random variables $X_1, X_2, \dots, X_n$ with mean $\mu$ and finite variance $\sigma^2$:
$$\frac{\bar{X}_n - \mu}{\sigma / \sqrt{n}} \xrightarrow{d} N(0, 1) \quad 	ext{as } n 	o \infty$$
The sample mean standardizes to a Standard Normal distribution regardless of the underlying distribution shape.

Question 19

Law of Total Expectation (Adam's Law)

Accepted Answer

$$E[E[X|Y]] = E[X]$$
The expected value of the conditional expectation $E[X|Y]$ equals the overall expected value $X$. Allows finding $E[X]$ by conditioning on $Y$.

Question 20

Law of Total Variance (Eve's Law)

Accepted Answer

$$Var(X) = E[Var(X|Y)] + Var(E[X|Y])$$
States that total variance of $X$ equals the expected conditional variance plus the variance of the conditional expectation ("EVVE").

Question 21

Poisson Process

Accepted Answer

A continuous-time stochastic process for counting arrivals occurring at constant rate $\lambda$.
1. Number of arrivals in interval length $t$ is $	ext{Pois}(\lambda t)$.
2. Inter-arrival times are independent $	ext{Expo}(\lambda)$ random variables.

Question 22

Beta-Binomial Conjugacy

Accepted Answer

If prior distribution is $p \sim 	ext{Beta}(a, b)$ and likelihood is $X|p \sim 	ext{Bin}(n, p)$, the posterior distribution is:
$$p | X=k \sim 	ext{Beta}(a + k, b + n - k)$$
This conjugate prior relationship simplifies Bayesian updating.

STAT110 - Probability Theory & Random Variables

Card Preview

Related Flashcard Sets

ECON201 - Intermediate Microeconomics & Game Theory

Rust Systems Programming & Ownership Model

NEUR101 - Neurobiology & Action Potentials

French A1/A2 - High-Frequency Vocabulary & Essential Grammar