Harvard STAT 110 - Probability & Random Variables Flashcards

Question 1

Naive Definition of Probability

Accepted Answer

If all outcomes in a finite sample space $S$ are equally likely, the probability of an event $A$ is:
$$P(A) = \frac{|A|}{|S|}$$
*Limitation:* Only applies when outcomes are finite and equally likely.

Question 2

Law of Total Variance (Eve's Law)

Accepted Answer

Decomposes variance into expected conditional variance and variance of conditional expectation:
$$	ext{Var}(X) = \mathbb{E}[	ext{Var}(X \mid Y)] + 	ext{Var}(\mathbb{E}[X \mid Y])$$

Question 3

Kolmogorov Axioms of Probability

Accepted Answer

For a sample space $S$ and events $A, A_1, A_2, \dots$:
1. **Non-negativity:** $P(A) \ge 0$
2. **Normalization:** $P(S) = 1$
3. **Countable Additivity:** If $A_1, A_2, \dots$ are disjoint, $P\left(\bigcup_{i=1}^{\infty} A_i\right) = \sum_{i=1}^{\infty} P(A_i)$

Question 4

Conditional Probability

Accepted Answer

The probability of event $A$ occurring given that event $B$ has occurred, provided $P(B) > 0$:
$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$

Question 5

Bayes' Rule

Accepted Answer

Relates the conditional and marginal probabilities of random events:
$$P(A \mid B) = \frac{P(B \mid A)P(A)}{P(B)}$$
Using Law of Total Probability for the denominator:
$$P(A_k \mid B) = \frac{P(B \mid A_k)P(A_k)}{\sum_i P(B \mid A_i)P(A_i)}$$

Question 6

Law of Total Probability (LTP)

Accepted Answer

If $B_1, B_2, \dots, B_n$ form a partition of the sample space $S$ (disjoint and $\bigcup B_i = S$), then for any event $A$:
$$P(A) = \sum_{i=1}^{n} P(A \mid B_i)P(B_i)$$

Question 7

Independence of Events

Accepted Answer

Two events $A$ and $B$ are independent if and only if:
$$P(A \cap B) = P(A)P(B)$$
Equivalently, $P(A \mid B) = P(A)$ when $P(B) > 0$. Pairwise independence does not imply mutual independence.

Question 8

Random Variable (Definition)

Accepted Answer

A **Random Variable (R.V.)** $X$ is a function mapping the sample space $S$ to the real line $\mathbb{R}$:
$$X: S 	o \mathbb{R}$$

Question 9

Cumulative Distribution Function (CDF)

Accepted Answer

The function $F_X(x)$ defined for all $x \in \mathbb{R}$ by:
$$F_X(x) = P(X \le x)$$
**Properties:** Non-decreasing, right-continuous, $\lim_{x 	o -\infty} F(x) = 0$, and $\lim_{x 	o \infty} F(x) = 1$.

Question 10

Expected Value & Linearity of Expectation

Accepted Answer

The mean/weighted average of a random variable. Discrete: $\mathbb{E}[X] = \sum_x x P(X=x)$. Continuous: $\mathbb{E}[X] = \int_{-\infty}^{\infty} x f_X(x) dx$.

**Linearity:** For *any* R.V.s $X$ and $Y$ (independent or not) and constants $a, b$:
$$\mathbb{E}[aX + bY] = a\mathbb{E}[X] + b\mathbb{E}[Y]$$

Question 11

Indicator Random Variable & Fundamental Bridge

Accepted Answer

An indicator variable $I_A$ equals 1 if event $A$ occurs and 0 otherwise.

**Fundamental Bridge:**
$$\mathbb{E}[I_A] = P(A)$$
Extremely useful for finding expected values via the sum of indicators.

Question 12

Variance and Standard Deviation

Accepted Answer

**Variance:** $	ext{Var}(X) = \mathbb{E}[(X - \mathbb{E}[X])^2] = \mathbb{E}[X^2] - (\mathbb{E}[X])^2$
**Properties:**
- $	ext{Var}(aX + b) = a^2 	ext{Var}(X)$
- If $X, Y$ are independent: $	ext{Var}(X + Y) = 	ext{Var}(X) + 	ext{Var}(Y)$

**Standard Deviation:** $\sigma_X = \sqrt{	ext{Var}(X)}$

Question 13

Bernoulli Distribution

Accepted Answer

Model for a single trial with success probability $p$.
- **Notation:** $X \sim 	ext{Bern}(p)$
- **PMF:** $P(X = 1) = p$, $P(X = 0) = 1 - p$
- **Expectation:** $\mathbb{E}[X] = p$
- **Variance:** $	ext{Var}(X) = p(1-p)$

Question 14

Binomial Distribution

Accepted Answer

Number of successes in $n$ independent $	ext{Bern}(p)$ trials.
- **Notation:** $X \sim 	ext{Bin}(n, p)$
- **PMF:** $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$ for $k \in \{0, 1, \dots, n\}$
- **Expectation:** $\mathbb{E}[X] = np$
- **Variance:** $	ext{Var}(X) = np(1-p)$

Question 15

Geometric Distribution & Memoryless Property

Accepted Answer

Number of failures before the first success in independent $	ext{Bern}(p)$ trials.
- **PMF:** $P(X = k) = (1-p)^k p$ for $k \in \{0, 1, 2, \dots\}$
- **Expectation:** $\mathbb{E}[X] = \frac{1-p}{p}$
- **Memoryless Property:** $P(X \ge s + t \mid X \ge s) = P(X \ge t)$

Question 16

Poisson Distribution

Accepted Answer

Models the number of rare events occurring in a fixed time/space interval.
- **Notation:** $X \sim 	ext{Pois}(\lambda)$
- **PMF:** $P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}$ for $k \in \{0, 1, 2, \dots\}$
- **Expectation & Variance:** $\mathbb{E}[X] = \lambda$, $	ext{Var}(X) = \lambda$

Question 17

Uniform Distribution (Continuous)

Accepted Answer

Equally likely outcomes over a continuous interval $[a, b]$.
- **Notation:** $X \sim 	ext{Unif}(a, b)$
- **PDF:** $f(x) = \frac{1}{b-a}$ for $x \in [a, b]$
- **Expectation:** $\mathbb{E}[X] = \frac{a+b}{2}$
- **Variance:** $	ext{Var}(X) = \frac{(b-a)^2}{12}$

Question 18

Normal (Gaussian) Distribution

Accepted Answer

Symmetric bell-shaped continuous distribution.
- **Notation:** $X \sim \mathcal{N}(\mu, \sigma^2)$
- **PDF:** $f(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp\left( -\frac{(x - \mu)^2}{2\sigma^2} \right)$
- **Standard Normal:** $Z = \frac{X - \mu}{\sigma} \sim \mathcal{N}(0, 1)$

Question 19

Exponential Distribution

Accepted Answer

Models waiting time between events in a Poisson process.
- **Notation:** $X \sim 	ext{Expo}(\lambda)$
- **PDF:** $f(x) = \lambda e^{-\lambda x}$ for $x \ge 0$
- **CDF:** $F(x) = 1 - e^{-\lambda x}$
- **Expectation & Variance:** $\mathbb{E}[X] = \frac{1}{\lambda}$, $	ext{Var}(X) = \frac{1}{\lambda^2}$

Question 20

Covariance and Correlation

Accepted Answer

**Covariance:** Measures linear association between $X$ and $Y$.
$$	ext{Cov}(X, Y) = \mathbb{E}[(X - \mathbb{E}[X])(Y - \mathbb{E}[Y])] = \mathbb{E}[XY] - \mathbb{E}[X]\mathbb{E}[Y]$$
**Correlation:** Scale-invariant version $ho(X, Y) = \frac{	ext{Cov}(X, Y)}{\sigma_X \sigma_Y} \in [-1, 1]$.

Question 21

Law of Total Expectation (Adam's Law)

Accepted Answer

The expected value of $X$ can be found by conditioning on $Y$:
$$\mathbb{E}[X] = \mathbb{E}[\mathbb{E}[X \mid Y]]$$
The outer expectation is taken over the distribution of $Y$, while the inner conditional expectation $\mathbb{E}[X \mid Y]$ is a function of $Y$.

Question 22

Law of Large Numbers (LLN)

Accepted Answer

Let $X_1, X_2, \dots, X_n$ be i.i.d. with mean $\mu$. As $n 	o \infty$, the sample mean $\bar{X}_n = \frac{1}{n} \sum_{i=1}^n X_i$ converges to the true mean $\mu$:
$$\bar{X}_n \xrightarrow{p} \mu \quad 	ext{(Weak LLN)}$$

Question 23

Central Limit Theorem (CLT)

Accepted Answer

Let $X_1, X_2, \dots, X_n$ be i.i.d. with mean $\mu$ and finite variance $\sigma^2$. As $n 	o \infty$, the standardized sample mean converges in distribution to standard normal:
$$\frac{\bar{X}_n - \mu}{\sigma / \sqrt{n}} \xrightarrow{d} \mathcal{N}(0, 1)$$

Harvard STAT 110 - Probability & Random Variables

Cards in this deck

Related Flashcard Sets

Harvard STAT110 - Introduction to Probability & Random Variables

STAT110 - Probability Theory & Random Variables

HSK 1 - Basic Chinese Vocabulary, Pinyin & Character Recognition

Functional Programming - Immutability, Pure Functions & Higher-Order Functions