Harvard STAT110 - Introduction to Probability & Random Variables Flashcards

Question 1

Sample Space and Events

Accepted Answer

The **sample space** $\Omega$ is the set of all possible outcomes of a random experiment. An **event** $A$ is a subset of the sample space ($A \subseteq \Omega$).

Question 2

Naive Definition of Probability

Accepted Answer

Defined as:
$$P(A) = \frac{|A|}{|\Omega|}$$
This definition assumes that the sample space $\Omega$ is finite and **all outcomes are equally likely**.

Question 3

Normal (Gaussian) Distribution

Accepted Answer

$X \sim N(\mu, \sigma^2)$: Continuous bell-curve distribution.
- **PDF**: $f(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp\left(-\frac{(x-\mu)^2}{2\sigma^2}ight)$
- **Standardization**: $Z = \frac{X - \mu}{\sigma} \sim N(0, 1)$
- **Mean**: $E[X] = \mu$, **Variance**: $	ext{Var}(X) = \sigma^2$

Question 4

Kolmogorov's Axioms of Probability

Accepted Answer

1. **Non-negativity**: $P(A) \ge 0$ for all events $A$.
2. **Normalization**: $P(\Omega) = 1$.
3. **Countable Additivity**: For any sequence of disjoint (mutually exclusive) events $A_1, A_2, \dots$, $P(\bigcup_{i=1}^{\infty} A_i) = \sum_{i=1}^{\infty} P(A_i)$.

Question 5

Conditional Probability

Accepted Answer

The probability of event $A$ occurring given that event $B$ has occurred, defined for $P(B) > 0$ as:
$$P(A|B) = \frac{P(A \cap B)}{P(B)}$$

Question 6

Law of Total Probability (LTP)

Accepted Answer

If $B_1, B_2, \dots, B_n$ form a partition of the sample space $\Omega$ (mutually exclusive and exhaustive with $P(B_i) > 0$), then for any event $A$:
$$P(A) = \sum_{i=1}^n P(A|B_i)P(B_i)$$

Question 7

Bayes' Theorem

Accepted Answer

Updates the prior probability $P(A)$ after observing evidence $B$:
$$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$
Expanded using LTP:
$$P(A|B) = \frac{P(B|A)P(A)}{\sum_{i} P(B|A_i)P(A_i)}$$

Question 8

Independence of Two Events

Accepted Answer

Events $A$ and $B$ are independent if and only if:
$$P(A \cap B) = P(A)P(B)$$
Equivalently, $P(A|B) = P(A)$ when $P(B) > 0$.

Question 9

PMF vs. PDF

Accepted Answer

- **Probability Mass Function (PMF)** $p_X(x) = P(X = x)$: Used for discrete random variables; outputs true probabilities.
- **Probability Density Function (PDF)** $f_X(x)$: Used for continuous random variables; gives density where $P(a \le X \le b) = \int_{a}^{b} f_X(x) dx$. Note that $P(X = x) = 0$ for any single point $x$.

Question 10

Cumulative Distribution Function (CDF)

Accepted Answer

Defined for any random variable $X$ as $F_X(x) = P(X \le x)$.
Key properties:
1. Non-decreasing ($x_1 \le x_2 \implies F(x_1) \le F(x_2)$)
2. $\lim_{x 	o -\infty} F(x) = 0$ and $\lim_{x 	o \infty} F(x) = 1$
3. Right-continuous

Question 11

Expectation (Expected Value)

Accepted Answer

The long-run weighted average of a random variable $X$:
- **Discrete**: $E[X] = \sum_x x \cdot P(X = x)$
- **Continuous**: $E[X] = \int_{-\infty}^{\infty} x \cdot f_X(x) dx$

Question 12

Linearity of Expectation

Accepted Answer

For any random variables $X$ and $Y$ and constants $a, b, c$:
$$E[aX + bY + c] = aE[X] + bE[Y] + c$$
**Crucial rule**: This property holds **regardless** of whether $X$ and $Y$ are independent.

Question 13

Indicator Variables & Fundamental Bridge

Accepted Answer

An **indicator variable** $I_A$ equals $1$ if event $A$ occurs and $0$ otherwise.
The **Fundamental Bridge** connects expectation to probability:
$$E[I_A] = P(A)$$
This simplifies complex counting and expected value problems by decomposing variables into sums of indicators.

Question 14

Law of the Unconscious Statistician (LOTUS)

Accepted Answer

Computes the expected value of a function of a random variable $g(X)$ without finding the distribution of $g(X)$ directly:
- **Discrete**: $E[g(X)] = \sum_x g(x)P(X=x)$
- **Continuous**: $E[g(X)] = \int_{-\infty}^{\infty} g(x)f_X(x)dx$

Question 15

Variance and Standard Deviation

Accepted Answer

**Variance** measures expected squared spread around the mean:
$$	ext{Var}(X) = E[(X - E[X])^2] = E[X^2] - (E[X])^2$$
**Standard Deviation**: $	ext{SD}(X) = \sqrt{	ext{Var}(X)}$

Question 16

Variance of a Sum & Covariance

Accepted Answer

For any random variables $X$ and $Y$:
$$	ext{Var}(X + Y) = 	ext{Var}(X) + 	ext{Var}(Y) + 2	ext{Cov}(X, Y)$$
where $	ext{Cov}(X,Y) = E[XY] - E[X]E[Y]$. If $X$ and $Y$ are **independent**, $	ext{Cov}(X,Y) = 0$, so $	ext{Var}(X + Y) = 	ext{Var}(X) + 	ext{Var}(Y)$.

Question 17

Binomial Distribution

Accepted Answer

$X \sim 	ext{Bin}(n, p)$: Models the number of successes in $n$ independent Bernoulli($p$) trials.
- **PMF**: $P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$ for $k \in \{0, 1, \dots, n\}$
- **Mean**: $E[X] = np$
- **Variance**: 	ext{Var}(X) = np(1-p)

Question 18

Geometric Distribution

Accepted Answer

$X \sim 	ext{Geom}(p)$: Models the number of failures before the first success in independent Bernoulli($p$) trials.
- **PMF**: $P(X = k) = (1-p)^k p$ for $k \in \{0, 1, 2, \dots\}$
- **Mean**: $E[X] = \frac{1-p}{p}$
- **Memoryless Property**: $P(X \ge s + t \mid X \ge s) = P(X \ge t)$

Question 19

Poisson Distribution

Accepted Answer

$X \sim 	ext{Pois}(\lambda)$: Models the count of rare events occurring at a constant rate $\lambda > 0$.
- **PMF**: $P(X = k) = \frac{e^{-\lambda} \lambda^k}{k!}$ for $k \in \{0, 1, 2, \dots\}$
- **Mean**: $E[X] = \lambda$
- **Variance**: $	ext{Var}(X) = \lambda$

Question 20

Continuous Uniform Distribution

Accepted Answer

$X \sim U(a, b)$: Equal probability density over the interval $[a, b]$.
- **PDF**: $f(x) = \frac{1}{b-a}$ for $x \in [a, b]$
- **Mean**: $E[X] = \frac{a+b}{2}$
- **Variance**: $	ext{Var}(X) = \frac{(b-a)^2}{12}$

Question 21

Exponential Distribution

Accepted Answer

$X \sim 	ext{Expo}(\lambda)$: Models the continuous waiting time between events in a Poisson process with rate parameter $\lambda > 0$.
- **PDF**: $f(x) = \lambda e^{-\lambda x}$ for $x \ge 0$
- **Mean**: $E[X] = \frac{1}{\lambda}$
- **Variance**: $	ext{Var}(X) = \frac{1}{\lambda^2}$
- **Memoryless**: $P(X > s + t \mid X > s) = P(X > t)$

Question 22

Central Limit Theorem (CLT)

Accepted Answer

Let $X_1, X_2, \dots, X_n$ be i.i.d. random variables with mean $\mu$ and finite variance $\sigma^2$. As $n 	o \infty$, the sample mean $\bar{X}_n$ converges in distribution to a Normal distribution:
$$\frac{\bar{X}_n - \mu}{\sigma / \sqrt{n}} \xrightarrow{d} N(0, 1)$$

Harvard STAT110 - Introduction to Probability & Random Variables

Cards in this deck

Related Flashcard Sets

HSK 1 - Basic Chinese Vocabulary, Pinyin & Character Recognition

Functional Programming - Immutability, Pure Functions & Higher-Order Functions

Neuroscience - Neuroanatomy & Brain Function

Microsoft Azure Administrator (AZ-104) - Compute, Storage & Networking