Harvard CS109 - Data Science & Applied Probability Flashcards

Question 1

Beta Distribution

Accepted Answer

A continuous probability distribution defined on $[0, 1]$ parameterized by $\alpha$ and $\beta$, commonly used as a prior for binomial probabilities:
$$f(p; \alpha, \beta) = \frac{1}{B(\alpha, \beta)} p^{\alpha-1} (1-p)^{\beta-1}$$
Mean: $E[p] = \frac{\alpha}{\alpha + \beta}$.

Question 2

Bayes' Theorem

Accepted Answer

A fundamental theorem in probability that updates the probability of a hypothesis $A$ given evidence $B$:
$$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$
Where $P(A)$ is the prior, $P(B|A)$ is the likelihood, $P(B)$ is the marginal probability, and $P(A|B)$ is the posterior probability.

Question 3

Probability Mass Function (PMF) vs. Probability Density Function (PDF)

Accepted Answer

• **PMF**: For discrete random variables, yields $P(X = x)$, where $\sum P(X = x) = 1$.
• **PDF**: For continuous random variables, yields the probability density $f(x)$ at $x$. Probability over an interval $[a, b]$ is $\int_a^b f(x) dx = 1$.

Question 4

Cumulative Distribution Function (CDF)

Accepted Answer

The probability that a random variable $X$ takes a value less than or equal to $x$:
$$F_X(x) = P(X \le x)$$
For continuous variables, $F_X(x) = \int_{-\infty}^x f(t) dt$, and $\frac{d}{dx} F_X(x) = f(x)$.

Question 5

Poisson Distribution

Accepted Answer

Models the count of independent events occurring in a fixed interval of time or space at a constant rate $\lambda > 0$:
$$P(X = k) = \frac{\lambda^k e^{-\lambda}}{k!}$$
Key properties: $E[X] = \lambda$ and $Var(X) = \lambda$.

Question 6

Normal (Gaussian) Distribution

Accepted Answer

A continuous symmetric distribution defined by mean $\mu$ and variance $\sigma^2$:
$$f(x) = \frac{1}{\sigma\sqrt{2\pi}} e^{-\frac{1}{2}\left(\frac{x-\mu}{\sigma}\ight)^2}$$
Under standard normal $Z \sim \mathcal{N}(0, 1)$, $Z = \frac{X - \mu}{\sigma}$.

Question 7

Central Limit Theorem (CLT)

Accepted Answer

States that the distribution of the sample mean $\bar{X}_n$ of $n$ independent and identically distributed (i.i.d.) random variables with mean $\mu$ and variance $\sigma^2$ approaches a normal distribution as $n \	o \infty$:
$$\bar{X}_n \sim \mathcal{N}\left(\mu, \frac{\sigma^2}{n}\ight)$$

Question 8

Law of Large Numbers (LLN)

Accepted Answer

States that as the number of independent trials $n$ increases, the sample mean $\bar{X}_n$ converges toward the true population expectation $E[X]$:
$$\lim_{n \	o \infty} P(|\bar{X}_n - \mu| < \epsilon) = 1$$
(Weak Law of Large Numbers).

Question 9

Maximum Likelihood Estimation (MLE)

Accepted Answer

A technique for estimating parameters $\	heta$ by maximizing the likelihood function $\mathcal{L}(\	heta; x) = \prod_{i=1}^n f(x_i | \	heta)$. In practice, we maximize the log-likelihood:
$$\hat{\	heta}_{MLE} = \arg\max_{\	heta} \sum_{i=1}^n \ln f(x_i | \	heta)$$

Question 10

Why perform Log-Likelihood optimization?

Accepted Answer

1. **Numerical Stability**: Converts products of small probabilities into sums, preventing floating-point underflow.
2. **Mathematical Ease**: Simplifies derivatives when applying calculus or gradient descent optimization algorithms.

Question 11

p-value Definition

Accepted Answer

The probability of obtaining a test statistic at least as extreme as the observed result, assuming the null hypothesis ($H_0$) is strictly true. It measures compatibility between observed data and $H_0$.

Question 12

Decision Boundary & Significance Level ($\alpha$)

Accepted Answer

The threshold set before testing (commonly $\alpha = 0.05$). 
• If $p \le \alpha$: Reject $H_0$ (statistically significant).
• If $p > \alpha$: Fail to reject $H_0$.

Question 13

Type I vs. Type II Errors

Accepted Answer

• **Type I Error ($\alpha$)**: False Positive. Rejecting $H_0$ when $H_0$ is actually true.
• **Type II Error ($\beta$)**: False Negative. Failing to reject $H_0$ when $H_0$ is false.
• **Statistical Power**: $1 - \beta$, the probability of correctly rejecting a false $H_0$.

Question 14

Confidence Interval (CI)

Accepted Answer

An interval estimate $[a, b]$ computed from sample data. A $95\%$ confidence level means that if the experiment were repeated indefinitely, $95\%$ of the calculated intervals would contain the true parameter $\	heta$.

Question 15

Components of Bayesian Inference

Accepted Answer

• **Prior $P(\	heta)$**: Belief about parameter $\	heta$ before observing data.
• **Likelihood $P(D|\	heta)$**: Probability of observing data $D$ given $\	heta$.
• **Posterior $P(\	heta|D)$**: Updated belief about $\	heta$ after observing data $D$.
• **Evidence / Marginal $P(D)$**: Standardizing constant $\int P(D|\	heta)P(\	heta)d\	heta$.

Question 16

Conjugate Prior

Accepted Answer

A prior distribution $P(\	heta)$ is conjugate to the likelihood $P(D|\	heta)$ if the resulting posterior distribution $P(\	heta|D)$ belongs to the exact same probability family as the prior.
*Example*: Beta prior + Binomial likelihood = Beta posterior.

Question 17

Non-parametric Bootstrap

Accepted Answer

A computational resampling method to estimate the sampling distribution of a statistic by repeatedly drawing samples of size $n$ **with replacement** from the observed sample data of size $n$.

Question 18

Naive Bayes Classifier

Accepted Answer

A probabilistic classification model based on Bayes' Theorem under the strong ('naive') assumption that features $X_1, ..., X_n$ are conditionally independent given class $Y$:
$$P(Y|X_1, ..., X_n) \propto P(Y) \prod_{i=1}^n P(X_i | Y)$$

Question 19

Expectation and Variance Properties

Accepted Answer

For constants $a, b$ and random variables $X, Y$:
• $E[aX + b] = a E[X] + b$
• $Var(aX + b) = a^2 Var(X)$
• $Var(X) = E[X^2] - (E[X])^2$
• If $X, Y$ are independent: $Var(X + Y) = Var(X) + Var(Y)$.

Question 20

Binomial Distribution

Accepted Answer

Models the number of successes $k$ in $n$ independent Bernoulli trials with probability $p$:
$$P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}$$
Properties: $E[X] = np$ and $Var(X) = np(1-p)$.

Harvard CS109 - Data Science & Applied Probability

Cards in this deck

Related Flashcard Sets

JLPT N2 - Upper-Intermediate Japanese Kanji, Grammar & Expressions

CompTIA Network+ (N10-008) - Networking Fundamentals & Subnetting

HSK 4 - Intermediate Chinese Vocabulary & Grammar

CompTIA Network+ (N10-008) - Networking Fundamentals & Protocols