Machine Learning - Supervised Learning & Neural Networks Flashcards

Question 1

Supervised Learning

Accepted Answer

A machine learning paradigm where an algorithm learns a mapping function $f: X \rightarrow Y$ from labeled training data consisting of input-output pairs $(x_i, y_i)$.

Question 2

Linear Regression & MSE

Accepted Answer

A linear model for continuous target variables. Parameters $	heta$ are trained by minimizing the Mean Squared Error (MSE) cost function: $J(	heta) = \frac{1}{2m} \sum_{i=1}^{m} (h_	heta(x^{(i)}) - y^{(i)})^2$.

Question 3

Logistic Regression

Accepted Answer

A classification algorithm that models the probability of a binary output $y \in \{0, 1\}$ using the sigmoid function: $P(y=1|x) = \sigma(	heta^T x) = \frac{1}{1 + e^{-	heta^T x}}$.

Question 4

Support Vector Machine (SVM) & Kernel Trick

Accepted Answer

A supervised model that finds an optimal hyperplane maximizing the decision boundary margin. The **kernel trick** maps non-linearly separable inputs into higher-dimensional feature spaces where they become linearly separable.

Question 5

Decision Trees & Impurity Metrics

Accepted Answer

Non-parametric models that partition feature space recursively. Node split quality is determined by **Gini Impurity** ($1 - \sum p_i^2$) or **Entropy** ($-\sum p_i \log_2 p_i$).

Question 6

Random Forest (Bagging)

Accepted Answer

An ensemble method combining multiple decision trees trained on random bootstrapped subsets of data and feature subsets (**Bootstrap Aggregating**), reducing variance without increasing bias.

Question 7

Gradient Boosting

Accepted Answer

An ensemble technique that builds decision trees sequentially, where each new tree is trained to predict the residual errors (pseudo-residuals) of the combined previous trees.

Question 8

Bias-Variance Tradeoff

Accepted Answer

The total error constraint in supervised models: $	ext{Total Error} = 	ext{Bias}^2 + 	ext{Variance} + 	ext{Irreducible Error}$. High bias causes **underfitting**; high variance causes **overfitting**.

Question 9

L1 vs. L2 Regularization

Accepted Answer

**L1 (Lasso)** adds penalty $\lambda \sum |	heta_i|$, producing sparse models (feature selection). **L2 (Ridge)** adds penalty $\lambda \sum 	heta_i^2$, shrinking weights smoothly to prevent extreme parameter values.

Question 10

Perceptron

Accepted Answer

The simplest neural network unit; computes a weighted sum of inputs plus bias and applies a step function $y = f(w^T x + b)$. It can only classify linearly separable data.

Question 11

Sigmoid Activation Function

Accepted Answer

S-shaped function defined as $\sigma(z) = \frac{1}{1 + e^{-z}}$. Maps inputs to $(0, 1)$, making it useful for binary probability outputs, but suffers from vanishing gradients for extreme values.

Question 12

ReLU (Rectified Linear Unit)

Accepted Answer

An activation function defined as $f(z) = \max(0, z)$. It introduces non-linearity without saturation for positive values, enabling faster gradient descent convergence.

Question 13

Tanh Activation Function

Accepted Answer

Zero-centered activation function defined as $	anh(z) = \frac{e^z - e^{-z}}{e^z + e^{-z}}$. Maps outputs to $(-1, 1)$, offering stronger gradient signal than Sigmoid.

Question 14

Binary Cross-Entropy Loss

Accepted Answer

Loss function measuring performance of binary classification models outputting probabilities: $L = -[y \log(\hat{y}) + (1 - y) \log(1 - \hat{y})]$.

Question 15

Forward Propagation

Accepted Answer

The process of passing input data through a neural network layer by layer (computing activations $a^{[l]} = g(W^{[l]} a^{[l-1]} + b^{[l]})$) to calculate the final output and loss.

Question 16

Backpropagation

Accepted Answer

Algorithm for training neural networks that computes the gradient of the loss function with respect to each weight using the differential calculus **chain rule**, propagating errors backward from the output layer.

Question 17

Stochastic Gradient Descent (SGD)

Accepted Answer

An optimization algorithm updating weights using the gradient of the loss for a single sample (or mini-batch) at each iteration: $	heta := 	heta - \alpha 
abla_	heta J(	heta)$.

Question 18

Adam Optimizer

Accepted Answer

An adaptive optimization technique combining **Momentum** (exponentially decaying average of past gradients) and **RMSProp** (exponentially decaying average of past squared gradients).

Question 19

Dropout Regularization

Accepted Answer

A technique during training where neurons are randomly zeroed out with probability $p$ on each forward pass, preventing complex co-adaptations and reducing overfitting.

Question 20

Vanishing & Exploding Gradient Problems

Accepted Answer

In deep networks, gradients can approach zero (**vanishing**) or grow exponentially (**exploding**) during backpropagation. Resolved using ReLU, residual connections, gradient clipping, and proper initialization (e.g., He or Xavier).

Question 21

Precision, Recall, and F1-Score

Accepted Answer

- **Precision**: $\frac{TP}{TP + FP}$ (accuracy of positive predictions)
- **Recall**: $\frac{TP}{TP + FN}$ (coverage of actual positive instances)
- **F1-Score**: $2 \cdot \frac{	ext{Precision} \cdot 	ext{Recall}}{	ext{Precision} + 	ext{Recall}}$

Question 22

Universal Approximation Theorem

Accepted Answer

A mathematical theorem stating that a feedforward neural network with a single hidden layer and a non-linear activation function can approximate any continuous function on a compact domain to arbitrary accuracy.

Machine Learning - Supervised Learning & Neural Networks

Cards in this deck

Related Flashcard Sets

JLPT N2 - Upper-Intermediate Japanese Kanji, Grammar & Expressions

CompTIA Network+ (N10-008) - Networking Fundamentals & Subnetting

HSK 4 - Intermediate Chinese Vocabulary & Grammar

CompTIA Network+ (N10-008) - Networking Fundamentals & Protocols