Machine Learning - Supervised Learning Algorithms Flashcards

Question 1

Supervised Learning

Accepted Answer

A machine learning paradigm where an algorithm learns a mapping function $f: X 	o Y$ from a labeled dataset containing input features ($X$) and corresponding target outputs ($Y$).

Question 2

Data Splitting: Training, Validation, and Test Sets

Accepted Answer

– **Training Set**: Portion of data used to fit the model parameters.
– **Validation Set**: Portion used for hyperparameter tuning and model selection to prevent overfitting.
– **Test Set**: Independent data used strictly for unbiased evaluation of final model performance.

Question 3

Linear Regression

Accepted Answer

A regression algorithm that models a continuous target variable $y$ as a linear combination of input features $x$: $y = \beta_0 + \beta_1 x_1 + \dots + \beta_n x_n + \epsilon$.

Question 4

Mean Squared Error (MSE) Cost Function

Accepted Answer

A common loss function for regression measuring average squared residuals between predicted $(\hat{y})$ and actual $(y)$ values: $J(	heta) = \frac{1}{2m} \sum_{i=1}^{m} (\hat{y}^{(i)} - y^{(i)})^2$.

Question 5

Gradient Descent

Accepted Answer

An iterative optimization algorithm used to minimize a cost function $J(	heta)$ by repeatedly taking steps proportional to the negative of the gradient: $	heta := 	heta - \alpha 
abla J(	heta)$, where $\alpha$ is the learning rate.

Question 6

Logistic Regression

Accepted Answer

A classification algorithm that estimates the probability $P(Y=1|X)$ by passing a linear combination of features through a non-linear sigmoid activation function.

Question 7

Sigmoid Function

Accepted Answer

An S-shaped mathematical activation function that maps any real value into a probability range between $0$ and $1$: $\sigma(z) = \frac{1}{1 + e^{-z}}$.

Question 8

Decision Tree

Accepted Answer

A non-parametric algorithm that breaks down a dataset into smaller subsets by recursively splitting features based on conditional rules, forming a tree structure of decision nodes and leaf nodes.

Question 9

Entropy

Accepted Answer

A metric that quantifies the degree of randomness or impurity in a dataset: $H(S) = -\sum_{i=1}^{C} p_i \log_2(p_i)$, where $p_i$ is the proportion of samples belonging to class $i$.

Question 10

Information Gain

Accepted Answer

The expected reduction in entropy caused by splitting a dataset $S$ on a given attribute $A$: $	ext{Gain}(S, A) = H(S) - \sum_{v \in 	ext{Values}(A)} \frac{|S_v|}{|S|} H(S_v)$.

Question 11

Gini Impurity

Accepted Answer

A criterion used by classification algorithms (like CART) to evaluate split quality based on class distribution: $I_G(p) = 1 - \sum_{i=1}^{C} p_i^2$. Lower values indicate higher purity.

Question 12

Random Forest

Accepted Answer

An ensemble learning method that constructs a multitude of decision trees using bootstrap aggregating (bagging) and random feature subsets, outputting the majority class vote or average prediction.

Question 13

Support Vector Machine (SVM)

Accepted Answer

A supervised learning algorithm that finds an optimal decision boundary (hyperplane) maximizing the margin between data points of different classes.

Question 14

Hyperplane and Margin (SVM)

Accepted Answer

– **Hyperplane**: An $N$-dimensional decision boundary defined by $w^T x + b = 0$.
– **Margin**: The distance between the separating hyperplane and the closest training samples (support vectors).

Question 15

Kernel Trick

Accepted Answer

A method in SVMs using kernel functions $K(x_i, x_j)$ to implicitly compute dot products in higher-dimensional feature spaces, allowing linear hyperplanes to solve non-linear classification problems.

Question 16

Overfitting vs. Underfitting

Accepted Answer

– **Overfitting**: Model fits training noise and yields poor generalization to unseen data (high variance).
– **Underfitting**: Model is too simple to capture underlying data structure (high bias).

Question 17

Bias-Variance Tradeoff

Accepted Answer

The balance between error from erroneous assumptions (**Bias**) and error from sensitivity to small fluctuations in training data (**Variance**). Total error = $	ext{Bias}^2 + 	ext{Variance} + 	ext{Irreducible Error}$.

Question 18

Accuracy

Accepted Answer

The proportion of correct predictions out of all predictions made: $	ext{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$. Can be misleading on class-imbalanced datasets.

Question 19

Precision vs. Recall

Accepted Answer

– **Precision**: Correct positive predictions out of all predicted positives: $\frac{TP}{TP + FP}$.
– **Recall (Sensitivity)**: Correct positive predictions out of all actual positives: $\frac{TP}{TP + FN}$.

Question 20

F1-Score

Accepted Answer

The harmonic mean of precision and recall, providing a balanced single evaluation metric: $F1 = 2 \cdot \frac{	ext{Precision} \cdot 	ext{Recall}}{	ext{Precision} + 	ext{Recall}}$.

Question 21

ROC Curve and AUC

Accepted Answer

– **ROC Curve**: Graphical plot of True Positive Rate (Recall) vs. False Positive Rate ($1 - 	ext{Specificity}$) across varying probability thresholds.
– **AUC**: Area Under the ROC Curve, measuring overall classification performance between $0$ and $1$.

Machine Learning - Supervised Learning Algorithms

Cards in this deck

Related Flashcard Sets

Korean TOPIK I - Essential Grammar & Vocab

French A1 - Essential Vocabulary & Grammar

Immunology - Adaptive & Innate Immunity

Multivariable Calculus - Vector Calculus & Optimization