Monash FIT9132 - Data Mining & Machine Learning Fundamentals Flashcards

Question 1

In Association Rule Mining ($A \Rightarrow B$), what is Support?

Accepted Answer

**Support** measures the proportion of total transactions in dataset $D$ that contain both itemsets $A$ and $B$:
$$	ext{Support}(A \Rightarrow B) = P(A \cup B) = \frac{	ext{count}(A \cup B)}{|D|}$$
It indicates how frequently a rule applies to the data.

Question 2

What is Data Cleaning in data preprocessing?

Accepted Answer

Data Cleaning involves filling in missing values, smoothing noisy data, identifying or removing outliers, and resolving data inconsistencies. Common strategies include **mean/median imputation**, binning, and regression-based predictions.

Question 3

What is the difference between Min-Max Normalization and Standardization (Z-score)?

Accepted Answer

**Min-Max Normalization** rescales data values to a specific range (typically $[0, 1]$):
$$v' = \frac{v - 	ext{min}_A}{	ext{max}_A - 	ext{min}_A}$$

**Standardization (Z-score)** rescales data to have a mean $\mu = 0$ and standard deviation $\sigma = 1$:
$$v' = \frac{v - \mu}{\sigma}$$

Question 4

What are the primary techniques used for Data Reduction?

Accepted Answer

Data Reduction reduces the volume of data while maintaining analytical integrity:
1. **Dimensionality Reduction**: Eliminating redundant features (e.g., PCA, feature selection).
2. **Numerosity Reduction**: Replacing original data with smaller representations (e.g., binning, clustering, sampling).
3. **Data Compression**: Encoding data transformation techniques.

Question 5

Compare Supervised Learning and Unsupervised Learning.

Accepted Answer

- **Supervised Learning**: The algorithm is trained on labeled data $(X, y)$ to learn a mapping function from input variables to a target variable (e.g., Classification, Regression).
- **Unsupervised Learning**: The algorithm finds inherent structures or patterns in unlabeled data $X$ without explicit target labels (e.g., Clustering, Association Rule Mining).

Question 6

How is Entropy calculated in Decision Trees, and what does it measure?

Accepted Answer

**Entropy** measures the degree of impurity or randomness in a dataset $S$:
$$H(S) = -\sum_{i=1}^{c} p_i \log_2(p_i)$$
where $p_i$ is the probability of an instance belonging to class $i$. Entropy is $0$ for completely pure sets and maximum when classes are equally likely.

Question 7

How is Information Gain calculated for Decision Tree splits?

Accepted Answer

**Information Gain** measures the expected reduction in entropy caused by splitting a dataset $S$ on attribute $A$:
$$	ext{Gain}(S, A) = H(S) - \sum_{v \in 	ext{Values}(A)} \frac{|S_v|}{|S|} H(S_v)$$
The attribute with the highest Information Gain is selected as the splitting node.

Question 8

What is Gini Impurity in Decision Tree algorithms like CART?

Accepted Answer

**Gini Impurity** measures the frequency with which a randomly chosen element from a set would be incorrectly labeled:
$$	ext{Gini}(D) = 1 - \sum_{i=1}^{m} p_i^2$$
Lower Gini impurity indicates higher purity. A node with Gini = 0 contains instances of only one class.

Question 9

How does the k-Nearest Neighbors (k-NN) classifier work?

Accepted Answer

k-NN is an instance-based lazy learning algorithm:
1. Calculates distance (e.g., Euclidean distance $d(x, y) = \sqrt{\sum_{i=1}^n (x_i - y_i)^2}$) between an unlabelled instance and all training examples.
2. Identifies the $k$ closest neighbors.
3. Assigns the majority class among these $k$ neighbors to the new instance.

Question 10

What is the core formula and conditional independence assumption of Naive Bayes?

Accepted Answer

Naive Bayes applies Bayes' Theorem:
$$P(Y|X) = \frac{P(X|Y)P(Y)}{P(X)}$$
It assumes that features $X_1, X_2, ..., X_n$ are **conditionally independent** given class $Y$, simplifying calculations to:
$$P(X_1, X_2, ..., X_n | Y) = \prod_{i=1}^n P(X_i | Y)$$

Question 11

Describe the algorithmic steps of K-Means Clustering.

Accepted Answer

1. **Initialization**: Choose $k$ initial centroids randomly.
2. **Assignment**: Assign each data point to its nearest centroid using a distance metric (e.g., Euclidean distance).
3. **Update**: Recalculate each cluster's centroid as the mean of all points assigned to it.
4. **Repeat**: Steps 2 and 3 until centroids stabilize (convergence) or maximum iterations are reached.

Question 12

In Association Rule Mining ($A \Rightarrow B$), what is Confidence?

Accepted Answer

**Confidence** measures how often item $B$ appears in transactions that contain item $A$:
$$	ext{Confidence}(A \Rightarrow B) = P(B|A) = \frac{	ext{Support}(A \cup B)}{	ext{Support}(A)}$$
It evaluates the reliability or certainty of the inference rule.

Question 13

In Association Rule Mining ($A \Rightarrow B$), how is Lift calculated and interpreted?

Accepted Answer

**Lift** measures how much more often $A$ and $B$ occur together than expected if they were statistically independent:
$$	ext{Lift}(A \Rightarrow B) = \frac{	ext{Confidence}(A \Rightarrow B)}{	ext{Support}(B)} = \frac{P(A \cup B)}{P(A)P(B)}$$
- $	ext{Lift} = 1$: $A$ and $B$ are independent.
- $	ext{Lift} > 1$: Positive correlation ($A$ increases likelihood of $B$).
- $	ext{Lift} < 1$: Negative correlation.

Question 14

What is the Apriori Property used in frequent itemset generation?

Accepted Answer

The **Apriori Property** states that **all non-empty subsets of a frequent itemset must also be frequent** (downward-closure property of support). If an itemset is infrequent, all of its supersets will also be infrequent and can be pruned immediately.

Question 15

Define True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN).

Accepted Answer

- **TP**: Actual Positive correctly predicted as Positive.
- **FP** (Type I Error): Actual Negative incorrectly predicted as Positive.
- **TN**: Actual Negative correctly predicted as Negative.
- **FN** (Type II Error): Actual Positive incorrectly predicted as Negative.

Question 16

What is Classification Accuracy, and when can it be misleading?

Accepted Answer

$$	ext{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN}$$
Accuracy measures total correct predictions. It can be highly misleading on **imbalanced datasets** (e.g., if 99% of samples are negative, a model predicting all negative gets 99% accuracy but fails to detect positives).

Question 17

What are Precision and Recall, and what are their formulas?

Accepted Answer

- **Precision**: Proportion of positive predictions that were truly positive.
$$	ext{Precision} = \frac{TP}{TP + FP}$$
- **Recall (Sensitivity)**: Proportion of actual positives correctly identified.
$$	ext{Recall} = \frac{TP}{TP + FN}$$

Question 18

What is the F1-Score, and why is it preferred over Accuracy for imbalanced data?

Accepted Answer

The **F1-Score** is the harmonic mean of Precision and Recall:
$$	ext{F1} = 2 	imes \frac{	ext{Precision} 	imes 	ext{Recall}}{	ext{Precision} + 	ext{Recall}} = \frac{2TP}{2TP + FP + FN}$$
It gives equal weight to precision and recall, serving as a better evaluation metric when dealing with imbalanced class distributions.

Question 19

Explain the difference between Overfitting and Underfitting.

Accepted Answer

- **Overfitting** (High Variance): Model learns the training data and noise too well. Performs exceptionally on training data but poorly on unseen test data.
- **Underfitting** (High Bias): Model is too simple to capture underlying data structures. Performs poorly on both training and test data.

Question 20

How does k-Fold Cross-Validation operate?

Accepted Answer

1. Split the dataset randomly into $k$ equal-sized folds.
2. For each iteration $i \in \{1, ..., k\}$, train the model on $k-1$ folds and validate on fold $i$.
3. Average the performance metrics across all $k$ iterations to get an overall evaluation score. This reduces evaluation variance and prevents overfitting during evaluation.

Monash FIT9132 - Data Mining & Machine Learning Fundamentals

Cards in this deck

Related Flashcard Sets

TypeScript - Modern JavaScript Development & Type Safety

Biochemistry - Metabolic Pathways & Regulation

JLPT N2 - Advanced Japanese Grammar, Vocabulary & Kanji

CPA Exam - AUD (Auditing and Attestation)