Quiz: Overfitting and regularization

What is overfitting in machine learning?

Overfitting occurs when a model learns the training data too well, capturing noise and details that do not generalize to new data.

True or False: Overfitting improves model performance on unseen data.

False. Overfitting typically results in poor performance on new data, as the model fails to generalize.

List two signs of overfitting.

- High accuracy on training data - Low accuracy on validation/test data

What happens when a model overfits?

The model becomes overly complex, making predictions based on noise rather than the actual data trends.

Fill in the blank: Overfitting leads to ________ in predictive performance.

decreased accuracy

Compare overfitting and underfitting.

Overfitting: models complex, learns noise. Underfitting: models too simple, misses trends.

What are the implications of overfitting?

Poor model generalization, high variance, and misleading performance metrics.

Describe a simple example of overfitting.

A model that perfectly predicts training data prices, but fails to predict new prices accurately.

What is the relationship between model complexity and overfitting?

Increased model complexity raises the risk of overfitting, as it can learn irrelevant details.

True or False: Regularization can help reduce overfitting.

True. Regularization techniques add constraints to the model, encouraging simpler solutions.

Identify a common consequence of overfitting.

Inability to perform well on unseen data.

What is cross-validation used for?

To assess how well a model generalizes to an independent dataset and detect overfitting.

What is L1 regularization?

L1 regularization adds a penalty equal to the absolute value of the coefficients. This encourages sparsity in the model, effectively reducing the number of features used.

What is L2 regularization?

L2 regularization adds a penalty equal to the square of the coefficients. This helps to shrink the coefficients but generally keeps all features in the model.

True or False: L1 regularization can lead to feature elimination.

True. L1 regularization can set some coefficients to zero, effectively removing those features from the model.

Compare L1 and L2 regularization.

- L1 can eliminate features - L2 shrinks coefficients - L1 results in sparse models - L2 keeps all features

What is dropout in neural networks?

Dropout randomly sets a fraction of the neurons to zero during training, which helps prevent overfitting by forcing the model to learn redundant representations.

Fill in the blank: In dropout, a common rate is ____.

0.5 (50% of neurons are dropped during training to prevent overfitting.)

What does data augmentation help with?

Data augmentation creates artificial data instances by transforming existing data, which helps reduce overfitting by providing more diverse training examples.

Cause → Effect: Using early stopping.

Cause: Monitor validation loss during training. Effect: Stop training when performance starts to degrade, preventing overfitting.

What is weight decay?

Weight decay is a form of L2 regularization where a small value is added to the loss function to penalize large weights, encouraging simpler models.

How does cross-validation help in regularization?

Cross-validation assesses model performance on different data splits, allowing for better tuning of regularization parameters and minimizing overfitting.

What is Elastic Net regularization?

Elastic Net combines L1 and L2 regularization techniques. It is useful when there are many correlated features.

True or False: Regularization always improves model accuracy.

False. Regularization can improve model generalization but may reduce accuracy on training data.

What is accuracy in model evaluation?

Accuracy is the ratio of correctly predicted instances to total instances. - Formula: Accuracy = \\frac{TP + TN}{TP + TN + FP + FN} - TP: True Positives, TN: True Negatives, FP: False Positives, FN: False Negatives.

True or False: High accuracy always indicates a good model.

False. High accuracy can be misleading, especially in imbalanced datasets where one class dominates.

What does precision measure?

Precision measures the accuracy of positive predictions. It is calculated as: Precision = \\frac{TP}{TP + FP}.

Recall is also known as what?

Recall is also known as Sensitivity or True Positive Rate. It measures the ability to identify all relevant instances.

Fill in the blank: F1 Score is the harmonic mean of ____ and ____.

F1 Score is the harmonic mean of Precision and Recall.

What is the purpose of the ROC curve?

The ROC curve illustrates the trade-off between True Positive Rate and False Positive Rate across different thresholds.

Compare precision and recall.

Precision focuses on the accuracy of positive predictions, while recall focuses on capturing all positive cases.

What is the AUC in ROC?

AUC stands for Area Under the Curve. It summarizes the ROC curve performance; a higher AUC indicates better model performance.

True or False: AUC of 1 indicates a perfect model.

True. An AUC of 1 signifies perfect classification, where all positive and negative instances are correctly identified.

What does the confusion matrix display?

The confusion matrix displays the counts of true positives, true negatives, false positives, and false negatives for a model's predictions.

Cause → Effect: High bias leads to ____.

High bias leads to underfitting, where the model fails to capture the underlying trends of the data.

What is a common issue with using only accuracy as a metric?

Using only accuracy can be problematic in imbalanced datasets, as it may not reflect the model's performance effectively.