Ensemble Evaluation: Stacking, Blending, and Bagging Quiz — Questions & Answers

Discover how well you understand ensemble evaluation techniques including stacking, blending, and bagging. Explore the principles, advantages, and typical use cases of these ensemble learning strategies to strengthen your data science foundations.

This quiz contains 10 questions. Below is a complete reference of all questions, answer choices, and correct answers. You can use this section to review after taking the interactive quiz above.

1. Question 1: Bagging Basics

   Which main goal does bagging primarily aim to achieve in an ensemble model?

   • Enhance interpretability
   • Decrease dataset size
   • Increase bias
   • Reduce variance
   Show correct answer

   Correct answer: Reduce variance

   Explanation: Bagging, or Bootstrap Aggregating, is mainly used to reduce the variance of predictions by averaging multiple models trained on different subsets of the data. Increasing bias is not the objective; in fact, bagging generally reduces variance without significantly increasing bias. Bagging does not directly enhance model interpretability nor does it aim to decrease the dataset size. The main benefit is improved stability and accuracy through variance reduction.

2. Question 2: Stacking Architecture

   In stacking, what is the purpose of the meta-model?

   • Generate random training data
   • Combine base learners' predictions
   • Preprocess raw input features
   • Simplify the training process
   Show correct answer

   Correct answer: Combine base learners' predictions

   Explanation: The meta-model in stacking is responsible for learning how to best combine the predictions of different base learners to produce a final output. It does not pre-process raw features or generate random training data; these steps are handled before or outside the stacking process. Stacking tends to be more complex than individual models, not necessarily simplifying the training process.

3. Question 3: Blending Versus Stacking

   How does blending differ from stacking when using holdout data sets?

   • Blending ignores meta-models, stacking does not
   • Blending always uses decision trees, stacking does not
   • Blending uses bagged models, stacking uses blended models
   • Blending uses a single holdout set, stacking uses cross-validation
   Show correct answer

   Correct answer: Blending uses a single holdout set, stacking uses cross-validation

   Explanation: Blending typically uses a fixed holdout set from the training data to generate predictions for the meta-model, whereas stacking usually employs cross-validation to produce out-of-fold predictions for more robust training. Neither technique is strictly tied to a particular model type or ignores meta-models completely. The distinction is in how the data is partitioned and used for second-level training.

4. Question 4: Bagging Example

   If you create 100 decision trees, each trained on a different random subset of your data and average their results, which ensemble technique are you using?

   • Bagging
   • Blending
   • Boosting
   • Stacking
   Show correct answer

   Correct answer: Bagging

   Explanation: Training multiple decision trees on different random subsets and aggregating their results is classic bagging. Boosting also uses multiple models but trains them sequentially with each new model focusing on previous errors. Stacking and blending involve combining model predictions, but their structures are more layered and not focused on simple averaging.

5. Question 5: Reducing Overfitting

   Which ensemble method is most effective at reducing overfitting caused by high variance in a model?

   • Stacking
   • Simple averaging
   • Bagging
   • Blending
   Show correct answer

   Correct answer: Bagging

   Explanation: Bagging is particularly effective for reducing overfitting due to high variance, as it averages out individual model errors from overfit models. Stacking and blending can help with generalization but are not primarily designed for variance reduction as bagging is. Simple averaging is less robust than bagging, as it doesn't always account for data diversity.

6. Question 6: Meta-model Training

   During the stacking process, what data does the meta-model use for training?

   • Predictions from base models on unseen data
   • Predictions from a single base model
   • The original raw features
   • Labels only
   Show correct answer

   Correct answer: Predictions from base models on unseen data

   Explanation: In stacking, the meta-model is trained using predictions from the base models on data they haven't seen during their own training, which helps prevent overfitting. The meta-model does not use the original raw features alone or the predictions from just one model. Using only labels is insufficient for this purpose.

7. Question 7: When to Use Stacking

   What is a key reason to choose stacking over a single model when evaluating ensemble approaches?

   • Stacking improves performance only for small datasets
   • Stacking is always faster to train
   • Stacking leverages diverse model strengths
   • Stacking works only with numeric features
   Show correct answer

   Correct answer: Stacking leverages diverse model strengths

   Explanation: Stacking is preferred when you want to combine the strengths of different models for potentially better performance. It is generally not faster to train due to its complexity. Stacking can be used with various feature types, not just numeric. Its benefits are not restricted to small datasets.

8. Question 8: Out-of-Bag Score Purpose

   In bagging, what does the out-of-bag (OOB) score estimate?

   • Highest possible accuracy
   • Best feature importance score
   • Model’s generalization performance
   • Lowest training error
   Show correct answer

   Correct answer: Model’s generalization performance

   Explanation: The out-of-bag score provides an unbiased estimate of generalization performance by evaluating the model on samples not included in each bootstrap iteration. It does not estimate the absolute highest accuracy, give feature importance measures directly, or represent the lowest possible training error. The OOB score is similar to cross-validation in spirit.

9. Question 9: Blending Holdout Size

   What is a recommended holdout set proportion when using blending in ensembles?

   • 100% of the test data
   • 0% (no holdout set)
   • 10-20% of the training data
   • 50% of the dataset
   Show correct answer

   Correct answer: 10-20% of the training data

   Explanation: Typically, blending sets aside about 10-20% of the training data as a holdout set for the second-layer model. Using 50% is generally too large, and 100% of test data should never be used for training. Not having a holdout set (0%) would defeat the purpose of blending. The chosen percentage balances data availability and the need for unbiased predictions.

10. Question 10: Ensemble Disadvantage

    Which is a potential disadvantage of using ensemble methods like stacking or bagging?

    • Increased computational complexity
    • Improved model stability
    • Higher predictive performance
    • Greater model interpretability
    Show correct answer

    Correct answer: Increased computational complexity

    Explanation: A common drawback of ensemble methods is that they often require more computational resources and time, as they involve training multiple models. Improved stability and predictive performance are usually benefits rather than disadvantages. Greater interpretability is typically reduced in ensembles compared to single models.