Feature Selection vs Feature Extraction: Key Differences Quiz — Questions & Answers

Explore the essential differences between feature selection and feature extraction in machine learning with this engaging quiz. Enhance your understanding of dimensionality reduction techniques, use cases, and core principles to boost your data preprocessing skills.

This quiz contains 10 questions. Below is a complete reference of all questions, answer choices, and correct answers. You can use this section to review after taking the interactive quiz above.

1. Question 1: Distinguishing core purpose

   What is the primary purpose of feature selection in data preprocessing?

   • To normalize all features to a standard scale
   • To create new features as combinations of old ones
   • To generate random features for more data
   • To choose the most relevant existing features for a model
   Show correct answer

   Correct answer: To choose the most relevant existing features for a model

   Explanation: Feature selection involves identifying and retaining the most important features from the original dataset, which helps improve model performance and reduce overfitting. Creating new combinations is feature extraction, not selection. Normalization adjusts scales but does not select features. Generating random features adds noise rather than relevance.

2. Question 2: Understanding feature extraction

   How does feature extraction typically modify data when applied to a dataset?

   • It splits the dataset into training and testing parts
   • It generates new features derived from existing ones
   • It deletes rows with missing values
   • It randomly selects a subset of samples
   Show correct answer

   Correct answer: It generates new features derived from existing ones

   Explanation: Feature extraction creates new variables by transforming or combining existing features, such as through mathematical or statistical methods. Deleting rows with missing values is data cleaning, not extraction. Dataset splitting is part of model evaluation. Random sample selection does not create new features.

3. Question 3: Approach to dimensionality

   If you want to reduce a dataset's number of input variables while keeping only the original columns, which technique should you use?

   • Data augmentation
   • Feature extraction
   • Target encoding
   • Feature selection
   Show correct answer

   Correct answer: Feature selection

   Explanation: Feature selection reduces dimensionality by picking a subset of the original columns, ensuring all selected variables remain as they were. Feature extraction generates new columns, not using only original ones. Data augmentation increases the dataset, and target encoding modifies categorical features based on target distributions.

4. Question 4: Nature of new features

   After applying feature extraction like Principal Component Analysis (PCA), how are the resulting features best described?

   • They directly represent physical measurements
   • They remain identical to the initial features
   • They are combinations of the original features and may lack direct interpretability
   • They are always categorical variables
   Show correct answer

   Correct answer: They are combinations of the original features and may lack direct interpretability

   Explanation: Feature extraction techniques like PCA create new features as mathematical combinations of original ones, often losing clear interpretability. These features are different from the initial ones. They are not always categorical, nor do they necessarily represent straightforward physical measurements.

5. Question 5: Handling correlated features

   When a dataset has several highly correlated features, which is a typical advantage of feature extraction over feature selection?

   • It simply removes duplicate features
   • It always improves accuracy regardless of data
   • It assigns random weights to all features
   • It can transform correlated features into uncorrelated components
   Show correct answer

   Correct answer: It can transform correlated features into uncorrelated components

   Explanation: Feature extraction methods like PCA can transform correlated features into a set of uncorrelated variables, often called principal components. Simply removing duplicates is not feature extraction's primary function. Assigning random weights is unrelated, and extraction does not always guarantee accuracy improvement.

6. Question 6: Interpretability focus

   Which technique typically preserves the interpretability of input features in the model's results?

   • Image augmentation
   • Label transformation
   • Feature extraction
   • Feature selection
   Show correct answer

   Correct answer: Feature selection

   Explanation: Feature selection keeps original features as inputs, making it easier to interpret which variables influence model outcomes. Feature extraction makes new combined features that can be hard to interpret. Image augmentation and label transformation handle data expansion and target changes, not feature interpretability.

7. Question 7: Real-world scenario

   In a facial recognition system, which technique would you use to reduce thousands of pixel values to a smaller set of meaningful numerical features?

   • Random oversampling
   • One-hot encoding
   • Feature extraction
   • Feature selection
   Show correct answer

   Correct answer: Feature extraction

   Explanation: Feature extraction can condense high-dimensional data (like pixels) into fewer, informative features using methods such as PCA or Fourier transforms. Feature selection would only pick certain pixels, potentially missing holistic patterns. Random oversampling deals with imbalances, and one-hot encoding turns categoricals into binaries.

8. Question 8: Conceptual difference

   What is a key conceptual difference between feature selection and feature extraction?

   • Both always produce the same output regardless of approach
   • Feature selection retains original features, while feature extraction creates new ones
   • Feature selection increases the number of features, while feature extraction decreases it
   • Feature selection can only be used with numerical data, while extraction cannot
   Show correct answer

   Correct answer: Feature selection retains original features, while feature extraction creates new ones

   Explanation: Feature selection keeps original columns, whereas feature extraction forms new, transformed features. Selection typically reduces the feature set, extraction can reduce or increase features. Both work with different data types, and their outputs vary by method used.

9. Question 9: Avoiding overfitting

   How can feature selection help reduce the risk of overfitting in machine learning models?

   • By decreasing the size of the training set
   • By removing irrelevant or redundant features to simplify the model
   • By increasing the number of features used for training
   • By adding synthetic noise to each feature
   Show correct answer

   Correct answer: By removing irrelevant or redundant features to simplify the model

   Explanation: Feature selection reduces overfitting by discarding features that do not contribute valuable information, which helps prevent models from learning unhelpful patterns. Increasing features or adding noise generally increases overfitting risk, while shrinking training data can hurt performance.

10. Question 10: Techniques and examples

    Which of the following is most often considered a feature extraction technique?

    • Variance Thresholding
    • Forward Selection
    • Principal Component Analysis (PCA)
    • Recursive Feature Elimination (RFE)
    Show correct answer

    Correct answer: Principal Component Analysis (PCA)

    Explanation: PCA is a classic feature extraction method, transforming existing features into new principal components. Recursive Feature Elimination, Variance Thresholding, and Forward Selection are common feature selection techniques, not extraction, as they remove or retain existing variables instead of creating new ones.