Scribd Logo
Upload

Welcome to Scribd!

  • 448 views

    Feature Engineering PDF

    Uploaded by

    Rutuparn Dalvi
    Feature engineering is the process of transforming raw data into features that better represent the underlying problem to improve machine learning results. It involves extracting relevant attributes from data, creating new features from existing ones, selecting important features, and reducing dimensionality. Some key techniques include feature extraction, creation, selection, and dimensionality reduction methods like PCA. Feature engineering is especially important when data is limited to avoid overfitting.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd

    Feature Engineering PDF

    Uploaded by

    Rutuparn Dalvi
    448 views19 pages
    Feature engineering is the process of transforming raw data into features that better represent the underlying problem to improve machine learning results. It involves extracting relevant attributes from data, creating new features from existing ones, selecting important features, and reducing dimensionality. Some key techniques include feature extraction, creation, selection, and dimensionality reduction methods like PCA. Feature engineering is especially important when data is limited to avoid overfitting.

    Original Title

    Feature-Engineering.pdf

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Feature engineering is the process of transforming raw data into features that better represent the underlying problem to improve machine learning results. It involves extracting relevant attributes from data, creating new features from existing ones, selecting important features, and reducing dimensionality. Some key techniques include feature extraction, creation, selection, and dimensionality reduction methods like PCA. Feature engineering is especially important when data is limited to avoid overfitting.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    448 views19 pages

    Feature Engineering PDF

    Uploaded by

    Rutuparn Dalvi
    Feature engineering is the process of transforming raw data into features that better represent the underlying problem to improve machine learning results. It involves extracting relevant attributes from data, creating new features from existing ones, selecting important features, and reducing dimensionality. Some key techniques include feature extraction, creation, selection, and dimensionality reduction methods like PCA. Feature engineering is especially important when data is limited to avoid overfitting.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Download as pdf or txt
    of 19

    Feature Engineering

    (In Machine Learning)


    What is a feature?

    An attribute (coordinate) of the observation (point)


    that is important from learning or prediction point
    of view

    Not all attributes are features

    2
    Examples of Features …
    • An attribute (in a table)
    • Line in an image
    • A phrase
    • A word count

    3
    What is Feature Engineering?

    A set of steps taken to present the original and / or


    transformed data to a machine learning strategy –
    such that inherent important structures in the data
    are exposed for the purpose of model creation

    4
    What is Feature Engineering?

    5
    Feature Engineering: when required, and not …
    • Feature engineering is required when …
    – Limited data is available
    • “Curse of dimensionality” if more features are considered in
    model building
    • Cases of over-fitting if there are more features and less data
    – Limited computation power

    • Feature engineering may not be required when …


    – Copious data availability (eg. images, server logs)
    – Computation power is not an issue (eg. cloud computing)
    – Most important: availability of universal function
    approximators
    • Artifical Neural Networks, Deep Learning Networks

    6
    The Feature Engineering Process
    Execution of the following steps

    1. Create / identify set of relevant features


    2. Fit a model and run validation tests
    3. Re-design or re-select features based on results of
    validation
    4. Perform step 2

    Repeat the process until ‘satisfactory’ results are


    obtained or there is no further improvement

    7
    Components of Feature Engineering
    • Feature Extraction
    • Feature Creation
    • Feature Selection
    • Dimensionality Reduction
    – PCA, SVD

    8
    Feature Extraction
    • Goal
    – To increase the level of abstraction
    – To reduce the total data sent into learning algorithms
    • Example
    – Edge detection in images
    – Curvature detection in 3d models
    – Number of concavities / convexities in 3D models
    – Identifying regions with same “colours”
    • Satellite imaging
    • Temperature based tool condition monitoring
    • MRI / X-Ray processing
    9
    Feature Extraction

    10
    Feature Extraction

    11
    Feature Creation
    • Goal: To create a set of attributes
    – based on domain knowledge or pre-processing /
    visualization
    – that are known to better describe the structure of the data
    to be processed
    • Example:
    – In linear regression, addition of new terms like ‘log’, ‘tanh’,
    ‘exp’, ‘sin’, square, cube, x1 * x2 (feature combinations),
    etc.
    – One-hot encoding: creation of dummy variables
    – Discretizing continuous attributes
    – Combining multiple attributes into one feature
    – Addition of new terms resulting from ‘feature extraction’
    12
    One-hot-encoding

    13
    Feature Selection
    • Goal: To reduce the total number of ‘features’ sent
    into the machine learning algorithm
    – To reduce model complexity and model computation time
    • Methods
    – Forward selection:
    • Start with minimal set and gradually add features
    – Backward selection:
    • Start with a maximal set and gradually reduce features
    – Filter
    • Based on analysis such as Pearson correlation coefficient
    – Embedded methods
    • LASSO (Least Absolute Shrinkage and Selection Operator: L1
    penalty), Ridge (L2), ElasticNet (L1+L2) Regression

    14
    Feature Selection: Forward Selection

    15
    Feature Selection: Forward Selection

    16
    Feature Selection: Backward Selection

    17
    Feature Selection: Backward Selection

    18
    Dimensionality Reduction
    • Goal:
    – To reduce number of features by identifying feature combinations
    • Example
    – Principal Component Analysis

    19

    You might also like