Feature Engineering in Machine Learning: Techniques, Tools and Python Implementation

Quick Overview: 

• Feature engineering in machine learning transforms raw data into useful features that help models identify patterns and make better predictions.

• Common techniques include missing value handling, one-hot encoding, feature scaling, log transformation, binning, date-time extraction, mathematical transformations, interaction features, and text feature engineering.

• Python libraries such as Pandas, NumPy, scikit-learn, Feature-engine, and tsfresh support different feature engineering and preprocessing tasks.

• In this blog you will learn feature engineering in machine learning in detail along with process, techniques, libraries used and implementation using Python.

Build practical AI and software engineering skills with the Executive Post Graduate Certificate in AI-Native Software Engineering. Learn to use AI across the software development lifecycle and prepare for the evolving demands of modern engineering roles.

What Is Feature Engineering in Machine Learning?

Feature engineering in machine learning is an important step while preparing the data for machine learning. 

It is the process of creating, changing and selecting features, called input variables, from raw data which helps to learn machine learning models better as raw data is often not ready to be used directly.

Let’s understand through a feature engineering example-  A dataset contains a customer's date of birth. Instead of directly using the date, you can create a new feature called Age. Similarly, a date-time value can be converted into features such as day, month, year, or day of the week. 

A good feature engineering for machine learning makes the complex data easy to understand for a ML model.

Why Is Feature Engineering Important?

Raw datasets contain missing values, inconsistent data, categorical information and features that are not useful for a model. The feature engineering in machine learning helps to transform this data in a more suitable format.

The feature engineering for machine learning is important because:

• It improves the model performance.

• It can help ML models  in identifying the useful patterns.

• It can convert raw data into meaningful variables.

• It can handle different types of data used by a Machine Learning model.

• It can also reduce unnecessary information.

• It can make the machine learning models more effective.

For example, instead of using separate Height and Weight features, you can create a BMI feature. This new feature may provide more useful information for certain prediction tasks. 

Now let’s understand how the feature engineering in machine learning works step by step.

Also Read: Linear Regression in Machine Learning: Types, Algorithm, and Implementation

Step-by-Step Feature Engineering Process

The feature engineering for machine learning process commonly involves the understanding of the datasets, preparing the features, creating useful variables and evaluating the impact of these variables on the model.

Feature engineering in machine learning follows these steps:

Step 1: Understand the Dataset

You have to start by understanding the available columns, their data types, distributions, and relationships with the target variable.

You can use techniques such as:

• Descriptive statistics

• Data visualization

• Correlation analysis

• Missing value analysis

Step 2: Identify Data Quality Issues

After understanding the dataset you have to check for:

• Missing values

• Duplicate records

• Incorrect data types

• Outliers

• Inconsistent values

• Irrelevant columns

These issues should be addressed before creating new features.

Step 3: Transform Existing Features

Then, you have to transform features into formats that machine learning algorithms can use effectively.

For example:

• Convert categorical values into numerical values.

• Scale numerical features when required.

• Apply logarithmic transformations to highly skewed variables.

• Extract useful information from dates.

Step 4: Create New Features

Then, you have to use the existing variables to create new features that may contain more useful information.

For example:

• Date of Birth: Age

• Purchase Date: Day, Month, Year

• Total Sales and Number of Orders: Average Order Value

• Height and Weight: BMI

Step 5: Evaluate the Features

At last, you will train the model with the engineered features and compare its performance with the original feature set. At this stage, not every new feature improves a model, some may add noise or unnecessary complexity.

Now that you have a good understanding of feature engineering basics, let’s explore the feature engineering techniques which are commonly used.

Also Read: K Means Clustering in Machine Learning: Algorithm, Steps, Examples, and Applications

Common Feature Engineering Techniques in Machine Learning

There are so many feature engineering techniques used depending on the type of data and machine learning problem. Some common techniques are:

1. Handling Missing Values

When you explore a dataset, you will find lots of missing values because it is common in real-life datasets. You can handle missing values by:

• Removing rows or columns

• Replacing missing numerical values with the mean

• Using the median

• Using the mode for categorical data

• Applying more advanced imputation methods

For example, missing age values can be replaced with the median age of the dataset.

Also Read: Logistic Regression in Machine Learning: Algorithm, Types, Examples, and Applications

2. One-Hot Encoding

The one hot encoding is the most common feature engineering techniques, which converts categorical values into separate binary columns.

Imagine in dataset, a City column contains:

• Delhi

• Mumbai

• Pune

One-hot encoding can create:

Here:

• 1 means the value is present.

• 0 means the value is not present.

This helps the ML model to understand categories without assuming that one category is bigger or smaller than another. For example, the model will not assume that Pune is greater than Delhi just because the categories are converted into numbers. 

3. Label Encoding

The label encoding technique converts categories into numbers. Each category gets a unique numerical value. 

For example:

• Low: 0

• Medium: 1

• High: 2

Here, the numbers represent an order. Low comes before Medium, and Medium comes before High.

The label encoding works good when categories have a clear order, such as:

• Low, Medium, High

• Beginner, Intermediate, Advanced

• Poor, Average, Good, Excellent

For categories that do not have an order, such as City or Colour, one hot encoding is usually more suitable.

Also Read: Clustering in Machine Learning: Types, Algorithms, Examples, and How It Works

4. Feature Scaling

Feature scaling changes numerical features so they are on a similar scale. For example, suppose a dataset has:

• Age: 18 to 60

• Salary: ₹20,000 to ₹10,00,000

The salary values are much larger than the age values. Some machine learning algorithms may give more importance to salary because of its larger numbers.

Two common methods of feature scaling are:

• Standardization: Changes the values so they have a mean close to 0 and a standard deviation close to 1.

• Normalization: Changes the values to a fixed range, usually between 0 and 1.

Feature scaling is useful for algorithms that are affected by the size of numerical values, such as KNN, SVM, and some neural networks.

5. Log Transformation

Log transformation is a feature engineering technique used to handle highly skewed numerical data.

For example, income data may contain many people with average incomes and a few people with very high incomes. These extremely high values can make the data uneven.

It is commonly used for features such as:

• Income

• Sales

• Population

• House prices

6. Binning

Binning is a feature engineering technique that groups continuous numerical values into different ranges or categories. This makes the data easier to understand and can help a model identify patterns within different groups.

For example, instead of using exact ages, you can group them as:

• 18–25: Young

• 26–40: Adult

• 41–60: Middle-aged

• 61+: Senior

7. Date and Time Feature Extraction

Date and time data can contain useful information for a machine learning model. You can extract specific parts of a date or time and use them as separate features.

For example: 2026-09-21 10:30 AM

This can be converted into:

• Year: 2026

• Month: September

• Day: 21

• Hour: 10

• Day of the week: Monday

• Weekend: No

These features can help a model identify patterns based on when an event happens. This technique is commonly used for sales, finance, transportation, and time-series data. 

Also Read: Supervised vs Unsupervised Learning: Differences, Examples, and Applications

8. Mathematical Transformations

Mathematical transformations use existing features to create new and more useful features. For example, you can calculate:

Profit = Revenue - Cost

Or:

Average Purchase = Total Purchase Amount / Number of Orders

These new features can give the machine learning model more useful information and help it identify patterns in the data.

9. Interaction Features

Interaction features in feature engineering combines two or more variables to capture relationships between them.

For example: House Size × Number of Rooms

It can create a feature that represents the relationship between property size and room count.

10. Text Feature Engineering

Text data can be transformed into numerical features using feature engineering techniques such as:

• Bag of Words

• TF-IDF

• Word embeddings

• Text length

• Word count

• Character count

For example, the number of words in a customer review can be used as a feature for a text classification model.

Now let’s understand this implementation of feature engineering in machine learning through Python which helps you to understand the topic practically as well.

How to Perform Feature Engineering in Python?

Python provides several libraries for feature engineering, including Pandas, NumPy, and scikit-learn. Let’s consider a simple dataset containing customer information:

import pandas as pd

data = {

    "Age": [22, 35, 42, 28],

    "Income": [30000, 55000, 70000, 45000],

    "City": ["Delhi", "Mumbai", "Pune", "Delhi"]

}

df = pd.DataFrame(data)

print(df)

Output: 

   Age  Income    City

0   22   30000   Delhi

1   35   55000  Mumbai

2   42   70000    Pune

3   28   45000   Delhi

1. Creating a New Feature

You can create a new feature using existing columns.

df["Income_per_Age"] = df["Income"] / df["Age"]

print(df)

Output:

   Age  Income    City  Income_per_Age

0   22   30000   Delhi     1363.636364

1   35   55000  Mumbai     1571.428571

2   42   70000    Pune     1666.666667

3   28   45000   Delhi     1607.142857

2. One-Hot Encoding in Python

Using Pandas:

df_encoded = pd.get_dummies(df, columns=["City"])

print(df_encoded)

Output:

   Age  Income  Income_per_Age  City_Delhi  City_Mumbai  City_Pune

0   22   30000     1363.636364        True        False       False

1   35   55000     1571.428571       False         True       False

2   42   70000     1666.666667       False        False        True

3   28   45000     1607.142857        True        False       False

3. Feature Scaling in Python

You can use StandardScaler from scikit-learn:

from sklearn.preprocessing import StandardScaler

scaler = StandardScaler()

df[["Age", "Income"]] = scaler.fit_transform(

    df[["Age", "Income"]]

)

print(df)

Output:

        Age    Income    City

0 -1.300723 -1.371989   Delhi

1  0.433574  0.342997  Mumbai

2  1.367427  1.371989    Pune

3 -0.500278 -0.342997   Delhi

Now that you have learned Python implementation, let’s apply the feature engineering in machine learning by yourself with exercise below.

Strengthen your data science skills with the Executive Post Graduate Certificate in Applied Data Science & Agentic AI. Learn to work with data, AI, and intelligent agents to build practical solutions for real-world business challenges.

Feature Engineering Exercise

Try this exercise to practise the feature engineering techniques you have learned.

Given the following dataset:

import pandas as pd

data = {

    "Age": [21, 29, 35, 42, 50],

    "Income": [25000, 40000, 55000, 75000, 90000],

    "City": ["Delhi", "Mumbai", "Delhi", "Pune", "Mumbai"],

    "Orders": [2, 4, 5, 8, 10],

    "Total_Spending": [5000, 12000, 18000, 32000, 50000]

}

df = pd.DataFrame(data)

print(df)

Your Tasks

1. Create a new feature
   

   • Create a column called Average_Order_Value.

   • Calculate it using:  Average_Order_Value = Total_Spending / Order
     

2. Apply One-Hot Encoding

   • Convert the City column into separate numerical columns using pd.get_dummies().

3. Create Age Groups

   • Create a new column called Age_Group.

   • Use these categories:

18–25 → Young

26–40 → Adult

41–60 → Middle-aged

4. Apply Feature Scaling

   • Use StandardScaler to scale the Age and Income columns.

5. Check the final dataset

   • Print the transformed DataFrame.

   • Identify which columns are original features and which ones you created.

Challenge: Try completing all five tasks without looking at the previous examples.

Now let’s understand what tools and libraries you can use with python for feature engineering in machine learning.

Popular Feature Engineering Tools and Libraries

You can use so many tools for feature engineering in machine learning including Pandas, NumPy, Scikit-learn and more. Let’s understand each of them in detail:

1. Pandas

Pandas is widely used for:

• Data cleaning

• Creating new columns

• Handling missing values

• Grouping data

• Working with dates

• Encoding categorical variables

2. NumPy

NumPy is a Python library used for numerical and mathematical operations. It can help you create, transform, and combine features.

NumPy is widely used for:

• Performing mathematical calculations

• Creating new features

• Applying mathematical functions

• Working with numerical arrays

• Handling numerical data efficiently

• Transforming existing features

3. Scikit-learn

Scikit-learn provides tools for:

• Standardization

• Normalization

• One-hot encoding

• Imputation

• Feature selection

• Feature transformation

• Building preprocessing pipelines

4. Feature-engine

Feature-engine is a Python library designed specifically for feature engineering and preprocessing tasks. It provides transformers for missing data, categorical variables, numerical variables, outliers, and feature selection.

5. tsfresh 

tsfresh is a Python library used to automatically extract useful features from time-series data.

It can calculate many features from raw time-series data, such as:

• Mean

• Maximum and minimum values

• Standard deviation

• Trend

• Variance

• Statistical patterns

Now let’s understand what are the advantages and disadvantages of feature engineering for machine learning.

Advantages and Disadvantages of Feature Engineering

Feature engineering can make data more suitable for machine learning, but it also requires careful decisions.

Advantages of Feature Engineering

• Can improve model performance.

• Converts raw data into meaningful features.

• Helps models identify useful patterns.

• Can make complex data easier to use.

• Supports different types of machine learning algorithms.

• Can reduce the impact of irrelevant information.

Disadvantages of Feature Engineering

• Can require significant time and domain knowledge.

• Poorly designed features can reduce model performance.

• Feature engineering can introduce data leakage.

• Too many features can increase model complexity.

• Some transformations may not generalise to new data.

• Different models may benefit from different types of features.

Conclusion

So, what is feature engineering in machine learning in simple terms? It is the process of transforming raw data into useful features that help a machine learning model learn patterns more effectively. It can involve creating new variables, transforming existing data, encoding categories, and extracting useful information.

Using the right feature engineering techniques can improve model performance and make data easier to work with. However, features should be created carefully to avoid unnecessary complexity, data leakage, and poor generalization.