Importance of Feature Engineering

• Improve accuracy: Choosing the right features helps the model learn better, leading to more accurate predictions.
• Reduce overfitting: Using fewer, more important features helps the model avoid memorizing the data and perform better on new data.
• Boost interpretability: Well‑chosen features make it easier to understand how the model makes its predictions.
• Enhance efficiency: Focusing on key features speeds up the model’s training and prediction process, saving time and resources.

Processes Involved in Feature Engineering

1. Feature Creation: Generate new features from domain knowledge or data patterns.
   • Domain‑specific: Based on industry knowledge.
   • Data‑driven: Recognize patterns in the data.
   • Synthetic: Combine existing features.
2. Feature Transformation: Adjust features to improve learning.
   • Normalization & Scaling – Align feature ranges.
   • Encoding – One‑hot encode categorical variables.
   • Mathematical transforms – Logarithmic for skewed data.
3. Feature Extraction: Reduce dimensionality (e.g., PCA) or aggregate features.
   • Dimensionality reduction preserves important information while shrinking feature space.
   • Aggregation & Combination simplify the model.
4. Feature Selection: Choose a subset of relevant features.
   • Filter methods – Correlation based.
   • Wrapper methods – Model‑performance based.
   • Embedded methods – Integrated into training.
5. Feature Scaling: Ensure equal contribution.
   • Min‑Max scaling to [0,1].
   • Standard scaling (mean 0, variance 1).

Common Techniques

1. One‑Hot Encoding

import pandas as pd

data = {'Color': ['Red', 'Blue', 'Green', 'Blue']}
df = pd.DataFrame(data)

df_encoded = pd.get_dummies(df, columns=['Color'], prefix='Color')

print(df_encoded)

2. Binning

import pandas as pd

data = {'Age': [23, 45, 18, 34, 67, 50, 21]}
df = pd.DataFrame(data)

bins = [0, 20, 40, 60, 100]
labels = ['0-20', '21-40', '41-60', '61+']

df['Age_Group'] = pd.cut(df['Age'], bins=bins, labels=labels, right=False)

print(df)

3. Text Data Preprocessing

import nltk
from nltk.corpus import stopwords
from nltk.stem import PorterStemmer
from sklearn.feature_extraction.text import CountVectorizer

texts = ["This is a sample sentence.", "Text data preprocessing is important."]

stop_words = set(stopwords.words('english'))
stemmer = PorterStemmer()
vectorizer = CountVectorizer()

def preprocess_text(text):
    words = text.split()
    words = [stemmer.stem(word) for word in words if word.lower() not in stop_words]
    return " ".join(words)

cleaned_texts = [preprocess_text(text) for text in texts]
X = vectorizer.fit_transform(cleaned_texts)

print("Cleaned Texts:", cleaned_texts)
print("Vectorized Text:", X.toarray())

4. Feature Splitting

import pandas as pd

data = {
  'Full_Address': [
    '123 Elm St, Springfield, 12345',
    '456 Oak Rd, Shelbyville, 67890'
  ]
}

df = pd.DataFrame(data)

df[['Street', 'City', 'Zipcode']] = df['Full_Address'].str.extract(
    r'([0-9]+\s[\w\s]+),\s([\w\s]+),\s(\d+)'
)

print(df)

Tools for Feature Engineering include Featuretools, TPOT, DataRobot, Alteryx, and H2O.ai.