特征工程在機器學習中的重要性不言而喻，恰當的特征工程能顯著提升機器學習模型性能。我們在 Github 上整理編寫了一份系統的特征工程教程，供大家參考學習。
項目位址：
https://github.com/YC-Coder-Chen/feature-engineering-handbook本文将探讨資料預處理部分：介紹了如何利用 scikit-learn 處理靜态的連續變量，利用 Category Encoders 處理靜态的類别變量以及利用 Featuretools 處理常見的時間序列變量。
目錄
特征工程的資料預處理我們将分為三大部分來介紹：
- 靜态連續變量
- 靜态類别變量
- 時間序列變量
本文将介紹 1.1 靜态連續變量的資料預處理。下面将結合 Jupyter，使用 sklearn，進行詳解。
1.1 靜态連續變量
1.1.1 離散化
離散化連續變量可以使模型更加穩健。例如，當預測客戶的購買行為時，一個已有 30 次購買行為的客戶可能與一個已有 32 次購買行為的客戶具有非常相似的行為。有時特征中的過精度可能是噪聲，這就是為什麼在 LightGBM 中，模型采用直方圖算法來防止過拟合。離散連續變量有兩種方法。
1.1.1.1 二值化
将數值特征二值化。
# load the sample data
from sklearn.datasets import fetch_california_housing
dataset = fetch_california_housing()
X, y = dataset.data, dataset.target # we will take the first column as the example later %matplotlib inline
import seaborn as sns
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
sns.distplot(X[:,0], hist = True, kde=True)
ax.set_title('Histogram', fontsize=12)
ax.set_xlabel('Value', fontsize=12)
ax.set_ylabel('Frequency', fontsize=12); # this feature has long-tail distribution
1.1.1.2 分箱
将數值特征分箱。
均勻分箱：
from sklearn.preprocessing import Binarizer
sample_columns = X[0:10,0] # select the top 10 samples
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
model = Binarizer(threshold=6) # set 6 to be the threshold
# if value <= 6, then return 0 else return 1
result = model.fit_transform(sample_columns.reshape(-1,1)).reshape(-1)
# return array([1., 1., 1., 0., 0., 0., 0., 0., 0., 0.])
from sklearn.preprocessing import KBinsDiscretizer
# in order to mimic the operation in real-world, we shall fit the KBinsDiscretizer
# on the trainset and transform the testset
# we take the top ten samples in the first column as test set
# take the rest samples in the first column as train set
test_set = X[0:10,0]
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
train_set = X[10:,0]
model = KBinsDiscretizer(n_bins=5, encode='ordinal', strategy='uniform') # set 5 bins
# return oridinal bin number, set all bins to have identical widths
model.fit(train_set.reshape(-1,1))
result = model.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([2., 2., 2., 1., 1., 1., 1., 0., 0., 1.])
bin_edge = model.bin_edges_[0]
# return array([ 0.4999 , 3.39994, 6.29998, 9.20002, 12.10006, 15.0001 ]), the bin edges
# visualiza the bin edges
fig, ax = plt.subplots()
sns.distplot(train_set, hist = True, kde=True)
for edge in bin_edge: # uniform bins
line = plt.axvline(edge, color='b')
ax.legend([line], ['Uniform Bin Edges'], fontsize=10)
ax.set_title('Histogram', fontsize=12)
ax.set_xlabel('Value', fontsize=12)
ax.set_ylabel('Frequency', fontsize=12); from sklearn.preprocessing import KBinsDiscretizer
# in order to mimic the operation in real-world, we shall fit the KBinsDiscretizer
# on the trainset and transform the testset
# we take the top ten samples in the first column as test set
# take the rest samples in the first column as train set
test_set = X[0:10,0]
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
train_set = X[10:,0]
model = KBinsDiscretizer(n_bins=5, encode='ordinal', strategy='quantile') # set 3 bins
# return oridinal bin number, set all bins based on quantile
model.fit(train_set.reshape(-1,1))
result = model.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([4., 4., 4., 4., 2., 3., 2., 1., 0., 2.])
bin_edge = model.bin_edges_[0]
# return array([ 0.4999 , 2.3523 , 3.1406 , 3.9667 , 5.10824, 15.0001 ]), the bin edges
# 2.3523 is the 20% quantile
# 3.1406 is the 40% quantile, etc.. # visualiza the bin edges
fig, ax = plt.subplots()
sns.distplot(train_set, hist = True, kde=True)
for edge in bin_edge: # quantile based bins
line = plt.axvline(edge, color='b')
ax.legend([line], ['Quantiles Bin Edges'], fontsize=10)
ax.set_title('Histogram', fontsize=12)
ax.set_xlabel('Value', fontsize=12)
ax.set_ylabel('Frequency', fontsize=12); 1.1.2 縮放
不同尺度的特征之間難以比較，特别是線上性回歸和邏輯回歸等線性模型中。在基于歐氏距離的 k-means 聚類或 KNN 模型中，就需要進行特征縮放，否則距離的測量是無用的。而對于任何使用梯度下降的算法，縮放也會加快收斂速度。
一些常用的模型：
注：偏度影響 PCA 模型，是以最好使用幂變換來消除偏度。
1.1.2.1 标準縮放（Z 分數标準化）
公式：
其中，X 是變量（特征），𝜇 是 X 的均值，𝜎 是 X 的标準差。此方法對異常值非常敏感，因為異常值同時影響到 𝜇 和 𝜎。
from sklearn.preprocessing import StandardScaler
# in order to mimic the operation in real-world, we shall fit the StandardScaler
# on the trainset and transform the testset
# we take the top ten samples in the first column as test set
# take the rest samples in the first column as train set
test_set = X[0:10,0]
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
train_set = X[10:,0]
model = StandardScaler()
model.fit(train_set.reshape(-1,1)) # fit on the train set and transform the test set
# top ten numbers for simplification
result = model.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([ 2.34539745, 2.33286782, 1.78324852, 0.93339178, -0.0125957 ,
# 0.08774668, -0.11109548, -0.39490751, -0.94221041, -0.09419626])
# result is the same as ((X[0:10,0] - X[10:,0].mean())/X[10:,0].std()) # visualize the distribution after the scaling
# fit and transform the entire first feature
import seaborn as sns
import matplotlib.pyplot as plt
fig, ax = plt.subplots(2,1, figsize = (13,9))
sns.distplot(X[:,0], hist = True, kde=True, ax=ax[0])
ax[0].set_title('Histogram of the Original Distribution', fontsize=12)
ax[0].set_xlabel('Value', fontsize=12)
ax[0].set_ylabel('Frequency', fontsize=12); # this feature has long-tail distribution
model = StandardScaler()
model.fit(X[:,0].reshape(-1,1))
result = model.transform(X[:,0].reshape(-1,1)).reshape(-1)
# show the distribution of the entire feature
sns.distplot(result, hist = True, kde=True, ax=ax[1])
ax[1].set_title('Histogram of the Transformed Distribution', fontsize=12)
ax[1].set_xlabel('Value', fontsize=12)
ax[1].set_ylabel('Frequency', fontsize=12); # the distribution is the same, but scales change
fig.tight_layout() from sklearn.preprocessing import MinMaxScaler
# in order to mimic the operation in real-world, we shall fit the MinMaxScaler
# on the trainset and transform the testset
# we take the top ten samples in the first column as test set
# take the rest samples in the first column as train set
test_set = X[0:10,0]
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
train_set = X[10:,0]
model = MinMaxScaler(feature_range=(0,1)) # set the range to be (0,1)
model.fit(train_set.reshape(-1,1)) # fit on the train set and transform the test set
# top ten numbers for simplification
result = model.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([0.53966842, 0.53802706, 0.46602805, 0.35469856, 0.23077613,
# 0.24392077, 0.21787286, 0.18069406, 0.1089985 , 0.22008662])
# result is the same as (X[0:10,0] - X[10:,0].min())/(X[10:,0].max()-X[10:,0].min()) # visualize the distribution after the scaling
# fit and transform the entire first feature
import seaborn as sns
import matplotlib.pyplot as plt
fig, ax = plt.subplots(2,1, figsize = (13,9))
sns.distplot(X[:,0], hist = True, kde=True, ax=ax[0])
ax[0].set_title('Histogram of the Original Distribution', fontsize=12)
ax[0].set_xlabel('Value', fontsize=12)
ax[0].set_ylabel('Frequency', fontsize=12); # this feature has long-tail distribution
model = MinMaxScaler(feature_range=(0,1))
model.fit(X[:,0].reshape(-1,1))
result = model.transform(X[:,0].reshape(-1,1)).reshape(-1)
# show the distribution of the entire feature
sns.distplot(result, hist = True, kde=True, ax=ax[1])
ax[1].set_title('Histogram of the Transformed Distribution', fontsize=12)
ax[1].set_xlabel('Value', fontsize=12)
ax[1].set_ylabel('Frequency', fontsize=12); # the distribution is the same, but scales change
fig.tight_layout() # now the scale change to [0,1] import numpy as np
from sklearn.preprocessing import RobustScaler
# in order to mimic the operation in real-world, we shall fit the RobustScaler
# on the trainset and transform the testset
# we take the top ten samples in the first column as test set
# take the rest samples in the first column as train set
test_set = X[0:10,0]
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
train_set = X[10:,0]
model = RobustScaler(with_centering = True, with_scaling = True,
quantile_range = (25.0, 75.0))
# with_centering = True => recenter the feature by set X' = X - X.median()
# with_scaling = True => rescale the feature by the quantile set by user
# set the quantile to the (25%, 75%)
model.fit(train_set.reshape(-1,1)) # fit on the train set and transform the test set
# top ten numbers for simplification
result = model.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([ 2.19755974, 2.18664281, 1.7077657 , 0.96729508, 0.14306683,
# 0.23049401, 0.05724508, -0.19003715, -0.66689601, 0.07196918])
# result is the same as (X[0:10,0] - np.quantile(X[10:,0], 0.5))/(np.quantile(X[10:,0],0.75)-np.quantile(X[10:,0], 0.25))
# visualize the distribution after the scaling
# fit and transform the entire first feature
import seaborn as sns
import matplotlib.pyplot as plt
fig, ax = plt.subplots(2,1, figsize = (13,9))
sns.distplot(X[:,0], hist = True, kde=True, ax=ax[0])
ax[0].set_title('Histogram of the Original Distribution', fontsize=12)
ax[0].set_xlabel('Value', fontsize=12)
ax[0].set_ylabel('Frequency', fontsize=12); # this feature has long-tail distribution
model = RobustScaler(with_centering = True, with_scaling = True,
quantile_range = (25.0, 75.0))
model.fit(X[:,0].reshape(-1,1))
result = model.transform(X[:,0].reshape(-1,1)).reshape(-1)
# show the distribution of the entire feature
sns.distplot(result, hist = True, kde=True, ax=ax[1])
ax[1].set_title('Histogram of the Transformed Distribution', fontsize=12)
ax[1].set_xlabel('Value', fontsize=12)
ax[1].set_ylabel('Frequency', fontsize=12); # the distribution is the same, but scales change
fig.tight_layout() from sklearn.preprocessing import PowerTransformer
# in order to mimic the operation in real-world, we shall fit the PowerTransformer
# on the trainset and transform the testset
# we take the top ten samples in the first column as test set
# take the rest samples in the first column as train set
test_set = X[0:10,0]
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
train_set = X[10:,0]
model = PowerTransformer(method='box-cox', standardize=True)
# apply box-cox transformation
model.fit(train_set.reshape(-1,1)) # fit on the train set and transform the test set
# top ten numbers for simplification
result = model.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([ 1.91669292, 1.91009687, 1.60235867, 1.0363095 , 0.19831579,
# 0.30244247, 0.09143411, -0.24694006, -1.08558469, 0.11011933]) # visualize the distribution after the scaling
# fit and transform the entire first feature
import seaborn as sns
import matplotlib.pyplot as plt
fig, ax = plt.subplots(2,1, figsize = (13,9))
sns.distplot(X[:,0], hist = True, kde=True, ax=ax[0])
ax[0].set_title('Histogram of the Original Distribution', fontsize=12)
ax[0].set_xlabel('Value', fontsize=12)
ax[0].set_ylabel('Frequency', fontsize=12); # this feature has long-tail distribution
model = PowerTransformer(method='box-cox', standardize=True)
model.fit(X[:,0].reshape(-1,1))
result = model.transform(X[:,0].reshape(-1,1)).reshape(-1)
# show the distribution of the entire feature
sns.distplot(result, hist = True, kde=True, ax=ax[1])
ax[1].set_title('Histogram of the Transformed Distribution', fontsize=12)
ax[1].set_xlabel('Value', fontsize=12)
ax[1].set_ylabel('Frequency', fontsize=12); # the distribution now becomes normal
fig.tight_layout() from sklearn.preprocessing import PowerTransformer
# in order to mimic the operation in real-world, we shall fit the PowerTransformer
# on the trainset and transform the testset
# we take the top ten samples in the first column as test set
# take the rest samples in the first column as train set
test_set = X[0:10,0]
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
train_set = X[10:,0]
model = PowerTransformer(method='yeo-johnson', standardize=True)
# apply box-cox transformation
model.fit(train_set.reshape(-1,1)) # fit on the train set and transform the test set
# top ten numbers for simplification
result = model.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([ 1.90367888, 1.89747091, 1.604735 , 1.05166306, 0.20617221,
# 0.31245176, 0.09685566, -0.25011726, -1.10512438, 0.11598074]) # visualize the distribution after the scaling
# fit and transform the entire first feature
import seaborn as sns
import matplotlib.pyplot as plt
fig, ax = plt.subplots(2,1, figsize = (13,9))
sns.distplot(X[:,0], hist = True, kde=True, ax=ax[0])
ax[0].set_title('Histogram of the Original Distribution', fontsize=12)
ax[0].set_xlabel('Value', fontsize=12)
ax[0].set_ylabel('Frequency', fontsize=12); # this feature has long-tail distribution
model = PowerTransformer(method='yeo-johnson', standardize=True)
model.fit(X[:,0].reshape(-1,1))
result = model.transform(X[:,0].reshape(-1,1)).reshape(-1)
# show the distribution of the entire feature
sns.distplot(result, hist = True, kde=True, ax=ax[1])
ax[1].set_title('Histogram of the Transformed Distribution', fontsize=12)
ax[1].set_xlabel('Value', fontsize=12)
ax[1].set_ylabel('Frequency', fontsize=12); # the distribution now becomes normal
fig.tight_layout() from sklearn.preprocessing import Normalizer
# Normalizer performs operation on each row independently
# So train set and test set are processed independently
###### for L1 Norm
sample_columns = X[0:2,0:3] # select the first two samples, and the first three features
# return array([[ 8.3252, 41., 6.98412698],
# [ 8.3014 , 21., 6.23813708]])
model = Normalizer(norm='l1')
# use L2 Norm to normalize each samples
model.fit(sample_columns)
result = model.transform(sample_columns) # test set are processed similarly
# return array([[0.14784762, 0.72812094, 0.12403144],
# [0.23358211, 0.59089121, 0.17552668]])
# result = sample_columns/np.sum(np.abs(sample_columns), axis=1).reshape(-1,1)
L2 正則化：
###### for L2 Norm
sample_columns = X[0:2,0:3] # select the first three features
# return array([[ 8.3252, 41., 6.98412698],
# [ 8.3014 , 21., 6.23813708]])
model = Normalizer(norm='l2')
# use L2 Norm to normalize each samples
model.fit(sample_columns)
result = model.transform(sample_columns)
# return array([[0.19627663, 0.96662445, 0.16465922],
# [0.35435076, 0.89639892, 0.26627902]])
# result = sample_columns/np.sqrt(np.sum(sample_columns**2, axis=1)).reshape(-1,1)
# visualize the difference in the distribuiton after Normalization
# compare it with the distribuiton after RobustScaling
# fit and transform the entire first & second feature
import seaborn as sns
import matplotlib.pyplot as plt
# RobustScaler
fig, ax = plt.subplots(2,1, figsize = (13,9))
model = RobustScaler(with_centering = True, with_scaling = True,
quantile_range = (25.0, 75.0))
model.fit(X[:,0:2])
result = model.transform(X[:,0:2])
sns.scatterplot(result[:,0], result[:,1], ax=ax[0])
ax[0].set_title('Scatter Plot of RobustScaling result', fontsize=12)
ax[0].set_xlabel('Feature 1', fontsize=12)
ax[0].set_ylabel('Feature 2', fontsize=12);
model = Normalizer(norm='l2')
model.fit(X[:,0:2])
result = model.transform(X[:,0:2])
sns.scatterplot(result[:,0], result[:,1], ax=ax[1])
ax[1].set_title('Scatter Plot of Normalization result', fontsize=12)
ax[1].set_xlabel('Feature 1', fontsize=12)
ax[1].set_ylabel('Feature 2', fontsize=12);
fig.tight_layout() # Normalization distort the original distribution 1.1.4 缺失值的估算
在實際操作中，資料集中可能缺少值。然而，這種稀疏的資料集與大多數 scikit 學習模型不相容，這些模型假設所有特征都是數值的，而沒有丢失值。是以在應用 scikit 學習模型之前，我們需要估算缺失的值。
但是一些新的模型，比如在其他包中實作的 XGboost、LightGBM 和 Catboost，為資料集中丢失的值提供了支援。是以在應用這些模型時，我們不再需要填充資料集中丢失的值。
1.1.4.1 單變量特征插補
假設第 i 列中有缺失值，那麼我們将用常數或第 i 列的統計資料（平均值、中值或模式）對其進行估算。
from sklearn.impute import SimpleImputer
test_set = X[0:10,0].copy() # no missing values
# return array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
# manully create some missing values
test_set[3] = np.nan
test_set[6] = np.nan
# now sample_columns becomes
# array([8.3252, 8.3014, 7.2574, nan, 3.8462, 4.0368, nan, 3.12 ,2.0804, 3.6912])
# create the test samples
# in real-world, we should fit the imputer on train set and tranform the test set.
train_set = X[10:,0].copy()
train_set[3] = np.nan
train_set[6] = np.nan
imputer = SimpleImputer(missing_values=np.nan, strategy='mean') # use mean
# we can set the strategy to 'mean', 'median', 'most_frequent', 'constant'
imputer.fit(train_set.reshape(-1,1))
result = imputer.transform(test_set.reshape(-1,1)).reshape(-1)
# return array([8.3252 , 8.3014 , 7.2574 , 3.87023658, 3.8462 ,
# 4.0368 , 3.87023658, 3.12 , 2.0804 , 3.6912 ])
# all missing values are imputed with 3.87023658
# 3.87023658 = np.nanmean(train_set)
# which is the mean of the trainset ignoring missing values 1.1.4.2 多元特征插補
多元特征插補利用整個資料集的資訊來估計和插補缺失值。在 scikit-learn 中，它以循環疊代的方式實作。
在每一步中，一個特征列被指定為輸出 y，其他特征列被視為輸入 X。一個回歸器适用于已知 y 的（X，y）。然後，回歸器被用來預測 y 的缺失值。這是以疊代的方式對每個特征進行的，然後對最大值插補回合重複進行。
使用線性模型（以 BayesianRidge 為例）：
from sklearn.experimental import enable_iterative_imputer # have to import this to enable
# IterativeImputer
from sklearn.impute import IterativeImputer
from sklearn.linear_model import BayesianRidge
test_set = X[0:10,:].copy() # no missing values, select all features
# the first columns is
# array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
# manully create some missing values
test_set[3,0] = np.nan
test_set[6,0] = np.nan
test_set[3,1] = np.nan
# now the first feature becomes
# array([8.3252, 8.3014, 7.2574, nan, 3.8462, 4.0368, nan, 3.12 ,2.0804, 3.6912])
# create the test samples
# in real-world, we should fit the imputer on train set and tranform the test set.
train_set = X[10:,:].copy()
train_set[3,0] = np.nan
train_set[6,0] = np.nan
train_set[3,1] = np.nan
impute_estimator = BayesianRidge()
imputer = IterativeImputer(max_iter = 10,
random_state = 0,
estimator = impute_estimator)
imputer.fit(train_set)
result = imputer.transform(test_set)[:,0] # only select the first column to revel how it works
# return array([8.3252 , 8.3014 , 7.2574 , 4.6237195 , 3.8462 ,
# 4.0368 , 4.00258149, 3.12 , 2.0804 , 3.6912 ]) 使用基于樹的模型（以 ExtraTrees 為例）：
from sklearn.experimental import enable_iterative_imputer # have to import this to enable
# IterativeImputer
from sklearn.impute import IterativeImputer
from sklearn.ensemble import ExtraTreesRegressor
test_set = X[0:10,:].copy() # no missing values, select all features
# the first columns is
# array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
# manully create some missing values
test_set[3,0] = np.nan
test_set[6,0] = np.nan
test_set[3,1] = np.nan
# now the first feature becomes
# array([8.3252, 8.3014, 7.2574, nan, 3.8462, 4.0368, nan, 3.12 ,2.0804, 3.6912])
# create the test samples
# in real-world, we should fit the imputer on train set and tranform the test set.
train_set = X[10:,:].copy()
train_set[3,0] = np.nan
train_set[6,0] = np.nan
train_set[3,1] = np.nan
impute_estimator = ExtraTreesRegressor(n_estimators=10, random_state=0)
# parameters can be turned in CV though sklearn pipeline
imputer = IterativeImputer(max_iter = 10,
random_state = 0,
estimator = impute_estimator)
imputer.fit(train_set)
result = imputer.transform(test_set)[:,0] # only select the first column to revel how it works
# return array([8.3252 , 8.3014 , 7.2574 , 4.63813, 3.8462 , 4.0368 , 3.24721,
# 3.12 , 2.0804 , 3.6912 ]) 使用 K 近鄰（KNN）：
from sklearn.experimental import enable_iterative_imputer # have to import this to enable
# IterativeImputer
from sklearn.impute import IterativeImputer
from sklearn.neighbors import KNeighborsRegressor
test_set = X[0:10,:].copy() # no missing values, select all features
# the first columns is
# array([8.3252, 8.3014, 7.2574, 5.6431, 3.8462, 4.0368, 3.6591, 3.12, 2.0804, 3.6912])
# manully create some missing values
test_set[3,0] = np.nan
test_set[6,0] = np.nan
test_set[3,1] = np.nan
# now the first feature becomes
# array([8.3252, 8.3014, 7.2574, nan, 3.8462, 4.0368, nan, 3.12 ,2.0804, 3.6912])
# create the test samples
# in real-world, we should fit the imputer on train set and tranform the test set.
train_set = X[10:,:].copy()
train_set[3,0] = np.nan
train_set[6,0] = np.nan
train_set[3,1] = np.nan
impute_estimator = KNeighborsRegressor(n_neighbors=10,
p = 1) # set p=1 to use manhanttan distance
# use manhanttan distance to reduce effect from outliers
# parameters can be turned in CV though sklearn pipeline
imputer = IterativeImputer(max_iter = 10,
random_state = 0,
estimator = impute_estimator)
imputer.fit(train_set)
result = imputer.transform(test_set)[:,0] # only select the first column to revel how it works
# return array([8.3252, 8.3014, 7.2574, 3.6978, 3.8462, 4.0368, 4.052 , 3.12 ,
# 2.0804, 3.6912]) 1.1.4.3 标記估算值
有時，某些缺失值可能是有用的。是以，scikit learn 還提供了将缺少值的資料集轉換為相應的二進制矩陣的功能，該矩陣訓示資料集中缺少值的存在。
from sklearn.impute import MissingIndicator
# illustrate this function on trainset only
# since the precess is independent in train set and test set
train_set = X[10:,:].copy() # select all features
train_set[3,0] = np.nan # manully create some missing values
train_set[6,0] = np.nan
train_set[3,1] = np.nan
indicator = MissingIndicator(missing_values=np.nan, features='all')
# show the results on all the features
result = indicator.fit_transform(train_set) # result have the same shape with train_set
# contains only True & False, True corresponds with missing value
result[:,0].sum() # should return 2, the first column has two missing values
result[:,1].sum(); # should return 1, the second column has one missing value
1.1.5 特征變換
1.1.5.1 多項式變換
有時我們希望在模型中引入非線性特征，進而增加模型的複雜度。對于簡單的線性模型，這将大大增加模型的複雜度。但是對于更複雜的模型，如基于樹的 ML 模型，它們已經在非參數樹結構中包含了非線性關系。是以，這種特性轉換可能對基于樹的 ML 模型沒有太大幫助。
例如，如果我們将階數設定為 3，形式如下：
from sklearn.preprocessing import PolynomialFeatures
# illustrate this function on one synthesized sample
train_set = np.array([2,3]).reshape(1,-1) # shape (1,2)
# return array([[2, 3]])
poly = PolynomialFeatures(degree = 3, interaction_only = False)
# the highest degree is set to 3, and we want more than just intereaction terms
result = poly.fit_transform(train_set) # have shape (1, 10)
# array([[ 1., 2., 3., 4., 6., 9., 8., 12., 18., 27.]]) 1.1.5.2 自定義變換
from sklearn.preprocessing import FunctionTransformer
# illustrate this function on one synthesized sample
train_set = np.array([2,3]).reshape(1,-1) # shape (1,2)
# return array([[2, 3]])
transformer = FunctionTransformer(func = np.log1p, validate=True)
# perform log transformation, X' = log(1 + x)
# func can be any numpy function such as np.exp
result = transformer.transform(train_set)
# return array([[1.09861229, 1.38629436]]), the same as np.log1p(train_set) 好了，以上就是關于靜态連續變量的資料預處理介紹。建議讀者結合代碼，在 Jupyter 中實操一遍。