我一直在尝试训练一个RandomForestRegressor模型,用于预测给定训练集中的房价数据到给定测试集。
import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_absolute_error
from sklearn.metrics import accuracy_score
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import OneHotEncoder
from sklearn.preprocessing import MaxAbsScaler
file='file:///F:/Download sort required/train.csv'
data=pd.read_csv(file)
data.dropna(axis=0,subset=['SalePrice'],inplace=True)
y=data.SalePrice
predictors=['LotArea','OverallQual','GrLivArea','GarageCars','TotRmsAbvGrd','Neighborhood','HouseStyle','YearBuilt','ExterQual','KitchenQual']
One_hot_encoded_predictors=['Neighborhood','HouseStyle','YearBuilt','ExterQual','KitchenQual']
X_uncoded=data[predictors]
#编码训练数据
X_uncoded=pd.get_dummies(X_uncoded,columns=One_hot_encoded_predictors)
X=X_uncoded
maxabsscaler=MaxAbsScaler()
X_max_abs=maxabsscaler.fit_transform(X)
model=RandomForestRegressor()
model.fit(X_max_abs,y)
test_file='file:///C:/Users/shand/Downloads/test.csv'
test_data=pd.read_csv(test_file)
X_uncoded_test=test_data[predictors]
X_uncoded_test=pd.get_dummies(X_uncoded_test,columns=One_hot_encoded_predictors)
X_test=X_uncoded_test
X_test.fillna(X_test.mean(),inplace=True)
X_max_abs_test=maxabsscaler.fit_transform(X_test)
predicted_prices=model.predict(X_max_abs_test)
my_submission = pd.DataFrame({'Id': test_data.Id, 'SalePrice': predicted_prices})
my_submission.to_csv('submission.csv', index=False)
我对分类特征进行了独热编码处理,接着应用了最大绝对值缩放器,因为大部分数据的范围在-1到1或0到1之间。但是在编译代码时,出现了以下错误-
> > 28 X_test.fillna(X_test.mean(),inplace=True)
> 29 X_max_abs_test=maxabsscaler.fit_transform(X_test)
> ---> 30 predicted_prices=model.predict(X_max_abs_test)
> 31 my_submission = pd.DataFrame({'Id': test_data.Id, 'SalePrice': predicted_prices})
> 32 my_submission.to_csv('submission.csv', index=False)
> > C:\Users\shand\Anaconda3\lib\site-packages\sklearn\ensemble\forest.py
> in predict(self, X)
> 683 """
> 684 # Check data
> --> 685 X = self._validate_X_predict(X)
> 686
> 687 # Assign chunk of trees to jobs
> > C:\Users\shand\Anaconda3\lib\site-packages\sklearn\ensemble\forest.py
> in _validate_X_predict(self, X)
> 353 "call `fit` before exploiting the model.")
> 354
> --> 355 return self.estimators_[0]._validate_X_predict(X, check_input=True)
> 356
> 357 @property
> > C:\Users\shand\Anaconda3\lib\site-packages\sklearn\tree\tree.py in
> _validate_X_predict(self, X, check_input)
> 374 "match the input. Model n_features is %s and "
> 375 "input n_features is %s "
> --> 376 % (self.n_features_, n_features))
> 377
> 378 return X
> > ValueError: Number of features of the model must match the input.
> Model n_features is 158 and input n_features is 151
在应用独热编码和最大绝对值缩放器后,模型的训练使用了158个特征。为什么虽然我对训练集和测试集数据应用了相同的转换,但仍然出现这个错误?如何修正这个错误?
PS-数据来源于-https://www.kaggle.com/c/house-prices-advanced-regression-techniques
回答:
正如您提到的,在编码后,训练数据和测试数据的列数不同。训练数据有158列,而测试数据只有151列。
#编码训练数据
X_uncoded=pd.get_dummies(X_uncoded,columns=One_hot_encoded_predictors)
X=X_uncoded
print(X.shape)
(1460, 158)
#编码测试数据
X_uncoded_test=pd.get_dummies(X_uncoded_test,columns=One_hot_encoded_predictors)
print(X_uncoded_test.shape)
(1459, 151)
这可能是因为测试数据的级别数量少于训练数据。请看下面的示例,来自pandas.get_dummies
您可以考虑在编码前先将训练和测试数据合并,然后在编码后再将它们分开为训练和测试数据,如这里所述