我正在使用5折交叉验证运行RandomizedSearchCV以寻找最佳参数。我有一个保留集(X_test
)用于预测。我的代码部分是:
svc= SVC(class_weight=class_weights, random_state=42)Cs = [0.01, 0.1, 1, 10, 100, 1000, 10000]gammas = [1e-1, 1e-2, 1e-3, 1e-4, 1e-5]param_grid = {'C': Cs, 'gamma': gammas, 'kernel': ['linear', 'rbf', 'poly']}my_cv = TimeSeriesSplit(n_splits=5).split(X_train)rs_svm = RandomizedSearchCV(SVC(), param_grid, cv = my_cv, scoring='accuracy', refit='accuracy', verbose = 3, n_jobs=1, random_state=42)rs_svm.fit(X_train, y_train)y_pred = rs_svm.predict(X_test)cm = confusion_matrix(y_test, y_pred)clfreport = classification_report(y_test, y_pred)print (rs_svm.best_params_)
现在,我有兴趣使用选定的参数通过独立运行的模型(不使用RandomizedSearchCV)来重现这个结果:
from sklearn.model_selection import TimeSeriesSplittcsv=TimeSeriesSplit(n_splits=5)for train_index, test_index in tcsv.split(X_train): train_index_ = int(train_index.shape[0]) test_index_ = int(test_index.shape[0]) X_train_, y_train_ = X_train[0:train_index_],y_train[0:train_index_] X_test_, y_test_ = X_train[test_index_:],y_train[test_index_:] class_weights = compute_class_weight('balanced', np.unique(y_train_), y_train_) class_weights = dict(enumerate(class_weights)) svc= SVC(C=0.01, gamma=0.1, kernel='linear', class_weight=class_weights, verbose=True, random_state=42) svc.fit(X_train_, y_train_) y_pred_=svc.predict(X_test)cm = confusion_matrix(y_test, y_pred_)clfreport = classification_report(y_test, y_pred_)
根据我的理解,分类报告应该是一样的,但在我运行后得到的结果是:
有没有人有建议为什么会发生这种情况?
回答:
根据您的第一个代码片段,您使用RandomizedSearchCV来寻找最佳超参数,您不需要再次进行分割;因此,在您的第二个代码片段中,您应该仅使用找到的超参数和类权重,使用整个训练集进行拟合,然后在测试集上进行预测:
class_weights = compute_class_weight('balanced', np.unique(y_train), y_train)class_weights = dict(enumerate(class_weights))svc= SVC(C=0.01, gamma=0.1, kernel='linear', class_weight=class_weights, verbose=True, random_state=42)svc.fit(X_train, y_train)y_pred_=svc.predict(X_test)cm = confusion_matrix(y_test, y_pred)clfreport = classification_report(y_test, y_pred)
讨论使用验证、训练和测试集的顺序可能有助于澄清过程…