我无法手动匹配LGBM的交叉验证得分。
这是一个最简可重现示例（MCVE）：
from sklearn.datasets import load_breast_cancerimport pandas as pdfrom sklearn.model_selection import train_test_split, KFoldfrom sklearn.metrics import roc_auc_scoreimport lightgbm as lgbimport numpy as npdata = load_breast_cancer()X = pd.DataFrame(data.data, columns=data.feature_names)y = pd.Series(data.target)X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)folds = KFold(5, random_state=42)params = {'random_state': 42}results = lgb.cv(params, lgb.Dataset(X_train, y_train), folds=folds, num_boost_round=1000, early_stopping_rounds=100, metrics=['auc'])print('LGBM\'s cv score: ', results['auc-mean'][-1])clf = lgb.LGBMClassifier(**params, n_estimators=len(results['auc-mean']))val_scores = []for train_idx, val_idx in folds.split(X_train): clf.fit(X_train.iloc[train_idx], y_train.iloc[train_idx]) val_scores.append(roc_auc_score(y_train.iloc[val_idx], clf.predict_proba(X_train.iloc[val_idx])[:,1]))print('Manual score: ', np.mean(np.array(val_scores)))我原本期望这两个交叉验证得分是相同的 – 我已经设置了随机种子，并且做了完全相同的事情。但它们却不同。
这是我得到的输出：
LGBM's cv score: 0.9851513530737058Manual score: 0.9903622177441328为什么？难道我没有正确使用LGBM的cv模块吗？
回答：
你将X分成了X_train和X_test。对于交叉验证，你将X_train分成了5折，而手动操作时，你将X分成了5折。也就是说，手动操作时你使用的数据点比交叉验证时多。
将results = lgb.cv(params, lgb.Dataset(X_train, y_train)改为results = lgb.cv(params, lgb.Dataset(X, y)
此外，可能存在不同的参数。例如，LightGBM使用的线程数会影响结果。在交叉验证过程中，模型是并行训练的。因此，用于交叉验证的线程数可能与你手动顺序训练时不同。
第一次修正后的编辑：
你可以使用以下代码，通过手动分割/交叉验证来获得相同的结果：
from sklearn.datasets import load_breast_cancerimport pandas as pdfrom sklearn.model_selection import train_test_split, KFoldfrom sklearn.metrics import roc_auc_scoreimport lightgbm as lgbimport numpy as npdata = load_breast_cancer()X = pd.DataFrame(data.data, columns=data.feature_names)y = pd.Series(data.target)X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)folds = KFold(5, random_state=42)params = { 'task': 'train', 'boosting_type': 'gbdt', 'objective':'binary', 'metric':'auc', }data_all = lgb.Dataset(X_train, y_train)results = lgb.cv(params, data_all, folds=folds.split(X_train), num_boost_round=1000, early_stopping_rounds=100)print('LGBM\'s cv score: ', results['auc-mean'][-1])val_scores = []for train_idx, val_idx in folds.split(X_train): data_trd = lgb.Dataset(X_train.iloc[train_idx], y_train.iloc[train_idx], reference=data_all) gbm = lgb.train(params, data_trd, num_boost_round=len(results['auc-mean']), verbose_eval=100) val_scores.append(roc_auc_score(y_train.iloc[val_idx], gbm.predict(X_train.iloc[val_idx])))print('Manual score: ', np.mean(np.array(val_scores)))结果是
LGBM's cv score: 0.9914524426410262Manual score: 0.9914524426410262造成差异的是这一行reference=data_all。在交叉验证过程中，变量的分箱（参考LightGBM文档）是使用整个数据集（X_train）构建的，而在你的手动循环中，它是在训练子集（X_train.iloc[train_idx]）上构建的。通过传递包含所有数据的数据集的引用，LightGBM将重用相同的分箱，从而得出相同的结果。
