### 在scikit的MLPClassifier（和其他分类器）中训练得分低

(更新：发布了最终发现作为单独的答案)

我开始尝试理解如何使用scikit模型进行训练。我已经尝试过使用像iris、MNIST等知名数据集——它们都是结构良好的数据，准备好被使用。这是第一次我尝试自己从原始数据构建模型，结果不尽如人意。

这里是数据的一个快照，让你了解字段而无需下载数据。

我的第一个实验很简单——尝试构建一个模型，根据“驾照州代码”和“年龄”来预测性别（M或F）。

import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn import metrics
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import learning_curve
from sklearn.model_selection import ShuffleSplit
import tensorflow.contrib.learn as skflow
from tensorflow.contrib.learn.python.learn.estimators import run_config
from sklearn.svm import SVC
import pickle, seaborn
def plot_learning_curve(estimator, title, X, y, ylim=None, cv=None,
                        n_jobs=1, train_sizes=np.linspace(.1, 1.0, 5)):
    #http://scikit-learn.org/stable/auto_examples/model_selection/plot_learning_curve.html
    plt.figure()
    plt.title(title)
    if ylim is not None:
        plt.ylim(*ylim)
    plt.xlabel("训练样本")
    plt.ylabel("得分")
    train_sizes, train_scores, test_scores = learning_curve(
        estimator, X, y, cv=cv, n_jobs=n_jobs, train_sizes=train_sizes)
    train_scores_mean = np.mean(train_scores, axis=1)
    train_scores_std = np.std(train_scores, axis=1)
    test_scores_mean = np.mean(test_scores, axis=1)
    test_scores_std = np.std(test_scores, axis=1)
    plt.grid()
    plt.fill_between(train_sizes, train_scores_mean - train_scores_std,
                     train_scores_mean + train_scores_std, alpha=0.1,
                     color="r")
    plt.fill_between(train_sizes, test_scores_mean - test_scores_std,
                     test_scores_mean + test_scores_std, alpha=0.1, color="g")
    plt.plot(train_sizes, train_scores_mean, 'o-', color="r",
             label="训练得分")
    plt.plot(train_sizes, test_scores_mean, 'o-', color="g",
             label="交叉验证得分")
    plt.legend(loc="best")
    plt.show()
#MAIN
crashes = pd.read_csv("crashes.csv", nrows=100000)
# 删除无用列
crashes.drop(["Year","Case Individual ID", "Case Vehicle ID", "Transported    By", "Injury Location", "Role Type"],
axis=1, inplace=True)
crashes = crashes [pd.notnull(crashes['Age'])]
crashes = crashes[crashes.Age >= 10 ] # 有年龄<10的——可能是垃圾数据。我认为他们不会开车
# 让我们删除空行的数据
crashes = crashes [pd.notnull(crashes['License State Code'])]
crashes = crashes [pd.notnull(crashes['Injury Severity'])]
crashes = crashes [pd.notnull(crashes['Safety Equipment'])]
crashes = crashes [pd.notnull(crashes['Sex'])]
# 将文本字段转换为数值
le = LabelEncoder()
crashes = crashes[crashes.columns[:]].apply(le.fit_transform)
crashes = crashes._get_numeric_data()
# 让我们绘制一个热图来显示相关性
corr = crashes.corr()
ax = seaborn.heatmap (corr, xticklabels=corr.columns.values,
    yticklabels=corr.columns.values, annot=True)
plt.setp( ax.xaxis.get_majorticklabels(), rotation=45 )
plt.setp( ax.yaxis.get_majorticklabels(), rotation=-45 )
plt.show()
crashes_train, crashes_test = train_test_split(crashes, test_size = 0.2)
Y_train = crashes_train['Sex']
X_train =  crashes_train[[ 'Age',  'License State Code']]
Y_test = crashes_test['Sex']
X_test =  crashes_test[[ 'Age', 'License State Code']]
names_train  = crashes_train.columns.values
print "train size ",len (X_train)
print "test size",len (X_test)
## cls = RandomForestClassifier(verbose = True)
#cls = MLPClassifier(hidden_layer_sizes=(10,10,10), max_iter=500, alpha=1e-4,
                  solver='sgd', verbose=10, tol=1e-4, random_state=1,
                  learning_rate_init=0.01)
#cls = tf.contrib.learn.DNNClassifier(feature_columns=feats, #                                               hidden_units=[50, 50, 50], #                                              n_classes=3)
###
cls = SVC(verbose = True)
print "Fitting..."
cls.fit(X_train, Y_train)
plot_learning_curve(cls,"Crash Learning", X_train, Y_train)
print("Training set score: %f" % cls.score(X_train, Y_train))
print("Test set score: %f" % cls.score(X_test, Y_test))

我尝试了多个模型（从RandomForest到SVC再到MLP等）——它们都得出了大约0.56的训练得分和0.6x的损失

最后，这是当前配置下为MLP生成的图表：