我在尝试对一个数值数据集实现逻辑回归分类器时遇到了问题。使用Tensorflow构建的模型无法获得良好的准确率和损失。为了排除数据本身的问题，我尝试使用了scikit-learn的LogisticRegression，结果得到了大幅改善。差异如此之大，我怀疑自己在Tensorflow方面犯了一些非常基本的错误…
数据预处理:
dt = pd.read_csv('data.csv', header=0)npArray = np.array(dt)xvals = npArray[:,1:].astype(float)yvals = npArray[:,0]x_proc = preprocessing.scale(xvals)XTrain, XTest, yTrain, yTest = train_test_split(x_proc, yvals, random_state=1)如果我现在使用sklearn进行逻辑回归:
log_reg = LogisticRegression(class_weight='balanced')log_reg.fit(XTrain, yTrain)yPred = log_reg.predict(XTest)print (metrics.classification_report(yTest, yPred))print ("Overall Accuracy:", round(metrics.accuracy_score(yTest, yPred),2))…我得到了以下混淆矩阵:
precision recall f1-score support 1 1.00 0.98 0.99 52 2 0.96 1.00 0.98 52 3 0.98 0.96 0.97 51 4 0.98 0.97 0.97 58 5 1.00 0.95 0.97 37 6 0.93 1.00 0.96 65 7 1.00 0.95 0.97 41 8 0.94 0.98 0.96 50 9 1.00 0.98 0.99 45 10 1.00 0.98 0.99 49 avg/total 0.98 0.98 0.98 500 Overall Accuracy: 0.98效果很好，对吧？下面是分割后的Tensorflow代码:
yTrain.resize(len(yTrain),10) #the labels are scores between 1 and 10yTest.resize(len(yTest),10)tf.reset_default_graph()X = tf.placeholder(tf.float32, [None, 8], name="input") Y = tf.placeholder(tf.float32, [None, 10])W = tf.Variable(tf.zeros([8, 10])) b = tf.Variable(tf.zeros([10])) out = (tf.matmul(X, W) + b)pred = tf.nn.softmax(out, name="output")learning_rate = 0.001training_epochs = 100batch_size = 200display_step = 1L2_LOSS = 0.01l2 = L2_LOSS * \ sum(tf.nn.l2_loss(tf_var) for tf_var in tf.trainable_variables())# Minimize error using cross entropycost = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits = out, labels = Y)) + l2# Gradient Descentoptimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)train_count = len(XTrain)#defining optimizer and accuracycorrect_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(Y, 1))accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))#----Training the model------------------------------------------saver = tf.train.Saver()history = dict(train_loss=[], train_acc=[], test_loss=[], test_acc=[])sess=tf.InteractiveSession()sess.run(tf.global_variables_initializer())for i in range(1, training_epochs + 1): for start, end in zip(range(0, train_count, batch_size), range(batch_size, train_count + 1,batch_size)): sess.run(optimizer, feed_dict={X: XTrain[start:end], Y: yTrain[start:end]}) _, acc_train, loss_train = sess.run([pred, accuracy, cost], feed_dict={ X: XTrain, Y: yTrain}) _, acc_test, loss_test = sess.run([pred, accuracy, cost], feed_dict={ X: XTest, Y: yTest}) history['train_loss'].append(loss_train) history['train_acc'].append(acc_train) history['test_loss'].append(loss_test) history['test_acc'].append(acc_test) if i != 1 and i % 10 != 0: continue print(f'epoch: {i} test accuracy: {acc_test} loss: {loss_test}')predictions, acc_final, loss_final = sess.run([pred, accuracy, cost], feed_dict={X: XTest, Y: yTest})print()print(f'final results: accuracy: {acc_final} loss: {loss_final}')现在我得到了以下结果:
epoch: 1 test accuracy: 0.41200000047683716 loss: 0.6921926140785217epoch: 10 test accuracy: 0.5 loss: 0.6909801363945007epoch: 20 test accuracy: 0.5180000066757202 loss: 0.6918861269950867epoch: 30 test accuracy: 0.515999972820282 loss: 0.6927152872085571epoch: 40 test accuracy: 0.5099999904632568 loss: 0.6933282613754272epoch: 50 test accuracy: 0.5040000081062317 loss: 0.6937957406044006epoch: 60 test accuracy: 0.5019999742507935 loss: 0.6941683292388916epoch: 70 test accuracy: 0.5019999742507935 loss: 0.6944747567176819epoch: 80 test accuracy: 0.4959999918937683 loss: 0.6947320103645325epoch: 90 test accuracy: 0.46799999475479126 loss: 0.6949512958526611epoch: 100 test accuracy: 0.4560000002384186 loss: 0.6951409578323364final results: accuracy: 0.4560000002384186 loss: 0.6951409578323364有什么想法吗？我尝试过初始化权重（参考这里的第二个答案：如何在TensorFlow上进行Xavier初始化），更改学习率、轮数、批量大小、L2损失等，但都没有显著效果。任何帮助都将非常感激…
回答：
我想我找到了问题的根源 – yTrain.resize和yTest.resize在逻辑和数学上都是愚蠢的，一旦我用one-hot编码数组替换它们（参考将索引数组转换为1-hot编码的numpy数组），一切都开始运作得更好。最终得到了与sk-learn相同的准确率（我想）！
