当我使用fit时，一切正常，但当我使用fit_generator时，出现了问题。
我使用回调方法在每个训练周期结束时查找混淆矩阵。
然而，从混淆矩阵获得的准确率和Keras输出的验证准确率不同。
我的代码如下。
metrics = Valid_checker(model_name, args.patience, (x_valid, y_valid), x_length_valid)
model.compile(optimizer=optimizers.RMSprop(lr=args.lr),
loss=[first_loss],
loss_weights=[1.],
metrics={'capsnet': 'accuracy'})
callback_list = [lr_decay, metrics]
model.fit_generator(
no_decoder_generator(x_train, y_train),
steps_per_epoch=len(x_train),
epochs=args.epochs,
validation_data=no_decoder_generator(x_valid, y_valid),
validation_steps=len(x_valid),
callbacks=callback_list,
#class_weight=class_weights,
verbose=1)Valid check 是我的回调方法。 no_decoder_generator 是我的解码器生成器。我的训练和验证的批次大小为1。
这是我的 Valid_check 类。（如下）
class Valid_checker(keras.callbacks.Callback):
def __init__(self, model_name, patience, val_data, x_length):
super().__init__()
self.best_score = 0
self.patience = patience
self.current_patience = 0
self.model_name = model_name
self.validation_data = val_data
self.x_length = x_length
def on_epoch_end(self, epoch, logs={}):
X_val, y_val = self.validation_data
if args.decoder==1:
y_predict, x_predict = model.predict_generator(no_decoder_generator(X_val, y_val), steps=len(X_val))
y_predict = np.asarray(y_predict)
x_predict = np.asarray(x_predict)
else:
y_predict = np.asarray(model.predict_generator(predict_generator(X_val), steps=len(X_val)))
y_val, y_predict = get_utterence_label_pred(y_val, y_predict, self.x_length )
cnf_matrix = get_accuracy_and_cnf_matrix(y_val, y_predict)[1]
val_acc_custom = get_accuracy_and_cnf_matrix(y_val, y_predict)[0]
war = val_acc_custom[0]
uar = val_acc_custom[1]
score = round(0.2*war+0.8*uar,2)
loss_message=''
# 使用UAR进行自定义ModelCheckpoint和提前停止
loss_message='loss: %s - acc: %s - val_loss: %s - val_acc: %s'%(round(logs.get('loss'),4), round(logs.get('acc'),4), round(logs.get('val_loss'),4), round(logs.get('val_acc'),4))
log('[Epoch %03d/%03d]'%(epoch+1, args.epochs))
log(loss_message)
log('Confusion matrix:')
log('%s'%cnf_matrix)
log('Valid [WAR] [UAR] [Custom] : %s [%s]'%(val_acc_custom,score))
if score > self.best_score :
model.save_weights(model_name)
log('Epoch %05d: val_uar_acc improved from %s to %s saving model to %s'%(epoch+1, self.best_score, score, self.model_name))
self.best_score = score
self.current_patience = 0
else :
self.current_patience+=1
# 提前停止
if self.current_patience == (self.patience-1):
self.model.stop_training = True
log('Epoch %05d: early stopping' % (epoch + 1))
return它应该等于Keras输出的val_acc和war。然而，数值不同。为什么会发生这种情况？我已经确认get_utterence_label_pred和get_accuracy_and_cnf_matrix没有问题。当我使用fit函数时，它运行得很好。
我的生成器如下。
def predict_generator(x):
while True:
for index in range(len(x)):
feature = x[index]
feature = np.expand_dims(x[index],-1)
feature = np.expand_dims(feature,0) # make (1,input_height,input_width,1)
yield (feature)
def no_decoder_generator(x, y):
while True:
indexes = np.arange(len(x))
np.random.shuffle(indexes)
for index in indexes:
feature = x[index]
feature = np.expand_dims(x[index],-1)
feature = np.expand_dims(feature,0) # make (1,input_height,input_width,1)
label = y[index]
label = np.expand_dims(label,0)
yield (feature, label)第1/70个周期
1858/1858 [==============================] – 558s 300ms/step – loss: 1.0708 – acc: 0.5684 – val_loss: 0.9087 – val_acc: 0.6244 [Epoch 001/070]
loss: 1.0708 – acc: 0.5684 – val_loss: 0.9087 – val_acc: 0.6244
混淆矩阵:
[[ 0. 28. 68. 4. ]
[ 0. 13.33 80. 6.67]
[ 0.96 2.88 64.42 31.73]
[ 0. 0. 3.28 96.72]]
验证 [WAR] [UAR] [自定义] : [62.44 43.62] [47.38]第2/70个周期 1858/1858 [==============================] – 262s 141ms/step – loss: 0.9526 – acc: 0.6254 – val_loss: 1.1998 – val_acc: 0.4537 [Epoch 002/070]
loss: 0.9526 – acc: 0.6254 – val_loss: 1.1998 – val_acc: 0.4537
混淆矩阵:
[[ 36. 12. 24. 28. ]
[ 20. 0. 46.67 33.33]
[ 4.81 0.96 24.04 70.19]
[ 0. 0. 0. 100. ]]
验证 [WAR] [UAR] [自定义] : [46.34 40.01] [41.28]
回答：
我通过使用序列而不是生成器解决了这个问题。
我可以在以下来源中找出这种现象发生的原因。
https://github.com/keras-team/keras/issues/11878
使用序列的一个简单示例如下所示。
https://medium.com/datadriveninvestor/keras-training-on-large-datasets-3e9d9dbc09d4
