我对深度学习模型非常新手，正在尝试使用LSTM训练一个多标签文本分类模型。我有大约2600条记录，包含4个类别。使用80%的数据进行训练，其余用于验证。
代码中没有复杂的内容，即我读取csv文件，对数据进行分词，然后输入模型。但是在3-4个epoch之后，验证损失大于1，而训练损失趋近于零。据我所知，这是过拟合的情况。为了克服这个问题，我尝试了不同的层，改变了单元数。但问题依然存在。如果我在1-2个epoch时停止，那么预测结果就会出错。
以下是我的模型创建代码：
ACCURACY_THRESHOLD = 0.75class myCallback(tf.keras.callbacks.Callback): def on_epoch_end(self, epoch, logs={}): print(logs.get('val_accuracy')) fname='Arabic_Model_'+str(logs.get('val_accuracy'))+'.h5' if(logs.get('val_accuracy') > ACCURACY_THRESHOLD): #print("\nWe have reached %2.2f%% accuracy, so we will stopping training." %(acc_thresh*100)) #self.model.stop_training = True self.model.save(fname) #from google.colab import files #files.download(fname) # The maximum number of words to be used. (most frequent)MAX_NB_WORDS = vocab_len# Max number of words in each complaint.MAX_SEQUENCE_LENGTH = 50# This is fixed.EMBEDDING_DIM = 100callbacks = myCallback()def create_model(vocabulary_size, seq_len): model = models.Sequential() model.add(Embedding(input_dim=MAX_NB_WORDS+1, output_dim=EMBEDDING_DIM, input_length=seq_len,mask_zero=True)) model.add(GRU(units=64, return_sequences=True)) model.add(Dropout(0.4)) model.add(LSTM(units=50)) #model.add(LSTM(100)) #model.add(Dropout(0.4)) #Bidirectional(tf.keras.layers.LSTM(embedding_dim)) #model.add(Bidirectional(LSTM(128))) model.add(Dense(50, activation='relu')) #model.add(Dense(200, activation='relu')) model.add(Dense(4, activation='softmax')) model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) model.summary() return modelmodel=create_model(MAX_NB_WORDS, MAX_SEQUENCE_LENGTH)_________________________________________________________________Layer (type) Output Shape Param # =================================================================embedding_4 (Embedding) (None, 50, 100) 2018600 _________________________________________________________________gru_2 (GRU) (None, 50, 64) 31680 _________________________________________________________________dropout_10 (Dropout) (None, 50, 64) 0 _________________________________________________________________lstm_6 (LSTM) (None, 14) 4424 _________________________________________________________________dense_7 (Dense) (None, 50) 750 _________________________________________________________________dropout_11 (Dropout) (None, 50) 0 _________________________________________________________________dense_8 (Dense) (None, 4) 204 =================================================================Total params: 2,055,658Trainable params: 2,055,658Non-trainable params: 0_________________________________________________________________model.fit(sequences, y_train, validation_data=(sequences_test, y_test), epochs=25, batch_size=5, verbose=1, callbacks=[callbacks] )如果能得到一个有效的解决过拟合的方法，将非常有帮助。你可以参考下面的Colab查看完整代码：
https://colab.research.google.com/drive/13N94kBKkHIX2TR5B_lETyuH1QTC5VuRf?usp=sharing
编辑：—我现在使用了用gensim创建的预训练嵌入层，但准确率下降了。此外，我的记录数量现在是4643。
附件中的代码如下：在这个代码中，’English_dict.p’是我使用gensim创建的字典。
embeddings_index=load(open('English_dict.p', 'rb'))vocab_size=len(embeddings_index)+1embedding_model = zeros((vocab_size, 100))for word, i in embedding_matrix.word_index.items(): embedding_vector = embeddings_index.get(word) if embedding_vector is not None: embedding_model[i] = embedding_vectormodel.add(Embedding(input_dim=MAX_NB_WORDS, output_dim=EMBEDDING_DIM, weights=[embedding_model],trainable=False, input_length=seq_len,mask_zero=True)) Model: "sequential_2"_________________________________________________________________Layer (type) Output Shape Param # =================================================================embedding_2 (Embedding) (None, 50, 100) 2746300 _________________________________________________________________gru_2 (GRU) (None, 50, 64) 31680 _________________________________________________________________dropout_2 (Dropout) (None, 50, 64) 0 _________________________________________________________________lstm_2 (LSTM) (None, 128) 98816 _________________________________________________________________dense_3 (Dense) (None, 50) 6450 _________________________________________________________________dense_4 (Dense) (None, 4) 204 =================================================================Total params: 2,883,450Trainable params: 137,150Non-trainable params: 2,746,300_________________________________________________________________如果我做错了什么，请告诉我。你可以参考上面的Colab进行参考。
回答：
