我尝试构建了一个序列到序列模型，用于根据最初的几个输入预测传感器信号随时间的变化（见下图）
模型运行得还不错，但我想要“调味一下”，尝试在两个LSTM层之间添加一个注意力层。
模型代码：
def train_model(x_train, y_train, n_units=32, n_steps=20, epochs=200, n_steps_out=1): filters = 250 kernel_size = 3 logdir = os.path.join(logs_base_dir, datetime.datetime.now().strftime("%Y%m%d-%H%M%S")) tensorboard_callback = TensorBoard(log_dir=logdir, update_freq=1) # 从输入数据中获取特征数量 n_features = x_train.shape[2] # 建立网络 # （您可以自由使用其他层的组合和参数） model = keras.models.Sequential() model.add(keras.layers.LSTM(n_units, activation='relu', return_sequences=True, input_shape=(n_steps, n_features))) model.add(keras.layers.LSTM(n_units, activation='relu')) model.add(keras.layers.Dense(64, activation='relu')) model.add(keras.layers.Dropout(0.5)) model.add(keras.layers.Dense(n_steps_out)) model.compile(optimizer='adam', loss='mse', metrics=['mse']) # 训练网络 history = model.fit(x_train, y_train, epochs=epochs, validation_split=0.1, verbose=1, callbacks=[tensorboard_callback]) return model, history我查看了文档，但有点迷惑。任何关于添加注意力层或对当前模型的评论都会受到欢迎
更新：在谷歌搜索后，我开始觉得自己之前理解错了，所以我重写了代码。
我正在尝试迁移我在这个GitHub存储库中找到的序列到序列模型。在存储库代码中展示的问题是根据一些早期样本预测随机生成的正弦波。
我有一个类似的问题，我正在尝试更改代码以适应我的需求。
差异：
- 我的训练数据形状为(439, 5, 20)，439个不同的信号，每个信号有5个时间步，每个时间步有20个特征
- 我没有在拟合数据时使用
fit_generator
超参数：
layers = [35, 35] # 编码器和解码器每层的隐藏神经元数量learning_rate = 0.01decay = 0 # 学习率衰减optimiser = keras.optimizers.Adam(lr=learning_rate, decay=decay) # 其他可能的优化器“sgd”（随机梯度下降）num_input_features = train_x.shape[2] # 每个时间步的输入维度。在这种情况下是一维信号。num_output_features = 1 # 每个时间步的输出维度。在这种情况下是一维信号。# 输入序列没有理由必须与输出序列具有相同的维度。# 例如，使用3个输入信号：消费者信心、通货膨胀和房价来预测未来的房价。loss = "mse" # 其他损失函数是可能的，请参阅Keras文档。# 此应用不需要正则化lambda_regulariser = 0.000001 # 如果正则化器为None，则不会使用regulariser = None # 可能的正则化器：keras.regularizers.l2(lambda_regulariser)batch_size = 128steps_per_epoch = 200 # batch_size * steps_per_epoch = 总训练样本数epochs = 100input_sequence_length = n_steps # 编码器使用的序列长度target_sequence_length = 31 - n_steps # 解码器预测的序列长度num_steps_to_predict = 20 # 测试模型时使用的长度编码器代码：
# 定义输入序列。encoder_inputs = keras.layers.Input(shape=(None, num_input_features), name='encoder_input')# 创建一个RNN单元列表，然后将它们连接成单一层# 与RNN层。encoder_cells = []for hidden_neurons in layers: encoder_cells.append(keras.layers.GRUCell(hidden_neurons, kernel_regularizer=regulariser, recurrent_regularizer=regulariser, bias_regularizer=regulariser))encoder = keras.layers.RNN(encoder_cells, return_state=True, name='encoder_layer')encoder_outputs_and_states = encoder(encoder_inputs)# 丢弃编码器输出，只保留状态。# 输出对我们没有兴趣，编码器的# 工作是创建描述输入序列的状态。encoder_states = encoder_outputs_and_states[1:]解码器代码：
# 解码器输入将设置为零（参见utils模块的random_sine函数）。# 不要担心输入大小为1，我将在下一个单元格中解释这一点。decoder_inputs = keras.layers.Input(shape=(None, 20), name='decoder_input')decoder_cells = []for hidden_neurons in layers: decoder_cells.append(keras.layers.GRUCell(hidden_neurons, kernel_regularizer=regulariser, recurrent_regularizer=regulariser, bias_regularizer=regulariser))decoder = keras.layers.RNN(decoder_cells, return_sequences=True, return_state=True, name='decoder_layer')# 将解码器的初始状态设置为编码器的输出状态。# 这是编码器-解码器的基本部分。decoder_outputs_and_states = decoder(decoder_inputs, initial_state=encoder_states)# 只选择解码器的输出（不是状态）decoder_outputs = decoder_outputs_and_states[0]# 应用一个带线性激活的密集层以将输出设置为正确的维度# 和比例（Keras中GRU的默认激活为tanh，我们的输出正弦函数可以大于1）decoder_dense = keras.layers.Dense(num_output_features, activation='linear', kernel_regularizer=regulariser, bias_regularizer=regulariser)decoder_outputs = decoder_dense(decoder_outputs)模型摘要：
model = keras.models.Model(inputs=[encoder_inputs, decoder_inputs], outputs=decoder_outputs)model.compile(optimizer=optimiser, loss=loss)model.summary()Layer (type) Output Shape Param # Connected to ==================================================================================================encoder_input (InputLayer) (None, None, 20) 0 __________________________________________________________________________________________________decoder_input (InputLayer) (None, None, 20) 0 __________________________________________________________________________________________________encoder_layer (RNN) [(None, 35), (None, 13335 encoder_input[0][0] __________________________________________________________________________________________________decoder_layer (RNN) [(None, None, 35), ( 13335 decoder_input[0][0] encoder_layer[0][1] encoder_layer[0][2] __________________________________________________________________________________________________dense_5 (Dense) (None, None, 1) 36 decoder_layer[0][0] ==================================================================================================Total params: 26,706Trainable params: 26,706Non-trainable params: 0__________________________________________________________________________________________________尝试拟合模型时：
history = model.fit([train_x, decoder_inputs],train_y, epochs=epochs, validation_split=0.3, verbose=1)我得到以下错误：
When feeding symbolic tensors to a model, we expect the tensors to have a static batch size. Got tensor with shape: (None, None, 20)我做错了什么？
回答：
这是对编辑后问题的回答
首先，当你调用fit时，decoder_inputs是一个张量，你不能用它来拟合你的模型。你引用的代码作者使用了一个零数组，所以你也必须这样做（我在下面的虚拟示例中这样做了）
其次，看看你的模型摘要中的输出层…它是三维的，所以你必须将目标管理为三维数组
第三，解码器输入必须是1个特征维度，而不是你报告的20
设置初始参数
layers = [35, 35]learning_rate = 0.01decay = 0 optimiser = keras.optimizers.Adam(lr=learning_rate, decay=decay)num_input_features = 20num_output_features = 1loss = "mse"lambda_regulariser = 0.000001regulariser = Nonebatch_size = 128steps_per_epoch = 200epochs = 100定义编码器
encoder_inputs = keras.layers.Input(shape=(None, num_input_features), name='encoder_input')encoder_cells = []for hidden_neurons in layers: encoder_cells.append(keras.layers.GRUCell(hidden_neurons, kernel_regularizer=regulariser, recurrent_regularizer=regulariser, bias_regularizer=regulariser))encoder = keras.layers.RNN(encoder_cells, return_state=True, name='encoder_layer')encoder_outputs_and_states = encoder(encoder_inputs)encoder_states = encoder_outputs_and_states[1:] # 只保留状态定义解码器（1个特征维度输入！）
decoder_inputs = keras.layers.Input(shape=(None, 1), name='decoder_input') #### <=== 必须是1decoder_cells = []for hidden_neurons in layers: decoder_cells.append(keras.layers.GRUCell(hidden_neurons, kernel_regularizer=regulariser, recurrent_regularizer=regulariser, bias_regularizer=regulariser))decoder = keras.layers.RNN(decoder_cells, return_sequences=True, return_state=True, name='decoder_layer')decoder_outputs_and_states = decoder(decoder_inputs, initial_state=encoder_states)decoder_outputs = decoder_outputs_and_states[0] # 只保留输出序列decoder_dense = keras.layers.Dense(num_output_features, activation='linear', kernel_regularizer=regulariser, bias_regularizer=regulariser)decoder_outputs = decoder_dense(decoder_outputs)定义模型
model = keras.models.Model(inputs=[encoder_inputs, decoder_inputs], outputs=decoder_outputs)model.compile(optimizer=optimiser, loss=loss)model.summary()Layer (type) Output Shape Param # Connected to ==================================================================================================encoder_input (InputLayer) (None, None, 20) 0 __________________________________________________________________________________________________decoder_input (InputLayer) (None, None, 1) 0 __________________________________________________________________________________________________encoder_layer (RNN) [(None, 35), (None, 13335 encoder_input[0][0] __________________________________________________________________________________________________decoder_layer (RNN) [(None, None, 35), ( 11340 decoder_input[0][0] encoder_layer[0][1] encoder_layer[0][2] __________________________________________________________________________________________________dense_4 (Dense) (None, None, 1) 36 decoder_layer[0][0] ==================================================================================================这是我的虚拟数据。与你的形状相同。注意decoder_zero_inputs，它与你的y具有相同的维度，但它是一个零数组
train_x = np.random.uniform(0,1, (439, 5, 20))train_y = np.random.uniform(0,1, (439, 56, 1))validation_x = np.random.uniform(0,1, (10, 5, 20))validation_y = np.random.uniform(0,1, (10, 56, 1))decoder_zero_inputs = np.zeros((439, 56, 1)) ### <=== 注意拟合
history = model.fit([train_x, decoder_zero_inputs],train_y, epochs=epochs, validation_split=0.3, verbose=1)Epoch 1/100307/307 [==============================] - 2s 8ms/step - loss: 0.1038 - val_loss: 0.0845Epoch 2/100307/307 [==============================] - 1s 2ms/step - loss: 0.0851 - val_loss: 0.0832Epoch 3/100307/307 [==============================] - 1s 2ms/step - loss: 0.0842 - val_loss: 0.0828在验证集上进行预测
pred_validation = model.predict([validation_x, np.zeros((10,56,1))])