我刚开始接触强化学习，尝试使用LSTM来训练一个空间入侵者代理。我尝试使用这篇论文中的网络，但一直遇到问题：
-如果我使用Conv2D，维度与LSTM不匹配，我会得到这个错误：
ValueError: Input 0 is incompatible with layer conv_lst_m2d_1: expected ndim=5, found ndim=4
这是代码：
self.model = Sequential() self.model.add(Conv2D(32,kernel_size=8,strides=4,activation='relu',input_shape=(None,84,84,1))) self.model.add(Conv2D(64,kernel_size=4,strides=2,activation='relu')) self.model.add(Conv2D(64,kernel_size=3, strides=1,activation='relu')) self.model.add(ConvLSTM2D(512, kernel_size=(3,3), padding='same', return_sequences=False)) self.model.add(Dense(4, activation='relu')) self.model.compile(loss='mse', optimizer=Adam(lr=0.0001)) self.model.summary()-如果我使用Conv3D，它输出一个5D张量，我无法使用单张图像作为输入：
ValueError: Error when checking input: expected conv3d_1_input to have 5 dimensions, but got array with shape (1, 84, 84, 1)
代码：
self.model.add(Conv3D(32,kernel_size=8,strides=4,activation='relu',input_shape=(None,84,84,1))) self.model.add(Conv3D(64,kernel_size=4,strides=2,activation='relu')) self.model.add(Conv3D(64,kernel_size=3, strides=1,activation='relu')) self.model.add(ConvLSTM2D(512, kernel_size=(3,3), padding='same', return_sequences=False)) self.model.add(Dense(4, activation='relu')) self.model.compile(loss='mse', optimizer=Adam(lr=0.0001)) self.model.summary()(编辑)
网络摘要（第二个网络）：
_________________________________________________________________Layer (type) Output Shape Param # =================================================================conv3d_1 (Conv3D) (None, None, 20, 20, 32) 16416 _________________________________________________________________conv3d_2 (Conv3D) (None, None, 9, 9, 64) 131136 _________________________________________________________________conv3d_3 (Conv3D) (None, None, 7, 7, 64) 110656 _________________________________________________________________conv_lst_m2d_1 (ConvLSTM2D) (None, 7, 7, 512) 10618880 _________________________________________________________________dense_1 (Dense) (None, 7, 7, 4) 2052 =================================================================数据输入形状为：(84, 84, 1)
回答：
你需要使用TimeDistributed Conv2D，它告诉你的网络将数据理解为时间序列（我猜这正是你想要的），并且会匹配LSTM的内部形状。
import tensorflow as tfmodel = tf.keras.Sequential()model.add(tf.keras.layers.Input(shape=(None,84,84,1)))model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Conv2D(32,kernel_size=8,strides=4,activation='relu')))model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Conv2D(64,kernel_size=4,strides=2,activation='relu')))model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Conv2D(64,kernel_size=3, strides=1,activation='relu')))model.add(tf.keras.layers.ConvLSTM2D(512, kernel_size=(3,3), padding='same', return_sequences=False))model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(4, activation='relu')))model.compile(loss='mse', optimizer=tf.keras.optimizers.Adam(lr=0.0001))model.summary()编译返回：
Model: "sequential_4"_________________________________________________________________Layer (type) Output Shape Param # =================================================================time_distributed_12 (TimeDis (None, None, 20, 20, 32) 2080 _________________________________________________________________time_distributed_13 (TimeDis (None, None, 9, 9, 64) 32832 _________________________________________________________________time_distributed_14 (TimeDis (None, None, 7, 7, 64) 36928 _________________________________________________________________conv_lst_m2d_3 (ConvLSTM2D) (None, 7, 7, 512) 10618880 _________________________________________________________________time_distributed_15 (TimeDis (None, 7, 7, 4) 2052 =================================================================Total params: 10,692,772Trainable params: 10,692,772Non-trainable params: 0_________________________________________________________________