我训练了一个模型,目的是生成如下句子:我将两个序列作为训练示例输入,x 是一个字符序列,y 是相同的序列但向后移动一个字符。该模型基于LSTM,并且使用tensorflow创建。
我的问题是:由于模型输入的是一定长度的序列(在我这里是50),我如何仅用单个字符作为种子来进行预测?我在一些例子中看到过,训练后他们通过输入单个字符来生成句子。
这是我的代码:
with tf.name_scope('input'): x = tf.placeholder(tf.float32, [batch_size, truncated_backprop], name='x') y = tf.placeholder(tf.int32, [batch_size, truncated_backprop], name='y') with tf.name_scope('weights'): W = tf.Variable(np.random.rand(n_hidden, num_classes), dtype=tf.float32) b = tf.Variable(np.random.rand(1, num_classes), dtype=tf.float32) inputs_series = tf.split(x, truncated_backprop, 1) labels_series = tf.unstack(y, axis=1) with tf.name_scope('LSTM'): cell = tf.contrib.rnn.BasicLSTMCell(n_hidden, state_is_tuple=True) cell = tf.contrib.rnn.DropoutWrapper(cell, output_keep_prob=dropout) cell = tf.contrib.rnn.MultiRNNCell([cell] * n_layers) states_series, current_state = tf.contrib.rnn.static_rnn(cell, inputs_series, \ dtype=tf.float32) logits_series = [tf.matmul(state, W) + b for state in states_series] prediction_series = [tf.nn.softmax(logits) for logits in logits_series] losses = [tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=labels) \ for logits, labels, in zip(logits_series, labels_series)] total_loss = tf.reduce_mean(losses) train_step = tf.train.AdamOptimizer(learning_rate).minimize(total_loss)
回答:
我建议你使用dynamic_rnn
替代 static_rnn
,前者会在执行时创建图形,允许你输入任意长度的序列。你的输入占位符可以是
x = tf.placeholder(tf.float32, [batch_size, None, features], name='x')
接下来,你需要一种方法来将初始状态输入到网络中。你可以通过向 dynamic_rnn
传递 initial_state
参数来实现这一点,例如:
initialstate = cell.zero_state(batch_sie, tf.float32)outputs, current_state = tf.nn.dynamic_rnn(cell, inputs, initial_state=initialstate)
这样,为了从单个字符生成文本,你可以一次向图形输入一个字符,每次传递前一个字符和状态,如下所示:
prompt = 's' # 起始字符,可以是任何字符inp = one_hot(prompt) # 预处理,因为你可能想输入独热编码向量state = Nonewhile True: if state is None: feed = {x: [[inp]]} else: feed = {x: [[inp]], initialstate: state} out, state = sess.run([outputs, current_state], feed_dict=feed) inp = process(out) # 从 out 中提取预测的字符并进行独热编码