无法理解PyTorch代码中的就地操作?

我在PyTorch中使用LSTM进行学习的实现如下:

https://gist.github.com/rahulbhadani/f1d64042cc5a80280755cac262aa48aa

然而,代码出现了就地操作错误

我的错误输出是:

/home/ivory/anaconda3/lib/python3.7/site-packages/ipykernel_launcher.py:10: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).  # Remove the CWD from sys.path while we load stuff.---------------------------------------------------------------------------RuntimeError                              Traceback (most recent call last)<ipython-input-86-560ec78f2b64> in <module>     27 linear = torch.nn.Linear(hidden_nums, output_dim)     28 ---> 29 global_loss_list = global_training(lstm2)<ipython-input-84-152890a3028c> in global_training(optimizee)      3     adam_global_optimizer = torch.optim.Adam([{'params': optimizee.parameters()},       4                                      {'params':linear.parameters()}], lr = 0.0001)----> 5     _, global_loss_1 = learn2(LSTM_Optimizee, training_steps, retain_graph_flag=True, reset_theta=True)      6       7     print(global_loss_1)<ipython-input-83-0357a528b94d> in learn2(optimizee, unroll_train_steps, retain_graph_flag, reset_theta)     43             # requires_grad=True. These are accumulated into x.grad for every     44             # parameter x. In pseudo-code: x.grad += dloss/dx---> 45             loss.backward(retain_graph = retain_graph_flag) #The default is False, when the optimized LSTM is set to True     46      47             print('x.grad: {}'.format(x.grad))~/anaconda3/lib/python3.7/site-packages/torch/tensor.py in backward(self, gradient, retain_graph, create_graph)    116                 products. Defaults to ``False``.    117         """--> 118         torch.autograd.backward(self, gradient, retain_graph, create_graph)    119     120     def register_hook(self, hook):~/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py in backward(tensors, grad_tensors, retain_graph, create_graph, grad_variables)     91     Variable._execution_engine.run_backward(     92         tensors, grad_tensors, retain_graph, create_graph,---> 93         allow_unreachable=True)  # allow_unreachable flag     94      95 RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [1, 10]] is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

我尝试追踪错误但没有成功。任何这方面的帮助将不胜感激。

谢谢。


回答:

我认为问题出在以下这行代码:

global_loss_list.append(global_loss.detach_())

PyTorch中就地操作的惯例是在函数名后面使用_(如detach_)。我认为你不应该进行就地分离。换句话说,将detach_改为detach

Related Posts

神经网络反向传播代码不工作

我需要编写一个简单的由1个输出节点、1个包含3个节点的…

值错误:y 包含先前未见过的标签:

我使用了 决策树分类器,我想将我的 输入 作为 字符串…

使用不平衡数据集进行特征选择时遇到的问题

我正在使用不平衡数据集(54:38:7%)进行特征选择…

广义随机森林/因果森林在Python上的应用

我在寻找Python上的广义随机森林/因果森林算法,但…

如何用PyTorch仅用标量损失来训练神经网络?

假设我们有一个神经网络,我们希望它能根据输入预测三个值…

什么是RNN中间隐藏状态的良好用途?

我已经以三种不同的方式使用了RNN/LSTM: 多对多…

发表回复

您的邮箱地址不会被公开。 必填项已用 * 标注