在实现混合量子LSTM模型时,模型出现了过拟合,导致准确率低。我尝试在nn.LSTM
中设置dropout = 1
,但没有改善。我使用了一个隐藏层。我该如何添加dropout层来减少过拟合?
模型参数:
input_dim = 16hidden_dim = 100layer_dim = 1output_dim = 1
模型类:
class LSTMModel(nn.Module): def __init__(self, input_dim, hidden_dim, layer_dim, output_dim): super(LSTMModel, self).__init__() self.hidden_dim = hidden_dim self.layer_dim = layer_dim self.lstm = nn.LSTM(input_dim, hidden_dim, layer_dim, dropout=1, batch_first=True, ) self.fc = nn.Linear(hidden_dim, output_dim) self.hybrid = Hybrid(qiskit.Aer.get_backend('qasm_simulator'), 100, np.pi / 2) def forward(self, x): h0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_() c0 = torch.zeros(self.layer_dim, x.size(0), self.hidden_dim).requires_grad_() x, (hn, cn) = self.lstm(x, (h0.detach(), c0.detach())) x = self.fc(x[:, -1, :]) x = self.hybrid(x) return T.cat((x, 1 - x), -1)
回答:
Pytorch的LSTM层接受dropout
参数作为层中节点被置零的概率。当你传入1时,它会将整个层置零。我猜你本来是想设定一个常规值,比如0.3或0.5。
正如@ayandas上面所说,它对每个层(除了最后一层)应用dropout(参见上面的链接),因此对于单层LSTM不起作用。如果你愿意,你总可以在LSTM层的输出处使用nn.dropout来应用你自己的dropout。