设$F \in \mathbb{R}^{S \times F}$为特征矩阵，我想使用带有自动求导的逻辑回归对其进行分类[1]。我使用的代码与下面的示例类似[2]。
我唯一想要改变的是，我有一个额外的权重矩阵$W$在$\mathbb{R}^{F \times L}$中，我想将其应用于每个特征。因此，每个特征都会与$W$相乘，然后输入到逻辑回归中。
是否可以使用自动求导同时训练$W$和逻辑回归的权重？
我尝试了以下代码，但不幸的是权重保持在0值。
import autograd.numpy as npfrom autograd import grad global inputs def sigmoid(x): return 0.5 * (np.tanh(x) + 1) def logistic_predictions(weights, inputs): # Outputs probability of a label being true according to logistic model. return sigmoid(np.dot(inputs, weights)) def training_loss(weights): global inputs # Training loss is the negative log-likelihood of the training labels. feature_weights = weights[3:] feature_weights = np.reshape(feature_weights, (3, 3)) inputs = np.dot(inputs, feature_weights) preds = logistic_predictions(weights[0:3], inputs) label_probabilities = preds * targets + (1 - preds) * (1 - targets) return -np.sum(np.log(label_probabilities)) # Build a toy dataset. inputs = np.array([[0.52, 1.12, 0.77], [0.88, -1.08, 0.15], [0.52, 0.06, -1.30], [0.74, -2.49, 1.39]]) targets = np.array([True, True, False, True]) # Define a function that returns gradients of training loss using autograd. training_gradient_fun = grad(training_loss) # Optimize weights using gradient descent. weights = np.zeros([3 + 3 * 3]) print "Initial loss:", training_loss(weights) for i in xrange(100): print(i) print(weights) weights -= training_gradient_fun(weights) * 0.01 print "Trained loss:", training_loss(weights)[1] https://github.com/HIPS/autograd
[2] https://github.com/HIPS/autograd/blob/master/examples/logistic_regression.py
回答：
常见的做法是将所有“向量化”的参数连接成决策变量向量。
如果您更新logistic_predictions以包括W矩阵，像这样
def logistic_predictions(weights_and_W, inputs): ''' Here, :arg weights_and_W: is an array of the form [weights W.ravel()] ''' # Outputs probability of a label being true according to logistic model. weights = weights_and_W[:inputs.shape[1]] W_raveled = weights_and_W[inputs.shape[1]:] n_W = len(W_raveled) W = W_raveled.reshape(inputs.shape[1], n_W/inputs.shape[1]) return sigmoid(np.dot(np.dot(inputs, W), weights))然后只需将traning_loss更改为（来自原始源示例）
def training_loss(weights_and_W): # Training loss is the negative log-likelihood of the training labels. preds = logistic_predictions(weights_and_W, inputs) label_probabilities = preds * targets + (1 - preds) * (1 - targets) return -np.sum(np.log(label_probabilities))