我想在MNIST数据集上使用Theano实现带有小批量处理的逻辑回归。我想尝试不同的优化器，因此我决定使用一个名为climin的库。Climin提供了几个函数，这些函数接收模型的参数、损失和/或损失的梯度，以及数据流作为参数。由于Theano函数需要编译，我设计了以下模型。

# 加载MNIST数据集
train_set_x, train_set_y = load_data(dataset)[0]
print('... building the model')
W = theano.shared(
        value = np.zeros((28 * 28, 10), dtype = theano.config.floatX),
        name = 'W',
        borrow = True
        ) # 权重维度 28 * 28, 10
b = theano.shared(
        value = np.zeros((10,), dtype = theano.config.floatX),
        name = 'b',
        borrow = True
        ) # 偏置
x = T.matrix('x') # 数据，表示为光栅化图像维度 28 * 28
y = T.ivector('y') # 标签，表示为[int]标签的一维向量维度 10
p_y_given_x             =  T.nnet.softmax(T.dot(x, W) + b)
y_pred                  =  T.argmax(p_y_given_x, axis=1)
NLL = -T.sum(T.log(p_y_given_x)[T.arange(y.shape[0]), y])
# negative_log_likelihood =  -T.mean(T.log(y_pred)[T.arange(y.shape[0]), y])
loss = theano.function(inputs = [ x, y ], outputs = NLL )
g_W = theano.function(inputs = [ x, y ], outputs = T.grad(cost=loss, wrt=W))
g_b = theano.function(inputs = [ x, y ], outputs = T.grad(cost=loss, wrt=b))
def d_loss_wrt_pars(parameters, inputs, targets):
    # climin将参数作为1-D拼接数组传递给
    # 这个函数，所以我需要
    # 解包参数数组
    W = parameters[:28 * 28 * 10].reshape((28 * 28, 10))
    b = parameters[28 * 28 * 10:].reshape((1, 10))
    return np.concatinate([g_W(x, y).flatten(), g_b(x, y)])
wrt = np.empty(7850) # 为参数分配空间（MNIST维度 28 * 28 * 10）
cli.initialize.randomize_normal(wrt, 0, 1) # 用随机数初始化参数
if batch_size is None:
    args = itertools.repeat(([train_set_x, train_set_y], {}))
    batches_per_pass = 1
else:
    args = cli.util.iter_minibatches([train_set_x, train_set_y], batch_size, [0, 0])
    args = ((i, {}) for i in args)
    batches_per_pass = train_set_x.shape[0] / batch_size
if optimizer == 'gd':
    opt = cli.GradientDescent(wrt, d_loss_wrt_pars, step_rate=0.1, momentum=.95, args=args)
else:
    print('unknown optimizer')
    return 1
print('... training the model')
for info in opt:
    if info['n_iter'] >= n_epochs and (not done_looping):
        break

遗憾的是，这只产生了以下结果：

Traceback (most recent call last):
  File "logreg/logistic_sgd.py", line 160, in <module>
    sys.exit(sgd_optimization_mnist())
  File "logreg/logistic_sgd.py", line 69, in sgd_optimization_mnist
    g_W = theano.function(inputs = [ x, y ], outputs = T.grad(cost=loss, wrt=W))
  File "/Users/romancpodolski/anaconda/lib/python2.7/site-packages/theano/gradient.py", line 430, in grad
    if cost is not None and isinstance(cost.type, NullType):

AttributeError: ‘Function’ object has no attribute ‘type’

有人知道如何使其工作并结合这两个库吗？

回答：

您的代码将一个已编译的函数（而不是Theano表达式）传递给了T.grad。将loss替换为NLL，您应该没问题

g_W = theano.function(inputs = [ x, y ], outputs = T.grad(cost=NLL, wrt=W))
g_b = theano.function(inputs = [ x, y ], outputs = T.grad(cost=NLL, wrt=b))

另外，您可能想尝试使用支持Theano的库（例如Lasagne）进行优化。