我正在使用从https://www.derivative-calculator.net计算得出的第一和第二阶导数来重建LightGBM的二元对数损失函数。

但是，我的图表与LightGBM中原始定义的实际图表不同。

为什么图表会有所不同？我计算导数的方式是否有误？

众所周知，loss = -y_true log(y_pred) - (1-y_true) log(1-y_pred)，其中y_pred = sigmoid(logits)

计算器对-y log(1/(1+e^-x)) - (1-y) log(1-1/(1+e^-x))的计算结果如下：

=

以及，

=

当我使用以下代码绘制上述内容时，

def custom_odds_loss(y_true, y_pred):    y = y_true    # ======================    # Inverse sigmoid    # ======================    epsilon_ = 1e-7    y_pred = np.clip(y_pred, epsilon_, 1 - epsilon_)    y_pred = np.log(y_pred/(1-y_pred))    # ======================        grad = -((y-1)*np.exp(y_pred)+y)/(np.exp(y_pred)+1)    hess = np.exp(y_pred)/(np.exp(y_pred)+1)**2        return grad, hess# Penalty chart for True 1s all the timey_true_k = np.ones((1000, 1))y_pred_k = np.expand_dims(np.linspace(0, 1, 1000), axis=1)grad, hess = custom_odds_loss(y_true_k, y_pred_k)data_ = {    'Payoff@grad': grad.flatten(),}pd.DataFrame(data_).plot(title='Target=1(G)|Penalty(y-axis) vs Probability/1000. (x-axis)');data_ = {    'Payoff@hess': hess.flatten(),}pd.DataFrame(data_).plot(title='Target=1(H)|Penalty(y-axis) vs Probability/1000. (x-axis)');

现在，LightGBM的实际图表，

def custom_odds_loss(y_true, y_pred):    # ======================    # Inverse sigmoid    # ======================    epsilon_ = 1e-7    y_pred = np.clip(y_pred, epsilon_, 1 - epsilon_)    y_pred = np.log(y_pred/(1-y_pred))    # ======================    grad = y_pred - y_true    hess = y_pred * (1. - y_pred)    return grad, hess# Penalty chart for True 1s all the timey_true_k = np.ones((1000, 1))y_pred_k = np.expand_dims(np.linspace(0, 1, 1000), axis=1)grad, hess = custom_odds_loss(y_true_k, y_pred_k)data_ = {    'Payoff@grad': grad.flatten(),}pd.DataFrame(data_).plot(title='Target=1(G)|Penalty(y-axis) vs Probability/1000. (x-axis)');data_ = {    'Payoff@hess': hess.flatten(),}pd.DataFrame(data_).plot(title='Target=1(H)|Penalty(y-axis) vs Probability/1000. (x-axis)');

回答：

在第二个函数中，你不需要反转sigmoid函数。

你看，你找到的导数可以简化为如下形式：

这种简化让我们无需反转任何东西，就能简单地找到梯度和二阶导数，如下所示：

def custom_odds_loss(y_true, y_pred):    grad = y_pred - y_true    hess = y_pred * (1. - y_pred)    return grad, hess