我想使用Keras和Tensorflow后端构建一个神经网络，使其输出一个L2归一化的向量。我尝试了以下方法，但不知为何输出并未被归一化：

import keras.backend as Kinput = Input(shape=input_shape)...dense7 = Dense(output_dim=3)(flatten6)l2_norm = Lambda(lambda  x: K.l2_normalize(x,axis=1))(dense7)return Model(input=input, output=l2_norm)

因此，这里输出的是一个三维向量，我希望确保这个向量的长度为1。谁能帮帮我？另外，能告诉我为什么我的解决方案失败了吗？

示例：

output: [  8.27677908e-08   2.64180613e-07  -2.81542953e-07]required: [ 0.20961709  0.6690619  -0.71303362]

模型摘要：

____________________________________________________________________________________________________Layer (type)                     Output Shape          Param #     Connected to                     ====================================================================================================input_1 (InputLayer)             (None, 224, 224, 3)   0                                            ____________________________________________________________________________________________________convolution2d_1 (Convolution2D)  (None, 112, 112, 64)  9472        input_1[0][0]                    ____________________________________________________________________________________________________batchnormalization_1 (BatchNormal(None, 112, 112, 64)  128         convolution2d_1[0][0]            ____________________________________________________________________________________________________maxpooling2d_1 (MaxPooling2D)    (None, 56, 56, 64)    0           batchnormalization_1[0][0]       ____________________________________________________________________________________________________convolution2d_2 (Convolution2D)  (None, 56, 56, 64)    4160        maxpooling2d_1[0][0]             ____________________________________________________________________________________________________batchnormalization_2 (BatchNormal(None, 56, 56, 64)    128         convolution2d_2[0][0]            ____________________________________________________________________________________________________convolution2d_3 (Convolution2D)  (None, 56, 56, 192)   110784      batchnormalization_2[0][0]       ____________________________________________________________________________________________________batchnormalization_3 (BatchNormal(None, 56, 56, 192)   384         convolution2d_3[0][0]            ____________________________________________________________________________________________________maxpooling2d_2 (MaxPooling2D)    (None, 28, 28, 192)   0           batchnormalization_3[0][0]       ____________________________________________________________________________________________________convolution2d_5 (Convolution2D)  (None, 28, 28, 96)    18528       maxpooling2d_2[0][0]             ____________________________________________________________________________________________________convolution2d_7 (Convolution2D)  (None, 28, 28, 16)    3088        maxpooling2d_2[0][0]             ____________________________________________________________________________________________________maxpooling2d_3 (MaxPooling2D)    (None, 28, 28, 192)   0           maxpooling2d_2[0][0]             ____________________________________________________________________________________________________convolution2d_4 (Convolution2D)  (None, 28, 28, 64)    12352       maxpooling2d_2[0][0]             ____________________________________________________________________________________________________convolution2d_6 (Convolution2D)  (None, 28, 28, 128)   110720      convolution2d_5[0][0]            ____________________________________________________________________________________________________convolution2d_8 (Convolution2D)  (None, 28, 28, 32)    12832       convolution2d_7[0][0]            ____________________________________________________________________________________________________convolution2d_9 (Convolution2D)  (None, 28, 28, 32)    6176        maxpooling2d_3[0][0]             ____________________________________________________________________________________________________merge_1 (Merge)                  (None, 28, 28, 256)   0           convolution2d_4[0][0]                                                                               convolution2d_6[0][0]                                                                               convolution2d_8[0][0]                                                                               convolution2d_9[0][0]            ____________________________________________________________________________________________________convolution2d_11 (Convolution2D) (None, 28, 28, 96)    24672       merge_1[0][0]                    ____________________________________________________________________________________________________convolution2d_13 (Convolution2D) (None, 28, 28, 32)    8224        merge_1[0][0]                    ____________________________________________________________________________________________________maxpooling2d_4 (MaxPooling2D)    (None, 28, 28, 256)   0           merge_1[0][0]                    ____________________________________________________________________________________________________convolution2d_10 (Convolution2D) (None, 28, 28, 64)    16448       merge_1[0][0]                    ____________________________________________________________________________________________________convolution2d_12 (Convolution2D) (None, 28, 28, 128)   110720      convolution2d_11[0][0]           ____________________________________________________________________________________________________convolution2d_14 (Convolution2D) (None, 28, 28, 64)    51264       convolution2d_13[0][0]           ____________________________________________________________________________________________________convolution2d_15 (Convolution2D) (None, 28, 28, 64)    16448       maxpooling2d_4[0][0]             ____________________________________________________________________________________________________merge_2 (Merge)                  (None, 28, 28, 320)   0           convolution2d_10[0][0]                                                                              convolution2d_12[0][0]                                                                              convolution2d_14[0][0]                                                                              convolution2d_15[0][0]           ____________________________________________________________________________________________________convolution2d_16 (Convolution2D) (None, 28, 28, 128)   41088       merge_2[0][0]                    ____________________________________________________________________________________________________convolution2d_18 (Convolution2D) (None, 28, 28, 32)    10272       merge_2[0][0]                    ____________________________________________________________________________________________________convolution2d_17 (Convolution2D) (None, 14, 14, 256)   295168      convolution2d_16[0][0]           ____________________________________________________________________________________________________convolution2d_19 (Convolution2D) (None, 14, 14, 64)    51264       convolution2d_18[0][0]           ____________________________________________________________________________________________________maxpooling2d_5 (MaxPooling2D)    (None, 14, 14, 320)   0           merge_2[0][0]                    ____________________________________________________________________________________________________merge_3 (Merge)                  (None, 14, 14, 640)   0           convolution2d_17[0][0]                                                                              convolution2d_19[0][0]                                                                              maxpooling2d_5[0][0]             ____________________________________________________________________________________________________convolution2d_21 (Convolution2D) (None, 14, 14, 96)    61536       merge_3[0][0]                    ____________________________________________________________________________________________________convolution2d_23 (Convolution2D) (None, 14, 14, 32)    20512       merge_3[0][0]                    ____________________________________________________________________________________________________maxpooling2d_6 (MaxPooling2D)    (None, 14, 14, 640)   0           merge_3[0][0]                    ____________________________________________________________________________________________________convolution2d_20 (Convolution2D) (None, 14, 14, 256)   164096      merge_3[0][0]                    ____________________________________________________________________________________________________convolution2d_22 (Convolution2D) (None, 14, 14, 192)   166080      convolution2d_21[0][0]           ____________________________________________________________________________________________________convolution2d_24 (Convolution2D) (None, 14, 14, 64)    51264       convolution2d_23[0][0]           ____________________________________________________________________________________________________convolution2d_25 (Convolution2D) (None, 14, 14, 128)   82048       maxpooling2d_6[0][0]             ____________________________________________________________________________________________________merge_4 (Merge)                  (None, 14, 14, 640)   0           convolution2d_20[0][0]                                                                              convolution2d_22[0][0]                                                                              convolution2d_24[0][0]                                                                              convolution2d_25[0][0]           ____________________________________________________________________________________________________convolution2d_27 (Convolution2D) (None, 14, 14, 112)   71792       merge_4[0][0]                    ____________________________________________________________________________________________________convolution2d_29 (Convolution2D) (None, 14, 14, 32)    20512       merge_4[0][0]                    ____________________________________________________________________________________________________maxpooling2d_7 (MaxPooling2D)    (None, 14, 14, 640)   0           merge_4[0][0]                    ____________________________________________________________________________________________________convolution2d_26 (Convolution2D) (None, 14, 14, 224)   143584      merge_4[0][0]                    ____________________________________________________________________________________________________convolution2d_28 (Convolution2D) (None, 14, 14, 224)   226016      convolution2d_27[0][0]           ____________________________________________________________________________________________________convolution2d_30 (Convolution2D) (None, 14, 14, 64)    51264       convolution2d_29[0][0]           ____________________________________________________________________________________________________convolution2d_31 (Convolution2D) (None, 14, 14, 128)   82048       maxpooling2d_7[0][0]             ____________________________________________________________________________________________________merge_5 (Merge)                  (None, 14, 14, 640)   0           convolution2d_26[0][0]                                                                              convolution2d_28[0][0]                                                                              convolution2d_30[0][0]                                                                              convolution2d_31[0][0]           ____________________________________________________________________________________________________convolution2d_33 (Convolution2D) (None, 14, 14, 128)   82048       merge_5[0][0]                    ____________________________________________________________________________________________________convolution2d_35 (Convolution2D) (None, 14, 14, 32)    20512       merge_5[0][0]                    ____________________________________________________________________________________________________maxpooling2d_8 (MaxPooling2D)    (None, 14, 14, 640)   0           merge_5[0][0]                    ____________________________________________________________________________________________________convolution2d_32 (Convolution2D) (None, 14, 14, 192)   123072      merge_5[0][0]                    ____________________________________________________________________________________________________convolution2d_34 (Convolution2D) (None, 14, 14, 256)   295168      convolution2d_33[0][0]           ____________________________________________________________________________________________________convolution2d_36 (Convolution2D) (None, 14, 14, 64)    51264       convolution2d_35[0][0]           ____________________________________________________________________________________________________convolution2d_37 (Convolution2D) (None, 14, 14, 128)   82048       maxpooling2d_8[0][0]             ____________________________________________________________________________________________________merge_6 (Merge)                  (None, 14, 14, 640)   0           convolution2d_32[0][0]                                                                              convolution2d_34[0][0]                                                                              convolution2d_36[0][0]                                                                              convolution2d_37[0][0]           ____________________________________________________________________________________________________convolution2d_39 (Convolution2D) (None, 14, 14, 144)   92304       merge_6[0][0]                    ____________________________________________________________________________________________________convolution2d_41 (Convolution2D) (None, 14, 14, 32)    20512       merge_6[0][0]                    ____________________________________________________________________________________________________maxpooling2d_9 (MaxPooling2D)    (None, 14, 14, 640)   0           merge_6[0][0]                    ____________________________________________________________________________________________________convolution2d_38 (Convolution2D) (None, 14, 14, 160)   102560      merge_6[0][0]                    ____________________________________________________________________________________________________convolution2d_40 (Convolution2D) (None, 14, 14, 288)   373536      convolution2d_39[0][0]           ____________________________________________________________________________________________________convolution2d_42 (Convolution2D) (None, 14, 14, 64)    51264       convolution2d_41[0][0]           ____________________________________________________________________________________________________convolution2d_43 (Convolution2D) (None, 14, 14, 128)   82048       maxpooling2d_9[0][0]             ____________________________________________________________________________________________________merge_7 (Merge)                  (None, 14, 14, 640)   0           convolution2d_38[0][0]                                                                              convolution2d_40[0][0]                                                                              convolution2d_42[0][0]                                                                              convolution2d_43[0][0]           ____________________________________________________________________________________________________convolution2d_44 (Convolution2D) (None, 14, 14, 160)   102560      merge_7[0][0]                    ____________________________________________________________________________________________________convolution2d_46 (Convolution2D) (None, 14, 14, 64)    41024       merge_7[0][0]                    ____________________________________________________________________________________________________convolution2d_45 (Convolution2D) (None, 7, 7, 256)     368896      convolution2d_44[0][0]           ____________________________________________________________________________________________________convolution2d_47 (Convolution2D) (None, 7, 7, 128)     204928      convolution2d_46[0][0]           ____________________________________________________________________________________________________maxpooling2d_10 (MaxPooling2D)   (None, 7, 7, 640)     0           merge_7[0][0]                    ____________________________________________________________________________________________________merge_8 (Merge)                  (None, 7, 7, 1024)    0           convolution2d_45[0][0]                                                                              convolution2d_47[0][0]                                                                              maxpooling2d_10[0][0]            ____________________________________________________________________________________________________convolution2d_49 (Convolution2D) (None, 7, 7, 192)     196800      merge_8[0][0]                    ____________________________________________________________________________________________________convolution2d_51 (Convolution2D) (None, 7, 7, 48)      49200       merge_8[0][0]                    ____________________________________________________________________________________________________maxpooling2d_11 (MaxPooling2D)   (None, 7, 7, 1024)    0           merge_8[0][0]                    ____________________________________________________________________________________________________convolution2d_48 (Convolution2D) (None, 7, 7, 384)     393600      merge_8[0][0]                    ____________________________________________________________________________________________________convolution2d_50 (Convolution2D) (None, 7, 7, 384)     663936      convolution2d_49[0][0]           ____________________________________________________________________________________________________convolution2d_52 (Convolution2D) (None, 7, 7, 128)     153728      convolution2d_51[0][0]           ____________________________________________________________________________________________________convolution2d_53 (Convolution2D) (None, 7, 7, 128)     131200      maxpooling2d_11[0][0]            ____________________________________________________________________________________________________merge_9 (Merge)                  (None, 7, 7, 1024)    0           convolution2d_48[0][0]                                                                              convolution2d_50[0][0]                                                                              convolution2d_52[0][0]                                                                              convolution2d_53[0][0]           ____________________________________________________________________________________________________convolution2d_55 (Convolution2D) (None, 7, 7, 192)     196800      merge_9[0][0]                    ____________________________________________________________________________________________________convolution2d_57 (Convolution2D) (None, 7, 7, 48)      49200       merge_9[0][0]                    ____________________________________________________________________________________________________maxpooling2d_12 (MaxPooling2D)   (None, 7, 7, 1024)    0           merge_9[0][0]                    ____________________________________________________________________________________________________convolution2d_54 (Convolution2D) (None, 7, 7, 384)     393600      merge_9[0][0]                    ____________________________________________________________________________________________________convolution2d_56 (Convolution2D) (None, 7, 7, 384)     663936      convolution2d_55[0][0]           ____________________________________________________________________________________________________convolution2d_58 (Convolution2D) (None, 7, 7, 128)     153728      convolution2d_57[0][0]           ____________________________________________________________________________________________________convolution2d_59 (Convolution2D) (None, 7, 7, 128)     131200      maxpooling2d_12[0][0]            ____________________________________________________________________________________________________merge_10 (Merge)                 (None, 7, 7, 1024)    0           convolution2d_54[0][0]                                                                              convolution2d_56[0][0]                                                                              convolution2d_58[0][0]                                                                              convolution2d_59[0][0]           ____________________________________________________________________________________________________averagepooling2d_1 (AveragePoolin(None, 1, 1, 1024)    0           merge_10[0][0]                   ____________________________________________________________________________________________________flatten_1 (Flatten)              (None, 1024)          0           averagepooling2d_1[0][0]         ____________________________________________________________________________________________________dense_1 (Dense)                  (None, 3)             3075        flatten_1[0][0]                  ____________________________________________________________________________________________________lambda_1 (Lambda)                (None, 3)             0           dense_1[0][0]                    ====================================================================================================Total params: 7328819

回答：

我找到了问题所在！

我使用Tensorflow作为后端，K.l2_normalize(x, axis)调用了tf.nn.l2_normalize(x, dim, epsilon=1e-12, name=None)。请注意，这个方法有一个额外的参数epsilon。这个方法如下所示：

with ops.name_scope(name, "l2_normalize", [x]) as name:   x = ops.convert_to_tensor(x, name="x")   square_sum = math_ops.reduce_sum(math_ops.square(x), dim, keep_dims=True)   x_inv_norm = math_ops.rsqrt(math_ops.maximum(square_sum, epsilon))return math_ops.mul(x, x_inv_norm, name=name)

因此，如果网络的输出包含小于epsilon的值（默认设置为1e-12），那么它就不会被正确归一化，这正是我遇到的情况。