我有这个数据框。
我正在尝试按照这个示例进行操作。
我想要预测的目标值是zg500
。我想要使用的另一个特征是tas
。
我想要创建特征列,以便将纬度和经度结合起来:
import numpy as npimport pandas as pdimport numpy as npimport tensorflow as tffrom tensorflow import feature_columndf = pd.read_csv('./df.csv')# 如果存在未命名列#df.drop(['Unnamed: 0'],# axis=1,# inplace=True)df.dropna(inplace=True)# 一个从Pandas数据框创建tf.data数据集的实用方法def df_to_dataset(dataframe, shuffle=True, batch_size=32): dataframe = dataframe.copy() labels = dataframe.pop('zg500') ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels)) if shuffle: ds = ds.shuffle(buffer_size=len(dataframe)) ds = ds.batch(batch_size) return dsbatch_size = 16 train_ds = df_to_dataset(df, batch_size=batch_size)feature_columns = []tas = feature_column.numeric_column("tas")latitude = feature_column.numeric_column("lats")longitude = feature_column.numeric_column("lons")bucketized_lat = feature_column.bucketized_column(latitude, boundaries=[0, 20, 40, 70])bucketized_lon = feature_column.bucketized_column(longitude, boundaries=[-45, -20, 0, 20, 60])feature_columns.append(tas)feature_columns.append(bucketized_lat)feature_columns.append(bucketized_lon)lat_lon = feature_column.crossed_column([bucketized_lat, bucketized_lon], 1000)lat_lon = feature_column.indicator_column(lat_lon)feature_columns.append(lat_lon)feature_layer = tf.keras.layers.DenseFeatures(feature_columns)
创建模型:
model = tf.keras.Sequential([ feature_layer, tf.keras.layers.Dense(10, activation='relu'), tf.keras.layers.Dense(1)])model.compile(optimizer='adam', loss='mse') history = model.fit(train_ds, epochs=2)
目前,我收到了NaN损失值:
10918/10918 [==============================] - 10s 861us/step - loss: nanEpoch 2/210918/10918 [==============================] - 10s 857us/step - loss: nan
另外,我想知道为什么使用df
数据框而不是train_ds
:
history = model.fit(df.iloc[:, [0, 2, 3]].values, df.iloc[:, 1].values, epochs=2)
会产生:
ValueError: ('We expected a dictionary here. Instead we got: ', <tf.Tensor 'IteratorGetNext:0' shape=(32, 3) dtype=float32>)
回答:
损失值出现nan
的原因是你的目标值处于极端范围内。它们从e^-32到e^31不等。你可以很容易地看到这一点。
df['zg500']'''0 -3.996248e-291 2.476790e+112 -1.010202e+083 -1.407987e-024 2.240596e-32 ... 1742 -1.682389e+111743 -4.802401e+001744 -3.480795e+311745 1.026754e+211746 1.790822e+23Name: zg500, Length: 1739, dtype: float64'''
针对这个问题,我们可以对目标值进行缩放。虽然这不是推荐的方法,但我们别无选择。下面是使用Standard Scaler
对目标值进行缩放的微小修改。
ss = StandardScaler()# 一个从Pandas数据框创建tf.data数据集的实用方法def df_to_dataset(dataframe, shuffle=True, batch_size=32): dataframe = dataframe.copy() labels = ss.fit_transform(dataframe['zg500'].values.reshape(-1,1)) ds = tf.data.Dataset.from_tensor_slices((dict(dataframe), labels)) if shuffle: ds = ds.shuffle(buffer_size=len(dataframe)) ds = ds.batch(batch_size) return ds
在做了这些修改后,训练模型的结果如下:
history = model.fit(train_ds, epochs=2)'''考虑使用函数式API重写这个模型。109/109 [==============================] - 1s 804us/step - loss: 27.0520Epoch 2/10109/109 [==============================] - 0s 769us/step - loss: 1.0166Epoch 3/10109/109 [==============================] - 0s 753us/step - loss: 1.0148Epoch 4/10109/109 [==============================] - 0s 779us/step - loss: 1.0115Epoch 5/10109/109 [==============================] - 0s 775us/step - loss: 1.0107Epoch 6/10109/109 [==============================] - 0s 915us/step - loss: 1.0107Epoch 7/10109/109 [==============================] - 0s 1ms/step - loss: 1.0034Epoch 8/10109/109 [==============================] - 0s 784us/step - loss: 1.0092Epoch 9/10109/109 [==============================] - 0s 735us/step - loss: 1.0151Epoch 10/10109/109 [==============================] - 0s 803us/step - loss: 1.0105'''