我正在训练一个模型来区分真实和虚假数据。我有一个真实数据集和一个虚假数据集，由于不知道如何在一个模型中同时训练这两个数据集，我现在分别为真实数据和虚假数据训练了两个模型。

问题是训练任何一个模型似乎都不可能，因为我遇到了这个错误：

ValueError: Input X contains infinity or a value too large for dtype('float16')

对于这段代码

import pandas as pdimport numpy as npfrom sklearn.model_selection import train_test_splitfrom sklearn.ensemble import RandomForestClassifierfrom sklearn.metrics import accuracy_scorefrom sklearn.utils import shufflefrom sklearn.preprocessing import MinMaxScalerfrom sklearn.preprocessing import RobustScalerfrom sklearn.metrics import classification_reportfrom sklearn.metrics import confusion_matrixscaler = RobustScaler()X = np.load("fake_x.npy")x_data_flat = X.reshape(X.shape[0], -1)x_data_scaled = scaler.fit_transform(x_data_flat)y = np.load("fake_y.npy")x_data_flat, y = shuffle(x_data_flat, y)X_train,X_test,y_train,y_test = train_test_split(x_data_scaled,y,test_size=0.1, train_size=0.1)clf = RandomForestClassifier()clf.fit(X_train, y_train)y_pred = clf.predict(X_test)accuracy = accuracy_score(y_test, y_pred)print("Accuracy:", accuracy)print("Classification Report:")print(classification_report(y_test, y_pred))conf_matrix = confusion_matrix(y_test, y_pred)print("Confusion Matrix:")print(conf_matrix)

数据确实太大了，x.shape = (750,1998,101) 而 y.shape = (750,496)。数据链接：https://drive.google.com/drive/folders/1EYnOIOWP17ALs-903ESFM-02_6VszJEx对于分类本身，我会使用输入的相似度得分，谁的相似度得分更高，输入就属于那个类别。

使用最初是real.npz但提取为real_x.npy和real_y.npy的2D频谱图以简化操作，模型应该能够识别输入是属于真实的还是属于我提取为fake_x.npy和fake_y.npy的虚假数据。

回答：

我将虚假和真实数据合并成一个矩阵，并创建了一个二进制向量y，其中1表示真实，0表示虚假。数据中有一些无效值（np.inf），我用float16允许的最大值替换了这些值。我训练了一个分类器，在验证集上获得了87%的分类准确率。

你可能还需要分出一个测试集。在我的代码中，我只处理了训练集和验证集。

输出：

Accuracy: 0.87Classification Report:              precision    recall  f1-score   support         0.0       0.84      0.89      0.87       141         1.0       0.90      0.85      0.87       159    accuracy                           0.87       300   macro avg       0.87      0.87      0.87       300weighted avg       0.87      0.87      0.87       300Confusion Matrix:[[126  15] [ 24 135]]

代码：

import pandas as pdimport numpy as npimport matplotlib.pyplot as pltfrom sklearn.model_selection import train_test_splitfrom sklearn.ensemble import ExtraTreesClassifierfrom sklearn.metrics import accuracy_scorefrom sklearn.utils import shufflefrom sklearn.preprocessing import MinMaxScalerfrom sklearn.preprocessing import RobustScalerfrom sklearn.metrics import classification_reportfrom sklearn.metrics import confusion_matrix## 加载真实和虚假数据，并堆叠成一个矩阵#data_real = np.load('real.npz')['x']data_fake = np.load('fake.npz')['x']data = np.concatenate([data_real, data_fake], axis=0)#目标向量：y=1对应真实，y=0对应虚假y = np.concatenate([np.ones(len(data_real)), np.zeros(len(data_fake))])#通过替换处理无穷大值X = np.nan_to_num(data, posinf=np.finfo(data.dtype).max)X = X.reshape(X.shape[0], -1) #扁平化#分割成训练和验证集X_train, X_test, y_train, y_test = train_test_split(    X, y, train_size=0.8, shuffle=True, random_state=0)#可选地降低维度# 这显著加速了clf.fit()步骤（从40秒减少到3秒）# 它还减少了内存消耗# 但准确率从大约87%下降到77%# 对于加速探索性分析可能有用reduce_dimensionality = Falseif reduce_dimensionality:    from sklearn.random_projection import SparseRandomProjection    projector = SparseRandomProjection(n_components='auto', random_state=0).fit(X_train)        X_train, X_test = [projector.transform(x) for x in [X_train, X_test]]#拟合分类器np.random.seed(0)clf = ExtraTreesClassifier(n_estimators=400)clf.fit(X_train, y_train)y_pred = clf.predict(X_test)accuracy = accuracy_score(y_test, y_pred)print("Accuracy:", accuracy)print("Classification Report:")print(classification_report(y_test, y_pred))conf_matrix = confusion_matrix(y_test, y_pred)print("Confusion Matrix:")print(conf_matrix)