我正在使用Scikit-learn进行监督机器学习。我有两个数据集。第一个数据集包含具有X特征和Y标签的数据。第二个数据集仅包含X特征，没有Y标签。我可以成功地对训练/测试数据执行LinearSVC，并获取测试数据集的Y标签。
现在，我想使用为第一个数据集训练的模型来预测第二个数据集的标签。我如何在Scikit-learn中使用第一个数据集的预训练模型来预测第二个数据集（未见标签）的标签？
来自我的尝试的代码片段：从下面的评论中更新的代码：
from sklearn.feature_extraction.text import TfidfVectorizerfrom sklearn.svm import LinearSVCfrom sklearn.model_selection import train_test_splitfrom sklearn.metrics import confusion_matriximport pandas as pdimport pickle# ----------- Dataset 1: for training ----------- ## Sample data ONLYsome_text = ['Books are amazing', 'Harry potter book is awesome. It rocks', 'Nutrition is very important', 'Welcome to library, you can find as many book as you like', 'Food like brocolli has many advantages']y_variable = [1,1,0,1,0]# books = 1 : y label# food = 0 : y labeldf = pd.DataFrame({'text':some_text, 'y_variable': y_variable })# ------------- TFIDF process -------------#tfidf = TfidfVectorizer()features = tfidf.fit_transform(df['text']).toarray()labels = df.y_variablefeatures.shape# ------------- Build Model -------------#model = LinearSVC()X_train, X_test, y_train, y_test= train_test_split(features, labels, train_size=0.5, random_state=0)model.fit(X_train, y_train)y_pred = model.predict(X_test)# Export modelpickle.dump(model, open('model.pkl', 'wb'))# Read the Modelmodel_pre_trained = pickle.load(open('model.pkl','rb'))# ----------- Dataset 2: UNSEEN DATASET ----------- #some_text2 = ['Harry potter books are amazing', 'Gluten free diet is getting popular']unseen_df = pd.DataFrame({'text':some_text2}) # Notice this doesn't have y_variable. This the is the data set I am trying to predict y_variable labels 1 or 0.# This is where the ERROR occursX_unseen = tfidf.fit_transform(unseen_df['text']).toarray()y_pred_unseen = model_pre_trained.predict(X_unseen) # error here: # ValueError: X has 11 features per sample; expecting 26print(X_unseen.shape) # prints (2, 11)print(X_train.shape) # prints (2, 26)# Looking for an output like this for UNSEEN data# Looking for results after predicting unseen and no label data. text y_variableHarry potter books are amazing 1Gluten free diet is getting popular 0不必使用我上面尝试过的pickle代码。我在寻找是否有人有建议，或者是否有任何预构建的函数可以从scikit进行预测？
回答：
如您所见，您的第一个tfidf将输入转换为26个特征，而您的第二个tfidf将它们转换为11个特征。因此，错误发生是因为X_train的形状与X_unseen不同。提示告诉您X_unseen中的每个观察值的特征数少于model训练时接收的特征数。
一旦您在第二个脚本中加载model，您就在为文本拟合另一个向量化器。也就是说，第一个脚本中的tfidf和第二个脚本中的tfidf是不同的对象。为了用model进行预测，您需要使用原始的tfidf来转换X_unseen。为此，您必须导出原始向量化器，在新脚本中加载它，并在将其传递给model之前用它来转换新数据。
### 在第一个程序中这样做# 导出模型和tfidfpickle.dump(model, open('model.pkl', 'wb'))pickle.dump(tfidf, open('tfidf.pkl', 'wb'))### 在第二个程序中这样做model = pickle.load(open('model.pkl', 'rb'))tfidf = pickle.load(open('tfidf.pkl', 'rb'))# 使用`transform`而不是`fit_transform`X_unseen = tfidf.transform(unseen_df['text']).toarray()# 在`X_unseen`上预测y_pred_unseen = model_pre_trained.predict(X_unseen)