我有如下所示的文本：
list1 = ["My name is xyz", "My name is pqr", "I work in abc"]以上将作为使用kmeans进行文本聚类的训练集。
list2 = ["My name is xyz", "I work in abc"]以上是我的测试集。
我已经构建了一个向量化器和模型，如下所示：
vectorizer = TfidfVectorizer(min_df = 0, max_df=0.5, stop_words = "english", charset_error = "ignore", ngram_range = (1,3))vectorized = vectorizer.fit_transform(list1)km=KMeans(n_clusters=2, init='k-means++', n_init=10, max_iter=1000, tol=0.0001, precompute_distances=True, verbose=0, random_state=None, copy_x=True, n_jobs=1)km.fit(vectorized)如果我尝试预测我的测试集”list2″的聚类结果：
km.predict(list2)我会得到以下错误：
ValueError: Incorrect number of features. Got 2 features, expected 5我被告知使用Pipeline来解决这个问题。所以我编写了以下代码：
pipe = Pipeline([('vect', vectorizer), ('vectorized', vectorized), ('kmeans',km )])但我得到了以下错误：
TypeError Traceback (most recent call last)/mnt/folder/Text_Mining/<ipython-input-14-321cabc3bf35> in <module>()----> 1 pipe = Pipeline([('vect', vectorizer), ('vectorized', vectorized), ('kmeans',km )])/usr/local/lib/python2.7/dist-packages/scikit_learn-0.13-py2.7-linux-x86_64.egg/sklearn/pipeline.pyc in __init__(self, steps) 87 raise TypeError("All intermediate steps a the chain should " 88 "be transforms and implement fit and transform"---> 89 "'%s' (type %s) doesn't)" % (t, type(t))) 90 91 if not hasattr(estimator, "fit"):TypeError: All intermediate steps a the chain should be transforms and implement fit and transform' (0, 2) 1.0(1, 4) 0.57735026919(1, 3) 0.57735026919(1, 1) 0.57735026919(2, 0) 1.0' (type <class 'scipy.sparse.csr.csr_matrix'>) doesn't)我认为可能是vectorized的输出没有实现fit和transform方法，但在这种特定情况下我该如何做呢？我是机器学习的新手。另外，如何从kmeans模型中获取标签？当我运行kmeans时，我可以通过使用km.labels_来访问聚类标签。如何在Pipeline中做类似的事情？
回答：
你已经非常接近了！跳过中间的显式vectorizer.fit()步骤，在管道中完成所有操作：
list1 = ["My name is xyz", "My name is pqr", "I work in abc"]list2 = ["My name is xyz", "I work in abc"]vectorizer = TfidfVectorizer(min_df = 0, max_df=0.5, stop_words = "english", ngram_range = (1,3))km = KMeans(n_clusters=2, init='k-means++', n_init=10, max_iter=1000, tol=0.0001, precompute_distances=True, verbose=0, random_state=None, copy_x=True, n_jobs=1)pipe = Pipeline([('vect', vectorizer), ('kmeans', km)])pipe.fit(list1)pipe.transform(list2)结果：
array([[0.70710678, 1.41421356],[0.70710678, 1.41421356]])
- 注意：为了在我的当前版本的sklearn（0.22.1）上运行，我调整了一些参数…已经过去几年了 🙂
