我正在尝试使用朴素贝叶斯文本分类器进行文本分类。我的数据格式如下，根据问题和摘录，我需要决定问题的所属主题。训练数据有超过20,000条记录。我知道SVM在这里可能是一个更好的选择，但我还是想使用sklearn库中的朴素贝叶斯。

{[{"topic":"electronics","question":"What is the effective differencial effective of this circuit","excerpt":"I'm trying to work out, in general terms, the effective capacitance of this circuit (see diagram: https://i.sstatic.net/BS85b.png).  \n\nWhat is the effective capacitance of this circuit and will the ...\r\n        "},{"topic":"electronics","question":"Outlet Installation--more wires than my new outlet can use [on hold]","excerpt":"I am replacing a wall outlet with a Cooper Wiring USB outlet (TR7745).  The new outlet has 3 wires coming out of it--a black, a white, and a green.  Each one needs to be attached with a wire nut to ...\r\n        "}]}

这是我目前尝试过的，

import numpy as npimport jsonfrom sklearn.naive_bayes import *topic = []question = []excerpt = []with open('training.json') as f:    for line in f:        data = json.loads(line)        topic.append(data["topic"])        question.append(data["question"])        excerpt.append(data["excerpt"])unique_topics = list(set(topic))new_topic = [x.encode('UTF8') for x in topic]numeric_topics = [name.replace('gis', '1').replace('security', '2').replace('photo', '3').replace('mathematica', '4').replace('unix', '5').replace('wordpress', '6').replace('scifi', '7').replace('electronics', '8').replace('android', '9').replace('apple', '10') for name in new_topic]numeric_topics = [float(i) for i in numeric_topics]x1 = np.array(question)x2 = np.array(excerpt)X = zip(*[x1,x2])Y = np.array(numeric_topics)print X[0]clf = BernoulliNB()clf.fit(X, Y)print "Prediction:", clf.predict( ['hello'] )

但正如预期的那样，我得到了ValueError: could not convert string to float。我的问题是如何创建一个简单的分类器来将问题和摘录分类到相关的主题中？

回答：

sklearn中的所有分类器都要求输入表示为固定维度的向量。对于文本，有CountVectorizer、HashingVectorizer和TfidfVectorizer，它们可以将你的字符串转换为浮点数的向量。

vect = TfidfVectorizer()X = vect.fit_transform(X)

显然，你需要以相同的方式对测试集进行向量化

clf.predict( vect.transform(['hello']) )

请参阅关于使用sklearn处理文本数据的教程。