I am training the NaiveBayesClassifier
in Python using sentences, and it gives me the error below. I do not understand what the error might be, and any help would be good.
I have tried many other input formats, but the error remains. The code given below:
from text.classifiers import NaiveBayesClassifier
from text.blob import TextBlob
train = [('I love this sandwich.', 'pos'),
('This is an amazing place!', 'pos'),
('I feel very good about these beers.', 'pos'),
('This is my best work.', 'pos'),
("What an awesome view", 'pos'),
('I do not like this restaurant', 'neg'),
('I am tired of this stuff.', 'neg'),
("I can't deal with this", 'neg'),
('He is my sworn enemy!', 'neg'),
('My boss is horrible.', 'neg') ]
test = [('The beer was good.', 'pos'),
('I do not enjoy my job', 'neg'),
("I ain't feeling dandy today.", 'neg'),
("I feel amazing!", 'pos'),
('Gary is a friend of mine.', 'pos'),
("I can't believe I'm doing this.", 'neg') ]
classifier = nltk.NaiveBayesClassifier.train(train)
I am including the traceback below.
Traceback (most recent call last):
File "C:\Users\5460\Desktop\train01.py", line 15, in <module>
all_words = set(word.lower() for passage in train for word in word_tokenize(passage[0]))
File "C:\Users\5460\Desktop\train01.py", line 15, in <genexpr>
all_words = set(word.lower() for passage in train for word in word_tokenize(passage[0]))
File "C:\Python27\lib\site-packages\nltk\tokenize\__init__.py", line 87, in word_tokenize
return _word_tokenize(text)
File "C:\Python27\lib\site-packages\nltk\tokenize\treebank.py", line 67, in tokenize
text = re.sub(r'^\"', r'``', text)
File "C:\Python27\lib\re.py", line 151, in sub
return _compile(pattern, flags).sub(repl, string, count)
TypeError: expected string or buffer
You need to change your data structure. Here is your
train
list as it currently stands:The problem is, though, that the first element of each tuple should be a dictionary of features. So I will change your list into a data structure that the classifier can work with:
Your data should now be structured like this:
Note that the first element of each tuple is now a dictionary. Now that your data is in place and the first element of each tuple is a dictionary, you can train the classifier like so:
If you want to use the classifier, you can do it like this. First, you begin with a test sentence:
Then, you tokenize the sentence and figure out which words the sentence shares with all_words. These constitute the sentence's features.
Your features will now look like this:
Then you simply classify those features:
This test sentence appears to be positive.