.arff files with scikit-learn?

Question

I would like to use an Attribute-Relation File Format with scikit-learn to do some NLP task, is this possible? How can use an .arff file with scikit-learn?

Answer 1

I really recommend liac-arff. It doesn't load directly to numpy, but the conversion is simple:

import arff, numpy as np
dataset = arff.load(open('mydataset.arff', 'rb'))
data = np.array(dataset['data'])

Answer 2

I found that scipy has a loader for arff files loadarff() to load them as numpy record arrays. I am not 100% sure that those arrays are suitable for direct consumption by scikit-learn but that should get your started.

Answer 3

Solution with scipy.arff

Code:

from scipy.io import arff
import pandas as pd

data = arff.loadarff('file.arff')
df = pd.DataFrame(data[0])
df.head()

Answer 4

Follow renatopp's answer: assume your data is the iris dataset, there should be 5 dimensional with last one is the class label column.

s = svm.SVC()
data_input = data[:,0:4]
labels = data[:,4] # this is the class column
s.fit(data_input, labels)

I think this is something you want.

Answer 5

If your "arff" file is a text file, try the following code instead:

import arff, numpy as np
dataset = arff.loads(open('mydataset.arff', 'rt'))
data = np.array(dataset['data'])