Export or Create a Dataset of the Imputed or Transformed Varibles Python

124 Views Asked by At

I was running KNN for my dataset, for which I had to impute the missing values and then transform the variables so that they can lie between 0 and 1.

I have to use this predicted results as inferred performance and make a TTD Model for the same.

When I use predict I can get the predicted probabilities but I am unable to transfer these results into the base dataset, so that it can be used to infer the performance.

Please find the sample code below -

train=pandas.read_csv("dev_in.csv")
y_train = train['Y']
w_train = train['WT']
x_train1 = train[[‘ABC’,’GEF’,’XYZ’]].replace(-1, numpy.NaN)
values = x_train1.values
imputer = Imputer()
#replacing with mean
x_train_trf = imputer.fit_transform(values) 
# count the number of NaN values in each column
print(numpy.isnan(x_train_trf).sum())
X_normalized = preprocessing.normalize(x_train_trf, norm='l2')

#similar data manipulations on test population
test=pandas.read_csv("oot_in.csv")
y_test = test['Y']
w_test = test['WT']
x_test1 = test[[‘ABC’,’GEF’,’XYZ’]].replace(-1, numpy.NaN)
print(numpy.isnan(x_test81).sum())
values_test = x_test1.values
imputer = Imputer()
#replacing with mean
x_test_trf = imputer.fit_transform(values_test) 
# count the number of NaN values in each column
print(numpy.isnan(x_test_trf).sum())
X_normalized_test = preprocessing.normalize(x_test_trf, norm='l2')

#fitting the KNN
knn = KNeighborsClassifier(n_neighbors=5, weights= 'distance', p=2)
knn.fit(X_normalized, y_train)

#checking prediction on the test population
y_pred_test = knn.predict(X_normalized_test) 
**test ['inferred'] = y_pred_test**
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-82-defc045e7eeb> in <module>()
----> 1 test ['inferred] = y_pred_test

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

The place where I try to create the variable inferred in the test dataset, I am getting the above error.

Your help will be greatly appreciated.

0

There are 0 best solutions below