Selection column in a dataframe in pandas apply min function

Question

I have n-dataframe in a list

df=[df_1, df_2, df_3, ...., df_n]

Where df_n is a dataframe in pandas (python). df_n is a variable of my keras-model.

X_train=[df_1_1,df_2_1,...,df_n_1]

Where:

df_1_1 is the first dataframe of the list (the first variable) and the first columns of this dataframe, his dataframe has m columns.

Each column of this dataframe if this variable applies a different type of smoothing or filter.

I have 100 column in each dataframe and I want to select the combination of columns (of different dataframes), the X_train than have min value in the score of my model.

score =  model.evaluate(X_test,Y_test)

X_test and Y_test are the last n occurrences of the selected columns.

There some library for selected this columns (neuronal networks, GA, colony ant, ...)?

How can I implement it?

Answer 1

What is your prediction task? Do you need a neural network or not? You are essentially looking at a feature selection problem here. You could use simpler models such as the lasso which will select columns using L1-regularization. Or you could use an ensembling technique such as random forests and consider the relative feature importances to select your columns. Perhaps have a look at scikit-learn.