Neither PowerTransformer nor QuantileTransformer is working for a feature

Question

I am working on famous bike-sharing dataset. The distribution of target variabe cnt is heavily skewed. I am sharing its distribution as well for my readers' convenience.

And I wanted to transform it similar to the transformations I made to other columns. When I use QuantileTransformer(output_distribution='normal'), it returns me the one same value for every value.

target_transformer = QuantileTransformer(output_distribution='normal')

# Apply the transformation to the target variable
y_transformed = quantile_transformer.fit_transform([y])
pd.Series(y_transformed.flatten()).value_counts()

Outputs:

-5.199338    16877 Name: count, dtype: int64

(So it transformed every value to -5.199338)

Instead of QuantileTransformer, I tried using PowerTransformer(), using this code:

target_transformer = PowerTransformer()

# Apply the transformation to the target variable
y_transformed = target_transformer.fit_transform([y.values])
pd.Series(y_transformed.flatten()).value_counts()

Outputs error:

BracketError: The algorithm terminated without finding a valid bracket. Consider trying different initial points.

Why is this happening and how to resolve it?

Answer 1

I think I had a similar problem as you, as I had BracketError when I tried to use QuantileTransformer or PowerTransformer. May I know what is your scikit-learn version? Mine used to be scikit-learn 1.2.2. If you look at https://github.com/scikit-learn/scikit-learn/issues/27499 it deals with this issue. It suggests upgrading to scikit-learn 1.3.1 to solve it. My conda couldn't find this version so I upgraded to scikit-learn 1.3.0 and it fixed the problem for me. Perhaps you can try that?