highlight outlier in each columns using panda style for loop

Question

i want to highlight my cell outlier with different condition minimum and maximum outlier for each column. this is my image of data.

num_cols = ['X','Y','FFMC','DMC','DC','ISI','temp','RH','wind','rain','area']

Q1 = dataset[num_cols].quantile(0.25)
Q3 = dataset[num_cols].quantile(0.75)
IQR = Q3 - Q1

lower = Q1 - 1.5 * IQR
upper = Q3 + 1.5 * IQR

i tried this code base on this solustion:

def highlight_outlier(df_):
    styles_df = pd.DataFrame('background-color: white',
                             index=df_.index,
                             columns=df_.columns)
    for s in num_cols:
        styles_df[s].apply(lambda x: 'background-color: yellow' if x < upper[s] or x < lower[s] else 'background-color: white')
    return styles_df

dataset_sort = dataset.sort_values("outliers")
dataset_sort.style.apply(highlight_outlier,axis=None)

also tried this code based on this solution:

def highlight_outlier(x):
    c1 = 'background-color: yellow'

    #empty DataFrame of styles
    df1 = pd.DataFrame('', index=x.index, columns=x.columns)
    #set new columns by condition
    for col in num_cols:
        df1.loc[(x[col] < upper), col] = c1
        df1.loc[(x[col] > lower), col] = c1
    return df1

dataset_sort = dataset.sort_values("outliers")
dataset_sort.style.apply(highlight_outlier,axis=None)

both failed. and how can i show only 5 data after styling? thank you

Answer 1

In your calculation lower und upper are of type pd.Series. Therefor you have to use an iterator in your loop inside the highlight_outlier() function to avoid an indexing problem. I used upper[i] below.

def highlight_outlier(x):
    c1 = 'background-color: yellow'

    #empty DataFrame of styles
    df1 = pd.DataFrame('', index=x.index, columns=x.columns)
    #set new columns by condition
    for i, col in enumerate(df.columns):
        df1.loc[(x[col] > upper[i]), col] = c1
        df1.loc[(x[col] < lower[i]), col] = c1
    return df1

Minimal Example

import pandas as pd
import numpy as np


df = pd.DataFrame({
    'a':np.random.randint(0,100,10),
    'b':np.random.randint(0,100,10),
})

Q1 = df[['a', 'b']].quantile(0.25)
Q3 = df[['a', 'b']].quantile(0.75)
IQR = Q3 - Q1

# here I set the values to some defaults to see any output
lower = [3, 5] # Q1 - 1.5 * IQR
upper = [97, 95] # Q3 + 1.5 * IQR

df.style.apply(highlight_outlier,axis=None)