Plotly: How to highlight certain periods in a pandas time series with rectangle shapes using a For Loop?

Question

I am trying to highlight timeperiods in Plotly. I seems like the best way is to use Shapes, like this, but in the real world you do not want to add each shape manually like in the example url. I am thinking a for loop is the best solution, but open for other (less computational) suggestions.

my script look like this:

np.random.seed(12345)
rows = 20
x = pd.Series(np.random.randn(rows),index=pd.date_range('1/1/2020', periods=rows)).cumsum()
df = pd.DataFrame({"index": x})

# add column showing what to shade
df["signal"] = df['index'] < 5

# plot index and highlight periods with Rectangle Shapes in Plotly
fig = px.line(df, x=df.index, y="index")

for row in df.iterrows():
    if df['signal'] == False:
        ply_shapes['shape_' + str(i)]=go.layout.Shape(type="rect",
                                                            x0=df.dato[i-1],
                                                            y0=0,
                                                            x1=df.dato[i],
                                                            y1=2000,
                                                            opacity=0.5,
                                                            layer="below"
                                                        )

lst_shapes=list(ply_shapes.values())
fig.update_layout(shapes=lst_shapes)
fig.show()

But this returns:

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Answer 1

There are several reasons why your code snippet will not work.

1. iterrows() returns an iterator containing the index of each row and the data in each row as a seres. To use this, you would have to replace

for row in df.iterrows():
    if df['signal'] == False:

with (for example):

for row in df.iterrows():
    if row[1]['signal'] == True:

Your error is raised bacause your are assigning a series of True, False with if df['signal'] == False: instead of a single value like if row[1]['signal' will do.

But only fixing this will not help you. At least not within the confinements of your snippet, because:

2. dato does not exist in your sample dataframe.

Similar questions have been asked and answered before. But since the solution will be similar for very different sounding questions, I've decided to make a custom solution for your use case as well.

---

The best approach will in either case depend heavily on how you identify and assign your highlighted periods within your time series. The following example will use random data and identify periods to highlight depending on a threshold. Just like in your question.

My suggestion will boil down to a function highLights() that will take a a plotly figure, a pandas series and a threshold as input, as well as a few other details (take a look at the docstring).

The highLights() function with some example input:

fig = highLights(fig = fig, variable = 'signal', level = 5, mode = 'above',
               fillcolor = 'rgba(200,0,200,0.2)', layer = 'below')

Plot

Complete code:

# imports
import numpy as np
import pandas as pd
import plotly.graph_objects as go
import plotly.express as px
import datetime

pd.set_option('display.max_rows', None)

# data sample
cols = ['signal']
nperiods = 200
np.random.seed(123)
df = pd.DataFrame(np.random.randint(-10, 12, size=(nperiods, len(cols))),
                  columns=cols)
datelist = pd.date_range(datetime.datetime(2020, 1, 1).strftime('%Y-%m-%d'),periods=nperiods).tolist()
df['dato'] = datelist 
df = df.set_index(['dato'])
df.index = pd.to_datetime(df.index)
df.iloc[0] = 0
df = df.cumsum().reset_index()

# plotly setup
fig = px.line(df, x='dato', y=df.columns[1:])
fig.update_xaxes(showgrid=True, gridwidth=1, gridcolor='rgba(0,0,255,0.1)')
fig.update_yaxes(showgrid=True, gridwidth=1, gridcolor='rgba(0,0,255,0.1)')


# function to set background color for a
# specified variable and a specified level
def highLights(fig, variable, level, mode, fillcolor, layer):
    """
    Set a specified color as background for given
    levels of a specified variable using a shape.
    
    Keyword arguments:
    ==================
    fig -- plotly figure
    variable -- column name in a pandas dataframe
    level -- int or float
    mode -- set threshold above or below
    fillcolor -- any color type that plotly can handle
    layer -- position of shape in plotly fiugre, like "below"
    
    """
    
    if mode == 'above':
        m = df[variable].gt(level)
    
    if mode == 'below':
        m = df[variable].lt(level)
        
    df1 = df[m].groupby((~m).cumsum())['dato'].agg(['first','last'])

    for index, row in df1.iterrows():
        #print(row['first'], row['last'])
        fig.add_shape(type="rect",
                        xref="x",
                        yref="paper",
                        x0=row['first'],
                        y0=0,
                        x1=row['last'],
                        y1=1,
                        line=dict(color="rgba(0,0,0,0)",width=3,),
                        fillcolor=fillcolor,
                        layer=layer) 
    return(fig)

fig = highLights(fig = fig, variable = 'signal', level = 5, mode = 'above',
               fillcolor = 'rgba(200,0,200,0.2)', layer = 'below')

fig.update_layout(template = 'plotly_dark')

fig.show()