finding conditional local minima values in time series python

1.3k Views Asked by At

For a time series dataset:

A, How do I find the local minima (nadir values) for each ID? (local mins)

B, How do I find any subsequent values that are 2 greater than each local minima. (local mins + 2)

import pandas as pd
df = pd.DataFrame({'id': [1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2], 'value': [8,5,3,2,1,2,3,5, 1.5, 3, 1, 1.5, 2, 3, 4, 0.4]})

For A, I was able to use the following code to find all the nadir/local minimum values from the dataset, but they are not grouped by each id. How do I modify this to group them by id?

nadir_min = df.value[(df.value.shift(1) > df.value) & (df.value.shift(-1) > df.value)]
nadir_min
Out[1]: 
4     1.0
8     1.5
10    1.0
Name: value, dtype: float64

For B, I would like to get back the subsequent values after the nadir/local minimums that are two greater than the nadir/local minimums. For the example data above I would get back:

index  id value 
 6      1   3.0
 13     2   3.0
 14     2   4.0 

Perhaps a conditional loop would do the trick as it can store each local min and compare the subsequent values if they are 2 greater than it. However, the working dataset is MASSIVE and would take too long to run so I am trying something like this:

df['min_plus2'] = (df['value'] >= nadir_min + 2) & (df.index > nadir_min_index)
1

There are 1 best solutions below

5
On

You do it with next code:

import pandas as pd
import numpy as np

df = pd.DataFrame({'id': [1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,2], 'value': [8,5,3,2,1,2,3,5, 1.5, 3, 1, 1.5, 2, 3, 4, 0.4]})    
df['loc_min'] = df.value[(df.value.shift(1) > df.value) & (df.value.shift(-1) > df.value)]
df['if_A'] = np.where(df['loc_min'].isna(), False, True)    
df['loc_min'].fillna(method='ffill', inplace=True)    
df['if_B'] = np.where(df['value'] - df['loc_min'] >= 2, True, False)

Answer for A:

df[df['if_A']==True]

Answer for B:

df[df['if_B']==True]