When one column cell has a value zero, make the value in another column zero and cells below it zero

440 Views Asked by At

I'm reading temperature data from a sensor that is cycling on and off in a dataframe df. Each time the sensor turns on, it takes roughly 5 rows of data to thermally equilibrate. I want to ignore the decreased temp values from the warm up time of the sensor in any statistics that would be run on the temperature column, and also ignore them when plotting. The three columns in the data frame are 'Seconds', 'Sensor_State', and 'Temperature'. I have created a fourth column called 'Sensor_Warmup_State' that is created with a loop, and turns all values to 0 after a 0 is detected in the 'Sensor_State' column in the next 5 cells. Then I multiply 'Temperature' by 'Sensor_Warmup_State' to get 'Processed_Temp'. This works, but I know there should be a more pythonic, faster way to do this, I just don't have the expertise yet.

Here's what I have. To create the dataframe:

import numpy as np
a=np.arange(1,21).tolist()
b = (np.zeros((2), dtype=int)).tolist()
c = (np.ones((18), dtype = int)).tolist()
d = b + c
e = [0,0,1,2,4,8,9,10,10,10,10,10,10,10,10,10,10,10,10,10]
data = {'Seconds': a, 'Sensor_State': d, 'Temperature': e}
df = pd.DataFrame.from_dict(data)
df['Sensor_Warmup_State'] = 0
df

Screenshot of df output

To create the final two columns:

NumOfRows = df['Sensor_State'].size
x=0
for index, value in df['Sensor_State'].iteritems():
    if (value == 0) & (index < NumOfRows-5):
        df['Sensor_Warmup_State'].iloc[index] = 0
    elif (value == 1) & (index < NumOfRows-5):
        df.loc[(index + 5), 'Sensor_Warmup_State'] = 1
df['Processed_Temp'] = df['Sensor_Warmup_State'] * df['Temperature']
df

Screenshot of df output with new columns

1

There are 1 best solutions below

0
octothor On

OP here, I figured out a better way by using .shift(), this is much simpler and 30% faster than looping through as initially outlined. I edited the starting dataframe to account for when the Sensor_State goes from 0 to 1, back to 0 and to 1 to account for these circumstances. Hope this helps someone:

In [1]:
import numpy as np
import pandas as pd

a=np.arange(1,24).tolist()
b=[0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1]
c = [0,0,1,2,4,8,9,10,10,10,0,0,0,0,0,2,4,8,10,10,10,10,10]
data = {'Seconds': a, 'Sensor_State': b, 'Temperature': c}
df = pd.DataFrame.from_dict(data)
df['Sensor_Warmup_State'] = 0
df

Out[1]: 
    Seconds  Sensor_State  Temperature  Sensor_Warmup_State
0         1             0            0                    0
1         2             0            0                    0
2         3             1            1                    0
3         4             1            2                    0
4         5             1            4                    0
5         6             1            8                    0
6         7             1            9                    0
7         8             1           10                    0
8         9             1           10                    0
9        10             1           10                    0
10       11             0            0                    0
11       12             0            0                    0
12       13             0            0                    0
13       14             0            0                    0
14       15             0            0                    0
15       16             1            2                    0
16       17             1            4                    0
17       18             1            8                    0
18       19             1           10                    0
19       20             1           10                    0
20       21             1           10                    0
21       22             1           10                    0
22       23             1           10                    0

The new code:

In [2]:
df['Sensor_Warmup_State'] = (df['Sensor_State'] == 1) &\
(df['Sensor_State'].shift(1) == 1) &\
(df['Sensor_State'].shift(2) == 1) &\
(df['Sensor_State'].shift(3) == 1) &\
(df['Sensor_State'].shift(4) == 1) &\
(df['Sensor_State'].shift(5) == 1)

df['Processed_Temp'] = df['Sensor_Warmup_State'] * df['Temperature']
df

Out[2]: 
    Seconds  Sensor_State  Temperature  Sensor_Warmup_State  Processed_Temp
0         1             0            0                False               0
1         2             0            0                False               0
2         3             1            1                False               0
3         4             1            2                False               0
4         5             1            4                False               0
5         6             1            8                False               0
6         7             1            9                False               0
7         8             1           10                 True              10
8         9             1           10                 True              10
9        10             1           10                 True              10
10       11             0            0                False               0
11       12             0            0                False               0
12       13             0            0                False               0
13       14             0            0                False               0
14       15             0            0                False               0
15       16             1            2                False               0
16       17             1            4                False               0
17       18             1            8                False               0
18       19             1           10                False               0
19       20             1           10                False               0
20       21             1           10                 True              10
21       22             1           10                 True              10
22       23             1           10                 True              10