subtract one day from a pandas dataframe date column

Question

Sorry for my lack of Pandas and datetime ability. I have tried but something seems to be just outside my current limited skillset.

I am using a date column (FinalStartPunch) which is typed to datetime64[ns] and if the Hour is at 0, then subtract one day from a datetime64[ns] column (date). If the FinalStartPunch hour is not a 0, then just copy the date column value as is.

sample data:

date         FinalStartPunch
6/27/2015   27JUN2015:14:15:00
7/23/2015   23JUL2015:13:31:00
7/23/2015   23JUL2015:18:43:00
8/15/2015   15AUG2015:18:35:00
8/15/2015   15AUG2015:23:30:00
8/16/2015   16AUG2015:00:00:00
1/30/2016   30JAN2016:18:25:00
1/30/2016   30JAN2016:23:52:00
1/31/2016   31JAN2016:00:00:00
8/13/2016   13AUG2016:18:30:00
8/13/2016   13AUG2016:23:58:00
8/14/2016   14AUG2016:00:00:00
1/28/2017   28JAN2017:18:30:00
1/28/2017   28JAN2017:23:57:00
1/29/2017   29JAN2017:00:00:00

Key code section:

df['New'] = df['date'] - pd.Timedelta(1, unit='D')  ### This one works
print(df.dtypes)

conds = [df['FinalStartPunch'].dt.hour == 0, df['FinalStartPunch'].dt.hour > 0]

choices = [df['date'] - pd.Timedelta(1, unit='D'), df['date']]

df['Date1'] = np.select(conds, choices, default=0)

The Error: invalid type promotion

The df['New'] works with the same code in the choices row which seems to be the problem causing the error.

Perhaps a Numpy issue and I need a different approach to doing this?

Any help - so greatly appreciated.

Answer 1

You can use .dt.hour == 0 with np.where() to identify rows where the hour is 0 and - pd.tseries.offsets.Day() to subtract a day.

df['date'] = pd.to_datetime(df['date'])
df['FinalStartPunch'] = pd.to_datetime(df['FinalStartPunch'], format='%d%b%Y:%H:%M:%S')
df['Date1'] = df['FinalStartPunch'].where((df['FinalStartPunch'].dt.hour != 0),
                                                    df['date'] - pd.tseries.offsets.Day())
df
Out[37]: 
         date     FinalStartPunch               Date1
0  2015-06-27 2015-06-27 14:15:00 2015-06-27 14:15:00
1  2015-07-23 2015-07-23 13:31:00 2015-07-23 13:31:00
2  2015-07-23 2015-07-23 18:43:00 2015-07-23 18:43:00
3  2015-08-15 2015-08-15 18:35:00 2015-08-15 18:35:00
4  2015-08-15 2015-08-15 23:30:00 2015-08-15 23:30:00
5  2015-08-16 2015-08-16 00:00:00 2015-08-15 00:00:00
6  2016-01-30 2016-01-30 18:25:00 2016-01-30 18:25:00
7  2016-01-30 2016-01-30 23:52:00 2016-01-30 23:52:00
8  2016-01-31 2016-01-31 00:00:00 2016-01-30 00:00:00
9  2016-08-13 2016-08-13 18:30:00 2016-08-13 18:30:00
10 2016-08-13 2016-08-13 23:58:00 2016-08-13 23:58:00
11 2016-08-14 2016-08-14 00:00:00 2016-08-13 00:00:00
12 2017-01-28 2017-01-28 18:30:00 2017-01-28 18:30:00
13 2017-01-28 2017-01-28 23:57:00 2017-01-28 23:57:00
14 2017-01-29 2017-01-29 00:00:00 2017-01-28 00:00:00

Answer 2

df.loc[df['FinalStartPunch'].dt.hour == 0, 'FinalStartPunch'] = df['FinalStartPunch'] - pd.Timedelta(1, unit='D')

You can just use .loc to search for those times, then subtract the day.