Formatting a Y2K date value in Databricks

Question

The date of Jan 01, 1901 converts to Jan 01, 2001 which is wrong. How is this done in Databricks?

%sql
Select '01-Jan-01' badDate, to_date('01-Jan-01','dd-MMM-yy') as date2, date_format('01-Jan-01','d-MMM-yy'),,date_format('01-Jan-01','dd-MM-yy'),date_format('01-01-01','dd-MM-yy');

Answer 1

Its unclear what you're asking but your date format string is incorrectly formated for one. You have dd-MMM-yy when it should be dd-MM-yy

Answer 2

Just a thought - since it is Databricks. Use a Python function to convert it using a cutoff date of '01-01-'+ year(date)

cutoff_date = pd.to_datetime('01-01-2023')
df.loc[df.date > cutoff_date, 'date'] -= pd.DateOffset(years=100)

Possible solution Easy way to fix wrong year (y2k bug) using pandas