I have a column in my koalas dataframe called purchase_date
. In databricks notebook, with runtime as 10.3, when I do the following lines of code, I get the dtype of the purchase_date
column as <U0
. I am not able to understand why this is happenning.
My code which caused this is as follows (in Databricks runtime 10.3):
import databricks.koalas as ks
print("Datatype of purchase_date before astype:" , my_ks_dataframe['purchase_date'].dtype) # Datatype of purchase_date before astype: object
# Using the astype
my_ks_dataframe['purchase_date'] = my_ks_dataframe['purchase_date'].astype('str')
print("Datatype of purchase_date before astype:" , my_ks_dataframe['purchase_date'].dtype) # Datatype of purchase_date after astype: <U0
I am not sure why I see this behaviour in Databricks runtime 10.3. When I execute the same code in Databricks runtime 8.1, I get the desired datatype for purchase_date as object
before and after astype usage.
# print result in Databricks runtime 8.1
Datatype of purchase_date before astype: object
Datatype of purchase_date after astype: object