Is there a cost associated with converting Koalas dataframe to Spark dataframe?

305 Views Asked by At

I know that pandas works "under the hood" with numpy arrays stored in dictionaries. In contrast, Koalas works with the underlying Spark framework. Does that mean that there is no extra cost associated with switching back and forth between Koalas and PySpark dataframes?

#convert to pyspark dataframe
df.to_spark()

#convert to kolas frame
koalas_df = ks.DataFrame(df)
0

There are 0 best solutions below