I'm writing a Python Transform and need to get the SparkSession so I can construct a DataFrame.
How should I do this?
I'm writing a Python Transform and need to get the SparkSession so I can construct a DataFrame.
How should I do this?
Copyright © 2021 Jogjafile Inc.
You can pass the SparkContext as an argument in the transform, which can then be used to generate the SparkSession.
This example can also be found in the Foundry documentation here: https://www.palantir.com/docs/foundry/transforms-python/transforms-python-api/#transform