I am new to PySpark, and I want to add a column to a Spark DataFrame from another DataFrame's column. The structure of the column I want to add is such that each row is an array of integers. I am receiving the following error when attempting to do so:
java.lang.RuntimeException: Unsupported literal type class java.util.ArrayList [61, 70, 78, 83, 77]
Can anyone advise on how to solve this issue?