I'm using the Redshift Unload command to unload a table to parquet into a target S3 bucket. I'm finding that my JSON columns - which are currently defined as a SUPER data type in redshift is not being serialized into a parquet struct.
What this it gets serialized as is a Byte String. I confirmed that the columns are in proper JSON format using redshifts is_valid_json command
Is there anyway to override this behavior and make the JSON column behave as a Parquet struct? I cannot find anything in the documentation about this issue. It seems odd to me that they would support Parquet format but lack the ability to unload to all of the parquet data types without mentioning it in the docs. (or I'm blind)
I tried a different redshift column types Super(resolves as a byte string) and Varchar(resolves as a string). Confirmed the column is valid JSON. I've scoured the docs for this being supported or not and cannot find anything. Also Googled this specific issue and cannot find anything on it.