Parquet - Specifying file path when using external key material

75 Views Asked by At

I have a use case where I have to encrypt my Parquet files. I implemented the KMSClient abstract class provided by Parquet CryptoFactory and have been able to encrypt and Decrypt the Parquet files and the DEK.

While the above is working as expected, I've been also trying to save the DEK externally using the property in EncryptionConfiguration(internal_key_material=True) but this ends up giving me an error like The Parquet file path must be specified when using external key material

My EncryptionConfig:

encryption_config = pe.EncryptionConfiguration(
        footer_key=key_name,
        column_keys={
            key_name: ["a", "b"],
        },
        plaintext_footer=True,
        internal_key_material=False,
        data_key_length_bits=256,
    )
    with pq.ParquetWriter(path,
                          table.schema,
                          encryption_properties=file_encryption_properties
) as writer:
        writer.write_table(table)

I provided the path where it must create the Parquet file, as per the doc it says it creates all other files in the same path. I can't figure out what I am missing here.

0

There are 0 best solutions below