I am using Parquet.Net library to generate parquet files from C# code. 4.15.0 version works fine, but after switching to latest 4.16.4 the files generated are not readable using ParquetViewer. It throws error "DELTA_BINARY_PACKED" encoding is not support. I found on the net that this can be disabled using ParquetOptions. However, I am not able to find this option as part of the ParquetWriter method.
This is the code snippet
private async Task ParquetWriteRowGroup(ParquetSchema schema, List<dynamic> data)
{
Console.WriteLine($"output file => {OutputFilePath}");
using (Stream fs = File.Open(OutputFilePath, FileMode.CreateNew))
{
using (ParquetWriter writer = await ParquetWriter.CreateAsync(schema, fs))
{
writer.CompressionMethod = CompressionMethod.Snappy;
using (ParquetRowGroupWriter groupWriter = writer.CreateRowGroup())
{
for (int i = 0; i < data.Count; i++)
{
await groupWriter.WriteColumnAsync(new DataColumn(schema.DataFields[i], data[i]));
}
}
}
}
}
Got caught out with this also try this: