I need to use Parquet-mr library to read from Parquet files programmatically in Java. I need to selectively read a few columns and skip other columns (For example, read 3 columns out of 500 columns). I can't seem to find any documentation on how to do that. Can someone please point me to one if there is any?
Documentation for Parquet-mr java library
1.8k Views Asked by User29519 At
1
There are 1 best solutions below
Related Questions in PARQUET
- Polars with Rust: Out of Memory Error when Processing Large Dataset in Docker Using Streaming
- I am facing issue with ParquetFileWriting n hdfs in flink where parquet file size is around 382 KB . I want the parquet file in MB
- Packages for reading parquets in NodeJS (2024)
- ADF Copy Activity from Source Azure Synapse Analytics Target ADLSGen2 Storage account
- Worth it to access data by blocks on modern OS/hardware?
- Does having large number of parquet files causes memory overhead while reading using Spark?
- Hive query on HUE shows different timestamp than programatically/on data
- Reading partitioned parquet files with Apache Beam and Python SDK
- Read the latest S3 parquet files partitioned by date key using Polars
- redshift spectrum type conversion from String to Varchar
- Azure error writing parquet to ADLS Gen 2
- Is there any way to stream to a parquet file in Ruby?
- AWS S3 Parquet data lake: How to best deploy aggregation Python script
- TensorFlowIO: Corrupted reads of pyspark compressed spark Parquet files
- parquet Incremental updates cause disordered reading in python
Related Questions in PARQUET-MR
- Access Pages, PageHeaders and encodings of a parquet file
- How to get the version of parquet from file header using parquet-tools?
- Parquet-mr - Enabling dictionnary on column increase row group number
- How do I find the parquet.writer.version from an existing parquet file?
- Does Apache Parquet support Custom Filter Predicate on Repeated values?
- Spark3.2 write parquet files in spark2.3.1 format
- AvroParquetWriter - addLogicalTypeConversion not working as expected (using version parquet-avro 1.12.3) - causing ClassCastException
- How should protobuf message with repeated fields be converted to parquet to be queried by Athena?
- How to get an efficient data ingestion solution using Java, Apache Arrow and Apache Parquet
- Is it possible to reopen ParquetWriter after close() is called?
- Parquet-MR library is throwing an exception while reading (FIXED_LEN_BYTE_ARRAY / UUID) column
- Process parquet file row-wise
- Sorted parquet files for query optimization
- read a parquet file using Java, but it works in local machine, and doesn't work in docker container
- can OOZIE Map-Reduce job save data in parquet format?
Trending Questions
- UIImageView Frame Doesn't Reflect Constraints
- Is it possible to use adb commands to click on a view by finding its ID?
- How to create a new web character symbol recognizable by html/javascript?
- Why isn't my CSS3 animation smooth in Google Chrome (but very smooth on other browsers)?
- Heap Gives Page Fault
- Connect ffmpeg to Visual Studio 2008
- Both Object- and ValueAnimator jumps when Duration is set above API LvL 24
- How to avoid default initialization of objects in std::vector?
- second argument of the command line arguments in a format other than char** argv or char* argv[]
- How to improve efficiency of algorithm which generates next lexicographic permutation?
- Navigating to the another actvity app getting crash in android
- How to read the particular message format in android and store in sqlite database?
- Resetting inventory status after order is cancelled
- Efficiently compute powers of X in SSE/AVX
- Insert into an external database using ajax and php : POST 500 (Internal Server Error)
Popular # Hahtags
Popular Questions
- How do I undo the most recent local commits in Git?
- How can I remove a specific item from an array in JavaScript?
- How do I delete a Git branch locally and remotely?
- Find all files containing a specific text (string) on Linux?
- How do I revert a Git repository to a previous commit?
- How do I create an HTML button that acts like a link?
- How do I check out a remote Git branch?
- How do I force "git pull" to overwrite local files?
- How do I list all files of a directory?
- How to check whether a string contains a substring in JavaScript?
- How do I redirect to another webpage?
- How can I iterate over rows in a Pandas DataFrame?
- How do I convert a String to an int in Java?
- Does Python have a string 'contains' substring method?
- How do I check if a string contains a specific word?
Unfortunately this is not documented too well. There are some examples you can check out here. These ones use the ExampleParquetWriter class from Parquet however, which was meant to be used as an example only. Nevertheless, it works.
The proper way to use Parquet would be either through one of the supported object models (like Avro, Thrift or Protobuf) or by implementing your own object model (which leads to the best performance). You can read more about object models here.