Read parquet partitioned file with dtplyr (lazy evalution) - R

328 Views Asked by At

I have those parquet files partitioned, so in multiple files.

I can read them with

df_test <- arrow::open_dataset('/path/topic.parquet',
                              partitioning=c("id")) 

Since I have to do a lot of transformation before really using it I wanted to have a lazy DF thanks to dtplyr.

df_test_lazy <- arrow::open_dataset('/path/topic.parquet',
                              partitioning=c("id")) %>% lazy_dt()

I have the error

Error in as.data.frame.default(x, ...) : cannot coerce class ‘c("FileSystemDataset", "Dataset", "ArrowObject", "R6")’ to a data.frame

How can I use lazy_dt() to read my data ?

0

There are 0 best solutions below