type 5 error while using MonetDBLite or readr

65 Views Asked by At

I am trying to write 80GB csv file onto the MonetDB using MonetDBLite using the following code.

mtcars[1:12,] %>% write_csv("mtcars1_12.csv")
mtcars[13:32,] %>% write_csv("mtcars13_32.csv")

setwd("path") # please use your folder path

dbdir <- "path" # provide path where you want to create the MonetDB

con <- dbConnect(MonetDBLite::MonetDBLite(), dbdir)

f <- function(x, pos) dbWriteTable(conn=con, name="mtcars1", value= x, header = T, append = T)

read_csv_chunked("mtcars1_12.csv", DataFrameCallback$new(f), chunk_size = 10)
read_csv_chunked("mtcars13_32.csv", DataFrameCallback$new(f), chunk_size = 10)

Now, this mtcars data works perfectly fine, but my actual data with exactly the same code throws the following error.

Evaluation error: Failed to insert data: MALException:embedded:Could not convert column 2 var_abc to type 5.

Unfortunately, I can't reproduce the above error using mtcars as I don't know what's causing it!

What is "convert column to type 5" issue and how can I fix this?

0

There are 0 best solutions below