read into R data frame from cognos 8 file (utf 16) with readr

1.2k Views Asked by At

I am attempting to read csv files generated by cognos 8 into r using readr.

Example file attached for reproducibility: Example csv file

The following python code works:

df = pd.read_table('csv_test.csv', encoding = 'utf-16')

I've tried the following in R, and none of them return the correct result. They either error (Incomplete multibyte sequence) or read in improperly (as nested lists or similar)

csv_data <- read_table('csv_test.csv')
csv_data <- read_table('csv_test.csv', locale = locale(encoding = 'UTF-16LE'))
csv_data <- read_tsv('csv_test.csv')
csv_data <- read_tsv('csv_test.csv', locale = locale(encoding = 'UTF-16LE'))

I used guess_encoding() to get the UTF-16LE, I also tried UTF-16.

1

There are 1 best solutions below

0
d.ellis On

As mentioned in comment by Gregor, there is an open issue with the readr package for this.

As a workaround, base package read_delim will work:

csv_data <- read.delim('csv_test.csv', stringsAsFactors = FALSE, fileEncoding = 'UTF-16LE')