Data Harvesting in R: Get nested lists, unlist, make edits, re-nest them back

153 Views Asked by At

the following code harvests data from a website. I retrieve a list of lists, I want to unlist one of the lists, edit it, then re-nest it back into the data into the form the data was received. Here is my code below, it fails one the re-nesting.

library(jsonlite)
library(plyr)
library(ckanr)
library(purrr)
library(dplyr)

ckanr_setup(url = "https://energydata.info/")
package_search(q = 'organization:world-bank-grou')$count
json_data2 <- fromJSON("https://energydata.info/api/3/action/package_search?q=organization:world-bank-grou", flatten = TRUE)
dat2 <- json_data2$result
str(dat2)


###########
#Get the datasets and unlist metadata
###########

df <- as.data.frame(json_data2$result$results)
Tags <- select(df, id, topic)

#Make some edits
Tags$topic <- tolower(Tags$topic)

res <- rbind.fill(lapply(Tags,function(y){as.data.frame(t(y),stringsAsFactors=FALSE)}))
res$V1 = paste0("Some edit:",res$V1)
res$V2 = paste0("Some edits:", res$V2)
res$V3 = paste0("Some edit:", res$V3)
res[res=="Some edit:NA"]<-NA
res$V1 <- gsub(" ", "_", res$V1) 
res$V2 <- gsub(" ", "_", res$V2)
res$V3 <- sub(" ", "_", res$V3)
res

###########
#Re-nest
###########


#turning res df back into list of lists
nestedList <- flatten(by_row(res, ..f = function(x) flatten_chr(x), .labels = FALSE)) #FAILS HERE

ERROR: Error in flatten(by_row(res, ..f = function(x) flatten_chr(x), .labels = FALSE)) : could not find function "by_row"

1

There are 1 best solutions below

0
On

Unclear from the question wording exactly what kind of list of lists you want to end up with, but maybe this is what you're looking for?

res %>%
  rowwise() %>% 
  as.list()

or

res %>%
  t() %>%
  as.data.frame() %>%
  rowwise() %>% 
  as.list()