unexpected behaviour when extracting from lists with purrr::map

Question

I'm having trouble understanding how purrr::map behaves, for instance, this works:

irisList <- list(iris, iris, iris)
carsList <- list(cars, cars, cars)
airqualityList <- list(airquality, airquality, airquality)

irisList %>% map("Species") %>% str(vec.len = 2)

> List of 3 : 
> $ : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 ...  
> $ : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 ...  
> $ : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 ...

carsList %>% map("speed") %>% str(vec.len = 2)

> List of 3
> $ : num [1:50] 4 4 7 7 8 ...
> $ : num [1:50] 4 4 7 7 8 ...
> $ : num [1:50] 4 4 7 7 8 ...

airqualityList %>% map("Ozone") %>% str(vec.len = 2)

> List of 3
> $ : int [1:153] 41 36 12 18 NA ...
> $ : int [1:153] 41 36 12 18 NA ...
> $ : int [1:153] 41 36 12 18 NA ...

and this also works:

mixedList <- list(iris, cars, airquality)

mixedList %>% map("Species") %>% str(vec.len = 2)
> List of 3
> $ : Factor w/ 3 levels "setosa","versicolor",..: 1 1 1 1 1 ...
> $ : NULL
> $ : NULL

Repeating but mapping for 'speed' or 'Ozone' gives the expected response. However,if I make it a bit more complicated things start to not return as I expected:

unNamedList <- list(list(iris, iris, iris),
                list(cars, cars, cars),
                list(airquality, airquality,airquality))
unNamedList %>%  map("Species")

> [[1]]
> NULL

> [[2]]
> NULL

> [[3]]
> NULL

According the the purrr user manual I should be able to do the following though if I have a named list:

namedIrisList <- list("iris1" = iris,
                      "iris2" = iris,
                      "iris3" = iris)

namedCarsList <- list("cars1" = cars,
                      "cars2" = cars,
                      "cars3" = cars)

namedAirqualityList <- list("aq1" = airquality,
                            "aq2" = airquality,
                            "aq3" = airquality) 

aNamedList <- list("flowers" = namedIrisList,
                   "autos" = namedCarsList,
                   "aq" = namedAirqualityList)

aNamedList %>% map("flowers","Species")

> $flowers
> NULL

> $autos
> NULL

> $aq
> NULL

So purrr::map 'knows' where 'Species' is, but doesn't return anything.

Since we know where 'Species' is located this should work

 aNamedList %>% map("flowers","iris1","Species")

But I get the same response.

> $flowers
> NULL

> $autos
> NULL

> $aq
> NULL

I think I'm not understanding how the extracting ability of map() is supposed to be used.

The context is that I'm dealing with a mixed list of bootstrapping results where the elements I want are in a large list of lists with different names and structures. I've been trying to access the 'list of list' components but it I keep getting 'Null' back from map().

Answer 1

You can use modify_depth to pull out the desired vectors from a list of lists. You indicate the depth, i.e., the level of list you want to work with. In your example, you are working on the second level lists.

From the documentation:

modify_depth(x, 0, fun) is equivalent to x[] <- fun(x)
modify_depth(x, 1, fun) is equivalent to x[] <- map(x, fun)
modify_depth(x, 2, fun) is equivalent to x[] <- map(x, ~ map(., fun))

So this code returns the either the Species column of NULL for every element of the nested lists:

aNamedList %>% modify_depth(2, "Species")

For a single element of a nested list (like iris1), you can use the names as you have been. However, the top level list is what you are looping through and so you won't refer to those name. When you check, for example, names(aNamedList[[1]]) returns "iris1" "iris2" "iris3" and not "flowers".

You can use

aNamedList %>% map("iris1", "Species")

to get the Species column for any nested list named iris1.

Answer 2

First get a hold of your data by viewing it. I use a combo of listviewer and and str.

listviewer::jsonedit(aNamedList) # interactive viewer
str(aNamedList, list.len = 10, max.level = 2) # play with list.len and max.level to adjust visible data

It's nice to look at just one chuck of data sometimes.

str(aNamedList[1]) # subset by 2 or 3 to get autos and aq
str(aNamedList['flowers']) # this also works with 'autos' and 'aq'

Use vectorized indexing to subset first for the 'flowers' list, then go to 'iris1', then 'species'. Like above, you can subset with int of chr strings

a <- map(aNamedList[1], c(1,5)) # all species from iris1
b <- map(aNamedList['flowers'], c('iris1', 'Species')) # identical to the line above
identical(a,b) # returns TRUE

---

What if we want all species in all lists under flowers (iris1, iris2, iris3 ...)

1. @aosmith's solution is excellent:

aNamedList %>% modify_depth(2, "Species")

2. Say you wanted only the flower information (not the null info from the other lists), you could first subset aNamedList to only include the flowers, then jump to level 2, and extract 'species'

aNamedList[1] %>% modify_depth(2, "Species")