R: Values saved into a list within %dopar% / foreach are not available downstream in global environment

393 Views Asked by At

I am trying to run the following code in parallel using dopar / foreach , but I can't figure out how to actually save the values into the list and have them appear in the global environment further down in the script.

I have that first line of code to initialize the seurat.object list. I am importing the list into the foreach. and assigning a new value to each of the list's elements in there too, using <<-, which should mean it will be saved into the global environment. Why is the updated seurat.objects list not preserved outside of the foreach?


1a. Scale only (without nUMI regression):

1b. Scale with nUMI regression and store in a new object:

seurat.objects <- list(scaled=NULL, scaled.regressed=NULL)

registerDoFuture()
cl <- makeCluster(2, outfile="")
plan(cluster, workers = cl)

result <- foreach(object=names(seurat.objects), 
           .export = ls(.GlobalEnv)) %dopar% {


   selectObject(object)

   if( ! file.exists(object.path)) {

         if(object == "scaled") {

             assign('seurat.objects[["scaled"]]', ScaleData(seurat.object, 
                    do.scale = T, do.center = T,  display.progress = F))

          }

         if(object == "scaled.regressed") {

          assign('seurat.objects[["scaled.regressed"]]',
          ScaleData(seurat.object,
          vars.to.regress = "nUMI", 
          do.scale = T, do.center = T, display.progress = F))

          }

      saveRDS(seurat.objects[[object]], file=object.path)

      } else { # Found scaled .Rds

               x <- readRDS(object.path)

              seurat.objects[[object]] <<- x

              rm(x)

        }

    }

stopCluster(cl)

The selectObject function is defined before the above code, as follows:

selectObject <- function(object) {

      if(object == "scaled") {
                             scaling <<- "_scaleOnly"
                             pca.result <<- "pca.scaled"
                             object.path <<- path.scaled.object
                             pca.result.path <<- paste0(clustering.path, "2_pca/pcaObject_", 
                                            age, scaling, ".Rds")
                            } 

      if(object == "scaled.regressed") {

                                scaling <<- "_scale_nUMIregress"
                                pca.result <<- "pca.scaled.regressed"
                                object.path <<- path.scaled.regressed.object
                                pca.result.path <<- paste0(clustering.path, "2_pca/pcaObject_", 
                                            age, scaling, ".Rds")
                            }
}

When I try to inspect the contents of seurat.objects, the list in which the data should have been stored, I get:

> seurat.objects

$scaled
NULL

$scaled.regressed
NULL
0

There are 0 best solutions below