Function arguments not seen in R future framework under RStudio's Background Jobs

Question

I want to use the package future and its sibling future.apply in order to execute a series of operations in parallel. I was able to successfully run everything interactively but given the size of the data, I want to user the Background Jobs tab in RStudio. Running the code below in a normal (interactive) RStudio session does not raise any error and I am happy with it.

library(future)

ncores = 10

# define the number of workers
plan(future::multisession(workers = ncores))

print("Done")
#> [1] "Done"

^{Created on 2023-01-19 by the reprex package (v2.0.1)}

Apparently, when launching the job from the Background Jobs tab in RStudio, future::multisession() fails to find the parameter ncores as specified in the environment. I thought the background job manager integrated in RStudio uses Rscript in the background and that this one looks at the same environment where the script sits but maybe I am wrong.

So, if I use the "Background Jobs" tab in RStudio and try to execute the same code, I get the following error:

Error in tweak.future(function (..., workers = availableCores(), lazy = FALSE,  : 
  object 'ncores' not found
Calls: sourceWithProgress ... eval -> plan -> do.call -> <Anonymous> -> tweak.future
Execution halted

I still think that the issue is related to what environment multisession() looks at. After a few attempts, it seems that environment() is the one I am looking for. As you can see, if I print the objects defined, ncores is there but...same error.

library(future)

ncores = 10

print(ls(environment()))

# define the number of workers
plan(future::multisession(workers = ncores, envir = environment()))

print("Done")

Any idea on how to solve this issue?

Below is the output of sessionInfo().

sessionInfo()
#> R version 4.1.2 (2021-11-01)
#> Platform: x86_64-apple-darwin17.0 (64-bit)
#> Running under: macOS Big Sur 10.16
#> 
#> Matrix products: default
#> BLAS:   /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.0.dylib
#> LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
#> 
#> locale:
#> [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> loaded via a namespace (and not attached):
#>  [1] digest_0.6.29   withr_2.5.0     magrittr_2.0.3  reprex_2.0.1   
#>  [5] evaluate_0.15   highr_0.9       stringi_1.7.6   rlang_1.0.6    
#>  [9] cli_3.6.0       rstudioapi_0.13 fs_1.5.2        rmarkdown_2.13 
#> [13] tools_4.1.2     stringr_1.4.0   glue_1.6.2      xfun_0.31      
#> [17] yaml_2.3.5      fastmap_1.1.0   compiler_4.1.2  htmltools_0.5.2
#> [21] knitr_1.39

^{Created on 2023-01-27 by the reprex package (v2.0.1)}

Answer 1

(Author of future here)

Apparently, when using Rscript, future::multisession() fails to find the parameter ncores as specified in the environment. ...

I cannot reproduce this with R 4.2.2 on Linux. If I copy your code into a test.R file and call, I get:

$ Rscript test.R
[1] "Done"

FWIW, the recommended way to set the plan is:

plan(multisession, workers = ncores)

but I'm not sure if that makes a difference.

Maybe you're using an old version of future? What's your sessionInfo()?