Using parallel package functions inside own R package

606 Views Asked by At

I created own R package with functions that use parallel functions like makeCluster, parLapply etc. However, they are much slower inside the package as used outside. There are slower initialization of cluster, and exporting objects... Do you have tips how to use properly parallel functions inside an own R package?

Example of using parallel:

cl <- parallel::makeCluster(parallel::detectCores()-1)
parallel::clusterExport(cl, varlist = c("data"), envir = environment())
parallel::clusterCall(cl, function() {library(myPackage)})
data_res <- parallel::parLapply(cl, 1:nrow(data), function(i) {
    tryCatch(myFun(data[i,]), error = function(err) {return(data.table(row = i))})
})
if(!is.null(cl)) {
  parallel::stopCluster(cl)
  cl <- c()
}
gc()

Thanks

0

There are 0 best solutions below