How long should a Monte Carlo bootstrap power analysis simulation in R take? Is it potentially hours? (1000 reps, 1000 bootstraps)

Question

I am using a Monte Carlo simulation to run a power analysis for a longitudinal mediation model. I'm using the power.boot function from the bmem package (lavaan).

I checked the code with only 5 reps/5 bootstrap to make sure it worked and it did.

Then I ran the code with 1000 reps, 1000 bootstrap as the package documentation recommends.

It's been over an hour now and it's still running - is this normal? How long is too long?

powermodel1 <-'

x2 ~ start(.6)*x1 + x*x1 
x3 ~ start(.6)*x2 + x*x2

m2 ~ start(.15)*x1 + a*x1 + start(.3)*m1 + m*m1 
m3 ~ start(.15)*x2 + a*x2 + start(.3)*m2 + m*m2

y2 ~ start(.5)*m1 + b*m2 + start(.3)*y1 + y*y1
y3 ~ start(.5)*m2 + b*m2 + start(.3)*y2 + y*y2 + start(0.05)*x1 + c*x1

x1 ~~ start(.15)*m1

x1 ~~ start(.15)*y1


y1 ~~ start(.5)*m1

'
indirect <- 'ab:=a*b'

N<-200

system.time(bootstrap<-power.boot(powermodel1, indirect, N, nrep=1000, nboot=1000, parallel = 'multicore'))

            summary(bootstrap)

Answer 1

Unfortunately it looks like it will take a while; ~8hrs on my system:

library(bmem)

powermodel1 <-'

x2 ~ start(.6)*x1 + x*x1 
x3 ~ start(.6)*x2 + x*x2

m2 ~ start(.15)*x1 + a*x1 + start(.3)*m1 + m*m1 
m3 ~ start(.15)*x2 + a*x2 + start(.3)*m2 + m*m2

y2 ~ start(.5)*m1 + b*m2 + start(.3)*y1 + y*y1
y3 ~ start(.5)*m2 + b*m2 + start(.3)*y2 + y*y2 + start(0.05)*x1 + c*x1

x1 ~~ start(.15)*m1

x1 ~~ start(.15)*y1


y1 ~~ start(.5)*m1

'
indirect <- 'ab:=a*b'

N<-200

system.time(bootstrap<-bmem::power.boot(powermodel1, indirect, N, nrep = 10, nboot = 10, parallel = 'multicore'))
system.time(bootstrap<-bmem::power.boot(powermodel1, indirect, N, nrep = 30, nboot = 30, parallel = 'multicore'))
system.time(bootstrap<-bmem::power.boot(powermodel1, indirect, N, nrep = 60, nboot = 60, parallel = 'multicore'))
system.time(bootstrap<-bmem::power.boot(powermodel1, indirect, N, nrep = 100, nboot = 100, parallel = 'multicore'))

library(tidyverse)
# Load the times from above into a dataframe
benchmark <- tibble(bootstraps = c(10, 30, 60, 100),
                    times = c(4.021, 30.122, 121.103, 311.236)) 

# Plot the points and fit a curve
ggplot(benchmark, aes(x = bootstraps, y = times)) +
  geom_point() +
  geom_smooth(se = FALSE, span = 5)

# Fit a model
fit <- lm(data = benchmark, times~poly(bootstraps,
                                       2, raw=TRUE))
newtimes <- data.frame(bootstraps = seq(100, 1000, length = 4))

# Predict the time it will take for larger bootstrap/rep values
predict(fit, newdata = newtimes)
>        1          2          3          4 
>  311.6829  4568.3812 13789.6754 27975.5655 

# Convert from seconds to hours
print(27975.5655/60/60)
>[1] 7.77099