Speeding up ggplot2-plotting for animated plots in big dataset

250 Views Asked by At

I built an animated plot using ggplot2 and magick from a large dataset containing tenthousands of entries.

Based on my post How to manage parallel processing with animated ggplot2-plot?, I'm now no longer concerned with speeding up the process for plotting, but the process of saving the plot by using parallel processing through snowfall, since this is most time-consuming part of the code. The problem is, saving the plot requires to go through all plots in the list, which makes the code really slow.

Here's a sample code demonstrating my problem:

library(parallel)
library(snowfall)
library(ggplot2)
library(magick)

# creating some sample data for one year
# 4 categories; each category has a specific value per day
set.seed(1)
x <- data.frame(
  rep(as.Date((Sys.Date()-364):Sys.Date(), origin="1970-01-01"),4),
  c(rep("cat01",length.out=365),
    rep("cat02",length.out=365),
    rep("cat03",length.out=365),
    rep("cat04",length.out=365)),
  sample(0:50,365*4, replace=TRUE)
)
colnames(x) <- c("date", "category", "value")
x$category <- factor(x$category)

# creating a cumulative measure making the graphs appear "growing"
x$cumsum <- NA
for(i in levels(x$category)){
  x$cumsum[x$category == i] <- cumsum(x$value[x$category == i])
}
x <- x[order(x$date),]

# number of cores
cores <- detectCores()

# clustering
sfInit(parallel = TRUE, cpus = cores, type = "SOCK")

# splitting data for plotting
datalist <- split(x, x$date)

# making everything accessible in the cluster
sfExportAll()
sfLibrary(ggplot2)
sfLibrary(magick)

# plotting
out <- sfLapply(datalist, function(data){
  plot <- ggplot(data)+
    geom_bar(aes(category, cumsum), stat = "identity")+
    # holding breaks and limits constant per plot
    scale_y_continuous(expand = c(0,0), 
                       breaks = seq(0,max(x$cumsum)+500,500), 
                       limits = c(0,max(x$cumsum)+500))+
    ggtitle(data$date)
plot
})
# accessing changed data
sfExportAll()

# opening magick-device
img <- image_graph(1000, 700, res = 96)

sfLapply(names(out), function(x){out[[x]]}) 

dev.off()

# animation
animation <- image_animate(img, fps = 5)
animation

# closing cluster
sfRemoveAll()
sfStop()

Any suggestions how to speed up the saving-process?

I'm concerned with the speed of these lines:

img <- image_graph(1000, 700, res = 96) 
sfLapply(names(out), function(x){print(out[[x]])}) 
dev.off()

Thank you!

0

There are 0 best solutions below