Is there a way to create a warc from a dynamic website with chromote and another R library?

56 Views Asked by At

I would like to save a page rendered with headless chrome using chromote to a warc file. Rendering the page works fine, but I am a bit stuck at saving it as a warc file. First I wanted to use jwatr but due to some policies on our laptops this is not possible.

Is there a function in R to which I can pass the rendered page and writes a proper warc file?

library(tidyverse)

library(chromote)

chromote:::find_chrome()

b1 <- ChromoteSession$new()

b1$view()
b1$Page$navigate(url)


#Give the page some time to render
Sys.sleep(20)


content <- b1$DOM$getDocument( )

page <- b1$DOM$getOuterHTML(content$root$nodeId)


# create warc file from page





0

There are 0 best solutions below