I'm trying to scrape US census data using RSelenium in RStudio but I'm having trouble navigating to a URL using Chrome browser.
I already have the Selenium Standalone Server running and the chrome driver setup as environment variable in system settings.
As a minimum working example, I'm able to implement the following commands:
> library(RSelenium)
> remDr <- remoteDriver(browserName = "chrome")
> remDr$open()
Running the last command gives me the following output:
[1] "Connecting to remote server"
$applicationCacheEnabled
[1] FALSE
$rotatable
[1] FALSE
$chrome
$chrome$userDataDir
[1] "C:\\Users\\rchoksi\\AppData\\Local\\Temp\\scoped_dir5908_26290"
$takesHeapSnapshot
[1] TRUE
$databaseEnabled
[1] FALSE
$handlesAlerts
[1] TRUE
$version
[1] "61.0.3163.79"
$platform
[1] "Windows NT"
$browserConnectionEnabled
[1] FALSE
$nativeEvents
[1] TRUE
$acceptSslCerts
[1] TRUE
$locationContextEnabled
[1] TRUE
$webStorageEnabled
[1] TRUE
$browserName
[1] "chrome"
$takesScreenshot
[1] TRUE
$javascriptEnabled
[1] TRUE
$cssSelectorsEnabled
[1] TRUE
$id
[1] "f28c4373e6063f797904f3e4a890d02b"
I believe the 'browserConnectionEnabled' property which is set to FALSE is creating the problem, but I may be wrong.
On running the following command, I get an error which is shown below.
> remDr$navigate("https://factfinder.census.gov/faces/nav/jsf/pages/index.xhtml")
Selenium message:unknown error: Runtime.executionContextCreated has invalid 'context': {"auxData":{"frameId":"5972.1","isDefault":true},"id":1,"name":"","origin":"://"}
(Session info: chrome=61.0.3163.79)
(Driver info: chromedriver=2.9.248315,platform=Windows NT 6.1 SP1 x86_64)
Error: Summary: UnknownError
Detail: An unknown server-side error occurred while processing the command.
Further Details: run errorDetails method
The session info is as shown below:
R version 3.4.0 (2017-04-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] RSelenium_1.7.3
loaded via a namespace (and not attached):
[1] httr_1.3.1 compiler_3.4.0 R6_2.2.2 assertthat_0.2.0
[5] tools_3.4.0 wdman_0.2.2 binman_0.1.0 curl_2.8.1
[9] Rcpp_0.12.12 jsonlite_1.5 caTools_1.17.1 openssl_0.9.7
[13] bitops_1.0-6 semver_0.2.0 XML_3.98-1.9
Any pointers on how to overcome this issue or suggestions for any other tools to implement the same will be appreciated.