I am trying to connect our local machine to Spark with livy method.
We tried with a couple of users, but we get this error:
library(sparklyr)
sc <- spark_connect(master = "link/to/spark/via/livy",
version = "2.4",
method = "livy",
config =
livy_config(
username = 'user',
password = "pwd"))
sc
We tried using CURL and the exit status was positive, so I do not get why this is not working, giving 404.
Error in livy_validate_master(master, config) : Failed to connect to Livy service at link/to/spark/via/livy. Failed to retrieve livy session (Client error: (404) Not Found):
Note: there is the tag RStudio because we used the procedure of RStudio -whom put out the code above.
Thanks in advance.