I'm running a Spark job that does some Http calls along the line. I decided to use a library Sttp
which requires an implicit
Http backend, in my case HttpURLConnectionBackend()
: an object apply which creates a new class instance, which is not serializable.
trait ApiUtils extends MediaTypes {
implicit val backend: SttpBackend[Identity, Nothing, NothingT] = HttpURLConnectionBackend()
def post = { ...
As a consequence, because my Spark job extends this trait
, all methods from this trait
inside the Spark job are not available on the executors.
As a fairly brainless workaround, I made the val
a def
, but what's the recommended way for sorting these things out?
The Spark job itself is a class
, not an object
. Reason for this is to mock/stub the mentioned Http backend inside the tests.