Integration testing an HTTP server with ZIO test suite

Question

I am trying to figure out the idiom for writing an integration test for an Http4s app that supports two end points. I'm starting the Main app class in a ZManaged by forking it on a new fiber and then doing interruptFork on release of the ZManaged. I then convert this to a ZLayer and pass it via provideCustomLayerShared() on the whole suite that has multiple testMs .

1. Am I on the right track here?
2. It doesn't behave as I expect it to :

• Although the httpserver managed in the way mentioned is provided to the suite that encloses both the tests, it is released after the first test and thus the second test fails
• The test suite never finishes and just hangs after executing both tests

Apologies for the half baked nature of the code below.

object MainTest extends DefaultRunnableSpec {

  def httpServer =
    ZManaged
      .make(Main.run(List()).fork)(fiber => {
        //fiber.join or Fiber.interrupt will not work here, hangs the test
        fiber.interruptFork.map(
          ex => println(s"stopped with exitCode: $ex")
        )
      })
      .toLayer

  val clockDuration = 1.second

  //did the httpserver start listening on 8080?
  private def isLocalPortInUse(port: Int): ZIO[Clock, Throwable, Unit] = {
    IO.effect(new Socket("0.0.0.0", port).close()).retry(Schedule.exponential(clockDuration) && Schedule.recurs(10))
  }

  override def spec: ZSpec[Environment, Failure] =
    suite("MainTest")(
      testM("Health check") {
        for {
          _ <- TestClock.adjust(clockDuration).fork
          _ <- isLocalPortInUse(8080)
          client <- Task(JavaNetClientBuilder[Task](blocker).create)
          response <- client.expect[HealthReplyDTO]("http://localhost:8080/health")
          expected = HealthReplyDTO("OK")
        } yield assert(response) {
          equalTo(expected)
        }
      },
      testM("Distances endpoint check") {
        for {
          _ <- TestClock.adjust(clockDuration).fork
          _ <- isLocalPortInUse(8080)
          client <- Task(JavaNetClientBuilder[Task](blocker).create)
          response <- client.expect[DistanceReplyDTO](
            Request[Task](method = Method.GET, uri = uri"http://localhost:8080/distances")
              .withEntity(DistanceRequestDTO(List("JFK", "LHR")))
          )
          expected = DistanceReplyDTO(5000)
        } yield assert(response) {
          equalTo(expected)
        }
      }
    ).provideCustomLayerShared(httpServer)
}

Output of the test is that the second test fails while the first succeeds. And I debugged enough to see that the HTTPServer is already brought down before the second test.

stopped with exitCode: ()
- MainTest
  + Health check
  - Distances endpoint check
    Fiber failed.
    A checked error was not handled.
    org.http4s.client.UnexpectedStatus: unexpected HTTP status: 404 Not Found

And whether I run the tests from Intellij on sbt testOnly, the test process keeps hung after all this and I have to manually terminate it.

Answer 1

In the end @felher 's Main.run(List()).forkManaged helped solve the first problem.

The second problem about the GET with a body being rejected from inside the integration test was worked-around by changing the method to POST. I did not look further into why the GET was being rejected from inside the test but not when done with a normal curl to the running app.

Answer 2

I think there are two things here:

ZManaged and acquire

The first parameter of ZManaged.make is the acquire function which creates the resource. The problem is that resource acquisition (as well as releasing them) is done uninterruptible. And whenever you do a .fork, the forked fiber inherits its interruptibility from its parent fiber. So the Main.run() part can actually never be interrupted.

Why does it seem to work when you do fiber.interruptFork? interruptFork doesn't actually wait for the fiber to be interrupted. Only interrupt will do that, which is why it will hang the test.

Luckily there is a method which will do exactly what you want: Main.run(List()).forkManaged. This will generate a ZManaged which will start the main function and interrupt it when the resource is released.

Here is some code which demonstrates the problem nicely:

import zio._
import zio.console._
import zio.duration._

object Main extends App {

  override def run(args: List[String]): URIO[ZEnv, ExitCode] = for {
    // interrupting after normal fork
    fiberNormal <- liveASecond("normal").fork
    _           <- fiberNormal.interrupt

    // forking in acquire, interrupting in relase
    _ <- ZManaged.make(liveASecond("acquire").fork)(fiber => fiber.interrupt).use(_ => ZIO.unit)

    // fork into a zmanaged
    _ <- liveASecond("forkManaged").forkManaged.use(_ => ZIO.unit)

    _ <- ZIO.sleep(5.seconds)
  } yield ExitCode.success

  def liveASecond(name: String) = (for {
    _ <- putStrLn(s"born: $name")
    _ <- ZIO.sleep(1.seconds)
    _ <- putStrLn(s"lived one second: $name")
    _ <- putStrLn(s"died: $name")
  } yield ()).onInterrupt(putStrLn(s"interrupted: $name"))

}

This will give the output:

born: normal
interrupted: normal

born: acquire
lived one second: acquire
died: acquire

born: forkManaged
interrupted: forkManaged

As you can see, both normal as well as forkManaged get interrupted immediately. But the one forked within acquire runs to completion.

The second test

The second test seems to fail not because the server is down, but because the server seems to be missing the "distances" route on the http4s side. I noticed that you get a 404, which is a HTTP Status Code. If the server were down, you would probably get something like Connection Refused. When you get a 404, some HTTP server is actually answering.

So here my guess would be that the route really is missing. Maybe check for typos in the route definition or maybe the route is just not composed into the main route.