Is there a way to combine two Scala TableDrivenPropertyChecks Tables into one?

Question

I want to combine two tables: one containing inputs and the other containing expected outputs into a single table that I can run through ScalaTest's TableDrivenProperyChecks framework. Below is essentially what I'm going for:

import org.scalatest.prop.TableDrivenPropertyChecks._

val inputs = Table(
    ("a", "b"),  // First tuple defines column names
    (  1,   2),  // Subsequent tuples define the data
    ( -1,   2),
)
    
val expectedOutputs = Table(
    ("addition", "subtraction"),  // First tuple defines column names
    (3,   -1),  // Subsequent tuples define the data
    (1,   -3),
)
    
forAll(inputs zip expectedOutputs) { (a, b, add, sub) =>
  sumFn(a, b) should equal (add)
  diffFn(a, b) should equal (sub)
}

I know that conventionally tests define the expected values in the same table as the inputs, but I find that to be confusing especially when tuples get long. This feels more elegant if possible.

I see that the org.scalatest.prop.TableFor2 extends scala.IndexedSeq, but I can't get it to play nicely with forAll. Basically the inputs zip expectedOutputs thing doesn't work.

Answer 1

Using tuples with many elements could be hard to read. It becomes harder in a collection. Table-driven property checks helps you to avoid duplicated code when you need to write the same tests with different values. You could have the expected output in the table, but I don't think it will be easier to read or maintain.

The article Knoldus - Table Driven Testing in Scala have a good example where instead of having the expected output in the table, it write tests that return the same value with different inputs:

Order.scala (model)

case class Order(quantity: Int, price: Int) {
  def isEmptyOrder: Boolean = quantity == 0 && price == 0
}

OrderValidation.scala (logic)

object OrderValidation {

  def validateOrder(order: Order): Boolean =
    order.isEmptyOrder || (validatePrice(order.price) && validateQuantity(order.quantity))

  private def validatePrice(p: Int): Boolean = p > 0
  private def validateQuantity(q: Int): Boolean = q > 0

}

OrderValidationTableDrivenSpec.scala (test)

import org.scalatest.FreeSpec
import org.scalatest._
import org.scalatest.prop.TableDrivenPropertyChecks

class OrderValidationTableDrivenSpec extends FreeSpec with TableDrivenPropertyChecks with Matchers {

  "Order Validation" - {
    "should validate and return false if" - {
      val orders = Table(
        ("statement"                        , "order"),
        ("price is negative"                , Order(quantity = 10, price = -2)),
        ("quantity is negative"             , Order(quantity = -10, price = 2)),
        ("price and quantity are negative"  , Order(quantity = -10, price = -2))
      )

      forAll(orders) {(statement, invalidOrder) =>
        s"$statement" in {
          OrderValidation.validateOrder(invalidOrder) shouldBe false
        }
      }
    }
  }
}

In this case, the table just have the name of the case that is being validated and then the input value, which is the Order in this case.

You can see the same approaach in the gist davidallsopp - PropertyTests.scala:

class Example extends PropSpec with PropertyChecks with ShouldMatchers {

  val examples =
    Table(
      "set",
      BitSet.empty,
      HashSet.empty[Int],
      TreeSet.empty[Int])

  // table-driven
  property("an empty Set should have size 0") {
    //forAll(examples) { set => set.size should be(0) }
    forAll(examples) { _.size should be(0) }
  }

  // table-driven, expecting exceptions
  property("invoking head on an empty set should produce NoSuchElementException") {
    forAll(examples) { set =>
      a[NoSuchElementException] should be thrownBy { set.head }
    }
  }

  // table-driven, expecting exceptions, with an alternative syntax
  property("again, invoking head on an empty set should produce NoSuchElementException") {
    forAll(examples) { set =>
      evaluating { set.head } should produce[NoSuchElementException]
    }
  }

  // A 2-column table
  val colours = Table(
    ("Name", "Colour"), // First tuple is the title row
    ("r", Color.RED),
    ("g", Color.GREEN),
    ("b", Color.BLUE))

  property("colours") {
    forAll(colours) { (name: String, colour: Color) =>
      colour.toString should include(s"$name=255") // e.g. java.awt.Color[r=0,g=255,b=0]
    }
  }

  // A bit more concise with a case statement (don't need explicit types)
  property("colours2") {
    forAll(colours) { case (name, colour) => colour.toString should include(s"$name=255") } // e.g. java.awt.Color[r=0,g=255,b=0]
  }

}

Each test case have different inputs but all of them expect the same output.

---

From the example provided in your question

val inputs = Table(
    ("a", "b"),
    (  1,   2),
    ( -1,   2),
)

forAll(inputs) { (a, b) =>
  sumFn(a, b) should equal (a + b)
  diffFn(a, b) should equal (a - b)
}

It doesn't make sense to use Table-driven property checks (I think it was just a dummy example to avoid complexity). If your real case is similar to something like that, it could be better to validate the logic of your code using Generator-driven property checks. This type of tests is useful when you need to validate some properties, such as commutative property.

// Generator-driven property
forAll { (a: Int, b: Int) =>
  (a + b) should be(b + a)
}

forAll { (a: Int, b: Int) =>
  whenever(a != b && (a != 0 || b != 0)) {
    (a - b) should not be(b - a)
  }
}

---

Table-driven and Generator-driven property checks comes from Property-based testing.

Answer 2

Thanks for the detailed answer above! The clarification on the semantics of Table-driven property checks is useful. I figured out how to achieve what I was after though. I need to create a new table by using the expanded output of the zip with map as the contents of that table, like so:

val inputsWithExpectedValues = Table(
  // header of new table
  ("a", "b", "addition", "subtraction"),
  // new rows of table
  inputs
    .zip(expectedOutputs)
    .map({
      // flatten out component columns
      case ((a, b), (addition, subtraction)) =>
        (a, b, addition, subtraction)
    }): _*
)