Multiple assertions in a Ruby Test::Unit test class skipped on first failure

Question

I've looked at several related questions and their answers, but none seem to quite address my issue.

My unit tests are typically data-driven by an external dataset, for ease of extension and maintenance. I test my primitives with all sorts of inputs. As an example, I have a #truthify method that evaluates its argument according to specific rules (not Ruby's usual ones). The set of test inputs keeps expanding.

So I have two test classes, one for testing that 'true' arguments are evaluated correctly, and one for 'false' arguments. Each class iterates through its set of inputs and makes an assertion in the loop. The first one that fails, however, aborts that test class' processing.

That is one of my more simple examples.

The last time I used Test::Unit (years ago), ISTR there was a way to have each failed assertion reported without aborting the test class.

Writing a separate test class for each test value, even if I simplify things with subclassing, is far too unwieldy and unmaintainable, and I feel it also violates KISS and DRY.

If there's no way to tell Test::Unit to continue after a failed assertion, is there a way to dynamically alter the list of test classes in the test case? That way I could at least have the case initialisation dynamically add all those individual test classes based on the values in the external dataset.

Answer 1

a way to have each failed assertion reported without aborting the test class

Each assertion would have to be in its own test method:

class MyTest < Test::Unit::TestCase
  10.times do |i|
    define_method "test_#{i}" do
      assert false, "Assertion was false."
    end
  end
end

10 tests, 10 assertions, 10 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
0% passed

---

a way to dynamically alter the list of test classes in the test case

You can selectively require files as needed:

# test/run.rb

require "test/unit"

# require whatever tests you want to run
require_relative "./my_test"

Test::Unit::AutoRunner.run

or regex test classes from command line:

$ ruby test/run.rb --testcase=/myte/i