What is the idiomatic way to run a Specification over multitple implementations with Spock?

Question

Let's say I have a Joiner interface:

interface Joiner {
    String join(List<String> l);
}

And two implementations of it:

class Java8Joiner implements Joiner {
    @Override String join(List<String> l) { String.join("", l); }
}

class GroovyJoiner implements Joiner {
    @Override String join(List<String> l) { l.join('') }
}

What is the best way to check that theses two implementation, or more, pass the same tests?

Basically, I want to run all the tests defined by JoinerSpecification with a Java8Joiner, then a GroovyJoiner, and so on...

class JoinerSpecification extends Specification {

    @Subject
    final def joiner = // Joiner instance here

    def "Joining #list -> #expectedString"(List<String> list, String expectedString) {

        expect:
        joiner.join(list) == expectedString

        where:
        list                | expectedString
        []                  | ''
        ['a']               | 'a'
        ['a', 'b']          | 'ab'
    }
}

It is perfectly fine to refactor JoinerSpecification if needed.

My primary goals are:

• Maintainable test code (with as little boilerplate as possible)
• Clear error messages (which test of which implementation is failing)
• Easy to run from the IDE

---

Edit 1 (15/06/2015)

Reworded my question & added some details to make it clearer.

Answer 1

What would be very useful here is comparison chaining but, as far as I know it's not implemented in groovy, see here. Since Subject purpose according to the docs is pure informational I've come up with the following ideas:

@Grab('org.spockframework:spock-core:0.7-groovy-2.0')
@Grab('cglib:cglib-nodep:3.1')

import spock.lang.*

class Test extends Specification {
    def 'attempt 1'() {
        given:    
        def joiners = [new GroovyJoiner(), new Java8Joiner()]

        expect:
        joiners.every { it.join(list) == expectedString }

        where:
        list                | expectedString
        []                  | ''
        ['a']               | 'a'
        ['a', 'b']          | 'ab'    
    }

    def 'attempt 2'() {
        given:    
        def gJoiner = new GroovyJoiner()
        def jJoiner = new Java8Joiner()

        expect:
        [gJoiner.join(list), jJoiner.join(list), expectedString].toSet().size() == 1

        where:
        list                | expectedString
        []                  | ''
        ['a']               | 'a'
        ['a', 'b']          | 'ab'    
    }
}

interface Joiner {
    String join(List<String> l);
}

class Java8Joiner implements Joiner {
    @Override String join(List<String> l) { String.join("", l); }
}

class GroovyJoiner implements Joiner {
    @Override String join(List<String> l) { l.join('') }
}

In the attempt 1 I'm just checking if every element on the list is equal to expectedString. It looks nice however on error it's not verbose enough. Any element may fail and there's no trace which one.

The attempt 2 method makes use of Set. Since all the results should be equal there'll be only one element left in the collection. In prints verbose info on test failure.

What also comes to my head is guava's ComparisonChain but no idea if it would be useful as well as it's another project dependency.

Answer 2

Unfortunately, there is no way to create a cartesian product of two lists declaratively in Spock. You have to define your own Iterable that will provide the values for your variables.

There is a more readable (using Spock tabular data definition) way to define the data, if you are willing to sacrifice terseness for readability. Let me know if that interests you. Otherwise here is a solution that lets you define everything declaratively, without duplication, but you lose the nice tabular definitions of Spock.

If your implementation is stateless, i.e. it does not keep state, you can get away with using @Shared instances of Joiner:

import spock.lang.Shared
import spock.lang.Specification
import spock.lang.Unroll


interface Joiner {
    String join(List<String> l);
}

class Java8Joiner implements Joiner {
    @Override
    String join(List<String> l) { String.join("", l);  }
}

class GroovyJoiner implements Joiner {
    @Override
    String join(List<String> l) { l.join('') }
}


class S1Spec extends Specification {
    @Shared
    Joiner java8Joiner = new Java8Joiner()
    @Shared
    Joiner groovyJoiner = new GroovyJoiner()

    List<Map> transpose(Map<List> mapOfLists) {
        def listOfMaps = [].withDefault { [:] }

        mapOfLists.each { k, values ->
            [values].flatten().eachWithIndex { value, index ->
                listOfMaps[index][k] = value
            }
        }
        listOfMaps
    }

    Iterable distribute(List<Joiner> joiners, Map<String, List> data) {
        def varsForEachIteration = transpose(data)

        new Iterable() {
            @Override
            Iterator iterator() {
                [joiners, varsForEachIteration].combinations().iterator()
            }
        }
    }

    @Unroll
    def "Joining with #joiner  #list -> #expectedString"() {
        expect:
        joiner.join(list) == expectedString

        where:
        [joiner, data] << distribute([java8Joiner, groovyJoiner], [
                list          : [[], ['a'], ['a', 'b']],
                expectedString: ['', 'a', 'ab']
        ])

        and:
        list = data.list
        expectedString = data.expectedString
    }

}

Answer 3

Opal's proposal is interesting but is impractical. Not being able to use then blocks is a showstopper.

Lets try a self-answer using another approach. Since the goal is to execute the same specification on several implementations of the same interface, why not try to use good old inheritance ?

abstract class AbstractJoinerSpec extends Specification {

    abstract Joiner getSubject()

    @Unroll
    def "Joining #list -> #expectedString"(List<String> list, String expectedString) {
        given:
        def joiner = getSubject()

        expect:
        joiner.join(list) == expectedString

        where:
        list                | expectedString
        []                  | ''
        ['a']               | 'a'
        ['a', 'b']          | 'ab'
    }
}


class Java8JoinerSpec extends AbstractJoinerSpec {
    @Override
    Joiner getSubject() { new Java8Joiner() }
}

class GroovyJoinerSpec extends AbstractJoinerSpec {
    @Override
    Joiner getSubject() { new Java8Joiner() }
}

Basically, the only abstract method is getSubject and all the inherited test cases are executed by Spock (I have not been able to find this feature in the documentation but it does work).

Pros

• It only requires creating a tiny class for each implementation and the specification code remains clean.
• The full power of Spock can be used
• Error messages and reports are clear (you know which test failed for which implementation)

Cons

• Running a given test for a given implementation from the IDE is not easy. You have to copy/paste the test case into the non abstract specification or remove the abstract modifiers and implement the getSubject method.

I thing the trade-off acceptable since you don't manually run/debug an implementation that often. Keeping the code clean, and having meaningful report worth this inconvenience.