Unitils vs Spring TestContext Framework

Question

I'm thinking about which testing framework to use for my spring+hibernate web application. Initially I spotted and got pretty interested in Unitils, but then I saw the standard Spring TestContext Framework and started having some doubts. I'm pretty new to the subject, but so far my understanding is the following:

Unitils pros:

• has a mix of various other testing frameworks i.e. EasyMock or DbUnit
• builds on top of the the frameworks, providing some additional value i.e. hibernate mapping tests or reflection testing utilities

Spring TestContext Framework pros:

• allows to use the standard Spring annotations when testing
• probably has a bigger user base and better support (this is my assumption, since Unitils seems to be basically created by two people)
• also has some of the Unitils features built-in (such as reflection testing utilities)
• if I need any of the Unitils 'bigger' features such as DbUnit or EasyMock I can just get regular DbUnit or EasyMock straight from the source - this allows me to always have the most current version of DbUnit/EasyMock without waiting for Unitils to release their new version with the current DbUnit/EasyMock

So for now I'm leaning more towards using Spring TestContext Framework. Does anyone have a bit more experience with these frameworks and could help me tilt in the right direction? What does Unitils actually offer on its own, except for integration of all these frameworks?

[EDIT] Just found out about a few more things provided by Unitils:

• some improvements to DbUnit like the @DataSet annotation (in Spring TestContext Framework it appears you have to write a bit more code on your own if you want to use it with DbUnit as described here: http://blog.zenika.com/index.php?post/2010/02/04/Testing-SQL-queries-with-Spring-and-DbUnit%2C-part-1)
• ability to disable database constraints in your tests

Here's a nice presentation on Unitils which convinced me to start using Unitils: http://www.javapolis.com/confluence/download/attachments/32918/C_14_09_04.pdf

I'm leaving the question open for the time being, maybe someone will come up with some other thoughts...

Answer 1

Unitils is a very good framework as you have noted.

Pros:

1. Unitils now uses Spring jars.
2. Unitils inject module works great in my view, though you can achieve something like that in spring too.

Cons: As you have said, I believe it lacks proper support in the Unitils forum. I can see many threads not answered for a long time. However stackoverflow has good support for it :-) except if you need additional features or bug fixes.

Personally, I find Unitils good for my testing.

Answer 2

We use Unitils and Spring's test framework on our project. With unitils one thing we ran into problems with was having more than one PersistenceUnit. Unitils does not support it. Of course spring does.

Another problem we ran into I posted about here: Using unitils/dbMaintain to maintain database, how to exclude scripts from being run?

Basically, unitils does not have all the latest features of each of the underlying frameworks such as DBMaintain because unitils uses "old" versions of the other frameworks under the covers.

That said, Unitils does have some great convenience methods like some of the ones provided in DatabaseUnitils. We particularly like updateSequences()

So for ultimate flexibility and/or if you anticipate a fairly complex code structure I would stick with Spring's test framework whenever possible.

In many other cases, unitils is a very handy tool.

And a little p.s... when possible I would recommend favoring unit tests and mocking frameworks like Mockito versus integration tests. I've always been an "integration test" heavy tester and on this last project it has turned into a bit of a maintenance challenge.