java continuous testing outside of IDE

Question

I have a bunch of Java unit tests, and I'd like to integrate a continuous testing framework into my codebase. Ideally, I would like to write a Maven / Ant target or bash script which would start running tests whenever the files it's watching change. I've looked at a couple of options so far (Infinitest, JUnit Max) but both of them appear to want to run as IDE plugins.

My motivation for using a CLI-only tool is that my coworkers use a broad set of text editors and IDEs, but I want to ensure that anyone can run the tests constantly.

EDIT: I did not consider Jenkins or other more typical CI solutions for several reasons:

• We already have a CI build tool for running unit and integration tests after every push.
• They hide the runtime of the tests (because they run asynchronously), allowing tests to become slower and slower without people really noticing.
• They usually only run tests if your repository is in some central location. I want unit tests to be running while I'm editing, not after I've already pushed the code somewhere. The sooner I run the tests, the sooner I can fix whatever mistake I made while editing. Our JavaScript team has loved a similar tool, quoting speedup of 3x for iterating on unit test development.

Answer 1

Why not use a CI tool like Jenkins to run your tests on every code change, as part of your overall CI build? It's easy to get Jenkins to poll your source control system and run a build or separate test job when a file changes.

Answer 2

Usual way is to use a Continuous Integration server such as Jenkins.

Have Jenkins poll your version control system every 15 minutes and it will build your project as it notices commits. You will never be far away from knowing that your source code works.

Answer 3

I would recommend to use a CI tool such as Jenkins, where you have the possibility not only to install on premise, but also to get a Jenkins cloud instance through a PaaS where you can easily test if this solution could meet or not your goal without spending too much time in the set-up process.

This PaaS provides some ClickStarts that you can use as a template for your own projects, on premise or on the cloud. It will generates you a Jenkins job fully set-up and working.

Some articles that you can take a look at are:

• Painless Maven Builds with Jenkins where you can see the dashboard you can get. You will see the maven tests which are passed per build and you can also get a graph showing the maven tests passed, skipped and failed.
• iOS dev: How to setup quality metrics on your Jenkins job? Although this article talks specifically about iOS, you can also get the same goal for a standard java maven project: Test coverage, Test results, Code duplication, Code metrics (LOC), ...

Answer 4

Yeah having a build server (like Bamboo, Cruise Control er TeamCity) and a build tool like Maven (along with surefire-plugin for TestNg/Junit and Failsafe-plugin for integration testing maybe using something like Selenium 2) is quite popular, because it's relatively trivial to setup (works almost out-of-the-box). :)

Answer 5

I am using a continuous polling for directory change solution for that. ( general code: http://www.qualityontime.eu/articles/directory-watcher/groovy-poll-watcher/ (in Hungarian, but source code is English) )

A customized solution for compiling nanoc based site. Review and customize to your need. (Groovy)

def job = {
  String  command = /java -jar jruby-nanoc2.jar -S nanoc compile/
  println "Executing "+command
  def proc = command.execute()
  proc.waitForProcessOutput(System.out, System.err)
}

params =  [
  closure: job,
  sleepInterval: 1000,
  dirPath: /R:\java\dev\eclipse_workspaces\project\help\content/
]

import groovy.transform.Canonical;

@Canonical
class AutoRunner{
  def closure
  def sleepInterval = 3000
  // running for 8 hours then stop automatically if checking every 3 seconds
  def nrOfRepeat = 9600 
  def dirPath = "."
  long lastModified = 0

  def autorun(){
    println "Press CTRL+C to stop..."
    println this
    def to_run = {
      while(nrOfRepeat--){
        sleep(sleepInterval)
        if(anyChange()){
          closure()
        }
      }
    } as Runnable
    Thread runner = new Thread(to_run)
    runner.start()
  }

  def boolean anyChange(){
    def max = lastModified
    new File(dirPath).eachFileRecurse {
      if(it.name.endsWith('txt') && it.lastModified() > max){
        max = it.lastModified()
      }
    }
    if(max > lastModified){
      lastModified = max
      return true
    }
    return false;
  }
}

new AutoRunner(params).autorun()