Is there a way to run a single benchmark with sbt-jmh?

Question

I am working on a big sbt project and there is some functionality that I want to benchmark. I decided that I will be using jmh, thus I enabled the sbt-jmh plugin.

I wrote an initial test benchmark that looks like this:

import org.openjdk.jmh.annotations.Benchmark

class TestBenchmark {

  @Benchmark
  def functionToBenchMark = {
    5 + 5
  }
}

However, when I try to run it with jmh:run -i 20 -wi 10 -f1 -t1 .*TestBenchmark.* I get java.lang.InternalError: Malformed class name. I have freshly rebuilt the project and everything compiles and runs just fine.

The first printed message says

Processing 6718 classes from /path-to-repo/target/scala-2.11/classes with "reflection" generator

I find it weird that the plugin tries to reflect the whole project (I guess including classes within the standard library). Before rebuilding I was getting NoClassDefFoundError, although the project was otherwise working well.

Since there are plenty of classes within the project and I cannot make sure that every little bit conforms to jmh's requirements, I was wondering if there's a way to overcome this issue and focus and reflect only the relevant classes that are annotated with @Benchmark?

My sbt version is 0.13.6 and the sbt-jmh version is 0.2.25.

Answer 1

So this is an issue with Scala and Class.getSimpleClassName.

Its not abonormal in Scala to have types like this:

object Outer {
   sealed trait Inner
   object Inner {
      case object Inner1 extends 
      case object Inner2 extends Inner
   }
}

With the above calling Outer.Inner.Inner1.getClass().getSimpleName() will throw the exception your seeing.

I don't think it uses the full project, but only for things that are directly referred to in the State or Benchmark.

Once I had my bench file written that way it worked.