How to properly run Eclemma coverage with Java

Question

We use non-java tests. Each one of them executes our tool which is written in Java. I'm trying to use Eclemma for creating the coverage report of the tests. Lets start with one test. We compile the code with build.xml. I would like somehow to create a coverage report for each test and then to merge them into one main report. I found out that Jacoco has CMD interface I could use in order to merge those reports. But I don't understand how do I run the tool with coverage package?

1. Which coverage package should I use? Is it Eclemma or Jacoco?
2. How do I run the tool with the coverage package? Should I add it into the build.xml file? Should I add it to the command line?

I'm a bit confused about the whole idea of coverage in Java. In dynamic langues such as Python and Perl, I just execute the code with the coverage module, which creates the coverage report.

The command we use to execute out tool:

gandu -vm /usr/pkgs/java/1.6.0.25-64/bin/java -configuration /.ganduData -data /.ganduData -configuration /ganduInternalConfig --session_id 1582722179

Should I add some options?

The build in build.xml:

<target name="pde-build" depends="clean, init">
    <java classname="org.eclipse.equinox.launcher.Main" fork="true" failonerror="true">
        <arg value="-application" />
        <arg value="org.eclipse.ant.core.antRunner" />
        <arg value="-buildfile" />
        <arg value="${eclipseLocation}/plugins/org.eclipse.pde.build_${pdeBuildPluginVersion}/scripts/productBuild/productBuild.xml" />
        <arg value="-Dtimestamp=${timestamp}" />
        <classpath>
            <pathelement location="${eclipseLocation}/plugins/org.eclipse.equinox.launcher_${equinoxLauncherPluginVersion}.jar" />
        </classpath>
    </java>
</target>

Should I add the following command?

<arg value="-autVMArgs" />      
<arg value="-Xmx800m;-XX:MaxPermSize=600M;-javaagent:${jacoco-agent-path}=destfile=${jacoco-report},append=true"/>

Update: Thanks to the two answer I have managed to create the report using the CLI. But it looks like it shows (almost) zero coverage. It shows red (uncovered) on printed lines that I see that have been executed (they were printed to the stdout).

The project contains a lot of packages but for first try I'm trying to create coverage report for a specific package. I ran:

gandu -vm /usr/pkgs/java/1.6.0.25-64/bin/java -configuration /.ganduData -data /.ganduData --session_id 1583967465 -vmargs -Xmx256m -Xms128m -javaagent:/jacoco_coverage/jacoco-0.8.5/lib/jacocoagent.jar

and then I ran:

/usr/pkgs/java/1.6.0.16-64/bin/java -jar /jacoco_coverage/jacoco-0.8.5/lib/jacococli.jar report jacoco.exec --classfiles /gandu_repo/com.core.gandu/bin/ --html temp_dir --sourcefiles /gandu_repo/com.core.gandu/src/

The output:

[WARN] Some classes do not match with execution data.
[WARN] For report generation the same class files must be used as at runtime.
[WARN] Execution data for class <PATH> does not match.
[INFO] Analyzing 8 classes.

I have trouble understanding how classfiles and sourcefiles. The file that I'm trying to get coverage on is Application. In the bin I have in some inner folder Application.class and in src I have in some inner folder Application.java. I even tried to add full path for the those files in classfiles and sourcefiles but still don't have coverage (all red). What could be the problem? The structure of the tool: The root contains alot of packages and each one has folder bin and src. Each src contains folders recursively with Java files. The bin has the same structure for class files.

Answer 1

As I know, the only place to add java agent is in the configuration of the unit tests.

Agent should be added to the JVM that executes application under test. Your confusion probably comes from the fact that usually unit tests are executed in the same JVM as code under test.

From your description unclear how JVM with the application is started.

However consider following src/Main.java as an example:

import java.io.BufferedReader;
import java.io.InputStreamReader;

public class Main {

  public static void main(String[] args) throws Exception {
    String input;
    try (BufferedReader reader = new BufferedReader(new InputStreamReader(System.in))) {
      input = reader.readLine();
    }
    if ("hello".equals(input)) {
      System.out.println("Hello, World!");
    }
  }

}

after compilation

javac -d classes src/Main.java

This application can be executed in JVM with JaCoCo Java Agent taken from JaCoCo zip distribution as following

java -javaagent:jacoco/lib/jacocoagent.jar -cp classes Main

Then tests are executed - e.g. in this case manual input of "hello".

After graceful termination of JVM, you'll get jacoco.exec file.

By passing this jacoco.exec, source and class files to the JaCoCo Command Line Interface you can create HTML report:

java -jar jacoco/lib/jacococli.jar report jacoco.exec --classfiles classes --sourcefiles src --html report

Which coverage package should I use? Is it Eclemma or Jacoco?

EclEmma is a plugin for Eclipse IDE based on JaCoCo.

If for testing you execute your application from Eclipse IDE then you can start it with JaCoCo using EclEmma. Here is the same example executed in Eclipse IDE:

However this is not suitable for other scenarios such as Ant builds.

How do I run the tool with the coverage package? Should I add it into the build.xml file? Should I add it to the command line?

This depends on how you execute your application - if for tests your application is executed by Ant, then you can use JaCoCo Ant Tasks for configuration of agent and generation of report. As an example here is build.xml for Ant that produces the same report for src/Example.java as the above commands:

<project xmlns:jacoco="antlib:org.jacoco.ant" name="Example Ant Build with JaCoCo" default="rebuild">

  <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">
    <classpath path="jacoco/lib/jacocoant.jar"/>
  </taskdef>

  <target name="clean">
    <delete dir="classes"/>
    <delete dir="report"/>
    <delete file="jacoco.exec"/>
  </target>

  <target name="compile">
    <mkdir dir="classes"/>
    <javac srcdir="src" destdir="classes" debug="true" includeantruntime="false"/>
  </target>

  <target name="run" depends="compile">
    <jacoco:agent property="jacocoAgent" />

    <java classname="Main" fork="true">
      <classpath path="classes"/>
      <jvmarg value="${jacocoAgent}"/>
    </java>
  </target>

  <target name="report" depends="run">
    <jacoco:report>
      <executiondata>
        <file file="jacoco.exec"/>
      </executiondata>
      <structure name="JaCoCo Ant Example">
        <classfiles>
          <fileset dir="classes"/>
        </classfiles>
        <sourcefiles encoding="UTF-8">
          <fileset dir="src"/>
        </sourcefiles>
      </structure>
      <html destdir="report"/>
    </jacoco:report>
  </target>

  <target name="rebuild" depends="clean,report"/>

</project>

If application is executed by some custom homemade tool, then maybe be better to add the agent to the command line directly.