Failed to execute goal org.scalatest:scalatest-maven-plugin:2.0.0:test (test)

Question

I am trying to build my Scala code using Maven POM and I have below plugin in my POM:

 <plugin>
                <groupId>org.scalatest</groupId>
                <artifactId>scalatest-maven-plugin</artifactId>
                <version>2.0.0</version>
 </plugin> 

When I try to do Maven build from Scala IDE (clean Install) I am facing below error:

Failed to execute goal org.scalatest:scalatest-maven-plugin:2.0.0:test (test)

But when checked in my local .m2 folder I could see relevant Jar,pom and sha1 files available there:

scalatest-maven-plugin-2.0.0.jar
scalatest-maven-plugin-2.0.0.pom

If anyone has encountered similar issue could you please help me in fixing this issue? Thanks

Detailed Error log:

--- scalatest-maven-plugin:2.0.0:test (test) @ streaming-base ---
[36mDiscovery starting. [0m
[36mDiscovery completed in 6 seconds, 175 milliseconds. [0m
[36mRun starting. Expected test count is: 12 [0m
[32mZookeeperLockSpec: [0m
[32mZookeeperLock [0m
[32m- should create lock path [0m
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:21810] WARN  org.apache.zookeeper.server.NIOServerCnxn  - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x1698ba52c8e0000, likely client has closed socket
    at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
    at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
    at java.lang.Thread.run(Unknown Source)
[33m- should not fail if lock path exists !!! IGNORED !!! [0m
[32m- should correctly release lock if job fails [0m
[32mResourceUtilSpec: [0m
[32mResource file reader [0m
[32m- correctly read and parse text file in resources [0m
[32m- fail with FileNotFoundException if there's no file in resources [0m
[32mString array extractor [0m
[32m- return expected value if index within bounds (full version) [0m
[32m- return default value if index is out of bounds [0m
[32m- return expected value if index within bounds (shortened version, empty string default) [0m
[32m- return empty string if index is out of bounds [0m
[32mJdbcToBqExportToolSpec: [0m
[ScalaTest-main-running-DiscoverySuite] WARN  org.apache.hadoop.util.Shell  - Did not find winutils.exe: {}
java.io.FileNotFoundException: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset. -see https://wiki.apache.org/hadoop/WindowsProblems
    at org.apache.hadoop.util.Shell.fileNotFoundException(Shell.java:528)
    at org.apache.hadoop.util.Shell.getHadoopHomeDir(Shell.java:549)
    at org.apache.hadoop.util.Shell.getQualifiedBin(Shell.java:572)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:669)
    at org.apache.hadoop.hive.conf.HiveConf$ConfVars.findHadoopBinary(HiveConf.java:2327)
    at org.apache.hadoop.hive.conf.HiveConf$ConfVars.<clinit>(HiveConf.java:365)
    at org.apache.hadoop.hive.conf.HiveConf.<clinit>(HiveConf.java:105)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Unknown Source)
    at org.apache.spark.util.Utils$.classForName(Utils.scala:230)
    at org.apache.spark.sql.SparkSession$.hiveClassesArePresent(SparkSession.scala:1063)
    at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:842)
    at com.kohls.bigdata.util.SparkSpecBase$class.beforeAll(SparkSpecBase.scala:29)
    at com.kohls.bigdata.dp.streaming.jdbc.JdbcToBqExportToolSpec.beforeAll(JdbcToBqExportToolSpec.scala:11)
    at org.scalatest.BeforeAndAfterAll$class.liftedTree1$1(BeforeAndAfterAll.scala:212)
    at org.scalatest.BeforeAndAfterAll$class.run(BeforeAndAfterAll.scala:210)
    at com.kohls.bigdata.dp.streaming.jdbc.JdbcToBqExportToolSpec.run(JdbcToBqExportToolSpec.scala:11)
    at org.scalatest.Suite$class.callExecuteOnSuite$1(Suite.scala:1210)
    at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1257)
    at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1255)
    at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
    at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
    at org.scalatest.Suite$class.runNestedSuites(Suite.scala:1255)
    at org.scalatest.tools.DiscoverySuite.runNestedSuites(DiscoverySuite.scala:30)
    at org.scalatest.Suite$class.run(Suite.scala:1144)
    at org.scalatest.tools.DiscoverySuite.run(DiscoverySuite.scala:30)
    at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
    at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1346)
    at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1340)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
    at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1011)
    at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1010)
    at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
    at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
    at org.scalatest.tools.Runner$.main(Runner.scala:827)
    at org.scalatest.tools.Runner.main(Runner.scala)
Caused by: java.io.FileNotFoundException: HADOOP_HOME and hadoop.home.dir are unset.
    at org.apache.hadoop.util.Shell.checkHadoopHomeInner(Shell.java:448)
    at org.apache.hadoop.util.Shell.checkHadoopHome(Shell.java:419)
    at org.apache.hadoop.util.Shell.<clinit>(Shell.java:496)
    ... 33 more

Answer 1

The error message mentions a likely issue and a site to go to for more details on how to fix it. (That site is a 404, but has a comment that points to the new location.)

https://cwiki.apache.org/confluence/display/HADOOP2/WindowsProblems

You need to download the appropriate Hadoop version of winutils.exe and place it on your Windows machine somewhere and set an environment variable pointing to that location. (You can also install a full native Hadoop version, but you may only need the winutils.exe executable.)

1. Download the winutils.exe for the version of Hadoop you are using from steveloughran/winutils on GitHub.

2. Create a folder where you want to store the executable.

3. Place the winutils.exe file in that folder within a /bin folder (e.g., C:\some\folder\bin\winutils.exe).

4. Set an environment variable called HADOOP_HOME using your preferred approach, set to the folder containing the /bin folder (not the /bin folder itself).

   For example, you can use PowerShell.

   Add-EnvironmentVariable -Name "HADOOP_HOME" -Value "C:\some\folder\"
   

   (Or you can add it from the Windows Environment Variables GUI.)

5. Depending on how you are building things, you may need to restart your terminal to pick up the new environment variable.