DEVHIDE
Login Or Sign up

Fail in Building Apache Tez from Source

107 Views Asked by At

I am setting up a cluster of nodes (CentOs 8) using Hadoop version 3.3.6. And I have tried to set up Apache Tez (version 0.9.2) for the execution engine of Hive (version 3.1.3) with Java 8. But after settin up hadoop version in pom.xml and hitting mvn clean package -DskipTests=true -Dmaven.javadoc.skip=true, it ends up being like this:

[INFO] 4 warnings
[INFO] -------------------------------------------------------------
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] tez 0.9.2 .......................................... SUCCESS [  1.240 s]
[INFO] hadoop-shim ........................................ SUCCESS [  1.435 s]
[INFO] tez-api ............................................ SUCCESS [  5.870 s]
[INFO] tez-build-tools .................................... SUCCESS [  0.105 s]
[INFO] tez-common ......................................... SUCCESS [  0.560 s]
[INFO] tez-runtime-internals .............................. SUCCESS [  0.871 s]
[INFO] tez-runtime-library ................................ SUCCESS [  2.554 s]
[INFO] tez-mapreduce ...................................... SUCCESS [  1.232 s]
[INFO] tez-examples ....................................... SUCCESS [  0.258 s]
[INFO] tez-dag ............................................ FAILURE [  2.530 s]
[INFO] tez-tests .......................................... SKIPPED
[INFO] tez-ext-service-tests .............................. SKIPPED
[INFO] tez-ui ............................................. SKIPPED
[INFO] tez-plugins ........................................ SKIPPED
[INFO] tez-protobuf-history-plugin ........................ SKIPPED
[INFO] tez-yarn-timeline-history .......................... SKIPPED
[INFO] tez-yarn-timeline-history-with-acls ................ SKIPPED
[INFO] tez-history-parser ................................. SKIPPED
[INFO] tez-aux-services ................................... SKIPPED
[INFO] tez-tools .......................................... SKIPPED
[INFO] tez-perf-analyzer .................................. SKIPPED
[INFO] tez-job-analyzer ................................... SKIPPED
[INFO] tez-javadoc-tools .................................. SKIPPED
[INFO] hadoop-shim-impls .................................. SKIPPED
[INFO] hadoop-shim-2.7 .................................... SKIPPED
[INFO] tez-dist ........................................... SKIPPED
[INFO] Tez 0.9.2 .......................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 17.025 s
[INFO] Finished at: 2023-09-07T19:24:55Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project tez-dag: Compilation failure
[ERROR] /usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine
[ERROR]
[ERROR] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project tez-dag: Compilation failure
/usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine

    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:213)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
Caused by: org.apache.maven.plugin.compiler.CompilationFailureException: Compilation failure
/usr/local/hadoop/apache-tez-0.9.2-src/tez-dag/src/main/java/org/apache/tez/state/StateMachineTez.java:[28,8] org.apache.tez.state.StateMachineTez is not abstract and does not override abstract method getPreviousState() in org.apache.hadoop.yarn.state.StateMachine

    at org.apache.maven.plugin.compiler.AbstractCompilerMojo.execute (AbstractCompilerMojo.java:858)
    at org.apache.maven.plugin.compiler.CompilerMojo.execute (CompilerMojo.java:129)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:208)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:154)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:146)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:954)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:288)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:289)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:229)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:415)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:356)
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :tez-dag

I've downloaded an older version but it was totally in vain due to some errors like this:

[ERROR] /usr/local/hadoop/apache-tez-0.9.0-src/tez-mapreduce/src/main/java/org/apache/tez/mapreduce/client/NotRunningJob.java:[89,29] no suitable method found for 

newInstance(org.apache.hadoop.yarn.api.records.ApplicationId,org.apache.hadoop.yarn.api.records.ApplicationAttemptId,java.lang.String,java.lang.String,java.lang.String,java.lang.String,int,

<nulltype>,org.apache.hadoop.yarn.api.records.YarnApplicationState,java.lang.String,java.lang.String,int,int,org.apache.hadoop.yarn.api.records.FinalApplicationStatus,

<nulltype>,java.lang.String,float,java.lang.String,<nulltype>)

How can I resolve this problem?

hadoop hive apache-tez
Original Q&A
2

There are 2 best solutions below

0
Avi On

Consider building newer version (if possible) like tez 10.1 and before building apply patches from bigtop repository

With this metod i succesfully build tez 10.1 on rocky8 (rhel compatible) and even created rpm for rhel8, although i was building against hadoop 3.3.4 and hive 3.1.3.

0
Sercan On

After Hadoop YARN devs have fixed this issue https://issues.apache.org/jira/browse/YARN-11395 and backported it to a few previous release versions, Tez devs hit the same issue and has fixed it here.

As you can see here, there is a new method in StateMachine interface which needs to be overridden in any class that implements it.

StateMachineTez also implements StateMachine interface. So, what you need to do is to override the new method in StateMachineTez class before building it;

  @Override
  public STATE getPreviousState() {
    return realStatemachine.getPreviousState();
  }

Related Questions in HADOOP

Related Questions in HIVE

Related Questions in APACHE-TEZ

Trending Questions

Popular # Hahtags

javascript python java c# php android html jquery c++ css ios sql mysql r reactjs

Popular Questions

.

Copyright © 2021 Jogjafile Inc.