Issue while reading Hbase in SPARK using rowkey

317 Views Asked by At

Aim: To read Hbase using Spark using rowkey

Error:
Exception in thread "main" java.lang.IncompatibleClassChangeError: Class org.apache.hadoop.hbase.client.RegionCoprocessorRpcChannel does not implement the requested interface com.google.protobuf.BlockingRpcChannel
    at org.apache.hadoop.hbase.protobuf.generated.AuthenticationProtos$AuthenticationService$BlockingStub.getAuthenticationToken(AuthenticationProtos.java:4512)
    at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:97)
    at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:137)
    at org.apache.hadoop.hbase.security.token.TokenUtil$1.run(TokenUtil.java:134)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1875)
    at org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:313)
    at org.apache.hadoop.hbase.security.token.TokenUtil.obtainToken(TokenUtil.java:134)
    at org.apache.hadoop.hbase.security.token.TokenUtil.addTokenForJob(TokenUtil.java:288)
    at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.initCredentials(TableMapReduceUtil.java:533)
    at org.apache.hadoop.hbase.spark.HBaseContext.<init>(HBaseContext.scala:71)


Error thrown while creating HbaseContext in below line of code:

    var hb:org.apache.hadoop.hbase.spark.HBaseContext =new HBaseContext(sc, HbaseConfig)

Things that I tried to resolve above issue: i believe above issue is because of common class named :AuthenticationProtos in jar files hbase-shaded-client & hbase-protocol that creates a conflict. maven dependencies for conflicting jars:

    <dependency>
            <artifactId>hbase-shaded-client</artifactId>
            <groupId>org.apache.hbase</groupId>
            <version>2.1.0-cdh6.3.4</version>
        </dependency>

<dependency>
            <artifactId>hbase-protocol</artifactId>
            <groupId>org.apache.hbase</groupId>
            <version>2.1.0-cdh6.3.4</version>
        </dependency>

if hbase-shaded-client is excluded from pom.xml , issue resolves .

However in my case i cant remove hbase-shaded-client dependency permanently as its already declared in project. my question is "Can these 2 dependencies hbase-protocol and hbase-shaded-client be used together ?"

0

There are 0 best solutions below