How to configure Hive Cli to automatically get the kerberos ticket and renew/request new if expires by it own

Question

Hi I am new to Hive and kerberos. I have some hive jobs which run more then life time of ticket. how can I configure hive so that when I start hive shell if ticket is not cached it automatically request for ticket. After acquiring ticket lets suppose if ticket expire is the middle then automatically acquire new one and also I may have simultaneous job running by same user so may be one cached ticket can be used by many jobs.

Any Solutions or direction to look upon will be highly appreciated. Thanks in Advance.

I am looking for a solution in which hive cli or shell can automatically acquire or renew Kerberos credentials.

Answer 1

What you need to look into is Java Authentication and Authorization Service (JAAS) It's how to enable java to use kerberos without adding anything to your code. Specifically here you might want to look at how beeline uses kerberos config as an example.

Create setEnv.sh file and save it inside "bin" folder. Paste below content inside it:

export HADOOP_HOME=/home/user/beeline/hadoop-2.5.1
export HIVE_HOME=/home/user/beeline/apache-hive-1.2.1-bin
export JAVA_HOME=/home/user/beeline/jre
PATH=$PATH:$HIVE_HOME/bin:$JAVA_HOME/bin
export HADOOP_OPTS="$HADOOP_OPTS -Dsun.security.krb5.debug=true -Djava.security.krb5.conf=/home/user/beeline/conf/krb5.conf -Djavax.security.auth.useSubjectCredsOnly=false -Djava.security.auth.login.config=/home/user/beeline/conf/jaas.conf"

jaas.conf File:

Create and save jaas.conf file under conf folder

Client {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=false
useTicketCache=true;
};

krb5.conf File:

Create and save krb5.conf File under conf folder. Modify this file as per your environment.

[logging]
default = FILE:~/krb5libs.log
kdc = FILE:~/krb5kdc.log
admin_server = FILE:~/kadmind.log
kdc_rotate = {"period"=>"1d", "versions"=>200}
admin_server_rotate = {"period"=>"1d", "versions"=>201}

[libdefaults]
    default_realm = DOMAIN.COM
    dns_lookup_realm = false
    dns_lookup_kdc = false
    forwardable = true
    renew_lifetime = 30d
    ticket_lifetime = 30d
    renewable = yes
    service = yes
    kdc_timeout = 5000
    default_tgs_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1
    default_tkt_enctypes = aes256-cts-hmac-sha1-96 aes128-cts arcfour-hmac-md5 des-cbc-crc des-cbc-md5 des-hmac-sha1
    allow_weak_crypto = yes
    udp_preference_limit = 1

[realms]
  DOMAIN.COM = {
     kdc = kdcserver.domain.com:88
     default_domain  = domain.com
    }

  [domain_realm]
    .domain.com = DOMAIN.COM 
    domain.com = DOMAIN.COM

[appdefaults]
  pam = {
      debug = false
      forwardable = true
      renew_lifetime = 36000
      ticket_lifetime = 36000
      krb4_convert = false
    }

It should be noted that the above config doesn't use a renewable kerberos ticket but that's just and example and you can make it renewable.