What happens if some hadoop map-reduce Job or some hdfs command is running and in the between kerberos ticket expires?

73 Views Asked by At

Lets say we have kerberos secured Hadoop Cluster. ( For example Kdc is configured to provide kerberos tickets with validity of 2 hour and renewal period of 3 hours after this so total 5 hours. ) Cluster also includes tools like Hive and Pig.

With respect to given scenario can any one help me understand the behaviors in each cases mentioned below(Note: Job: Map-Reduce Job, Job Submitted Through Hive Or Pig)

  1. Lets say job takes 4 hours to run, I am triggering the job, for next two hours user who triggered the job has valid ticket, job runs fine after two hours Job is out of valid kerberos ticket but it has renewal period left, Will job renew it? Will it stop execution? Is there any property to enable or disable such automatically renewal behavior.
  2. Lets say job takes 10 hours to run, I am trigerring the job, for next two hours user who triggered the job has valid ticket, lets say this time after two hours we don't have renewal period left, Will Job continue? Will Job fail with authentication error. Will Job request for new ticket?
  3. Let say Job takes 10 hours to run And Kdc is configured to provide 2 hours ticket. we setup a cron job which runs at every 6 hours and gets a new ticket(Technically User will not run out of valid kerberos ticket any time), Will new generated ticket cause any problem of the running job through old ticket?

I want to understand working behavior of this Jobs as well as tools like kerberos, hive and pig.

0

There are 0 best solutions below