Azkaban : Conditional executions in the workflow

Question

I have a requirement to inject a conditional execution in my workflow. For ex: If a particular condition is met, then a particular workflow should be executed. If not, a different workflow should be executed.

From my understanding, there is no direct feature in Azkkaban that supports this. Only Oozie supports it. Wanted to know how Azkkaban users are dealing with this? I am sure everyone must be facing this in some form or the other.

Thanks, Kranthi

Answer 1

You can use a trigger plugin in Azkaban to do that. Here's a example of a simple one I came up with: https://gist.github.com/feliperazeek/faae2fed6aef780442a4.

You can implement your own Condition implementation.

Answer 2

Azkaban's roadmap says that this feature is there in their future roadmap only and hence this cannot be configured so easily. Only option left is to handle this programatically.

If anyone has better suggestions, please let me know.

thanks, Kranthi

Answer 3

FWIW, we do this in our Azkaban projects by checking a condition in a shell script. The script then uses the Azkaban API to trigger another workflow if the condition is met.

#!/bin/bash

# check some condition
check=`…`;

if [ ! -z "$check" ]; then

  # Authenticate ourselves to Azkaban, capturing the session id
  auth=`curl -k -X POST --data "action=login&username=my_user&password=my_password" https://localhost:8443 | jq '.["session.id"]' | sed -e 's/\"//g'`;

  # Run the workflow using the Auth session id, capturing the response
  run=`curl -k --get --data "session.id=$auth" --data 'ajax=executeFlow' --data 'project=my_project' --data "flow=my_flow" https://localhost:8443/executor  | jq '.message'`;

  # If $run response does not contain "successfully" it failed
  if [[ $run = *successfully* ]]; then 
      echo "'$3': RUN STARTED"; 
  else 
      echo "'$3': RUN ERROR"; 
      exit 1; 
  fi; 

else …

You need to be aware that the triggered workflow runs in it's own execution.