Use Airflow to schedule Spring Batch jobs

Question

I am trying to schedule Spring Batch jobs on on-prem Kubernetes using Airflow but couldn't find enough documentation.

I am looking for some documentation with steps to follow to schedule a Spring Batch job.

We also have pyspark jobs which also use Airflow and I need to set dependencies between them for triggering.

TIA!!

Answer 1

You did not mention how you launch your Spring Batch app, but if you package it in an uber-jar and run with with something like java -jar myjob.jar, then you can follow the following approach:

Does Airflow support jar file?

The scheduling configuration is defined in this part:

dag = DAG(
    task_id = 'runjar', 
    schedule_interval = None, #manually triggered 
    default_args = args)