Create dependency between CRON jobs in Kubernetes docker-compose

Question

In my current project there are multiple CRON jobs scheduled in Kubernetes. Each cron job has a specific start time scheduled. But as per a new requirement we need to create dependency between jobs so that instead of a job waiting till it's scheduled time, it should start immediately whenever the previous job is completed.

For example let's say we have 4 CRON jobs(a1, a2, a3, a4) scheduled to start at 1pm, 2pm, 3pm and 4pm. But Let's say a2 is completed within 10mins(started 2pm and finished at 2.10pm). a3 instead of waiting till 3pm, can it be triggered at 2.10pm ? If yes how can we achieve this.

My current docker-compose.kubernetes.yml file looks like this -

version: "3.9"

services:
  purelit-run-migrations:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 9 1 1 *"
    deploy:
      resources:
        limits:
          memory: "300M"
          cpus: '0.4'
        reservations:
          cpus: '0.4'
          memory: "300M"
    command: ["/app/purelit/bin/python" ,"/app/utils/init_schema.py", "--script_start_index","105", "--script_end_index","105"]

  a1-purelit-copy-sftp-files:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    # restart: on-failure
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 8 * * *"
    deploy:
      resources:
        limits:
          memory: "4000M"
          cpus: '1.0'
        reservations:
          cpus: '1.0'
          memory: "4000M"
    command: ["/app/purelit/bin/python" ,"/app/utils/copy_files.py"]

  a2-purelit-supply-loader:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    # restart: on-failure
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 9 * * *"
    deploy:
      resources:
        limits:
          memory: "4000M"
          cpus: '1.0'
        reservations:
          cpus: '1.0'
          memory: "4000M"
    command: ["/app/purelit/bin/python" ,"/app/loader/supply_data/supply_data_pipeline_runner.py"]

  a3-purelit-supply-delta-export:
    image: artifactory.castecosys.com:8443/docker-python-debian:1.1.0
    # restart: on-failure
    env_file:
      - standard-model-environment
    labels:
      service: 'purelit'
      type: 'cron'
      kubernetes.schedule: "30 10 * * *"
    deploy:
      resources:
        limits:
          memory: "4000M"
          cpus: '1.2'
        reservations:
          cpus: '1.2'
          memory: "4000M"
    command: ["/app/purelit/bin/python" ,"/app/delta/export/etl_supply_to_app_data_sync_pipeline_runner.py", "--source","all"]

What configuration we can do in the above code to achieve the CRON job dependency ?

If not CRON job, then how it can be achieved otherwise.

Answer 1

This can be achieved with CronJob being able to start/create other jobs with RBAC:

CronJob:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: cron-job
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          serviceAccountName: job-service-account
          containers:
          - name: cron-container
            image: your-cron-image:tag
            command: ["sh", "-c", "your-cron-command && kubectl create job --from=cronjob/cron-job some-job"]

Other job(or jobs):

apiVersion: batch/v1
kind: Job
metadata:
  name: some-job
spec:
  template:
    spec:
      serviceAccountName: job-service-account
      containers:
      - name: some-container
        image: your-image:tag
        command: ["your-command"]

RBAC (needed to allow cronjob to create other jobs):

apiVersion: v1
kind: ServiceAccount
metadata:
  name: job-service-account

---

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: job-role
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["get", "list", "create"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: job-role-binding
subjects:
- kind: ServiceAccount
  name: job-service-account
  namespace: default
roleRef:
  kind: Role
  name: job-role
  apiGroup: rbac.authorization.k8s.io