how to extract all the scripts present in all dags in airflow

48 Views Asked by At

We have a requirement where we need to give steps to the client to extract all the airflow dag scripts in thier environment in a single file so that we can analyze it manually.

I got thread wherin we can extract the list of all dag names but our requirment to get the underlying script of all airflow dag present

Get List of all the dags in python

Found couple of ways to extract it :

  1. All python scripts which is used in dag will be stored in airflow--project-->dags-->datasets folder where airflow is insatlled.
  2. All DAG files will have this import statment "from airflow import DAG",So i have created a unix script to fetch all python files in all sub folders which has this string and put it in a single output file along with file name.

Are these the correct way to do? could you please confirm

1

There are 1 best solutions below

0
Tatiana A On

Airflow REST API provides a simpler way to do that:

  1. Get all dags
  2. Get file token for each dag
  3. Get source code for each file token