We have a requirement where we need to give steps to the client to extract all the airflow dag scripts in thier environment in a single file so that we can analyze it manually.
I got thread wherin we can extract the list of all dag names but our requirment to get the underlying script of all airflow dag present
Get List of all the dags in python
Found couple of ways to extract it :
- All python scripts which is used in dag will be stored in airflow--project-->dags-->datasets folder where airflow is insatlled.
- All DAG files will have this import statment "from airflow import DAG",So i have created a unix script to fetch all python files in all sub folders which has this string and put it in a single output file along with file name.
Are these the correct way to do? could you please confirm
Airflow REST API provides a simpler way to do that: