Enable Data Profiling on Apache Airflow

Question

Data profiling although present in documentation is not visible in a fresh installation. Is there a way to enable it, or it has been deprecated? Installed version is apache-airflow==1.10.3 in case it helps.

Answer 1

Data profile is default UI when install airflow v1.10.3.

If you can not see data profile menu, you might need to double check if 3rd party customized airflow for you. Airflow use flask as web framework. you can go to the related folder to see if it has been modified. the configuration file is at [you airflow source code folder]\www\app.py

---

Thanks for vote for the answer. The default app.py section related to dataprofiling is below:

 with app.app_context():
        from airflow.www import views

        admin = Admin(
            app, name='Airflow',
            static_url_path='/admin',
            index_view=views.HomeView(endpoint='', url='/admin', name="DAGs"),
            template_mode='bootstrap3',
        )
        av = admin.add_view
        vs = views
        av(vs.Airflow(name='DAGs', category='DAGs'))

        if not conf.getboolean('core', 'secure_mode'):
            print("create_app", __file__)
            av(vs.QueryView(name='Ad Hoc Query', category="Data Profiling"))
            av(vs.ChartModelView(
                models.Chart, Session, name="Charts", category="Data Profiling"))
        av(vs.KnownEventView(
            models.KnownEvent,
            Session, name="Known Events", category="Data Profiling"))

As you can see from above code, it related to secure mode. if not conf.getboolean('core', 'secure_mode'): you might need check you secure mode configuration as well.

Note: pls check if the secure_mode in airflow.cfg is configurated properly.

the data profiling can only be showed in secure_mode = False.

# If set to False enables some unsecure features like Charts and Ad Hoc Queries.
# In 2.0 will default to True.
secure_mode = False

Answer 2

Airflow 2 disabled Data Profiling due to security reasons: (Breaking changes section)

Due to security concerns, the new web server will no longer support the features in the Data Profiling menu of the old UI, including Ad Hoc Query, Charts, and Known Events.