airflow initdb: cannot import name 'Pendulum' from 'pendulum'

Question

I installed airflow within one of my Anaconda envs named engdados. When I execute the command airflow initdb I'm getting the following error: airflow initdb: cannot import name 'Pendulum' from 'pendulum'. The full trace back is shown below:

(engdados) guilherme@Athena-LNX:~$ airflow initdb
Traceback (most recent call last):
  File "/home/guilherme/anaconda3/envs/engdados/bin/airflow", line 25, in <module>
    from airflow.configuration import conf
  File "/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/airflow/__init__.py", line 47, in <module>
    settings.initialize()
  File "/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/airflow/settings.py", line 403, in initialize
    configure_adapters()
  File "/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/airflow/settings.py", line 319, in configure_adapters
    from pendulum import Pendulum
ImportError: cannot import name 'Pendulum' from 'pendulum' (/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/pendulum/__init__.py)
(engdados) guilherme@Athena-LNX:~$ service start mysql$
start: unrecognized service
(engdados) guilherme@Athena-LNX:~$ service mysql start$
Usage: /etc/init.d/mysql start|stop|restart|reload|force-reload|status|bootstrap
(engdados) guilherme@Athena-LNX:~$ airflow initdb
Traceback (most recent call last):
  File "/home/guilherme/anaconda3/envs/engdados/bin/airflow", line 25, in <module>
    from airflow.configuration import conf
  File "/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/airflow/__init__.py", line 47, in <module>
    settings.initialize()
  File "/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/airflow/settings.py", line 403, in initialize
    configure_adapters()
  File "/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/airflow/settings.py", line 319, in configure_adapters
    from pendulum import Pendulum
ImportError: cannot import name 'Pendulum' from 'pendulum' (/home/guilherme/anaconda3/envs/engdados/lib/python3.8/site-packages/pendulum/__init__.py)

The problem is: the pendulum is installed! When I execute the conda list command I can see the Pendulum there as follows:

 Name                    Version                   Build  Channel

pendulum                  2.1.2                    pypi_0    pypi

What I've checked so far:

Is the engdados environment activated? Yes

Is the Pendulum installed on Anaconda environment? Yes

The version of Pendulum the Anaconda shows is different of the one showed in conda list (1.4.4). Why?

I have no idea what is going on. Thanks in advance.

Answer 1

In pendulum version 2, the class pendulum.Pendulum is replaced with pendulum.DateTime.

Your version of airflow is expecting pendulum 1.x but your environment has 2.x.

You may be able to fix this by making a new env and installing airflow 2.0 (which uses pendulum 2.x). If you must use airflow < 2.0, you will need to pin pendulum to < 2.0 (e.g. using pip constraints).

Answer 2

Also if you using Pendulum in your code, for example in custom Operators you can add

try:
    from pendulum import DateTime as Pendulum
except ImportError:
    from pendulum import Pendulum