exeuction_timeout is failing the task but not sending email

Question

I want to fail the task if task duration exceeds a certain time for which I configured execution_timeout in every task of the dag. However, the task is failing, but I am not getting email notification of it. Does anyone know the reason? Any help would be appreciated.

default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime.today() - timedelta(days = 2),
    'email':['[email protected]'],
    'email_on_failure': True
}

[2022-10-27 03:06:42,587] {logging_mixin.py:112} INFO - [2022-10-27 03:06:42,585] {timeout.py:42} ERROR - Process timed out, PID: 15847
[2022-10-27 03:06:42,588] {bash_operator.py:140} INFO - Sending SIGTERM signal to bash process group
[2022-10-27 03:06:42,602] {taskinstance.py:1145} ERROR - Timeout, PID: 15847
Traceback (most recent call last):
  File "/home/airflow_user/.local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 978, in _run_raw_task
    result = task_copy.execute(context=context)
  File "/home/airflow_user/.local/lib/python3.7/site-packages/airflow/operators/bash_operator.py", line 124, in execute
    for line in iter(self.sub_process.stdout.readline, b''):
  File "/home/airflow_user/.local/lib/python3.7/site-packages/airflow/utils/timeout.py", line 43, in handle_timeout
    raise AirflowTaskTimeout(self.error_message)
airflow.exceptions.AirflowTaskTimeout: Timeout, PID: 15847

I can see the task turning to red. But no email is sent even with the email_on_failure config.

Answer 1

You have not configured your smtp in airflow.cfg, you need configure it:

[smtp]
smtp_host = #your smtp host
smtp_starttls = True
smtp_ssl = False
# Example: smtp_user = airflow
smtp_user = # your email to send
# Example: smtp_password = airflow
smtp_password = # App password
smtp_port = # your smtp port
smtp_mail_from = # your email to send
smtp_timeout = 30
smtp_retry_limit = 5

Example, if you are using gmail:

[smtp]
smtp_host = smtp.gmail.com
smtp_starttls = True
smtp_ssl = False
# Example: smtp_user = airflow
smtp_user = [email protected]
# Example: smtp_password = airflow
smtp_password = #app password
smtp_port = 25
smtp_mail_from = [email protected]
smtp_timeout = 30
smtp_retry_limit = 5

Note: you can take your app password in here: https://support.google.com/mail/answer/185833?hl=en

Gmail port can be 25, 465 or 587