Airflow Rendered Template changes when task starts running?

Question

I'm having a very weird airflow bug.

Problem

I have a dag that has a bash operator as step 1 and a KubernetesPodOperator as step 2. The issue is regarding the KubernetesPodOperator. Basically, I was giving the task image X for quite some time, I recently changed the image the task is receiving to Y. The issue I'm having is within TaskInstanceDetails the image is correct: Y. However, in the Rendered Template, the image starts correct X, however, as soon as the task starts running, it changes the image to Y.

I know this is very vague, I can't provide a whole lot more, but I'm just more looking for possibilities of things that could be happening, as I'm out of ideas.

What I've Tried

• Delete Serialized Dags from DB
• Delete Rendered Task Details from DB
• Airflow db reset
• Airflow db init (After nuking the whole thing)
• Deleting the EC2 nodes and trying with new ones

EDIT

So, I tried runnning airflow tasks render dag_id task_id execution_date and the result here is image X !! Image Y is only superimposed on dag runs.

Answer 1

Answering my own question here in case anyone runs into this issue. Very simple fix... I carelessly had a different image name for the workers to be run with on the kubernetes_pod_template file. Changing that solved the issue.