Astronomer.Cosmos -> AirFlow -> Dbt (DbtTaskGroup) error

401 Views Asked by At

I'm getting a weird error that I'm really struggling to troubleshoot.

So I'm using Astronomer.Cosmos with Docker to link Airflow and DBT. It's pushing the data to my postgres localhost.

my connection details are:

conn_id=cams_db profile_args {"dbname":"testing_dbt", "schema": "raw_csvs"}

As you can see, nothing complex (JaffleShop, testing the cosmos package).

my Postgres Connection in Airflow Localhost:8080 is just:

cams_db host - host.docker.internal port - 5433 login

My error:

Database Error in model orders (models/orders.sql) improper relation name (too many dotted names): raw_csvs.testing_dbt.raw_csvs.orders__dbt_backup

As you can see, it's trying to pass the staging tables (but not the staging views) from jaffleshop with schema.dbname.schema.table....

This doesn't happen with the seeds or any of the db views.

Any idea why this is happening?

I've tried removing the schema name from my airflow connection and just passing it in and it still generates this error.

I'm guessing this is a stupidly easy fix, but I can't see what line of code I need to change lol

I've tried editing all the schema variables, and connections. I also tried passing everything in the profile-config instead, however, nothing fixes it.

0

There are 0 best solutions below