I am working on a project with Airbyte where we are ingesting tables from mysql into another database for transformations. We have some partitioned data and I'm wondering how to achieve this in the extract/load process as opposed to the transformation layer, because we want to dump all of the partitions in a single raw table before transformation.
We have tried researching this and are having a bit of a hard time finding any information on this, even though it has to be a pretty common use case to ingest partitioned data.
We have looked into doing this at the transformation step, unfortunately this sort of muddies the ELT vs ETL distinction and kind of negates a lot of the benefits of just dumping the data into a single table.
We have tried researching airbyte's site and they have no information about ingesting partitioned tables, only information about partitioning as a concept.