CubeJS pre aggregation tables being duplicated upon refresh

707 Views Asked by At

I'm trying to use CubeJS to speed up queries using the pre aggregation caching. I have defined the following cube:

cube({
  // assume all measures and dimensions are defined as necessary
  ...
  preAggregations: {
    report: {
      type: `rollup`,
      measureReferences: [totalValue],
      dimensionReferences: [
       name,
       portfolio,
       country
      ],
      refreshKey: {
        every: `1 day`
      },
      external: true
   }
}

The raw daya is stored in Redshift and the pre aggregation is stored in a RDS Aurora Postgresql. The issue is that upon a refresh of the pre aggregation, the report pre aggregation table is duplicated rather than replacing the previous, old cached table in RDS. I would like to avoid having stale data being built up and also avoid manual maintenance of the databases. Is there a way in CubeJS to change the behavior of how the pre aggregation table is managed during a refresh?

1

There are 1 best solutions below

0
On

It is intended behavior. Please see https://cube.dev/docs/pre-aggregations#pre-aggregations-garbage-collection to learn more about rationale here.