PgBackRest backup multiple clusters in mutliple S3 bucket

462 Views Asked by At

I manage 3 PostgreSQL clusters each composed of a Leader node and two Replicas. I have to use PgBackRest to manage these backups. PgBackRest runs on its own node. However, reading the documentation it seems that the concept of Stanza exists to manage multiple clusters backups. Here a typical conf file:

[mydb1]
pg1-host=10.2.10.1
pg1-port=5432
pg1-path=/<my path>
pg1-host-user=postgres
pg1-socket-path=/tmp
pg2-host=10.2.10.2
pg2-port=5432
pg2-path=/<my path>
pg2-host-user=postgres
pg2-socket-path=/tmp
pg3-host=10.2.10.3
pg3-port=5432
pg3-path=/<my path>
pg3-host-user=postgres
pg3-socket-path=/tmp

Now I would like to add other two clusters in the config file:

[mydb2]
...
[mydb3]
...

but I want to add their backups in a different S3 bucket (so no same bucket and different stanza, but a different bucket and different stanza). Is this possible?

I noticed that in the pgbackrest.conf typical global configuration is like this:

[global]
log-level-file=detail
process-max=4
backup-standby=y
start-fast=y

repo1-cipher-pass=XXXXXXXXXX
repo1-cipher-type=aes-256-cbc
repo1-retention-full=120
repo1-retention-diff=72
repo1-path=/repo
repo1-s3-bucket=db-bucket-mydb1
repo1-s3-endpoint=s3....
repo1-s3-key=***************
repo1-s3-key-secret=*********************
repo1-s3-region=<my region>
repo1-type=s3

where it is possible to define only an S3 IP and one S3 bucket. I tried to add repo2-xxxx and repo3-xxxx but with no success. Does anyone know if there is a way to store the three back-ups in three different S3 buckets? If so, how?

1

There are 1 best solutions below

0
On

Move the repo configuration keys from global to stanza section of the pgbackrest config file:

[mydb2]
pg1-host=10.2.10.1
pg1-port=5432
pg1-path=/<my path>
pg1-host-user=postgres
pg1-socket-path=/tmp
repo1-cipher-pass=XXXXXXXXXX
repo1-cipher-type=aes-256-cbc
repo1-retention-full=120
repo1-retention-diff=72
repo1-path=/repo
repo1-s3-bucket=db-bucket-mydb2
repo1-s3-endpoint=s3....
repo1-s3-key=***************
repo1-s3-key-secret=*********************
repo1-s3-region=<my region>
repo1-type=s3

[mydb3]
pg1-host=10.2.10.1
pg1-port=5432
pg1-path=/<my path>
pg1-host-user=postgres
pg1-socket-path=/tmp
repo1-cipher-pass=XXXXXXXXXX
repo1-cipher-type=aes-256-cbc
repo1-retention-full=120
repo1-retention-diff=72
repo1-path=/repo
repo1-s3-bucket=db-bucket-mydb3
repo1-s3-endpoint=s3....
repo1-s3-key=***************
repo1-s3-key-secret=*********************
repo1-s3-region=<my region>
repo1-type=s3

So this way different buckets, db-bucket-mydb2 and db-bucket-mydb3 can be defined.

Same would apply to local repositories as well. Totally different local path could be used if desired.