How to take jfrog artifact registry backup to s3

Question

We are using jfrog artifact registry to store our artifacts.Our current artifact registry size is 1.7TB and we taking the backup also in the same server and its taking another 2TB and our server reaches to 95% of it limits.

Our jfrog version is - Commercial license 7.4.1

We are trying to take the backup and store that in s3, so that if any disasters happens we can restore it.

As we reaches the 95% we want to move our backups to s3.

Is there any way where we can take daily backup and move that to s3 without any data loss ? With current infra limitation in our project we can't increase the size of the server.

Answer 1

Not sure which commercial license you are on, but Artifactory supports AWS S3 as binarystore. You don't need to backup and store the backup to S3, all your artifacts can be directly stored on S3.

S3 object storage requires JFrog Container Registry, Pro, Enterprise X, or an Enterprise+ license.

To Configure Artifactory to Use S3

update the following on binarystore.xml

<config version="2">
    <chain>
        <provider id="cache-fs" type="cache-fs">
            <provider id="s3-storage-v3" type="s3-storage-v3"/>
        </provider>
    </chain>
    <provider id="s3-storage-v3" type="s3-storage-v3">
       <endpoint>s3.amazonaws.com</endpoint>
       <bucketName>bucketName</bucketName>
       <path>pathPrefix</path>
       <region>s3Region</region>
       <identity>yourIdentity</identity>
       <credential>yourCredentials</credential>
    </provider>
</config>

Migrate your filestore to S3

To make sure your filestore migration completes successfully without corrupting files, we recommend configuring Artifactory to do this migration for you automatically:

To do so, you need to create the following links in $JFROG_HOME/artifactory/var/data/artifactory/eventual/ (create it if the eventual folder does not exist - it is created automatically when the eventual binary provider is applied via an Artifactory restart with an updated binarystore.xml):

A link with the name _add that points to the $JFROG_HOME/artifactory/var/data/artifactory/filestore directory

A link with the name _pre that points to the $JFROG_HOME/artifactory/var/data/artifactory/filestore/_pre directory

With this setting, as soon as Artifactory starts up, it will automatically move your complete filestore over to your S3 provider. Artifactory validates each file uploaded to S3 and removes the file from the local storage if the transfer is successful.