JFrog Xray is very slow on the initial DB sync

Question

I'm trying to execute an initial DB sync for Xray to an external (Azure) PostgreSQL database.

It ran acceptedly for about an1.5 hrs., and then froze up at 90% (something to do with IOPS limits on Azure PG Database) after pushing about 50 G.B into the database.

So, I aborted it and restarted the server, and doubled the IOPS allocation.

But now, it runs VERY slow, apparently started from the beginning, although the Database size is still at around 50 GB, and I see something, that looks like failure messages in console.log.

"[jfxr ] [WARN ] [job_service:174] [main] Node is offline or does not exist, cannot be job manager".

Does anyone know what that is?

Answer 1

The pattern appears to be indicating exhaustion of the Xray resources (especially on the database's end). It is recommended to have the minimum requirements as described here for the Xray and its database. From the attached link, check the requirements section for Xray that detailly describes the minimum resource requirements based on the size of the environments.

Answer 2

This is usually happening when the disk for the Database is limited. Especially on IOPS. The disk you have for the database is very small compared to the requirements. The bare minimum installation requires:

Xray and DB: 500 GB (SSD, 3000 IOPS)

My recommendation is to provide a larger and faster, with less IOPs limitations for the disk.

https://www.jfrog.com/confluence/display/JFROG/System+Requirements#SystemRequirements-RequirementsMatrix