We work with a PostgreSQL database (source database) housing large tables, some with multiple millions of records, used for populating a Jasper database (also PostgreSQL). The Jasper server establishes connections with the source database to populate the corresponding Jasper tables. We have SQL scripts that connect to the source database, executing queries to fill the Jasper database. However, from time to time and seemingly randomly, certain loads fail for specific tables, and our logs display the following message: "Parallel load timed out after x ms, stopping the load threads."
The issue doesn't come from the specified timeout duration, as under normal circumstances, the query takes less than 1 minute to execute and retrieve the data, which is way below the defined timeout.
We execute multiple loads concurrently, in a single dataload. We tried to alter the order of loads executed in the same dataload, this didn't change the result. Loading problematic tables each one in a separate dataload shows fewer timeout errors, without solving entirely the problem. Also, we optimize queries for enhanced performance, still the random timeout persists.
Have any of you encountered a similar situation? If so, how did you address and resolve this "Parallel load timed out" error? Any insights or recommendations would be greatly appreciated.
Thank you very much.