I have created a job in Talend to capture the data from Salesforce and putting it in the snowflake as a target. For this I have followed the below architecture.
tsaleforceinputComponent -----> tsnowflakebulkoutput -----> tsnowflakeBulkExec
I am using the query mode in tsaleforcecomponent and trying to capture the Opportunity Table data. The table is having around 4 million records. The current job first copies the data into the azure blob as staging area using tsnowflakebulkloutput and then it is copied to snowflake using tsnowflkaebulkexec component.
The job is executed at remote server with below config:
RAM: 16 GB
vCore: 4
The current time taken by this is around 11 hours.
Is there any way to enhance the performance to either in job or in remote server to take less time as the other tools in the market is taking very less time.
Please let me know if some more details are required. Thanks