Salesforce -Fire apex trigger only after complete data load

Question

So here is the issue We are loading data into a CustomObject__c using DataLoader. Usually the no of records that are passed are 3. Also, if there is any issue with the data passed, they run the dataloader again and pass the corrected data. Now, the older data has to be deleted. So, I am handling it in before insert code and calling a batch in after insert code.

Here is the code for my trigger:

trigger TriggerCustom on CustomObject__c (before insert, after insert) {
  List<CustomObject__c> customobjectlist = [Select Id from CustomObject__c WHERE CreatedDate = TODAY ];
    if (Trigger.isBefore) {
        delete exchlisttoday;
        
    }
    if(Trigger.isAfter)
    {
         BatchApex b = BatchApex();    
            Database.executebatch(b);
    }
}

This was designed keeping in mind they pass only 3 records at a time. However, now they want to pass more than 200 records using data loader. How can I modify my trigger so that it fires only after one single dataload is completed (for e.g. if they pass 1000 records at once, the trigger has to fire only after the 1000 records are completely inserted

Answer 1

Trigger will not know when you are done, after 3, 203 or 10000 records (you can use bulk api to load large volumes, they'll be chunked into 10K packets but still - triggers will work 200 at a time).

If you have scripted data load - maybe you can update something else as next step. Another object (something dummy that has just 1 record) and have trigger on this?

If you have scripted data load - maybe you can query the Ids and then pass them to delete operation which would run before the upload task. This becomes bit too much for poor little data loader but Talend, Informatica, Azure Data Factory, Jitterbit etc proper ETL tools could do it. (although deleting before is bit brave... what if the load fails? You're screwed... Maybe delete should be after successful update)

Maybe you can guarantee that last record in your daily load will have some flag set and in the trigger - look for that flag?

Maybe you can schedule the batch to run every hour. You can't do it easily from UI but you can write the cron expression and schedule as 1-liner in dev console. In the Schedulable's execute() make it check if there was anything loaded today and if there was even single record - trigger the batch?