How to handle FTE queued transfers

Question

I have fte monitor with a '*.txt' as trigger condition, whenever a text file lands at source fte transfer file to destination, but when 10 files land at source at a time then fte is triggering 10 transfer request simultaneously & all the transfers are getting queued & stuck.

Please suggest how to handle this scenarios

Answer 1

Set your "Batch together the file transfers..." to a value more than 10:

Max Batch Size = 100

Answer 2

Check the box that says "Batch together the file transfers when multiple trigger files are found in one poll interval" (screen three).

Make sure that you set the maxFilesForTransfer in the agent.properties file to a value that is large enough for you, but be careful as this will affect all transfers.

You can also set monitorMaxResourcesInPoll=1 in the agent.properties file. I don't recommend this for 2 reasons: 1) it will affect all monitors 2) it may make it so that you can never catch up on all the files you have to transfer depending on your volume and poll interval.

Answer 3

Ok, I have just tested this case:

I want to transfer four *.xml files from directory right when they appear in that directory. So I have monitor set to *.xml and transfer pattern set to *.xml (see commands bellow).

Created with following commands:

fteCreateTransfer -sa AGENT1 -sm QM.FTE -da AGENT2 -dm QM.FTE -dd c:\\workspace\\FTE_tests\\OUT -de overwrite -sd delete -gt /var/IBM/WMQFTE/config/QM.FTE/FTE_TEST_TRANSFER.xml c:\\workspace\\FTE_tests\\IN\\*.xml

fteCreateMonitor -ma AGENT1 -mn FTE_TEST_TRANSFER -md c:\\workspace\\FTE_tests\\IN -mt /var/IBM/WMQFTE/config/TQM.FTE/FTE_TEST_TRANSFER.xml -tr match,*.xml

I got three different results depending on configuration changes:

1) just as commands are, default agent.properties:

1. in transfer log appeared 4 transfers
2. all 4 transfers tryed to transfer all four XML files
3. 3 of them with partial success because agent could't delete source file
4. with success that transfered all files and deleted all source files

Well, with transfer type File to File, final state is in fact ok - four files in destination directory because the previous file are overwritten. But with File to Queue I got 16 messages in destination queue.

2) fteCreateMonitor command modified with parameter "-bs 100", default agent.properties:

1. in transfer log , there is only one transfer
2. this transfer is with partial success result
3. this transfer tryed to transfer 16 files (each XML four times)
4. agent was not able to delete any file, so source files remained in source directory

So in sum I got same total amount of files transfered (16) as in first result. And not even deleted source files.

3) just as commands are, agent.properties modified with parameter "monitorMaxResourcesInPoll=1":

1. in transfer log , there is only one transfer
2. this transfer is with success result
3. this transfer tryed to transfer four files and succeeded
4. agent was able to delete all source files

So I was able to get expected result only with this settings. But I am still not sure about appropriateness of the monitorMaxResourcesInPoll parameter set to "1".

Therefore for me the answer is: add

monitorMaxResourcesInPoll=1

to agent.properties. But this is in collision with other answers posted here, so I am little bit confused now.

tested on version 7.0.4.4