In apache nifi can getsftp be a subsequent processor

Question

I have a file containing a bunch of folder names in it. After using getfile to get the data from the file, I'm splitting the data into strings in different queue and each queue contains the name of the folder in the attribute of flowfile.

Now I want to pass those queues to getsftp processor where I'll use $FolderName attribute in the Remote Path property of the processor to fetch data based on it.

Here the problem is that when I'm trying to connect any queue's success or failure to getsftp processor, it is not allowing me to connect to it. Here I researched and came to know that getsftp can't be subsequent processor of any other processor and only it can be parent of processors.

Is there any way to make getsftp a subsequent processor of any other processor? If no then is there any way where I can store the values of queues and then use that values in the Remote Path property of the processor to fetch data based on it?

Answer 1

No processor in NiFi can read a SFTP $foldername dynamically from a flow-file attribute and read all the file contents under the folder. You have to do some workaround - I have listed 2 workarounds as 2 separate answers.
I always use the combination of ListSFTP and FetchSFTP(which accepts an incoming connection). For your use case (you don't need ListSFTP), just connect the queue from your screenshot to the FetchSFTP processor and you should be good to go.
Note: FetchSFTP can only read contents of an SFTP file. So this will only work if you know the SFTP $path and $filename variables (i.e., complete name of the SFTP file to be read) I have explained the usage of ListSFTP and FetchSFTP processors as a separate answer.

Answer 2

If you want to read all the files from an SFTP folder, use ListSFTP processor (specify your $folderName in the Remote Path property that currently has a .). This will return you one flowFile for every file under the folder (flowFile will have attributes ${path} and ${filename}). Use these attributes in FetchSFTP processor

Here is the FetchSFTP processor properties you were interested

This is our current PROD setup (which is working fine for years)