I want to copy files from one folder to another folder in data lake using ADF pipelines. Ex : a/b/c/d. TO a/b Here a,b,c,d are folders here I don't want to copy c,d folders .I have to copy the files inside those folders to 'b' folder.
I created a pipeline using Get Metadata , For each and in For Each I used copy activity.But here I am able to copy files with folder itself .I'm failing to remove folders.
I reproduced your scenario follow the below steps:
In my demo container I have nested folders like a/b/c/d under d folder I have 3 files as below.
To copy files from folder to folder I took
Get metadata
activity to get list of files from folder. Dataset for Get Metadata: Get Metadata settings:Then I took
for-each
activity and passed the output ofGet Metadata
activities output to it.copy activity
inside for each activity and created source dataset with filename parameter In file name gave dynamic value as@dataset().filename
In copy activity source gave dynamic value for dataset property filename as@item().name
Now created sink dataset witha/b
directories only and passed it to sinkOutput