Until Activity in Azure Data Factory executes even when the condition doesn't match

Question

I have used an until activity to copy files in a sequential order. For this I have created a variable and assigned the value 0 and compared it against the number of files available in the data lake. The expression used in the until activity checks if the variable values is greater than the number of files, till the variable value becomes greater than the count of files, the loop executes. If no files found in the data lake then the variable value would be set to 1 and the the count of files would be 0. But what happens is , even if the value of variable is 1 and count of files is zero the until activity activity tries to execute the inner activities. Any solution for this? The condition activity used to set the variable value: @equals(activity(lookup).output.firstrow.NumberOfFiles,0) then set variable var1 =1 else 0 the expression used in until activity : @greater(int(variables('var1')),activity(lookup).output.firstrow.NumberOfFiles) Inside the until activity: Set the variable to increment the var1 value by 1

If any suggestions, would be really helpful

Regards, Sandeep

Answer 1

According to Microsoft documentation,the Until activity provides the same functionality that a do-until looping structure provides in programming languages.

It will first execute inner activities one time ,and then according to the result of expression to decide to loop or break.

So in your situation,your inner activities will execute once and then get the true value and break the until activity.

If your don't want to execute once,you can use another If Condition activity and let var1 = 1 don't execute the util activity.

Answer 2

If no files found in the data lake then the variable value would be set to 1 and the the count of files would be 0.

the expression used in until activity : @greater(int(variables('var1')),activity(lookup).output.firstrow.NumberOfFiles)

The Until activity is working as intended.

The expression used in the until activity checks if the variable values is greater than the number of files, till the variable value becomes greater than the count of files, the loop executes.

Unfortunately, your assumption about how it works is incorrect. The Until loop will always run at least once. At the end of each iteration it will then test its condition to see if it should run again. It does not check the condition first, as you described.

The fix is to use a ForEach activity instead of an Until activity. A ForEach will always run the number of times you need, and it saves you from having to track the loop condition with variable manipulation.

Answer 3

I wanted to provide a better solution here should the goal be "Do not execute the Until activity even once".

If you want to avoid executing ANY of the inner activities of your Until activity, then you want to have an IF activity -inside your Until activity-.

Then you put all of the activities you expected to run inside of that IF activity. So you get something like...

varOuterUntilSkipCondition = true; //if true, skip Until activities.
varMyCounter = 0;

Until -> condition: loop until varMyCounter = 20

    Until Inner Activity

        If varOuterUntilSkipCondition = false Then execute TRUE activities.
            -- execute your main process activities.
            -- set varMyCounter += 1; //proceed till we hit 20 and exit Until.

        If varOuterUntilSkipCondition = true Then  execute FALSE activities.
            -- set varMyCounter = 20; //to exit the Until entirely without executing
                                        any activities.

So you end up with...

• An Outer Variable controlling if you execute your Until at all.
• An Until to loop through the activities needed.
• An If condition inside of the Until to put all activities into, thus controlling any execution of activities, preventing one or more executions.
• A true side of the IF condition that executes your processes activities.
• A false side of the IF condition that sets the Until activities exit requirement, there by skipping all process oriented activity execution.

Hope that helps as it made me scratch my head a bit... Kind of a wonky way to use an "if" statement to manage a loop but, meh. It works.

Wish I had screen shots but I only had time to do a write up. Let me know if ya have questions.