I have created a data flow in Data Factory. Step 1. Read the parquet file. Step 2. Aggregate the file to get the Max(DateField) Step 3. Use a derived column to write in a Value. Step 4. Alter row task with Value and the DateField. Step 5. Sink select the Watermark table to update.
The flow updates the value, but it isn't putting in the max value. The date value is incorrect. Any ideas? Flow_image
max()aggregate function doesn't work on date/string format type. You must pass any column which contains numerical values. Date is not a valid input on which you can apply max function. There is no maximum date term.Instead you can filter the timestamp and get the latest or oldest date using ADF.
Refer this answer by @Leon to know how to implement the same.