Getting Data Lake Metadata in Azure Data Factory's Data Flow

550 Views Asked by At

I want to add the timestamp of copying parquet files to my dataframe in data flow as a derived column.
In source module I can filter parquet files by last modified which makes me think that it should be possible to access files' metadata including copied timestamp through derived column transformations, but I couldn't find anything for it in Microsoft documentation.
enter image description here

1

There are 1 best solutions below

1
On BEST ANSWER

There is no function can get the last modified time in the data flow expression.

As a workaround, you can create a Get Metadata activity to get that and then pass it's value to a parameter in your data flow.

enter image description here

The expression:@activity('Get Metadata1').output.lastModified

enter image description here