Purview advanced resource sets pattern not matching

103 Views Asked by At

I'm attempting to correctly group IoT ingested data in my Data Lake with Purview.

An example path is:

/100/101/IOT00002/2023/07/14/IOT00002-Received-2023-07-14T10:15:12.3310000Z-MID-568536ffd9c84797a4ca42d9b8b69802.json

Purview, by default, will create a resource set per folder with a device name in it. (IOT00002).

I've enabled Pattern Rules and loaded a match I believe will match (and applied it to the correct data source), on fresh rescan it doesn't apply the pattern.

What is wrong with my pattern when compared to the above path?

{mid:int}/{dt:int}/{folder:string}/{year:int}/{month:int}/{day:int}/{deviceid:string}-Received-{timestamp:timestamp}-MID-{messageid:guid}.json

enter image description here

1

There are 1 best solutions below

0
Damo On

The issue was the timezone format.

2023-07-14T10:15:12.3310000Z

It need to be compatible with https://learn.microsoft.com/en-gb/azure/purview/how-to-resource-set-pattern-rules#syntax