I am building a data flow within azure data factory and I would like to apply some GDPR masking rules within the flow.
What I would like to do is the following: In the Derived Column (or other component) I would like to match my input columns with a reference array and for the columns that matches between my input and reference array I would like to replace/mask those values.
Power point over the data flow and what I would like to do
I have tried some IN and regex functions but I have not gotten it yet. Anyone that know how and if this is possible?
Update: I might have got somewhere with the SELECT component. However, there's something that I don't quite get:
Let's say that I have a data flow parameter called ColumnsToMask of the type string[]. I define the variable content as ['a', 'b']. (a and b are two of my input columns.)
In the SELECT component I add a rule based mapping á: in($ColumnsToMask, name) That don't work for some reason. However, this works: in(['a', 'b'], name)
(By work I mean that I get the matching columns added to my output.)
Anyone knows what I am doing wrong setting my parameter?
Update2.5 Changed the text to a picture in an effort to hopefully explain it a bit better:
How come the evaluated expression works but not the expression itself?
So when I use the evaluated expression everything works like I would like it to but when I try the variable that holds the value it for some reason does not work. What should I change?
Your parameter definition would look something like this: