Data Masking - 2. Dynamically selecting the column to be Masked

Highlighted
Five Stars

Data Masking - 2. Dynamically selecting the column to be Masked

 I am using  tDataMasking component to mask the input columns of a delimited fie. My requirement here is to mask 1000+ files, each with different schema, using Talend job which will identify the column to be masked dynamically for each file. In other words, I don’t want to select the column to be masked  from tDatamasking dropdown for each file. Please let me know if we can achieve this  using tDataMasking or other Talend components. 

Highlighted
Employee

Re: Data Masking - 2. Dynamically selecting the column to be Masked

Hi,

 

    You will have to use Dynamic schema in this case. You can use a metadata table where you can specify the requirement to mask the data using a flag value. During the fetching of metadata, if the flag value is true, you can mask the data using custom code. An example to do dynamic masking can be explored from below blog.

 

https://www.talend.com/blog/2019/10/31/magically-shuffle-columns-talend-dynamic-schema/

 

    You will have to modify the flow by adding the masking requirements to it. 

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Highlighted
Five Stars

Re: Data Masking - 2. Dynamically selecting the column to be Masked

Thank you Nikhil. This post is really helpful and I can now make use of dynamic schema functionality to read various different files.

 

However, I need further help with Masking step. I wanted to know which component can be used now for masking. The reason I am not picking tDataMasking is due to the column name selection through dropdown. 

 

Is there any java function which can be used inside tJavaRow to perform static and bijective masking?

Highlighted
Employee

Re: Data Masking - 2. Dynamically selecting the column to be Masked

Hi,

 

   Since you are planning for dynamic data masking, tDataMasking will not be the right component for you. stackoverflow will be a good place to get java custom code for data masking.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog