Dynamic metadata properties definition (Using context / environment variables or something else)

Highlighted
Four Stars

Dynamic metadata properties definition (Using context / environment variables or something else)

Hello,

When creating a file metadata (xml, excel, ...) we must define file's path.

My issue is that I work on D drive of my computer while other developers are working on other drives, so metadata file's path properties are different between some developers.

One of the things that matters the most to us is the schema generated by metadata which will be used inside jobs. But metadata properties are also useful because even if we set "Property Type" to "Built-In" of components created using metadata, properties of components will keep properties of the metadata and we'll only have to modify the properties that must be updated (to fit our needs) but the others will stay as they are in the metadata.

So, if someone wants to update metadata, all the properties including path of developer environment should be set dynamically to prevent re-updating all metadata for each different working environments. But how to do that ?

From what I've understand in the following post, it is not possible to do that using context variables: https://community.talend.com/t5/Archive/Using-context-variables-in-metadata/m-p/168922

(But since this post dates from 2008, I prefer to ask the question again).

So, I tried to use environment variables in file's path property, but it doesn't work either.

The question here is not whether it's possible to do it using context or environment variables (I doubt that), but how to do it?


Accepted Solutions
Five Stars

Re: Dynamic metadata properties definition (Using context / environment variables or something else)

Hello,

step1:
create two excels one in d drive(say excel1) and one for c drive(say excel2)
and keep some data in

step2:
Go to create a new connection in metadata(say connection1) and give the connection name,purpose and
excel details

step3:
select export as context -> create a new repository context -> Give new name and purpose for the context
->

step 4:
To add a new context, click the context symbol in the upper right corner.
The window that pops up is used to manage contexts. To create a new context, click New,
enter the name of the context, in our example context2, and click Ok
To rename the Default context, select it, click Edit, enter context1 
and click Ok. When you’re done editing, click Ok.
After the window closes, you’ll see that an extra column appeared. Enter the
connection data of the 2nd environment(other drive) in the context2 column and click Finish.

Note: if unable to edit ,click finish go to job-> go to required input or output components on which u want your context to work -> select context variable in context section
-> select the respective context and say ok -> now go to context, coloumn
and edit the  connection detail
and say finish

step5:
now in components on which u are working in job simply select connection1(by selecting property type
as repository) and selecting connection1

step6:
now in the context tab of the job select the environment context as context1 first
then u can see the operation being performed in environment which was meant for context1
similarly in the drop down switch to context2 and we can see data operation
in context2

 

Plz mark kudos if u find it helpful.


All Replies
Five Stars

Re: Dynamic metadata properties definition (Using context / environment variables or something else)

Hello,

step1:
create two excels one in d drive(say excel1) and one for c drive(say excel2)
and keep some data in

step2:
Go to create a new connection in metadata(say connection1) and give the connection name,purpose and
excel details

step3:
select export as context -> create a new repository context -> Give new name and purpose for the context
->

step 4:
To add a new context, click the context symbol in the upper right corner.
The window that pops up is used to manage contexts. To create a new context, click New,
enter the name of the context, in our example context2, and click Ok
To rename the Default context, select it, click Edit, enter context1 
and click Ok. When you’re done editing, click Ok.
After the window closes, you’ll see that an extra column appeared. Enter the
connection data of the 2nd environment(other drive) in the context2 column and click Finish.

Note: if unable to edit ,click finish go to job-> go to required input or output components on which u want your context to work -> select context variable in context section
-> select the respective context and say ok -> now go to context, coloumn
and edit the  connection detail
and say finish

step5:
now in components on which u are working in job simply select connection1(by selecting property type
as repository) and selecting connection1

step6:
now in the context tab of the job select the environment context as context1 first
then u can see the operation being performed in environment which was meant for context1
similarly in the drop down switch to context2 and we can see data operation
in context2

 

Plz mark kudos if u find it helpful.

Four Stars

Re: Dynamic metadata properties definition (Using context / environment variables or something else)

Thank you very much for these detailed explanations that helped me to solve this problem.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog