multiple implicit context loads from multiple files

Six Stars

multiple implicit context loads from multiple files

Hi

 

When you open a project you can set the project level implicit context load to read from a single file.

 

I was wondering if you can be more fine grained than this if you want to keep context values in different files. 

 

Project level implicit context load only allows you to load from one file.

 

I was wondering how implicit context load works when you have reference projects.

 

Can you specify implicit context load in each reference project with different file paths in each reference project?

 

So reference project1 implicit context loads from file1 and reference project2 implicit context loads from file2.

 

Then the parent project could reference both reference projects and implicitly load context variables from both files?

 

Does Talend support this?

 

Is there another way of using multiple files?

 

thanks

 

n

Sixteen Stars

Re: multiple implicit context loads from multiple files

You can be clever here. A while ago I discovered that you can use Routines to configure your Implicit Context Load parameters. What I have done is configure my contexts to be loaded with different values depending upon the machine they are run on. I have set up System Environment Variables to identify the environment, etc, and that tells my ImplicitContextLoad Routine (a routine I wrote) where to get my contexts from. It's actually really simple. To see how it works set up a Routine for yourself. Just configure simple Methods to return hardcoded values for your Implicit Context Load params. Use these Routine Methods in your Implicit Context Load settings. Once you see it working, you can experiment to suit your requirements exactly. This hidden functionality opens A LOT of doors.

Six Stars

Re: multiple implicit context loads from multiple files

Hi 

That sounds interesting thanks for that. Smiley Happy

 

"Use these Routine Methods in your Implicit Context Load settings." - how would I do this?

 

Project level Implicit Context Load settings only allow file or db use?

 

Cheers

n

Sixteen Stars

Re: multiple implicit context loads from multiple files

The parameters that you set for file or db use you populate with your new Routine. For example, my Implicit Context Load config is setup as "From Database". My "Host" parameter has this code....

routines.ImplicitContextLoadUtils.getImplicitContextParameterValue("TalendContextHost",rootPid,pid)

..., my "Db Name" parameter has this code....

routines.ImplicitContextLoadUtils.getImplicitContextParameterValue("TalendContextDbName",rootPid,pid)

,etc. Ignore the actual methods as these are specific to my requirements, but these are methods NOT literal values or context variables I am using. This means that when the job runs, the routine Methods are called first, they return the appropriate values to log in to the database I want the context variables from and then everything else runs.

Six Stars

Re: multiple implicit context loads from multiple files

awesome!

 

I know you could get access to context variables in those settings but being able to call routines is news to me! (Hope that functionality remains Smiley Happy )


Many thanks for that. Will give that a go. Might test the reference project implicit context loads as well to see if that works.

 

Cheers

n

Six Stars

Re: multiple implicit context loads from multiple files

Hi

 

Reference project implicit context loads for reference project do not happen. you can only have one  project implicit context load, the parent project...

 

So I'm looking further at calling routines in those fields.

We want to encrypt our DB context vales and have the select query use a key in the select statement to decrypt these values..(not as good as talend using a key to decrypt the value at run time but anyway...)

 

Can you add routines in the "Table Name" and "Query Condition" implicit context load fields?

 

Cheers

 

Highlighted
Sixteen Stars

Re: multiple implicit context loads from multiple files

You can have routines called in all Implicit Context Load parameters.

 

For encryption take a look at JASYPT  (http://www.jasypt.org/). It allows you to use encrypted values in a file, etc, simply by wrapping by wrapping them in ENC( ) tags. I use this functionality as well.

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.