multiple implicit context loads from multiple files

Six Stars

multiple implicit context loads from multiple files

Hi

 

When you open a project you can set the project level implicit context load to read from a single file.

 

I was wondering if you can be more fine grained than this if you want to keep context values in different files. 

 

Project level implicit context load only allows you to load from one file.

 

I was wondering how implicit context load works when you have reference projects.

 

Can you specify implicit context load in each reference project with different file paths in each reference project?

 

So reference project1 implicit context loads from file1 and reference project2 implicit context loads from file2.

 

Then the parent project could reference both reference projects and implicitly load context variables from both files?

 

Does Talend support this?

 

Is there another way of using multiple files?

 

thanks

 

n


Accepted Solutions
Community Manager

Re: multiple implicit context loads from multiple files

Since passwords are Strings (or can be if you are encrypting them), you can simply extrapolate from the following code as an example of how you would encrypt/decrypt those Strings.

 

import org.jasypt.encryption.pbe.StandardPBEStringEncryptor; 
//Configure encryptor class

StandardPBEStringEncryptor encryptor = new StandardPBEStringEncryptor();

encryptor.setAlgorithm("PBEWithMD5AndDES");

encryptor.setPassword("BOB");




//Set the String to encrypt and print it

String stringToEncrypt = "Hello World";

System.out.println(stringToEncrypt);




//Encrypt the String and store it as the cipher String. Then print it

String cipher = encryptor.encrypt(stringToEncrypt);

System.out.println(cipher);




//Decrypt the String just encrypted and print it out

System.out.println(encryptor.decrypt(cipher));

 The Implicit Context Load will load the Context Variables behind the scenes. To alter the encrypted values, all you need to do is place a tPreJob into your job with a tJava component attached. You can then decrypt there. The rest of the job will start after this has been done.


All Replies
Community Manager

Re: multiple implicit context loads from multiple files

You can be clever here. A while ago I discovered that you can use Routines to configure your Implicit Context Load parameters. What I have done is configure my contexts to be loaded with different values depending upon the machine they are run on. I have set up System Environment Variables to identify the environment, etc, and that tells my ImplicitContextLoad Routine (a routine I wrote) where to get my contexts from. It's actually really simple. To see how it works set up a Routine for yourself. Just configure simple Methods to return hardcoded values for your Implicit Context Load params. Use these Routine Methods in your Implicit Context Load settings. Once you see it working, you can experiment to suit your requirements exactly. This hidden functionality opens A LOT of doors.

Six Stars

Re: multiple implicit context loads from multiple files

Hi 

That sounds interesting thanks for that. Smiley Happy

 

"Use these Routine Methods in your Implicit Context Load settings." - how would I do this?

 

Project level Implicit Context Load settings only allow file or db use?

 

Cheers

n

Highlighted
Community Manager

Re: multiple implicit context loads from multiple files

The parameters that you set for file or db use you populate with your new Routine. For example, my Implicit Context Load config is setup as "From Database". My "Host" parameter has this code....

routines.ImplicitContextLoadUtils.getImplicitContextParameterValue("TalendContextHost",rootPid,pid)

..., my "Db Name" parameter has this code....

routines.ImplicitContextLoadUtils.getImplicitContextParameterValue("TalendContextDbName",rootPid,pid)

,etc. Ignore the actual methods as these are specific to my requirements, but these are methods NOT literal values or context variables I am using. This means that when the job runs, the routine Methods are called first, they return the appropriate values to log in to the database I want the context variables from and then everything else runs.

Six Stars

Re: multiple implicit context loads from multiple files

awesome!

 

I know you could get access to context variables in those settings but being able to call routines is news to me! (Hope that functionality remains Smiley Happy )


Many thanks for that. Will give that a go. Might test the reference project implicit context loads as well to see if that works.

 

Cheers

n

Six Stars

Re: multiple implicit context loads from multiple files

Hi

 

Reference project implicit context loads for reference project do not happen. you can only have one  project implicit context load, the parent project...

 

So I'm looking further at calling routines in those fields.

We want to encrypt our DB context vales and have the select query use a key in the select statement to decrypt these values..(not as good as talend using a key to decrypt the value at run time but anyway...)

 

Can you add routines in the "Table Name" and "Query Condition" implicit context load fields?

 

Cheers

 

Community Manager

Re: multiple implicit context loads from multiple files

You can have routines called in all Implicit Context Load parameters.

 

For encryption take a look at JASYPT  (http://www.jasypt.org/). It allows you to use encrypted values in a file, etc, simply by wrapping by wrapping them in ENC( ) tags. I use this functionality as well.

Six Stars

Re: multiple implicit context loads from multiple files

Hi

 

Still deciding over files vs database and read your excellent post at https://www.talend.com/blog/2019/02/28/best-practices-for-using-context-variables-with-talend-part-3...

 

I am concerned about having to store context values that are credentials in plain text in any db table that the implicit context load reads...At least with a file any sensitive context values sit only on the server.

 

Would be good if the values in the context table could be encrypted/decrypted too Smiley Happy

 

Cheers

 

Nomit

 

 

Community Manager

Re: multiple implicit context loads from multiple files

Hi @n999,

 

I cover this in my second blog on Context Variables. I had meant to write it all into 1 big blog, but thought that it would be better to break it down. The link to the second blog is here https://www.talend.com/blog/2019/02/19/best-practices-for-using-context-variables-with-talend-part-2...

I use JASYPT to encode password fields in text files (it could also be used for database values).

 

I hope this helps.

 

For my other blogs, just click on my name at the top of the blog :-)

Six Stars

Re: multiple implicit context loads from multiple files

Hi - Thanks for responding.

"I use JASYPT to encode password fields in text files (it could also be used for database values)."

I can see how this would work with a text file but not with database values. Talend requires a table with "key" and "value" for the columns and these are selected by a hidden query you can't edit. How would you 

1) either add JASYPT into this select statement or

2) pass the selected values through a routine using JASYPT?

Cheers

 

Community Manager

Re: multiple implicit context loads from multiple files

Since passwords are Strings (or can be if you are encrypting them), you can simply extrapolate from the following code as an example of how you would encrypt/decrypt those Strings.

 

import org.jasypt.encryption.pbe.StandardPBEStringEncryptor; 
//Configure encryptor class

StandardPBEStringEncryptor encryptor = new StandardPBEStringEncryptor();

encryptor.setAlgorithm("PBEWithMD5AndDES");

encryptor.setPassword("BOB");




//Set the String to encrypt and print it

String stringToEncrypt = "Hello World";

System.out.println(stringToEncrypt);




//Encrypt the String and store it as the cipher String. Then print it

String cipher = encryptor.encrypt(stringToEncrypt);

System.out.println(cipher);




//Decrypt the String just encrypted and print it out

System.out.println(encryptor.decrypt(cipher));

 The Implicit Context Load will load the Context Variables behind the scenes. To alter the encrypted values, all you need to do is place a tPreJob into your job with a tJava component attached. You can then decrypt there. The rest of the job will start after this has been done.

Six Stars

Re: multiple implicit context loads from multiple files

Ahh okay - I'd get a handle on the encrypted values in a pre-job.

cheers for all your help!

Community Manager

Re: multiple implicit context loads from multiple files

No problem at all :-)

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend API Designer – Technical Overview

Take a look at this technical overview video of Talend API Designer

Watch Now

Getting Started with APIs

Find out how to get started with APIs

Read