context management in Talend Server

Highlighted

context management in Talend Server

hi,

i would like to understand the concept of contexts that are used across multiple jobs and projects.

while using the open-source, i have always used to maintain my context values in a DB and load these using a:
flat-files with initial context variables - including the DB details holding the contexts, and
DB tables with every other variable used in an environment.

assuming that i have DEV, TEST, PROD environments.

what is the approach that would make managing context in different environments?

thanks,


Note: how should i refer to Talend server environment?
Community Manager

Re: context management in Talend Server

Hi
We usually set context variable with different value for different contexts, for example, see pic1.
what is the approach that would make managing context in different environments?

It always be a default context, if you don't specify the context, it will use the default context.
If the job is execute within Talend Studio, you can select the context from the list on the right top side of Run viewer. see pic2
If you export the job script, you can specify the context by modifying the job script. For example:
%~d0
cd %~dp0
java -Xms256M -Xmx1024M -cp classpath.jar; talenddemosjava.test_0_1.test --context=dev %*

If you add task on Job Conductor, select the context which you want to deploy from the Context list. see pic3
----------------------------------------------------------
Talend | Data Agility for Modern Business

Re: context management in Talend Server

thanks for that Shong,

how would i use your approach when i have to share a context across Multiples (across) projects?

when using multiple projects, i have always used a DB to maintain context variables for that reason.

a case in point, details for the database connection used to collect Talend Job's logging, statistics, and messages.


but i would like to see if there is an alternative.

regards,


Nicolas
Community Manager

Re: context management in Talend Server

If you are using talend enterprise data integration, you can create a reference project, define the global context in reference project?the general project can refer to the reference project and use the same context variables.
----------------------------------------------------------
Talend | Data Agility for Modern Business

Re: context management in Talend Server

It seems that i would be loading a lot of:
contexts
routines
jobs (templates)
jobs (tasks - used as-is)

in this share project.

i will try it and report back.

THANKS!
One Star

Re: context management in Talend Server

If you use the reference project approach, you need to keep something in mind. Once deployed in job conductor, if a configuration changes, you will have to open the job again in studio to get the new configuration.

Now my experience is mostly with v4.2.3 of TAC/TIS, so this may have changed in v5.x

This approach is convenient for development/deployment, but not for changes. Imagine if you have 100 jobs and a password has changed ...

I do not know what the best approach is, but I'm looking into storing the configuration data in a database (using, say, jasypt to encrypt passwords) and then having a joblet that can be called at the beginning of a job to load the data. This approach has its own problems, such as additional work up front.

Re: context management in Talend Server

i have gone back to my old ways..

as a note - v5 contexts work the same way

decided to avoid Talend context altogether.

my jobs 1 (single) context and depending from the server where they are running they use a different context SET of variables.

this SET changes based on the environment.

i am using a similar setup of having a job to load these parameters at the beginning of the Job (orchestrator) that coordinates calls to other jobs - and then 'passes parameters' to child jobs.
One Star

Re: context management in Talend Server

Or you can put your context variables in a file and load them at the start of the job.

Re: context management in Talend Server

How to use a formula in context group in mapping?
Example : i am having two context parameters Start_Date and End_Date and Formula
              as  Start_Date;2015-05-25
                   End_Date;2015-09-20
                   FormulaTalendDate.diffDate(row5.EDD,row6.start_date,"dd") 
           so i have to use this Formula context variable in tMap !
i can use Formula as string but not returning result from it ! ie; it taking as string but not executing in tMap as a calculation .
Please let me know how to use the calculation which is defined in context to use in tMap or any where in Job ?

Re: context management in Talend Server

if i use context parameters file with one parameter 
 Result = 25/3   
The Result variable i want to use in job where ever i want to ?

context_file -----> contextLoad -----> Source -----> tMap -------->  Target

Result=25/3                                Id  ,Name         Id  ,Name , Result           Id  Name , Result 
                                               101 , ABC      101 , ABC  ,  8                101 , ABC  ,  8
                                               102 , XYZ      102 , XYZ  ,  8                102 , XYZ  ,  8
Community Manager

Re: context management in Talend Server

Hi archana


It treats the formula as a string, so you can't write an expression as a context variable, you should use it directly on tMap. 

About your second question,  what's the data type of Result variable defined in the context view?
----------------------------------------------------------
Talend | Data Agility for Modern Business

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog