One Star

Context_param in properties file for execution in commandLine

Hello,
I have a job with 2 parameters.
Actually, I execute the job by doing this :
jobname.bat --context_param param1=C:\TalendFiles\test.csv --context_param param2=toto
Is there possible to make Talend replace context parameters automatically with properties file?
For example , I make a properties file with my two parameters in and I give the path to the properties file in the commandLine?
example :
jobname.bat --context_properties C:\TalendFiles\job.properties
Do you know of it's possible?
Thank's for advance.
14 REPLIES
One Star

Re: Context_param in properties file for execution in commandLine

Hi,
Try
jobname.bat --context_param params_file=C:\TalendFiles\job.properties
One Star pti
One Star

Re: Context_param in properties file for execution in commandLine

jobname.bat --context_param params_file=C:\TalendFiles\job.properties

With this syntax, do you have to use a tContextLoad component on params_file inside your job, or does Talend load the file automatically?
Thanks
One Star pti
One Star

Re: Context_param in properties file for execution in commandLine

I think I have found the answer to my own question - I will describe it here in case it is useful to others. There are two possible solutions. In either case, you must create a context variable called params_file in your job (note that the name params_file is arbitrarily chosen).
One solution is to use a tContextLoad in each job. Place a tFileInputDelimited component that reads from the named in context.params_file, then connect it to a tContextLoad component.
An easier solution is to use the Implicit Context Load option in the job settings or project. To do this, go to the job properties view when you have the job open, and click the "Extra" tab. Check the "Implicit tContextLoad" option, and choose "From File". Then in the file path, you can enter context.params_file.
If you want to load the same context for many jobs, you can use the Project Settings to set the implicit context load settings jobs. To do this, go to File -> Edit Project Properties, and Choose Job Settings -> Implicit Context Load. Configure the settings here as above. Then, go to the Job Settings -> Use Project Settings and select all the jobs that you want to have these settings. When you click OK or Apply, the settings will be applied to all of the jobs.
This appears to work for me. Anyone should free to correct me if I am mistaken on any point.

Re: Context_param in properties file for execution in commandLine

Hi
thanks for sharing your solutions.
Do you know if one can use relative path when specifying location for the params file?
Here's why I ask you that:
absolute project folder position might be different in development environment, test env., production env.
very likely, the params file will be stored somewhere under the project folder.
so the absolute position for this file will change in different environment, but its relative position could be kept the same.
funny enough, one of the parameters to store in the file would be the current absolute project root path.
Cheers
Giuseppe
One Star pti
One Star

Re: Context_param in properties file for execution in commandLine

Hi Giuseppe,
You can indeed use a relative file path. The file name that you set in the implicit context load settings will be relative to the current directory when the job runs. The batch/shell scripts that are generated to run a job are set up to change to the job's root directory, so that will always be the current directory unless you do something different. For example, I run the job with the parameter:
--context_param context_file=../context.properties

and the context.properties file is located in the directory above all the job folders.
Hope this helps!

Re: Context_param in properties file for execution in commandLine

A-ha. Thanks for your feedback.
It looks to me as what you're saying is strictly related to the scenario (so to say): <<run job from exported script and passing context variable value on command line>>
What about instead the scenario: <<run job from within IDE and setting context variable value in IDE dialog window" ?
Regards
One Star pti
One Star

Re: Context_param in properties file for execution in commandLine

For running the job within the IDE, I don't know what the working directory is - it's probably somewhere in your workspace. I just used an absolute path.

Re: Context_param in properties file for execution in commandLine

For that I figured it out:
it's the TOS executable directory (i.e. somewhere under C:\program files (x86)\ ... \TOSvXYZ ... \bin ...).
I was just wondering if you could specify in the IDE some path relative to project position, but it looks like the answer is no.
As you say, in the IDE (as in development environment) I might use an absolute path for the context variable, and then in test and production environment I can set the same variable from command line.
Ok.
Thanks for all explanations.
Cheers
Giuseppe
One Star

Re: Context_param in properties file for execution in commandLine

I just wanted to say this is a really useful thread. I've never been happy about defining context values directly within the jobs (especially for Production), so thank you, pti, for coming up with a better way of doing this.

Re: Context_param in properties file for execution in commandLine

not sure if it is what you are looking for
i use a OS variable to keept the location of my properties file
set the job to load implicity context from the project menu (file > edit project settings > context loading > )
and i enter System.getenv("TALEND_INITIAL_CONTEXT") - System.getenv works on both windows and linux.
Seven Stars

Re: Context_param in properties file for execution in commandLine

If you want to load the same context for many jobs, you can use the Project Settings to set the implicit context load settings jobs.

If all jobs have implicit context load so they can be run individually using context variables from a file but there is also a master job with implicit context load that calls all the other individual jobs, does the context file get read by each child job or just once for the master?

Re: Context_param in properties file for execution in commandLine

hi alevy
I think there are many other threads in forum related to what you're asking (something about passing context from parent job to child job), so you might be better looking there. Anyway, the answer should be "yes to the second": if every job (parent and child) has the implicit context load set to on, then the context file is read by each child job and by the parent job too.
ciao
One Star

Re: Context_param in properties file for execution in commandLine

Tks pti for the properties file explanation.
One Star

Re: Context_param in properties file for execution in commandLine

Hi,
I am not able to access Hbase.It is showing 
: org.apache.hadoop.hbase.client.ZooKeeperRegistry - Can't retrieve clusterId from Zookeeper
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
I am not  able to figure out the hbase server and port.
Please help me on this.