I like to know the different types of approaches followed in setting the
values for context variables ,
DB connection information is configured
and how the migration of code is done between different environments?
Reason why asking is, in one project and I made most of the configurable values as context variables and driven the context load tContextLoad.
In my recent project which is using different approach,
DB connections are configured in Metadata section and they are used across all the jobs.
Value for the Context variables are defined in the group (no file load ,db load mechanism).
In this approach, when support team move the code from Dev to test environment they manually open each and every job refresh connection details and convert variables group (dev to test).
My question is whether the same kind of behavior is still observed in the recent versions??
The best way of working with context variables is to have only ONE context (Default) and then load them for every job. Either load them from a file from a standard location using a tContextLoad (the easiest way) or use the implicit context load. You can also load them from a database with the Implicit Context Load functionality. You can also make use of some real dynamic and easy to configure-by-server solutions by making use of operating system environment variables and the implicit context load. This is possibly one of the most powerful methods and means you can literally compile a job and rely on the server configuration to control it. This means whatever server your job is run on will dictate its context variables. I talk about this briefly here https://community.talend.com/t5/Design-and-Development/Dynamic-context-variable-when-promoting-jobs-...
... and here....
I agree richard @rhall_2_0 . When i was working Open Studio , I followed the same approach. This system which now i am working is old system making use of context groups (different context environments DEV, TEST 1 , TEST 2, PREPROD ,PROD)
You are saying having one context environment is effective way then purpose of having that different context environment option context subject to questioning ?
You can do what I have suggested with Context Groups. In reality Context Groups just enable grouping for ease of use. They do not change how the Context variables work terrifically.
Having multiple Contexts (DEV, TEST, PROD, etc) is useful from a development perspective. A developer can easily switch to try things out, debug, etc. But it causes many issues. If you use a tRunJob and do not transmit your context variables automatically, you can be stuck with mix of Contexts. This can be catastrophic if PROD is involved. Which Context do you default to? When you configure the TAC you need to ensure you have set the correct Context. There is too much to go wrong. It can also lead to having to change your code between DEV and TEST, which Talend have done a lot to try and move away from (with Nexus, etc). The Contexts should be retired in my opinion and you should only be left with one, Default.
Using the mechanism I have described, a developer can build once and completely forget about Context Variables apart from where he/she needs to use them. If they are driven by operating system environment variables (pointing to a file or a database) in the Implicit Context Load config, they simply need to ensure their development machine has the appropriate OS environment variable settings and they are done. Once the job is compiled, Contexts do not matter. It's all handled by the server the job will run on. I urge all my clients to use a variant of this because it really does save a lot of time and effort.
Yes I agree. We are planing to migration from 5.2.x to 6.X version. If I move the code with same design logic i.e. having context group with multiple environments and then again we need to manually interact to refresh contexts for each environment.
@rhall_2_0 : Do we have any talend documentation stating your approach as best practice..
I do not believe there is any documentation of this approach as a "best practice" from Talend. I stumbled upon using operating system environment variables and retrieving them with code in the Implicit Context Load config a while ago. I had a client who wanted their context variables stored in a database (to be loaded implicitly), but wanted the database configuration settings stored in a file and obfuscated/encrypted so that a user finding the file on the server could not easily use the credentials. I ended up using JASYPT to encrypt the flat file data (used to hold the Implicit Context Load settings) and as such needed to be able to run Java methods between calling the Implicit Context Load parameters and the values being used. It turned out that these parameters allow for calling Java methods.
I shall write up this method as a blog/tutorial when I get the chance and see if Talend want to publish it.
Try Talend Cloud free for 30 days.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.