I'm trying to configure some simple jobs as below.
Here I'm trying to fetch the DB connection parameters from the external property files. So for that I've designed the job as below.
But on executing the job it tries to get the connection to the DB even before loading the properties file, due to which the job fails.
I'm using Big Data sandbox 6.3 as it's seen in the screenshot.
A quick help will be much appreciated.
Are you sure the context values are getting loaded properly?
The Job design looks fine. I think there's an issue with fetching the values from the external properties file.
Make sure the context name that you declared and the key column values in the external properties file are exactly matching.
Also you can use the tJava component to print the context value before the MySQL component starts; just to see if the values are what you want it to be.
Thanks for the Response.
Yes. it's picking the proper file. even the values are getting loaded properly. I checked the same by displaying the context variables using tJava component.
The problem is job is trying to make the SQL connection even before the context load. Which should not be as per the job design. But, unfortunately, that's how it's execution order.
I am sure you must have tried this but try to see if you can remove all the links and connect them again. Remove all the twarn and tdie components for now. Just connect the context load subjob to the mysql component and try again.
If that doesn't fix the issue, try to create a new job and copy the subjobs to it. It could be a weird bug of some kind.
Also if it's possible, can you share a screenshot of how you configured your mysql component?
Could figure out the reason.
Actually I was trying to do the error logging at the same DB which was being used for data extraction. For which I referred the context parameters. But Talend tries to get the connection for error (Stat, Logs and Meter) logging even before loading the context parameters from the files.
Now, could someone advise me whether I could use the dynamic data-logging parameters.
Thanks in advanced.
could you try loading the context values by connecting it with a pre-job component?
That might work.
That too does not work. But you can modify the data directly at the properties file within your executable file structure.
Can you show us how your DB connection is configured please? This should be working. Are you sure that your context variables are set correctly? Can you switch off the DB component and just use a System.out call in a tJava in place of the DB component?
Sorry for the very late response.
Actually I found the root cause.
In this job, I used to keep the context parameters empty and used to fill by reading from external file. But I was also using Talend Job level logging (from menu->File->Edit Project Properties->Job Settings->Stat & Logs). This connection was being initiated even before the job could read the external file. And that's why it was failing.
I could resolve this issue by changing the deployment scripts, which will replace the parameters directly at context files.
Thank you all for your valuable inputs.
Ah I see. Another way of getting around this is to use a "wrapper job" where the context variables are set and passed to the job this problem is about. Simply set up a simple job which retrieves the contexts as you wish, then drop this job into it and pass the contexts over. This way the contexts will be set before anything happens in the child job.
Watch the recorded webinar!
Introduction to Talend Open Studio for Data Integration.
Test drive Talend's enterprise products.
Practical steps to developing your data integration strategy.