I have a scenario, where i'm designing a job using "dynamic schema" and calling that child job in multi threads under the parent job in enterprise edition.
To be more clear, parent Job has two trun components, where same child job is called in multi threads. Child job fetch the data from different tables and writes to a file.
I'm stuck with few issues. Kindly help me.
a)Is it best approach to call same job in multi-instance? Or do we have any other best solution?
b)My desired date format in ouput should be "yyyy-MM-dd". So, in dynamic schema, selected date format as "yyyy-MM-dd". But while running the job in multi-instance, i end up seeing the files created with timestamp(yyyy-MM-dd hh24.MM.ss). However, without multi-threads(sequentially), file is created with proper date format(yyyy-MM-dd).
Could you please post your current job setting screenshots on forum which will be helpful for us to understand your work flow?
Parent job where single child job is called with different parameters.
Child job: This job will read the sql file with the help of dynamic schema.
Have you already checked the option "Multi thread execution" option?
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.