How to create generic job in Talend

One Star

How to create generic job in Talend

Hi

I am new to Talend and I have a requirement to create a generic job which will process multiple tables data.
All the jobs will have the same pattern.

Oracle Table --> tmap(where I will write some DQ functions like is_null, business values check) --> output file

Each of the oracle tables will be downloaded at a different time based on the upstream job dependency.
Can you suggest how this can be achieved?

Thanks
Sriram
Four Stars

Re: How to create generic job in Talend

Talend works with the help of metadata... If your metadata is changing for every table, then you can't make it generic...

Try providing two scenario with sample metadata or data of input and what to achieve?

generic job could be a sample job implemented and you make it as a template... You make this job as benchmark for the development of other jobs and then create a duplicate copy of this job to use it for different table...

Vaibhav
Moderator

Re: How to create generic job in Talend

Hi,

It seems you want to measure the quality of oracle table data? Have you checked talend Data Quality Products?
http://www.talend.com/products/data-quality?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: How to create generic job in Talend

xdshi wrote:
Hi,

It seems you want to measure the quality of oracle table data? Have you checked talend Data Quality Products?
http://www.talend.com/products/data-quality?

Best regards
Sabrina

Hi Sabrina

Thanks for the reply. I am not looking at measuring the quality of the Oracle table data. Our data comes from a web interface and it is bound to have lots of data quality issues as the front end does not check them tightly.

We just want to filter out all the records in a generic way based on rules on particular fields from N tables.

Our oracle DB for this subject area has 40+ tables.

We just want to design a generic flow like how we can do a parameterised generic graph in AB Initio.

Input table ---> Business or DQ rules as per requirements ---> Land the file

Is there any way to write the DQ or business rules in a file and then activate it based on the input table passed?

Thanks
Sriram
One Star

Re: How to create generic job in Talend

Any thoughts?
Five Stars

Re: How to create generic job in Talend

Is there any solution to it? Do we need to create that many jobs to unload all the tables?

Sriram how did you achieve this?

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog