reuse job or other option to get data from multiple jobs? plz help

Seven Stars

reuse job or other option to get data from multiple jobs? plz help

how to reuse job in other job

I want to create a tracking table job which output data to mysql db,  which are having following columns 

1. datetime (when job ran)

2. number of rows in column 'email' (from source db job1)

3. number of rows in column 'phone' (from source db job1)

4. number of accepted rows in column 'email Accepted' (from job3)

5. number of rejected rows in column 'emailRejacted' (from job3)

6. number of accepted rows in column 'phone Accepted' (from job3)

7. number of rejected rows in column 'phone Rejacted' (from job3)

8. total time taken in 'time finished' to finish the job 

 

there are few criteria on which email and phone number are accepted/rejected in csv files.

i thought of reusing the jobs, is there any best possible way.

Community Manager

Re: reuse job or other option to get data from multiple jobs? plz help

This is a VERY good practice which I use all of the time. You create a suite of jobs like that (to do different things) and include them in each of your jobs to serve a business use case. This allows things like logging (as described by you), error handling and many other requirements, to handled easily. 

 

One thing I would suggest is to split the job you describe in your post into working in 2 ways (Start and End). The values can be supplied as parameters (context variables within the job). So at the beginning of your main job, you supply that child job with a "START" parameter. The child job would then create a record in your table with a start time, etc, etc. Then (using a tPostJob in your main job) connect a version of the same child job at the end supplying an "END" parameter. It will then supply an end time and any other metrics you want to provide it.

If you base all of your job designs around some logic like that, you can pretty much forget about logging and other stuff like that while just focusing on building your business logic. 

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download