I want to create a tracking table job which output data to mysql db, which are having following columns
1. datetime (when job ran)
2. number of rows in column 'email' (from source db job1)
3. number of rows in column 'phone' (from source db job1)
4. number of accepted rows in column 'email Accepted' (from job3)
5. number of rejected rows in column 'emailRejacted' (from job3)
6. number of accepted rows in column 'phone Accepted' (from job3)
7. number of rejected rows in column 'phone Rejacted' (from job3)
8. total time taken in 'time finished' to finish the job
there are few criteria on which email and phone number are accepted/rejected in csv files.
i thought of reusing the jobs, is there any best possible way.
This is a VERY good practice which I use all of the time. You create a suite of jobs like that (to do different things) and include them in each of your jobs to serve a business use case. This allows things like logging (as described by you), error handling and many other requirements, to handled easily.
One thing I would suggest is to split the job you describe in your post into working in 2 ways (Start and End). The values can be supplied as parameters (context variables within the job). So at the beginning of your main job, you supply that child job with a "START" parameter. The child job would then create a record in your table with a start time, etc, etc. Then (using a tPostJob in your main job) connect a version of the same child job at the end supplying an "END" parameter. It will then supply an end time and any other metrics you want to provide it.
If you base all of your job designs around some logic like that, you can pretty much forget about logging and other stuff like that while just focusing on building your business logic.
Join us at the Community Lounge.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Learn how media organizations have achieved success with Data Integration
Create systems and workflow to manage clean data ingestion and data transformation.