I want to create a reuseable component or use any existing component and capture details required to populate logs table. Details include source name, target name, count of the records, job status, no of columns, date and time of the load etc etc.
Can i use this components at project level, so all the job runs are captured?
Can anyone help me with this.
Talend provides user with Stat&log in project setting and Stat&log in job view.
Stat&log in project setting is working on the whole project (Parent job and child jobs, and output them on file/DB but no context setting).
Stat&log in job view is working on the current job (you should set Stat&log of job view in both parent job and child jobs and output them on file/DB with context setting.
If you want to modify schema of log/stat file, you have to use a single component.
For example: tlogcather-->tMap(modify schema)-->t<xx>output..
For more information, please have a look at this online documentation:
Let us know if it helps.
Thanks for the reply @xdshi But i don't think log or stats catcher will get the source name, target name or count.. correct me if i'm wrong.
The feature of Stat&log in talend have tFlowMeterCatcher and tFlowMeter, which operates as a log function triggered by the use of a tFlowMeter component in the Job.
You will be able to get the actual number of rows being processed(count) from it.
Feel free to let us know if this scenario helps.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema