The Definitive Guide to Data Quality
- the tStatCatcher component does not log the number of rows processed
- the tStatCatcher component does not log actual time spent in a component, just the total wallclock time from start to finish (which makes it much harder to debug which component needs optimization)
Starting job topic5996 at 14:00 31/03/2009.
tPerlFlex_1 duration 770 ms (0 seconds), 100000 runs, average : 7 microseconds, min : 6 microseconds, max: 1796 microseconds, speed: 129870 rows/second
tMysqlOutput_1 10721 ms (10 seconds), 100000 runs, average : 107 microseconds, min : 95 microseconds, max: 10112 microseconds, speed: 9327 rows/second
execution time: 28309 milliseconds
Job topic5996 ended at 14:00 31/03/2009.
Starting job topic5996 at 14:10 31/03/2009.
tPerlFlex_1 duration 705 ms (0 seconds), 100000 runs, average : 7 microseconds, min : 5 microseconds, max: 3548 microseconds, speed: 141843 rows/second
tMysqlOutput_1 1877 ms (1 second), 100000 runs, average : 18 microseconds, min : 6 microseconds, max: 13739 microseconds, speed: 53276 rows/second
execution time: 18152 milliseconds
Job topic5996 ended at 14:11 31/03/2009.
- the tStatCatcher component does not allow the timestamp in UTC
- And one more thing - do you think it would be a good idea to modify tStatCatcher to be able to "poll" data after certain amount of time and give the status of the job? Currently if some long-running component starts - there is no way to see the progress of the job. This may be a configurable parameter in IDE.
That's right, this is the tFlowMeter + tFlowMeterCatcher job
Try Talend Cloud free for 30 days.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.