The Definitive Guide to Data Quality
- the tStatCatcher component does not log the number of rows processed
- the tStatCatcher component does not log actual time spent in a component, just the total wallclock time from start to finish (which makes it much harder to debug which component needs optimization)
Starting job topic5996 at 14:00 31/03/2009.
tPerlFlex_1 duration 770 ms (0 seconds), 100000 runs, average : 7 microseconds, min : 6 microseconds, max: 1796 microseconds, speed: 129870 rows/second
tMysqlOutput_1 10721 ms (10 seconds), 100000 runs, average : 107 microseconds, min : 95 microseconds, max: 10112 microseconds, speed: 9327 rows/second
execution time: 28309 milliseconds
Job topic5996 ended at 14:00 31/03/2009.
Starting job topic5996 at 14:10 31/03/2009.
tPerlFlex_1 duration 705 ms (0 seconds), 100000 runs, average : 7 microseconds, min : 5 microseconds, max: 3548 microseconds, speed: 141843 rows/second
tMysqlOutput_1 1877 ms (1 second), 100000 runs, average : 18 microseconds, min : 6 microseconds, max: 13739 microseconds, speed: 53276 rows/second
execution time: 18152 milliseconds
Job topic5996 ended at 14:11 31/03/2009.
- the tStatCatcher component does not allow the timestamp in UTC
- And one more thing - do you think it would be a good idea to modify tStatCatcher to be able to "poll" data after certain amount of time and give the status of the job? Currently if some long-running component starts - there is no way to see the progress of the job. This may be a configurable parameter in IDE.
That's right, this is the tFlowMeter + tFlowMeterCatcher job