Long running job issue with Talend 6.2 enterprise edition/BigData
Hi, I am running a big data batch job(Map reduce) job with 58 million records, which is taking 5 hours to complete. There are few joins. The one thing I noticed that for every row, it is writing like below. So it is taking lot of time to complete. Running job: job_1483713701028_0784 map 0% reduce 0% 1|1|0.0|0.0 <== This is getting printed in log, how to block this from printing??? 1|1|0.0|0.0 1|1|0.0|0.0 1|1|0.0|0.0 1|1|0.0|0.0 1|1|0.0|0.0 Please note that I am reading a parquet file. I filtered the records and try to display only one record in the console, even then I am getting the above number of displays of "1|1|0.0|0.0" I have increased Map and reduce memory to 5gb and 10GB. But not much improvement in reduction in the run time. Question: 1) Why is this getting printed? How to fix this? Appreciate your help!!!