GC Overhead limit exceeded on server

Seven Stars

GC Overhead limit exceeded on server

Hi,

I am loading around 7.5 million records from a db and after transformation the size of the records doubles to 15 million. But during execution my job got failed with Error "GC Overhead Limit exceeded on server". I am putting xmx - 10240 MB and xms - 1024 MB. Please find my job design below.

 

tmssqlinput-->tJavaRow-->tExtractJsonFields --> tMap --> tDenormalize --> tJavaRow --> tMSSQLOutput

 

At tExtractJSONFields data is getting double and at tDenormalize I am merging 2 records to One. I am getting this error at tDenormalize. Is there any better solution to this flow because tDenormalize is holding complete and then passing it one by one.

 

 

Best Regards,

 

Abhishek

 

Moderator

Re: GC Overhead limit exceeded on server

Hello,

Could you please try with 4096 or bigger if You have this memory free on Talend Machine?

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: GC Overhead limit exceeded on server

Hi xdishi,
Xmx is already 10240 MB, almost twice as that of 4096.
Or should I increase the xms value ? But I think increasing the memory is not a permanent solution because in our case records count can increase to 40 - 50 millions. In that case we need to again increase the memory.
What I am looking for is a permanent solution. Can we process the records in batches. I mean instead of reading all the records in one go, can we do it in batches.

Best Regards,

Abhishek