slow talend

One Star

slow talend

Hello,
I am using Talend BIg data.But i am facing problem now.Talend has become very slow now. I dont know why it has become like that.
Previously it was very fast.
I am trying to load 10000 rows but it taking 400seconds for that.
Could you please tell me what is the problem behind this???
Regards,
Vinuta
Moderator

Re: slow talend

Hi,
Is it slow just for your specific job or the whole studio performance is worse than before?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: slow talend

Hello,
The studio performance is worse than before.
If I load the job it is coming as GC overhead limit exceeded.
Regards,
Vinuta
Seventeen Stars

Re: slow talend

hi all,
you haven't got enought memory  to do the Job.
GC overhead is about the fact that JVM spent to many time to his garbage collector.
It's due to too many java object created for the RAM available.
Increase memory or (better way) refactoring your job to optimize data management.
regards
laurent
One Star

Re: slow talend

Hi,
can you please tell me how to do that??
Regards,
Vinuta
One Star

Re: slow talend

Hello,
I have installed java environment 1.7.0_71 version for windows 8.1. Can i use this version for Talend Bigdata???
Vinuta
One Star

Re: slow talend

hello,
I have a job that looks like:
tredshiftinput > tMap >> tmysqlOutput

My file has  million rows and I get to  loaded I get the error. What can I do differently to avoid this Java heap space error?
Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space
at java.util.LinkedList.listIterator(Unknown Source)
at java.util.AbstractList.listIterator(Unknown Source)
at java.util.AbstractSequentialList.iterator(Unknown Source)
at routines.system.RunStat.sendMessages(RunStat.java:261)
at routines.system.RunStat.run(RunStat.java:225)
at java.lang.Thread.run(Unknown Source)
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Unknown Source)
at java.lang.Class.getDeclaredField(Unknown Source)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl.<init>(Unknown Source)
at java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(Unknown Source)
at java.sql.SQLException.<clinit>(Unknown Source)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1811)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
disconnected
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:372)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:252)
at project_1.r_m_refr_0_1.r_m_refr.tRedshiftInput_1Process(r_m_refr.java:1046)
at project_1.r_m_refr_0_1.r_m_refr.runJobInTOS(r_m_refr.java:1800)
at project_1.r_m_refr_0_1.r_m_refr.main(r_m_refr.java:1657)
Regards,
Vinuta
Moderator

Re: slow talend

Hi,

Exception in thread "Thread-0" java.lang.OutOfMemoryError: Java heap space


Have you already checked KB articles:https://help.talend.com/pages/viewpage.action?pageId=190513217 and https://community.talend.com/t5/Migration-Configuration-and/OutOfMemory-Exception/ta-p/21669?content... ?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Moderator

Re: slow talend

Hi,
I have installed java environment 1.7.0_71 version for windows 8.1. Can i use this version for Talend Bigdata???

Please check document about: TalendHelpCenter:Compatible Platforms
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: slow talend

Hi all,
I have checked these blogs. For windows 8.1 64 bit operating system oracle java 7 is supported.so, I have installes java 1.7.1_71 version.
But If I try to check the version and incresing memory in command prompt it is coming as java version 1.8 is required. I do not know whether 1.8 is compatible for Talend big data??
What should I do now??Can you please tell me??
Regards,
Vinuta
Moderator

Re: slow talend

Hi,
So far, talend don't support for Java 8.TalendHelpCenter:Java 8 Support
Could you please follow up document TalendHelpCenter:Exception: outOfMemory to allocate more memory for your job?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: slow talend

Hi,
I do not  know about job script. Can I allot 4096M memory for oracle java 7. is that supports??
I am using talend bigdata version 5.6. is that any problem with this version??
Regards,
Vinuta
One Star

Re: slow talend

hello,
I increased the memory also.but i have not done optimization since I am using only one file.
But still its not loading. It ll come as starting but rows are not loading.
I do not know what is the problem.
Talend big data version : 5.6
java version : 1.7.0_71
windows 8.1- 64 bit operating system
Could you please tell me????
Regards,
Vinuta
Moderator

Re: slow talend

Hi,
Is there any error info printed on console? Is it still " java.lang.OutOfMemoryError: Java heap space"? Or the error which is mentioned in your another topic: http://www.talendforge.org/forum/viewtopic.php?id=39802?
Best regards
Sabrina 
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: slow talend

Hi Vinuta,
- Are you using any lookup in tMap? if yes, you can opt for using FileSystem for lookup data storage in place of memory...
- Whether your output DB is on same machine where redshift is available? if not, then network delay could be an issue
- What are other processes are consuming memory when you execute the talend job and how much memory is available for talend?
- What is the batchsize in output,  try tweaking that..
- Have you tried restarting Talend?
Vaibhav