Java java.lang.OutOfMemoryError Error

Four Stars

Java java.lang.OutOfMemoryError Error

I'm getting the following exception while running my job. The job is simple... it takes two inputs (from tPostgreSQLInput components) through a tMap. The result is stored as a delimted TXT file. Where should I be looking to rectify this? Our admin has stated that he has already given the Java VM the max allocation of memory.
### Job STARTED at 2009/01/22 12:13:25 (jobId=lrpusmu3, jobExecutionId=2hSY6uaM) ###
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.String.replace(String.java:1879)
at java.net.URLClassLoader$1.run(URLClassLoader.java:191)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:299)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1309)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:192)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:451)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:336)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:235)
at dataextraction.partmaster_0_2.PartMaster.tPostgresqlInput_1Process(PartMaster.java:3361)
at dataextraction.partmaster_0_2.PartMaster.runJobInTOS(PartMaster.java:5611)
at dataextraction.partmaster_0_2.PartMaster.main(PartMaster.java:5491)
### Job ENDED WITH ERROR at 2009/01/22 12:14:51 (jobId=lrpusmu3, jobExecutionId=2hSY6uaM) ###

Re: Java java.lang.OutOfMemoryError Error

you might try using the "Enable stream" option on your main row to your tmap. This can be found under "Advanced settings".
Lookup tables will need to be loaded into memory.

edit for clarification.
Four Stars

Re: Java java.lang.OutOfMemoryError Error

I may be missing something, but my MAIN row tPostgreSQLInput component only has the settings: "Use Cursor", "Trim all the String/Char columns", and "tStatCatcher Statistics" options under Advanced Options. I'm using version 2.4.1. Was the Enable Stream option introduced after that?
Employee

Re: Java java.lang.OutOfMemoryError Error

"Enable stream" is the MySQL equivalent of Postgres "Enable Cursor"
Checking this option in your component should help you to reduce memory consuption on the main input.
This is not the only solution but it can be done as a first try.

Inyour lookup do you load all the columns of your table or did you select only the really required columns ?
HTH,
Four Stars

Re: Java java.lang.OutOfMemoryError Error

Ok. On my MAIN row tPostgreSQLInput, I checked the "Use Cursor" option and left the value at "1000". I'm still getting the same exception.
Four Stars

Re: Java java.lang.OutOfMemoryError Error

What would the other things I can try to eliminate this exception? Here's what we've done thusfar:
1) Allocated 2GB of memory to the Java VM
2) Checked the "Use Cursor" option (with a setting of 10,000) on the largest database input table (which was the MAIN row into the tMap)
Thanks.
One Star

Re: Java java.lang.OutOfMemoryError Error

What's the settings in your .ini file?
-vmargs
-Xms40m
-Xmx500m
-XX:MaxPermSize=128m
Bluesin
Four Stars

Re: Java java.lang.OutOfMemoryError Error

The settings are exactly as you show...
-vmargs
-Xms40m
-Xmx500m
-XX:MaxPermSize=128m
The file is TISPE-linux-gtk-x86.ini
Thanks.

Re: Java java.lang.OutOfMemoryError Error

you might try tweaking the Xmx500m value to a little higher.
Also, removing unused columns from the schema can help reduce memory consumption.
Four Stars

Re: Java java.lang.OutOfMemoryError Error

It looks like bumping the config and specifying the exact columns I needed (instead of SELECT *) did the trick. Thanks.