Five Stars rm
Five Stars

Postgres Cursor Size calculation

Data was read from postgres & written to file. Table size is 1.8GB with 1050000 records and has columns around 125. 
>>Assigned JVM as -Xms256M -Xmx1024M. Job failed due to out of memory. Postgres is keeps all the result set in physical memory until the query completes. So entire JVM was occupied and getting out of memory issue. Please correct if understanding is wrong.
>>Enabled Cursor option and kept the value as 100000 and JVM as -Xms256M -Xmx1024M. Job failed with java.lang.OutOfMemoryError: Java heap space
   I don't understand the concept here. Cursor used here denotes the fetch size of rows. In my case 100000 was set. So 100000 will be fetched and stored in physical memory and it will be pushed to file. Then occupied memory will be released and next batch will be fetched. Please correct if i'm wrong.
  Considering my case, with 1050000 records it occupies 1.8GB. Each record occupies 1.8KB of size. 100000*1.8=180000KB. So entire size is just 175MB. Why job is not running with 1GB JVM? Someone please help me with, how this process works?
>>Some record got dropped after setting the cursor option. Cannot trace the problem in that.
Five Stars rm
Five Stars

Re: Postgres Cursor Size calculation

Some thoughts will be helpful.