One Star

csv file to redshift

Hi,
I am trying to load data from csv file to redshift.
Tfileinputdelimited >> tmap >> tredshiftoutput.
But, I am getting these errors:
Starting job red_mysqltest at 17:19 09/12/2014.
connecting to socket on port 3890
connected
Exception in component tRedshiftOutput_1
org.postgresql.util.PSQLException: An I/O error occured while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:283)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:332)
at project_1.red_mysqltest_0_1.red_mysqltest.tFileInputDelimited_1Process(red_mysqltest.java:1449)
at project_1.red_mysqltest_0_1.red_mysqltest.runJobInTOS(red_mysqltest.java:2247)
at project_1.red_mysqltest_0_1.red_mysqltest.main(red_mysqltest.java:2091)
Caused by: java.io.EOFException
at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:276)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1654)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
... 6 more
disconnected
Job red_mysqltest ended at 17:47 09/12/2014.
Please tell me why these errors are coming??
Regards,
Vinuta
11 REPLIES
Moderator

Re: csv file to redshift

Hi,
Have you already checked if there is something wrong with network bandwidth? And also check if there is some restriction on data transmission over certain limit(Firewall issue)?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: csv file to redshift

Hi,
 My internet connection has some problem.It keeps on disconnect every half an hour.But sometimes Internet connection will be proper.But, still I get like this error. Is that error because of Internet connection???
I do not know about firewall issue??
Can you please explain me more about that??How to fix it??
I am Getting error as for input string : "http". How to resolve this???
Regards,
Vinuta
One Star

Re: csv file to redshift

Hello,
While try to load data from csv file to redshift I got these errors:
Starting job red_mysqltest at 13:44 10/12/2014.
connecting to socket on port 3430
connected
Exception in component tRedshiftOutput_1
org.postgresql.util.PSQLException: ERROR: /rds/bin/padb.1.0.867/data/exec/58/0: failed to map segment from shared object: Cannot allocate memory
  Detail: 
  -----------------------------------------------
  error:  /rds/bin/padb.1.0.867/data/exec/58/0: failed to map segment from shared object: Cannot allocate memory
  code:      1015
  context:   dlopen(/rds/bin/padb.1.0.867/data/exec/58/0,RTLD_LAZY)
  query:     4234372
  location:  exec_plan.cpp:2213
  process:   padbmaster
  -----------------------------------------------
at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2096)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1829)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:510)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:386)
at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:332)
at project_1.red_mysqltest_0_1.red_mysqltest.tFileInputDelimited_1Process(red_mysqltest.java:1056)
at project_1.red_mysqltest_0_1.red_mysqltest.runJobInTOS(red_mysqltest.java:1802)
at project_1.red_mysqltest_0_1.red_mysqltest.main(red_mysqltest.java:1646)
disconnected
Job red_mysqltest ended at 14:14 10/12/2014.
Please tell me the solution.
Regards,
Vinuta
Moderator

Re: csv file to redshift

Hi,
We cannot get this issue before. On which official version did you get that? What's your OS? Is the redshift DB connection successful in DB metadata?
Can you connect to your Redshift DB successfully by client without using Talend Tool?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: csv file to redshift

Hello,
Talend version:5.6.
OS: Windows 8.1
Job design:
tfileinputdelimited >> tmap >> tredshiftoutput
Yes. Redshift connection is successful in Meatdata.
Yes. I can connect to my redshift DB successfully without using Talend tool.
I am also getting for input string 'http' error.
I have selected the option trim columns in tfileinputdelimited component. But its not working.
Please help me to resolve the both errors.
Regards,
Vinuta
Moderator

Re: csv file to redshift

Hi,
How did you set your tRedShiftOutput component? What's the rows per insert(commit every)? Any data transmission restrictions/limits in your DB? Could you please show us more job setting screenshots?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: csv file to redshift

Hello,
commit size: 100
There is no data transmission limits.

Can we use any other component instead of tredshiftoutput component ???
Regards,
Vinuta
Moderator

Re: csv file to redshift

Hi,
Have you tried to check out "Die on error" on tRedShiftOutput component to see if there is any error info printed on console?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: csv file to redshift

Hi,
I have not select the box die on error.
Can we use other component to load data?? CAn you please tell me??
Regards,
Vinuta
Moderator

Re: csv file to redshift

Hi,
Can you use tRedShiftRow to insert data into Redshift DB?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: csv file to redshift

Hi,
Can you please explain me more about tredshiftrow??
How to load the data into tredshiftrow??