Error with tHDFSOutput

One Star

Error with tHDFSOutput

Hello, I have a very simple job. I am using Talend open studio for Big Data to load some data from DB2 to HDFS. I can see my file being created in HDFS but I do not see any data being loaded. I do see the following error messages on my Talend console.
I have attached a screenshot of my job. We are using Teradata Big Data Appliance with Hortonworks 1.3 distribution.
Is there something obvious that stands out to anyone?

Thanks in Advance

[statistics] connecting to socket on port 3457
[statistics] connected
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Exception in createBlockOutputStream 39.7.56.5:50010 java.net.ConnectException: Connection timed out: no further information
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Abandoning blk_-4452200433289838787_3453406
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Excluding datanode 39.7.56.5:50010
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Exception in createBlockOutputStream 39.7.56.6:50010 java.net.ConnectException: Connection timed out: no further information
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Abandoning blk_4915658027028762783_3453406
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Excluding datanode 39.7.56.6:50010
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Exception in createBlockOutputStream 39.7.56.7:50010 java.net.ConnectException: Connection timed out: no further information
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Abandoning blk_2314892590263004540_3453406
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Excluding datanode 39.7.56.7:50010
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Exception in createBlockOutputStream 39.7.56.8:50010 java.net.ConnectException: Connection timed out: no further information
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Abandoning blk_2361477977883183480_3453406
[INFO ]: org.apache.hadoop.hdfs.DFSClient - Excluding datanode 39.7.56.8:50010
[WARN ]: org.apache.hadoop.hdfs.DFSClient - DataStreamer Exception: java.io.IOException: Unable to create new block.
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3815)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2600(DFSClient.java:2986)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:3231)

[WARN ]: org.apache.hadoop.hdfs.DFSClient - Error Recovery for blk_2361477977883183480_3453406 bad datanode[0] nodes == null
[WARN ]: org.apache.hadoop.hdfs.DFSClient - Could not get block locations.
Moderator

Re: Error with tHDFSOutput

Hi,

Have you tried to check out "Trim column" option in Advanced settings of tDB2Input to avoid empty string?
Could you please use tFixedFlowInput -> tHDFSOutput to see if data can be loaded into HDFS successfully?

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Error with tHDFSOutput

xdshi, I have tried out both trim and also  tFixedFlowInput -> tHDFSOutput  to see whether I can load something into hdfs. I got the same initial error. It appears that we have an issue with some kind of security setting. 
Anyone seen this kind of error? Is there a workaround?

Thanks

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog