Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

unable to append & overwrite the existing file in thdfsoutput

Highlighted
Five Stars

unable to append & overwrite the existing file in thdfsoutput

Hi all,

 

am facing the issue which unable to overwrite & append the existing file have placed in the "file name" in thdfsoutput .

For every time it was creating new file (FILE_1 to create success) if i applied new filename in the "file name path" option but again want to append this (FILE_1) it was showing this  below error.

 

Exception in component tHDFSOutput_1 (FL_tsgal005255_r)
java.io.IOException: DataStreamer Exception:
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:577)
Caused by: java.nio.channels.UnresolvedAddressException
at sun.nio.ch.Net.checkAddress(Net.java:101)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:622)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1601)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1342)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1295)
at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:463)
[FATAL]: godrej_test.fl_tsgal005255_r_0_1.FL_tsgal005255_r - tHDFSOutput_1 DataStreamer Exception:

Highlighted
Moderator

Re: unable to append & overwrite the existing file in thdfsoutput

@saibabu ,check the below post may be the same issue in your case.

https://www.talendforge.org/forum/viewtopic.php?id=50662

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Employee

Re: unable to append & overwrite the existing file in thdfsoutput

Hi,

 

    What is the value you are using in Action?

image.png

 

Are you facing this issue in this file or all files?

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now