One Star

HDFS Append

Hi,
I have a use case to keep appending data to a HDFS file. Does someone know how we can append using Talend?
Thanks,
Ramakrishna
5 REPLIES
Moderator

Re: HDFS Append

Hi Ramakrishna,
Have you already checked option "Append" on TalendHelpCenter:tHDFSOutput component?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: HDFS Append

for for client , because this file is already being created by on xdshi  ]Hi Ramakrishna,
Have you already checked option "Append" on TalendHelpCenter:tHDFSOutput component?
Best regards
Sabrina
One Star

Re: HDFS Append

hi guys i am having same issue ... as given below.
Exception in component tHDFSOutput_3
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException): Failed to create file for on client , because this file is already being created by on
I have used append in all the hdfs output component still having same issue.
Moderator

Re: HDFS Append

Hi,
Did you config "append" parameter in hdfs core file(on server)?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: HDFS Append

Hi sabrina, 
i am facing the same issue for the append functionality in HDFS output !! i have tried with two HDFS output writing to the same file and as well one HDFS output (appending) writing the file twice (in which i am iterating the job twice). is there any other way around for this ?