Trying to Load tables data in to hdfs cluster but it is loading an empty file only

Five Stars

Trying to Load tables data in to hdfs cluster but it is loading an empty file only

I have extracted data from rest api and converted json into column fields and then tried to load the data into hdfsoutput. The connection is established and path file is created but there is no data received into it. I have deactivated the hdfsoutput component and tried to run it and i can see all the data coming until hdfsoutput. can anyone please correct me?

hdfs.png

Moderator

Re: Trying to Load tables data in to hdfs cluster but it is loading an empty file only

Hello,

Please use the "onsubjobok" connector to connect your tprejob-->HDFSConnection-->tDie and the treatclient-->tMap-->thash subjobs.

Best regard

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Moderator

Re: Trying to Load tables data in to hdfs cluster but it is loading an empty file only

Hello,

Is there any update for your issue? Feel free to post your issue here.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Trying to Load tables data in to hdfs cluster but it is loading an empty file only

Hi, I have already created the job extracting API from trestclient to thdfsoutput as shown above. so can I extract the data from another API using trestclient within the same job to join both api data and extract the required fields to the same hdfs output?

Tags (1)
Moderator

Re: Trying to Load tables data in to hdfs cluster but it is loading an empty file only

Hello,

Are your initial issue fixed? The issue that trying to Load tables data in to hdfs cluster but it is loading an empty file only?

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Trying to Load tables data in to hdfs cluster but it is loading an empty file only

Hi sabrina,

The above issues was fixed. The actual problem was with the datanode in the hdfs cluster. The job was perfectly working as posted above in the pic. 

Thanks