How to stream log file from server into Hadoop ? Incrementally ?

Five Stars

How to stream log file from server into Hadoop ? Incrementally ?

We have server Log file data need to push into Hadoop using talend. 
We tried :
 regex pattern to directly load the server log data into hadoop
thdfsconnection
|
|
tfileinputregex -->thdfsoutput


But it did not work. As it is not outputting anything in target hadoop. Please suggest if we have alternate way to do this.

Attached is console output after job run :
ConsoleOutput.txt_20170219-0358.txt
Any help on it would be much appreciated .
Moderator

Re: How to stream log file from server into Hadoop ? Incrementally ?

Hi,
For this error "Could not locate executable null\bin\winutils.exe  in the Hadoop binaries.", could you please take a look at this Kb article about:TalendHelpCenter:The missing winutils.exe program in the Big Data Jobs to see if this solution is OK with you.
Best regards
Sabrina 
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: How to stream log file from server into Hadoop ? Incrementally ?

Issue is i  have to Parse the Log data and Output Into HDFS.