Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

Logging multiple log console outputs to a csv file

Highlighted
Six Stars

Logging multiple log console outputs to a csv file

Hi,
  I'm unsure about what the best approach would be to collect log console output data to a csv file.

At the moment I'm dumping all of the console output to a file and the resulting txt is, which I want as a csv with headers

tJava_1 code is :

java.io.File logFile = new java.io.File("_logs/"+projectName+".log");
java.io.PrintStream ps = new java.io.PrintStream(new java.io.BufferedOutputStream(new java.io.FileOutputStream(logFile)));

System.setOut(ps);
System.setErr(ps);



How can I get this into a csv?

Header > status code (200 or 400) | person_id |          contact_id           |      Full Name      | Error Message (if applicable)
Data >           200                        150014236      001O000000lJwuEIAS         Jono Bridges
                    400                                             00128000003KXkJAAW        Acme              A surname is required



P.S. - The tFileOutputDelimited components just appended each console output in the first column and wasn't as good as the tjava txt file which at least grouped the data together.
 


Highlighted
Community Manager

Re: Logging multiple log console outputs to a csv file

At the beginning of job, write the header information to the csv file first. eg:
tFixedFlowInput--main--tFileOutputDelimted
   |
other job parts.
on tFixedFlowInput, define the header columns. 
on other job parts, catch the error message and customize them, append them to the csv file.
----------------------------------------------------------
Talend | Data Agility for Modern Business
Highlighted
Six Stars

Re: Logging multiple log console outputs to a csv file

Hi Shong,
Thanks for your reply.
I've defined the header via tFixedFlowInput --- main ---> tFileOutputDelimited at beginning of job, and it creates a csv with the headers i've defined.
However I'm not understanding how I insert the data under their respective header fields from log console at different points in the flow.
Can you please explain this in more detail. thanks.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog