tFileInputJSON only returns one row

Four Stars

tFileInputJSON only returns one row

Hi I am very new to this component and having problems trying to parse some simple JSON.

 

My JSON file has sever thousand rows like this.

{"timestamp":"2018-04-12 06:05:07,242","level":"INFO","message":"Application syncronisation"}

 

I have tried many combinations and the only settings that work are to use:

 

Read by JsonPath

Loop Json query "$"

 

and the json query in the mappings set to:

".timestamp"

".level"

".message"

 

This correctly retrieves the data but only the first row in the file?

 

Other changes cause an unable to parse error.

 

What am I doing wrong?

 

Many thanks in advance.


Accepted Solutions
Nine Stars

Re: tFileInputJSON only returns one row

Please try below

 

tTime1.PNGtTime2.PNG

Regards,

Veeru Boppudi

All Replies
Nine Stars

Re: tFileInputJSON only returns one row

Hi,

Is it possible to provide more sample records and expected output?

 

Regards,

Veeru Boppudi
Four Stars

Re: tFileInputJSON only returns one row

Hi Veeranjaneyulu 

below is more rows from the log file, each row is a logging entry.

 

{"timestamp":"2018-04-12 06:05:07,197","level":"INFO","message":"Application syncronisation "}
{"timestamp":"2018-04-12 06:05:07,197","level":"INFO","message":"Updating  sync information for ."}
{"timestamp":"2018-04-12 06:05:07,212","level":"INFO","message":"Team: 4810"}
{"timestamp":"2018-04-12 06:05:07,663","level":"ERROR","message":"There is no information"}
{"timestamp":"2018-04-12 06:05:07,666","level":"INFO","message":"Done"}
{"timestamp":"2018-04-12 06:05:07,666","level":"INFO","message":"Application syncronisation "}
{"timestamp":"2018-04-12 06:05:07,666","level":"INFO","message":"Updating sync information "}
{"timestamp":"2018-04-12 06:05:07,670","level":"INFO","message":"Team: 4810"}
{"timestamp":"2018-04-12 06:05:08,512","level":"INFO","message":"Posting  item to store"}
{"timestamp":"2018-04-12 06:05:09,930","level":"INFO","message":"Done"}
{"timestamp":"2018-04-12 06:05:09,930","level":"INFO","message":"Application syncronisation "}
{"timestamp":"2018-04-12 06:05:09,930","level":"INFO","message":"Updating sync information for ."}
{"timestamp":"2018-04-12 06:05:09,934","level":"INFO","message":"Team: 4810"}
{"timestamp":"2018-04-12 06:05:10,021","level":"INFO","message":"Done"}

 

I want to extract the start and end timestamps of a pair of log entries so I can assess the processing time of a job at different times of the day.

 

The files had in excess of 10,000 rows but only the first row is processed. The last image is where I dumped the output to a tLogRow.

 

I would expect to see around 5,000 rows of data in the file with the start and end times and the log message.

 

Many thanks

 

Dave

One.jpgtwo.jpgthree.jpgfour.jpg

Nine Stars

Re: tFileInputJSON only returns one row

Please try below

 

tTime1.PNGtTime2.PNG

Regards,

Veeru Boppudi
Four Stars

Re: tFileInputJSON only returns one row

Brilliant, Than you

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.