Help to read the header of a file

One Star

Help to read the header of a file

Hello friends,
I'm having a little trouble trying to figure it out how to read the header of a file in a certain way and the rest of it in a different way.
I have a file like this:
------------------------------------------------------------------------------------------------------------
*CRAP DATA : YYEAAP
@ INSI LAT LONG ELEV TAV AMP REFHT WNDHT
YYEAAP -3.1 10 4 5 6 -99 -88
@DATI SAAA TBBB TCCC RDDD
00001 5 10 15 20
00002 33 7 3 1
00003 34 8 4 31
00004 35 9 6 134
00005 36 10 7 15
...
------------------------------------------------------------------------------------------------------------
I know how to read the data since the 5 row using the tFileInputFullRow and setting the header to 5 and them using a tMap and a tExtractDelimitedFields to format the data.
But I need to read also the first part to join everything in the end to insert into my database.
My question is... how do I set on the tMap which row goes to one tool(tExtractDelimitedFields) and which row goes to another tool?
Thank you so much
Moderator

Re: Help to read the header of a file

Hi tfandrade,
Could you give us your expected result. Do you want to separate your source file into two parts?
One part:
@ INSI LAT LONG ELEV TAV AMP REFHT WNDHT
YYEAAP -3.1 10 4 5 6 -99 -88
the other part:
@DATI SAAA TBBB TCCC RDDD
00001 5 10 15 20
00002 33 7 3 1
00003 34 8 4 31
00004 35 9 6 134
00005 36 10 7 15
Need your confirmation, thanks.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Help to read the header of a file

Hello xdshi,
I expect to have something like this:
YYEAAP | 00001 | 5 | 10 | 15 | 20
YYEAAP | 00002 | 33 | 7 | 3 | 1

Thank you again
Moderator

Re: Help to read the header of a file

Hi,
I know how to read the data since the 5 row using the tFileInputFullRow and setting the header to 5 and them using a tMap and a tExtractDelimitedFields to format the data.
But I need to read also the first part to join everything in the end to insert into my database.
My question is... how do I set on the tMap which row goes to one tool(tExtractDelimitedFields) and which row goes to another tool.

What you need is adding a column "YYEAAP" before your "5 row ". Component tFilterColumns can achieve your goal.
I have designed a demo job to explain it clearly.
see my screenshots.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seventeen Stars

Re: Help to read the header of a file

hi all,
have a look at this post (http://www.talendforge.org/forum/viewtopic.php?id=6441) to retrieve "dynamically" column name.
hope it helps
regards
laurent
One Star

Re: Help to read the header of a file

Hi Sabrina,
I did implement what you wrote and it went perfect but the things is that I need to get value in a dynamic way because I will have about 1.000 of those files do import and they all come with that format I wrote.
So I need to figure it a way to get the first value of the 4 row and jump to 6 row and get all the other data.
The output that you sent to me it was right I just need to get the value INSI dynamically.
I try to use what kzone wrote but I couldn't find a way to combine it with what I have.
Thank you so much for everybody's help.
Thales
One Star

Re: Help to read the header of a file

It would be almost like saying:
- on row 4 you get only the first value => INSI.
- from row 5 you start to get the values DATI SAAA TBBB TCCC RDDD
- in the end I concatenate INSI | DATI | SAAA | TBBB | TCCC | RDDD
Can I do that?
Thank you again.
One Star

Re: Help to read the header of a file

You could split your file into 3 parts using @ as the row delimiter using tFileInputFullRow. This would give you header, INSI and DATI. In your tmap you can put a condition to output1 for strings starting with INSI and output2 strings starting with DATI. You could then process each of these outputs with their correct schemas.
One Star

Re: Help to read the header of a file

Thank you people so much.
I finally got it to work like it is on the pic.
But now I'm running into some other problem.
How do I get all the data together? I've tried tMap, tUnique... everything!
Because my outputs now are:
1 - On the first tLogRow
YYEAAP | 1
2 - On the second tLogRow
values for: DATI SAAA TBBB TCCC RDDD
But I need my output to be:
1 | YYEAAP | DATI
1 | YYEAPP | SAA
1 | YYEAPP | TBB

Is there a way that I can do that?
Thank you,
Thales
http://www.talendforge.org/forum/img/members/79895/mini_104419_Captura de Tela 2013-03-12
One Star

Re: Help to read the header of a file

I did what you said but now I have three outputs from the tMap with a big string.
How do I format that so I can have it row by row? The only way I knew how to do it was by tFileInputFullRow.
Thank you,
Thales
You could split your file into 3 parts using @ as the row delimiter using tFileInputFullRow. This would give you header, INSI and DATI. In your tmap you can put a condition to output1 for strings starting with INSI and output2 strings starting with DATI. You could then process each of these outputs with their correct schemas.
One Star

Re: Help to read the header of a file

Output them to tFileOutputDelimited instead of tLogRow and process them with the correct schema.
One Star

Re: Help to read the header of a file

Yesterday trying I did output them into two files and them started a new subjob opening the files.
But which palette do I use to do that:
One file has:
7LAG;2
The other file has:
11,8;22,7;99,9;27,7
12,3;74,8;76,3;99,9
...
I want them to be:
7LAG;2;11,8;22,7;99,9;27,7
7LAG;2;12,3;74,8;76,3;99,9
...

Observation: the first file will always have only 1 line.
Thank you so much!
One Star

Re: Help to read the header of a file

What do you mean by palette? You just have tFileInputDelimiteds to read the files. Connect to first process with OnSubjobOK.