Header and trailer validation of csv file

One Star

Header and trailer validation of csv file

Hi,
I have .csv file having multiple colmns,i need to do hader and trailer validation on csv file.
first row of csv file consisting name of column ,which should go in header file.and
last row of csv file consist total count of rows(eg.100) in file ,the count shoud goes in trailer file,
so how to do that header and trailer validation.
layout of csv file
col1 | col2 | col 3 | ...|.... } :this row goes to in header.csv file
data1 | data2 | data3 |..........
.
. :this rows(total 90 rows) goes to in maindata.csv file
.
90 :this count should goes to trailer.csv file

So will you please suggest that ,how to achive this?
Moderator

Re: Header and trailer validation of csv file

Hi,
For your requirement, you can control the num of "header" and "footer" on tFileInputDelimited.
To make it clear, I have designed a demo job.
The input file:
id,name,sex
1,shong,m
2'sabrina,f
3,Elise,f
The expected result is:
outfile 1
1,shong,m
outfile2
3,Elise,f
Please see my screenshots for details
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Header and trailer validation of csv file

Hi,
Many thanks for your quick reply. Actually my requirement is a little bit different.
The file will be something like -
Name Age Mobile This is the Header in my case. Meaning we do not have any value for Header.
abc 22 97453
hkh 33 86541
hfk 16 90686
lmn 19 56312
4 The data 4 is the count of no of recrods we have which is just an example and may vary
And so this is the trailer.

We want to validate that count mentioned in trailer ie 4 in our example case and the actual num of records present in the file ie 4 in our case matches or not.
So it is something like to first take the last record of the file into 1 file and store the value. Then do a count of the remaining records to get a count of 4 and then do a match.
Please let me know if you got it.
Thanks
One Star

Re: Header and trailer validation of csv file

Check out the screenshots below
Edit: Instead of tLogRow, place a tFileOutputDelimited component.
If the footer has more than 1 row, then in the tJava row (say for example 3 rows), you can specify
String footer_rows = ((Integer)globalMap.get("tFileRowCount_1_COUNT")-2)+".."+(Integer)globalMap.get("tFileRowCount_1_COUNT");
Five Stars

Re: Header and trailer validation of csv file

Hi,

 

I am generating a .csv file using tFileOutputDelimited component. 

My requirement is to append a footer while generating the file. (In my generated csv file i need extra row at the end).

 

Please suggest.

Moderator

Re: Header and trailer validation of csv file

Hello @MNK_2019 

Could you please elaborate your case with an example with input and expected output values?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog