Four Stars

remove non static header from a file

Hi,

I have a use case where a file comes to my system daily has multiple headers.

The no. of header counts are not same for each file. They can be 1,5,10, etc but there is an identification that says where the header ends.

 

E.g  Inside the file, here in the below example the record begins with "Cards_14Q1_1"  then headers, end of header(EOH), actual data header followed by actual data elements

 

Cards_14Q1_1
20180331
Month-End
45678098
Feed Name| Load Status|Date
TN-MT-34671| WAITING| 
TN-MT-34871| WAITING| 
NR-PP-14678|WAITING|  
TN-MT-34872| WAITING|

EOH

AccId|Branch Address|Legal Entity|Cross Border

XB124|920|100|N

67894|920|110|N

B1000|857|100|Y

 

Expected output after removing the headers:

XB124|920|100|N

67894|920|110|N

B1000|857|100|Y

 

Please advise.

 

Thanks,

Brat

1 ACCEPTED SOLUTION

Accepted Solutions
Nine Stars

Re: remove non static header from a file

Please try below for removing header

RemoveHeader3.PNG

Regards,

Veeru Boppudi
7 REPLIES
Nine Stars

Re: remove non static header from a file

Hi,

 

Are you looking for the following output?

 

RemoveHeader1.PNGRemoveHeader2.PNG

 

flag: row2.line.equals("Cards_14Q1_1")?false:row2.line.equals("AccId|Branch Address|Legal Entity|Cross Border")?true:Var.PreFalg 

Regards,

Veeru Boppudi
Four Stars

Re: remove non static header from a file

Hi Veera,
Thanks for your quick reply. Yes that is my expected output.

I have further query what if the first record for each file does not start with value “Cards_14Q_1” meaning the first record can be any value which is unknown.

Will the below provided solution work?

Thanks for the help!
Four Stars

Re: remove non static header from a file

Also I would like to remove the header from my output.
Nine Stars

Re: remove non static header from a file

try with this

flag=row2.line.equals("AccId|Branch Address|Legal Entity|Cross Border")?true:Var.PreFalg==null?false:Var.PreFalg 

 

Regards,

Veeru Boppudi
Nine Stars

Re: remove non static header from a file

Please try below for removing header

RemoveHeader3.PNG

Regards,

Veeru Boppudi
Four Stars

Re: remove non static header from a file

thanks Veera. It went through!

 

Four Stars

Re: remove non static header from a file

Hi,

 

I have same issue and I am new to Talend. The way we are hard coding the Header Record is not going to solve my problem. The logic I want to implement is remove all the records preceding EOH and one record after EOH. Is there any dynamic Solution? I am thinking of the following, but not able to design.

 

1. Assign Record Number to each Record.

2. Find out the Record Number with "EOH"  Say EOHRECNUM (Any other record having EOH in between field will not qualify, eg GEOHYDRO)

3. Write an expression to filter record number less than equals to EOHRECNUM + 1

 

Best,

Dhiren