how to use an extracted row as a pattern to check other extracted rows

Highlighted
Six Stars

how to use an extracted row as a pattern to check other extracted rows

Hi

I'd like to use an extracted row as a pattern in order to check others rows and eliminate duplicates variable in order to keep only variable which does not appear in my previous extraction. I have a csv file as an input with several path.

/Default/Usines/Documents/Articles/Mois/
/Default/Usines/Documents/Articles/Diverses/
/Default/Usines/Documents/Articles/Factures/Payees
/Default/Usines/Documents/Articles/Factures/Factures_Comptable/ClintsPrivees

1.- I extract the first row as: 

/Default/Usines/Documents/Articles/Mois/

/Default/Usines/

/Default/Usines/Documents/

/Default/Usines/Documents/Articles/

/Default/Usines/Documents/Articles/Mois/

2.- I'd like to use that extract in order to check the followings rows extraction. In order to remove duplicates variables. So I will have as a result:

/Default/Usines/

/Default/Usines/Documents/

/Default/Usines/Documents/Articles/

/Default/Usines/Documents/Articles/Mois/

/Diverses/   -> the second row

/Factures/
/Factures/Payees/  -> Third row

/Factures_Comptable/
/Factures_Comptable/ClintsPrivees   -> Fourth row

the attached pictured show how I think my job should be. But the filter on tMap doesn't really work.

Thank you for your help.

Warm regards


Accepted Solutions
Nine Stars

Re: how to use an extracted row as a pattern to check other extracted rows

Hello @stvn 

 

follow below steps ,

 

Step 1) Normalise all records like you want to do for Line 1 and add a record number e;g;

 

1;/Default/Usines/

1;/Default/Usines/Documents/

1;/Default/Usines/Documents/Articles/

1;/Default/Usines/Documents/Articles/Mois/

2;/Default/Usines/

----

 

Step2 ) 2.1° Using tMap , Read from 2nd record and compare it will all record created from Step 1 untill Preveous record ID 

e.g. Read 4th Line , compare 4th line with all the record  till ID is 3 from Step1.

 

2.2° if  Input record contains  lookup path ,  get the rest of sting 

e.g. for /Default/Usines/Documents/Articles/Factures/Factures_Comptable/ClintsPrivees

contains /Default/Usines/Documents/ so rest of sting is /Articles/Factures/Factures_Comptable/ClintsPrivees

contains /Default/Usines/so rest of sting is  /Documents/Articles/Factures/Factures_Comptable/ClintsPrivees

contains /Default/Usines/Documents/Articles//Factures  so rest of sting is /Factures_Comptable/ClintsPrivees

 

2.3° get the record having smallest of {rest of string } 

for above example it will be /Factures_Comptable/ClintsPrivees

 

So at the end of Step 2 you should have 

/Diverses/
/Factures/Payees
/Factures_Comptable/ClintsPrivees

 

Step 3 ) Normalize step2 output as you did for  step 1

 

/Diverses/

/Factures
/Factures/Payees

/Factures_Comptable
/Factures_Comptable/ClintsPrivees

 

Hope this will help 

 

 

 

 

 

Regards
Abhishek KUMAR

All Replies
Nine Stars

Re: how to use an extracted row as a pattern to check other extracted rows

Hello @stvn 

 

follow below steps ,

 

Step 1) Normalise all records like you want to do for Line 1 and add a record number e;g;

 

1;/Default/Usines/

1;/Default/Usines/Documents/

1;/Default/Usines/Documents/Articles/

1;/Default/Usines/Documents/Articles/Mois/

2;/Default/Usines/

----

 

Step2 ) 2.1° Using tMap , Read from 2nd record and compare it will all record created from Step 1 untill Preveous record ID 

e.g. Read 4th Line , compare 4th line with all the record  till ID is 3 from Step1.

 

2.2° if  Input record contains  lookup path ,  get the rest of sting 

e.g. for /Default/Usines/Documents/Articles/Factures/Factures_Comptable/ClintsPrivees

contains /Default/Usines/Documents/ so rest of sting is /Articles/Factures/Factures_Comptable/ClintsPrivees

contains /Default/Usines/so rest of sting is  /Documents/Articles/Factures/Factures_Comptable/ClintsPrivees

contains /Default/Usines/Documents/Articles//Factures  so rest of sting is /Factures_Comptable/ClintsPrivees

 

2.3° get the record having smallest of {rest of string } 

for above example it will be /Factures_Comptable/ClintsPrivees

 

So at the end of Step 2 you should have 

/Diverses/
/Factures/Payees
/Factures_Comptable/ClintsPrivees

 

Step 3 ) Normalize step2 output as you did for  step 1

 

/Diverses/

/Factures
/Factures/Payees

/Factures_Comptable
/Factures_Comptable/ClintsPrivees

 

Hope this will help 

 

 

 

 

 

Regards
Abhishek KUMAR

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads