Five Stars

Management of merged data from different user for same file

Hi there,

 

I'm using desktop version of Talend data preparation, in which I'm able to clean few rows fom csv file based on duplicate values. Now, it will be good if I explain my concern with Example :

 

Morning:

1. The client provide us csv sheet where there are 2000 rows.

2. I extract this file in dataset, and clean some data by applying some functions

e.g. Deleted 4 rows having city = 'ABC',

3. total rows = 1996

4. I save this file to preparation.

5. Either export this file or let it as it is there in preparation.

 

Evening:

1. The same client sent same csv file with some more additional rows, say 3000, i.e. 1000 rows added to previous 2000 rows.

2. In this file there are total 8 rows with city = 'ABC'

3. I again applied same function to clean data, delete rows with city = ABC, (4 rows I already deleted from previous sheet)

4. Total rows = 2992

5. Export

 

 And the process goes in same manner, which is definitely not an efficient way. 

 

How can we save and automate these rules again and again ?

Is there any solution (tool) provided by Talend for this scenario ? 

  • Data Integration
  • Dataprep
  • Talend Activity Monitoring
1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: Management of merged data from different user for same file

Hi,

 

What you are looking for is the ability to reapply an existing preparation to another dataset with the same structure, right? If so, that is doable natively in Data Prep - see https://community.talend.com/t5/Data-Quality-and-Preparation/Multiple-Datasets-with-the-same-structu...

 

Let me know if that matches your needs.

 

Regards,

 

Gwendal

2 REPLIES
Employee

Re: Management of merged data from different user for same file

Hi,

 

What you are looking for is the ability to reapply an existing preparation to another dataset with the same structure, right? If so, that is doable natively in Data Prep - see https://community.talend.com/t5/Data-Quality-and-Preparation/Multiple-Datasets-with-the-same-structu...

 

Let me know if that matches your needs.

 

Regards,

 

Gwendal

Five Stars

Re: Management of merged data from different user for same file

Yes, it does.
We want Talend Data Preparation to be use to access Microsoft Azure cloud. So please provide any suggestion or link.

Many thanks for your help Smiley Happy