From Thursday, July, 9, 3:00 PM Pacific,
our Community site will be in
read-only mode
through Sunday, July 12th.
Thank you for your patience.

Inserting number of columns into several csv files based upon the values in count column of input csv file

Highlighted
Six Stars

Inserting number of columns into several csv files based upon the values in count column of input csv file

Hello community,

 

I have a csv file which consists of several columns. There is also a column called count having values like 1,2,3,5, etc. specific to each record. Now at the output, there would be multiple csv files and the number of columns at each of the csv file would depend upon the values of 'count' column. Consider the below scenario for further details.

input csv -

id;name;sal;count
1;abc;100;2
2;def;200;1
3;ghi;400;3

 

output csv -

(Taking 1st row into consideration)

 

As the value of 'count' here is 2 therefore, output csv will have 

id;name
1;abc
2;def
3;ghi

ie., only the first 2 columns.

 

Please help me achieve this scenario.

 

Thanks in advance!

 

Best Regards,

Dipanjan

 

Highlighted
Moderator

Re: Inserting number of columns into several csv files based upon the values in count column of input csv file

@dipanjan93,if i understand correctly, you just want to take the first two columns of the input file is that correct?

 

you can just filter the columns which you do not want to populate using tFilterColumns or tJavarow or tMap.

Manohar B
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Six Stars

Re: Inserting number of columns into several csv files based upon the values in count column of input csv file

@manodwhb Well not really. Number of columns at the output csv file would be dependent upon the 'count'. As in the example you can see for the 1st record count is 2 therefore 2 columns are there at the output. Similarly, for the second row the value of count column is 1 so the output shall have 1 columns. So on and so forth.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now