CSV file with random Column position

Six Stars

CSV file with random Column position

Hi,

 

I want to read a CSV file from shared drive and process the columns in the file.

 

Name of the columns in the file are same but position on the column is getting changed.

 

E.g.

File1 has columns as "Username","Model","Drive",

File2 can have "Username","Drive", "Model"

 

I want to read the corresponding column and process the data.

 

Could you please let me know how can i handle this situation.

 

 

Highlighted
Community Manager

Re: CSV file with random Column position

I have written a tutorial on precisely this here....

https://www.rilhia.com/tutorials/dynamic-column-order

Forteen Stars

Re: CSV file with random Column position

Yes, it great tutorial!

and I was happy to read it when it was written

 

the only one problem - it really work with only few columns, than start will be more and more complicated (just because a lot of information on screen and a lot of coding)

 

The more universal solutions:

- pack this logic in java routine which loop over any number of columns. Like parse header and sort columns in all other rows by etalon schema (see p.3)

- convert csv to json, again by routine or by any of command line tools, then parse JSON and not worry about order (I use this)

- and final - the best news :-). All what explained in p.1 - already do the great component from Jan Lolling -  tFileInputTextFlat on one of project I extract  with this component  - more than hundreds columns for KPI from source file with 2000+ columns , work perfect!

 

 

-----------

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog