One Star

Dealing with changing order of columns in CSV

Hi,
I am trying to read a CSV file using a tFileInputDelimited. The first row of the CSV file contains the fieldnames. I extracted the schema from a test file and saved it in the repository. I am able to read the test file and process it within Talend.
However, the problem is that the order of columns in the input CSV can be different on each execution. The fieldnames are always the same, though. When the ordering is different, Talend assigns the wrong values based on the position rather than on the (infered) fieldname. I have tried the approach with a custom tJavaRow routine as explained in: http://www.talendforge.org/exchange/tos/extension_view.php?eid=177 . And I've got this working in the end. However, it seems a bit complex to deal with this issue.
Is there a simpler solution to deal with changing orders in CSV files? Am I missing something obvious? It just seems like a natural thing for a CSV reader to be able to assign values to fields based on the fieldnames in the first headerrow.
Thank you Smiley Happy
Bas
2 REPLIES
One Star

Re: Dealing with changing order of columns in CSV

rbaldwin's job on the Talend Exchange, "Example job for the dynamic schema routine", seems to be the best approach.
I wrote a blog post on this subject describing an alternative implementation. I'm using the same algorithm: create a data structure of header information, consult the data structure when processing. However, my example uses different Java and Talend components. There's a video walking through the job.
http://bekwam.blogspot.com/2011/06/dynamic-schemas-in-talend-open-studio.html
-Carl
Six Stars

Re: Dealing with changing order of columns in CSV

I know this is very old but I am searching for rbaldwin's job for dynamic schema routine. If anyone has it or the link to where I can find it, please let me know.