Four Stars

Validate the CSV file schema dynamically?

Please let me know how to validate the csv file schema dynamically. Please let me know any one have implemented this in Talend open studio.

12 REPLIES
Ten Stars

Re: Validate the CSV file schema dynamically?

open csv file adding one column more.
this one have to be empty.

Francois Denis

Don't forget to tag when it's "solved"!

Four Stars

Re: Validate the CSV file schema dynamically?

Please can you explain briefly. 

Ten Stars

Re: Validate the CSV file schema dynamically?

in a csv file

col1;col2;col3

a;b;c

if you open and get the first line (column name)

you an count column (=3)

 

on a second time if you read a line like this one: e;f;g;h

there is something wrong.

 

a csv file (first level) is never invalid col1=e col2=f col3=g;h

you can add coma enclosure (CSV Options)

 

the same line can be wrong

"e";"f";"g";"h" --> ???

"e";"f";"g;h" --> ok

 

think about what is an invalid csv file?

 

that's all

 

Francois Denis

Don't forget to tag when it's "solved"!

Four Stars

Re: Validate the CSV file schema dynamically?

Sure . Let me try and get back to u. Thank You.

Four Stars

Re: Validate the CSV file schema dynamically?

@fdenis

Please let me know if u have created any sample job for this. So i can create the job based on that.

 

 

Ten Stars

Re: Validate the CSV file schema dynamically?

I can not share pics.
but you are going to success.

Francois Denis

Don't forget to tag when it's "solved"!

Four Stars

Re: Validate the CSV file schema dynamically?

Actually my file headers are

emp_id,name,lastname,age,designation

evrytime the column positions will be changing. we dont know how the columns will be in csv file. So how can we achive in this position. We need to validated column header dynamically.

Ten Stars

Re: Validate the CSV file schema dynamically?

read csv using dynamic type then using java, check all columns are present , then re-order then.

Francois Denis

Don't forget to tag when it's "solved"!

Four Stars

Re: Validate the CSV file schema dynamically?

@fdenis

I am not able to make the column as dynamic type. can u share the java code to check all the columns are present.

 

Ten Stars

Re: Validate the CSV file schema dynamically?

here is a sample:
https://help.talend.com/reader/g8zdjVE7fWNUh3u4ztO6Dw/GhoMo1chGG3zr0yi8VM__g

Francois Denis

Don't forget to tag when it's "solved"!

Four Stars

Re: Validate the CSV file schema dynamically?

I m using Talend open source m not able to find tWriteDynamicFields component. i dint find the tWriteDynamicFields component in SourceForge.net . Please let me know is there any other way.

Ten Stars

Re: Validate the CSV file schema dynamically?

Samples are only samples. They are not your solution.

this sample use dynamic columns.
you can use them for your task.

an other way is to read only the first (headers) row of your file read the full row and split the result string.

good luck

Francois Denis

Don't forget to tag when it's "solved"!