[resolved] Dynamic column mapping in TIS

Five Stars

[resolved] Dynamic column mapping in TIS

hi,
can we pick up a csv data irrespective of the mapping,
e.g. i have 3 cols coming frm csv: name age sex
i want csv to mysql
i can do it the first tym but what if the column order is not constant. it can come in any sequence other tym
like age sex name etc.
can it be done by dynamic schema ? if so how ?

Accepted Solutions
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi,
Dynamics schema can deal with your issue as your described
See the related reference https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/2...
However, dynamic schema is only available in Talend Enterprise subscription version.
Best regards
Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

All Replies
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi,
Dynamics schema can deal with your issue as your described
See the related reference https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/2...
However, dynamic schema is only available in Talend Enterprise subscription version.
Best regards
Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Dynamic column mapping in TIS

thks sabrina
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi,
Feel free to post your issue on forum, you are welcome.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Dynamic column mapping in TIS

1 more question dynamic columns can be mapped directly to the target using Pass-through mode can i have an example of this ?
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi,
There is no more examples about dynamic schema scenario except for the example in article I have given to you.
But for your requirement, I have designed a demo job for you.
The work flow is: tFileInPutDelimited-->tMySqlOutPut and the source file is:
name;age;sex
sabrina;26;f
shong;30;m
john;28;m
See my screenshots for more details. Hope it will help you.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Dynamic column mapping in TIS

k but i mean what is the cols fro the csv are not in order
like name sex age instead of name age sex
in that case is there a way in dynamic schema we can map the fields
thks & regards,
djugal
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi,
Do you mean there are three columns name,age and sex which arranged randomly.
Dynamic schema don't care the order and it just map the source file into target file. If the source file order is name,age and sex, the target file will correspond to that. If the source file order is age, name and sex, the target file must be age, name and sex.
If I misunderstand your meaning, would you mind giving more description to me?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Dynamic column mapping in TIS

ok so i guess dynamic schema is not the thing i a looking for i guess,
see my prob is
i have a input file coming every hour in the schema : name sex age
mysql table has schema: name sex age
i have to run the job every hour to append the data.
my question was suppose at a time i got the schema of the input file in irregular sequence say name age sex
how can my job handle this jumble without errors. can this be achieved via dynamic schema ?
Five Stars

Re: [resolved] Dynamic column mapping in TIS

i found this link of bekwam's blog correct for my scenario
http://bekwam.blogspot.in/2011/06/dynamic-schemas-in-talend-open-studio.html
thks & regards,
djugal
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi djugal,
Dynamic schema works for your case even though the sequence of columns varies in the source file.
BTW, the link you provided cannot be opened. Would you mind pasted the details content into the forum. I will appreciate it very much. Thanks a lot.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Dynamic column mapping in TIS

hi sabrina,
can u elaborate how would dynamic schema work in my case ?
regards,
djugal
Five Stars

Re: [resolved] Dynamic column mapping in TIS

Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi djugal,
Let's still use your example:
1: In a first hour, inputting " name;age;sex" into your target DB, and then the data will be inserted into a table.
2: Here comes the second hour, inputting "age;name;sex" into your target DB, and then the data will be inserted into the existed table created by the first hour.
Source file for a first hour:
name;age;sex
sabrina;26;f
shong;30;m
john;28;m
Source file for the second hour:
age;name;sex
45;york;m
23;dan;m
18;mary;f
The actual result is : see pic
When the table has been created, the structure in DB is fixed. But dynamic schema will arrange your input data into the corresponding structure ignoring the order.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Dynamic column mapping in TIS

Hi,
i have the same job structure as your example except oracle output,
filedelim -----> oracloutput
i am getting this error while executing in the logrow component:
ORA-00947: not enough values
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi,
ORA-00947: not enough values

How many columns in your existed table? I think this issue is produced by missing insert column. For example, there are 3 columns age,sex,name in your table, but your input is just age,sex.
You can search it on Mr.Goolge to get more info about this issue description.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: [resolved] Dynamic column mapping in TIS

Worked like a charm...............
Thanks Sabrina
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi,

Thanks for your feedback.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: [resolved] Dynamic column mapping in TIS

Hey,
This post is little old, but i wanted to know something related to it.
What if the column names are also different? say name in one file and names in other. can we map it dynamically(after reading a file or something)?
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi ksultania,
Here is a response in your another forum http://www.talendforge.org/forum/viewtopic.php?id=37180, could you please take a look at it?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: [resolved] Dynamic column mapping in TIS

Hi xdshi,
I have a question related to dynamic schema. I am loading a file into mysql table. schema is defined as dynamic.
tfileinputdelimted-> tmap -> tmysqloutput
suppose, my input file has 2 columns- id , name. my table also has same columns. (ID int and Name String).
Problem is, when my ID is null in input file, my job is not inserting the records and throwing an error. (can not insert ' ' in a int column ). I think it is reading those null also as blank string (' ').
Is there any solution to this issue?
Thanks in advance.
Moderator

Re: [resolved] Dynamic column mapping in TIS

Hi shruti04mittal,
What's your expected result for your null input? Do you want to insert it into your mysql table?
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.