One Star

[resolved] Generic Talend Job

I want create a generic talend job with File Component , Tmap and Target Db Component.I have multiple files to load in multiple target tables with one talend job.But the metadata for each source and target are different.Let me know.
19 REPLIES
One Star

Re: [resolved] Generic Talend Job

For example:
for instance,
Table1.csv -> Generic_Talend_Job -> Oracle Table 1
Table2.csv -> Generic_Talend_Job -> Oracle Table 2
and so on...
Community Manager

Re: [resolved] Generic Talend Job

Hi
With community version, you have to create many subjobs for each file. With commercial subscription version, you are able to define a dynamic schema to map different structure, so that you only need one general Talend job for all files. For more information, please read this page:
https://help.talend.com/search/all?query=dynamic+schema+

 

Regards
Shong

----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] Generic Talend Job

Thanks Shong
One Star

Re: [resolved] Generic Talend Job

Shong,
I have got the commercial version of talend.Used dynamic schema.It work very fine for the source files to db tables without any transformation logic.How can i achieve the adding transformation logic like nulls etc for individual columns in the Dynamic Schema?
Let me know.I have 10 sources file and 10 tables.Created one job to load this 10 tables using dynamic schema.But i want add different transformation logic for 10 sources.How to i achieve this?
Regards
Vasanth
Community Manager

Re: [resolved] Generic Talend Job

Hi
But i want add different transformation logic for 10 sources.How to i achieve this?

What's the transformation would you like to do for the column? Below is an example job shows you how to get the column Name, column value of the dynamic schema during the job execution, for example:
the table has the following structure and data:
id;name
1;shong
2;elise
begin code of tJavaFlex:
int ncount=0;
String columnName="";
DynamicMetadata column;

main code of tJavaFlex:
ncount=row7.c.getColumnCount();
column=row7.c.getColumnMetadata(ncount-1);
columnName=column.getName();
String columnValue=(String)row7.c.getColumnValue(columnName);
row7.c.setColumnValue(ncount-1, "Hello "+columnName);
row9.c=row7.c;

It generates a new file with the following result:
id;name
1;Hello shong
2;Hello elise
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: [resolved] Generic Talend Job

Hi Shong,
I tried to use your code sample in a tJavaFlex, but I get a compile error telling me that DynamicMetadata cannot be resolved to a type. I see that we can import libraries directly into the tJavaFlex using its advanced settings, but I could not figure out which libraries to import before using this DynamicMetadata type. Can you or somebody else help me with this?
Thanks,
Douglas

begin code of tJavaFlex:
int ncount=0;
String columnName="";
DynamicMetadata column;

Shong
Seven Stars

Re: [resolved] Generic Talend Job

Does you tJavaFlex have a row with a Dynamic column coming in or going out?
One Star

Re: [resolved] Generic Talend Job

Hi alevy,
Thank you for your answer!
I'm not using dynamic columns and now I realize that this example is specific for dynamic columns.

Douglas
Does you tJavaFlex have a row with a Dynamic column coming in or going out?
One Star

Re: [resolved] Generic Talend Job

After applying this solution. I figured that in Dynamic schema, the whole row is read as string with column value separated using comma(,). Please correct me if I am wrong.
Can we control which column is mapped to which column in the output component?
Please feel free to ask if any other details are needed to answer the question.
One Star

Re: [resolved] Generic Talend Job

Hi Shong,
I tried to use your code sample in a tJavaFlex, but I get a compile error telling me that DynamicMetadata cannot be resolved to a type. I see that we can import libraries directly into the tJavaFlex using its advanced settings, but I could not figure out which libraries to import before using this DynamicMetadata type. Can you or somebody else help me with this?
Thanks,
Douglas
What  id c in the code? I am getting an error "c cannot be resolved"
begin code of tJavaFlex:
int ncount=0;
String columnName="";
DynamicMetadata column;

Shong

One Star

Re: [resolved] Generic Talend Job

What is c in the code? I am getting an error "c cannot be resolved"

ncount=row7.c.getColumnCount();
column=row7.c.getColumnMetadata(ncount-1);
One Star

Re: [resolved] Generic Talend Job

c is the single column of type Dynamic, that must be created in the schema of MySQLInput component
One Star

Re: [resolved] Generic Talend Job

Hi Talend Experts, I am using talend enterprise version, I am aware of very basics of talend, I have requirement to create generic job, which takes source table name as input (I may setup this through context variable to pass table name) and generate csv files. I have lot many tables that I need to set up jobs to generate and send to Amazon S3. Please let me know if you have any demo examples that help me to proceed through.
I was looking through this thread found below link but it does not exist.
https://help.talend.com/pages/viewpage. … Id=5671283
Thanks, Pundalik
Community Manager

Re: [resolved] Generic Talend Job

Hi Pundalik
You can use tTableList  to iterate all tables and pass the current table name to tMysqlInput (take mysql database for example in this case), define a Dynamic column on the schema, get each column name on a tJavaRow and se
----------------------------------------------------------
Talend | Data Agility for Modern Business
Six Stars

Re: [resolved] Generic Talend Job

Well this is the common issue with the Talend architecture. It is fundamentally not designed for configuration based integration. Of course you are always free to customize it with your custom Java implementation to achieve such design. Still dynamic schema helped little bit, but it works of course only in the way where source and target structures are same.
One Star

Re: [resolved] Generic Talend Job

Shong,
I have got the commercial version of talend.Used dynamic schema.It work very fine for the source files to db tables without any transformation logic.How can i achieve the adding transformation logic like nulls etc for individual columns in the Dynamic Schema?
Let me know.I have 10 sources file and 10 tables.Created one job to load this 10 tables using dynamic schema.But i want add different transformation logic for 10 sources.How to i achieve this?
Regards
Vasanth

Hi Vasanth,
Could you please share the generic Talend job details to load tables dynamically. I have same requirement in my project.
Regards,
Lavanya
One Star

Re: [resolved] Generic Talend Job

Hi All,
Can anybody tell me how to create dynamic schema using a text file where all the field names,datatype, length are present.
My requirement is 
I have two files with different formats.For these two files i have to create a header and trailer for a set of records where header and trailer format is different for these two files.  I have to create a generic module for header and trailer creation.
Hence i want to define the header structure of one file in one text file and the header structure of another file in another text file.
Using these two text files, during run time schema needs to be created in talend.
Is there any way? Please help me on this. 
One Star

Re: [resolved] Generic Talend Job

Hi All,
Can anybody tell me how to create dynamic schema using a text file where all the field names,datatype, length are present.
My requirement is 
I have two files with different formats.For these two files i have to create a header and trailer for a set of records where header and trailer format is different for these two files.  I have to create a generic module for header and trailer creation.
Hence i want to define the header structure of one file in one text file and the header structure of another file in another text file.
Using these two text files, during run time schema needs to be created in talend.
Is there any way? Please help me on this. 
One Star

Re: [resolved] Generic Talend Job

Hi
With community version, you have to create many subjobs for each file. With commercial subscription version, you are able to define a dynamic schema to map different structure, so that you only need one general Talend job for all files. For more information, please read this article:

Shong

Hi shong,
I got the exact same scenario. but i'm not being able to see the link you provided.
it's saying 
"The page you were trying to reach could not be found. This could be because:

The page does not exist.
The page exists, but you do not have permission to view it."