I have one scenario,
I want to load all tables of data in to respective target tables through one talend job dynamically..please help me on this.
all tables are different schema structure. ---------------------------
--------- These tables schema structure same as src1,src2 tables structure
tables are same like src1---A,B,C,D,E,F,.............X,Y,Z.
all tables are diffrerent schema structure
i want load union the data of src1,src2 tables data then load to target respective tables in single talend job dynamically for all tables.
my source and target databases are MSSQL...I want to create one talend job instead of creating multiple jobs for all tables..
Please suggest me.Thanks..all
Hi guys the above scenario like as,
I have 100 tables with different metadata and i need to develop a single job with loads all these tables to target system we Need to run 10 times for all tables or at a time if so please guide me out.
Use dynamic column type. However, this only works if you do not need to do transformation, since you do not know the schema and data type upfront.
Else use the Template functionality in the Studio to generate all your jobs https://help.talend.com/reader/itosd4jFDXxrm6qULuPrbg/rpFV93Uc4LKksvXQFtSdWg It will generate many jobs, but it take 5 mins instead of several days.
Hello Iburatally Employe Thanks for you response.
I am using Talend Open Studio..So in this there is no option for "Create job from templates".
Please give some solutuion for TOS.
I created source tables passing iteratively.
Pleas find attached screen shot.
You can achieve this using dynamic column load. Following are the steps
1. Create a list of your table names in one csv file
2. Read that file in tFileInputDelimited
3. Use tFlowToIterate and create global variable in with tablenames as value from file
4. Use tmysqlinput and set table name as global variable name and in schema select as built in and set only one row and select type as dynamic .
5. do same for another database input flow
6. use tunite to make union
7. connect to target where specify table name as global variable from tFlowToIterate and schema as built in and then create only one row in schema with type as dynamic.
Key here is to pass table names list using tFlowToIterate and defining dynamic column in built in schema
You can use following link for reference to dynamic column
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Thanks for you response..
I am using Talend Open Studio(TOS)..In this there is no option dynamic rt?
How i can ..? Please
Can anyone please help me here?
My requirement is to load data from MySQL/Oracle to Hive.
1) I want to load multiple tables with different structure in one talend job.
2) I am using Talend open studio so not having feature of Dynamic datatype.
3) Is there any way to achieve it in talend open studio version?
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema