Job template & dynamic table structure

One Star

Job template & dynamic table structure

Hello all,

 

We have to create several job doing exactly the same thing, but with differents input/output table.

 

The job will run as follow:

  1. Get an ID (LoadID) from an API;
  2. Get data on an input table;
  3. Insert these data, including the LoadID, on an output table (the field wich will contain the LoadID exists in all of the output table);
  4. Approve the insert via an API call;

 

We need to create 5 or 6 times this job, with a different input and output table. To me, the best practice would be to create a "generic job", passing input and output table as parameters to the job.

 

The problem is these input/output table don't have the same data structure.

 

Does anybody have an idea on how to dynamically specify the table structures?

 

Thanks in advance for your help,

 

Talend version: Talend Data Integration v6.3.1

Database type: Postgresql (> v10)

Moderator

Re: Job template & dynamic table structure

Hello,

Talend Dynamic schema feature will allow you to design schema with an unknown column structure (unknown name and number of columns).

Could you please let us know if this article helps?

https://community.talend.com/t5/Design-and-Development/How-to-process-changing-data-structure/ta-p/2...

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog