How to process changing data structure?


This article introduces the dynamic schema feature. Dynamic schema allows you to design Jobs with an unknown column structure (unknown name and number of columns). If necessary, dynamic columns can be mapped directly to the target using Pass-through mode.


The main application of this functionality is a replication scenario or a simple one-to-one mapping of many columns. This feature makes designing these types of Jobs easy. For example, if you need to migrate a whole database with hundreds of tables, you can do so without explicitly including the table structure, using a single Job.


This article demonstrates the dynamic schema feature through two examples:

  • Read a table with dynamic schema
  • Read a file with dynamic schema

These examples are available in the attached Zip file.



This procedure is compatible with all versions of Talend subscription products.


List of components that support the dynamic schema feature

Not every component supports the dynamic schema feature. Refer to the article Which components provide the Dynamic Schema feature? to know which components provide the dynamic schema feature.



Read a table with a dynamic schema

This example will extract data from a table called person. The table has many columns and contains the following example data. The objective is to write this data to a file.

id name gender email department entry_date
1 Jeffe Boy R&D 2012-05-15
2 Shong Boy Community 2011-08-07
3 Johnson Girl HR 2014-04-16
4 Joyce Girl Test 2012-09-10
5 Truman Boy Marketing 2013-02-27


In Talend Open Studio for Data Integration, the community version, you have to define each column in the schema to map the real table structure. If the table structure is unknown, it is impossible to read the table.


In Talend subscription products, there is a Dynamic type in the Type list, as seen in the following figure:



Using this type, you will need to define only one column with the Dynamic type, instead of defining all columns.


Example Job

Create an example Job called ReadTableDataWithDynamicSchema. The Job contains two components:

tMysqlInput_1: read person data from the database.

tFileOutputDelimited_1: write the data to a text file with a header.


The detailed Job settings can be seen in the following figures:



tMysqlInput_1 settings:



tMysqlInput_1 schema:



tFileOutputDelimited_1 settings:



Execute the Job

Execute the Job and open the generated file (in this case, it is D:/file/out.txt). Verify that all data is written to the file with the correct column names. The data shows as follows:



Read a file with dynamic schema

Here is another example to explain how to read data from a file with the dynamic schema feature. In this example, you will read the file that was generated by the above scenario.


Note: When reading data from a file with a dynamic schema, the first line of the file must be the header that will be read as the column names at run time. In this example, the first line of the file is the header.



Example Job

Create an example Job called ReadFileDataWithDynamicSchema. The Job contains two components:

tFileInputDelimited_1: read data from the text file.

tMysqlOutput_1: create a table in database and insert the data.



tFileInputDelimited_1 settings:



tMysqlOutput_1 settings:



Execute the Job

Execute the Job. Verify that a table called person1 was created in the database and the data was inserted.


Version history
Revision #:
7 of 7
Last update:
‎02-24-2019 11:18 PM
Updated by:
Labels (2)
Six Stars

The above feature if we have to implement for multiple tables and if i have to launch the same job instances fr multiple tables ,will it work?


Tfiledelimitedinput-->tflowtoiterate-->iterate --Trunjob (which have the job which uses the DB connection and loading to file).Will this flow run one by one or Trunjob instance willbe fired in parellel..


Th sis just not to run the table load one by one ,i just want to kick off the jobs in parallel i will have 100 tables to be loaded.

Community Manager

Hi sanjana,

Our Community Discussion forums are monitored more closely than the Shared Knowledge areas, so you might want to ask your question on the appropriate forum (perhaps

If you need technical support, please log in to our Support portal at

Hope that helps,


Two Stars



I did the same as shown above example. extracted the data from the oracle database using dynamic schema and include header option. 

As my table was already created so i tried to load the data but it saying not a valid month. 


Why i am getting this error ?