My use case:
Using oracle cdc component in redo log mode to export data into CSV. Created a generic job instead of creating a separate job for each CDC table.
Get the table list (metadata) using Oracle Input component and extract the latest data (delta changes) for those table.
tOracleInput (Read config_table metadata) -> tFlowIterate (Iterate config_table) ==> tOracleCDC (Retrieve delta changes for those table records selected using tOracleInput component) -> tFileOutputDelimeted (Output the delta changes in CSV Output)
My Oracle metadata:
table1 schema: (CDC table)
table2 schema: (CDC table)
As you can see, my CDC tables schemas are different.
Configuration in tOracleCDC:
CDC mode: Log
Table using CDC: (String)globalMap.get("Table_name") (This Table_name attribute is passed from tOracleInputComponent)
Events to catch: Insert, Update, Delete (Selected all 3)
Schema using CDC: Built-In with attribute as "columnName" with "Dynamic" type (This is where the problem occurs. tOracleCDC doesn't support Dynamic type)
Please can someone help. is there any alternative solution to set the schema dynamically so that tOracleCDC component can use it.
Thanks in advance!
Here exists a new feature jira issue.
There are 2 potential options to solve this 'industrialisation' requirement, namely:-
Both would typically require custom development / delivery work either from Talend Professional Services or Talend Engineering directly in my view.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Part 2 of a series on Context Variables
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema