I’m looking for some technical insight on the Talend tool to figure out if:
To provide a summary of my query:
We have a main table and 3 other tables on the Oracle DB side which are Joined in order to create JSON data and PostgreSQL database is the source database which stores these JSON.
All we need to test is that any changes made to any of these tables on the oracle side are reflected exactly with the same formats, fields and values in corresponding tables in PostgreSQL database and the correct element is updated in JSON.
The time taken for this entire operation needs to be measured too.
I have to provide a plan to my team on how we can achieve this and timeline for the same with the help of Talend
Kindly let me know, if anyone has done anything similar to this in their projects and can provide some insight.
Thanks & Regards,
Synchronization works best when you can find your changed records quickest. With Talend, you can implement CDC (change data capture) feature.
Here is online document about:TalendHelpCenter:Change Data Capture (CDC): publish/subscribe
Note: You can capture the DML changes with CDC feature, including inserted, updated or deleted records, but it is unable to capture DDL changes.
Introduction to Talend Open Studio for Data Integration.
Practical steps to developing your data integration strategy.
Create systems and workflow to manage clean data ingestion and data transformation.