I'm trying to capture SQL Server table changes in real-time and send them to a Kafka topic.
I know that I can use a CDC table and connect it to Kafka directly (CDC table -> Kafka), but is there another alternative using Talend (something like CDC table -> Talend -> Kafka, etc)?
I've tried using Talend's CDC feature, but it seems that it only works for batch jobs and not real-time... Do correct me if I'm wrong though.
any solution would be not a real-time (unless you not design sending a message direct from triggers), all will be - loop by schedule or infinite loop
so, you could run Talend job in loop - 1 sec delay or less
or you could check in internet SQL CDC to Kafka and find few other solution for read SQL Server CDC tables, but it still will be not a real real-time
Yes, I am aware that I can create real-time streams from SQL Server directly to Kafka, but I was wondering whether it's possible to do it in a more user-friendly way with Talend.
Thanks for the input!
as I know - not, all what you can do it schedule job in a loop (schedule or design Job as infinite loop)
with some other tools it same - you must define query interval
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Learn how to do cool things with Context Variables
Find out how to migrate from one database to another using the Dynamic schema
Pick up some tips and tricks with Context Variables