MySQL CDC Capture - understanding logic

Twelve Stars

MySQL CDC Capture - understanding logic

Hi Team,
I never use Talend CDC for sync databases because prefer use other methods, but now one friend ask me to help and I meet some problem, which not expect.
Connections - created, schemas created
Subscriber created, triggers - all ok
Next step create Job - capture "changes" and split for 3 flow - Insert/Update/Delete
also clean
Leave Job for sometime with regular scheduling ... and it crash by null pointer exception, let go investigate

All Inserts - work fine
All Updates - work also fine, and it work fine - if records was Updated twice - it catch only 1 Update, Perfect!!!
Deletes - as well, until ...

Until at the time between iteration same records was - Updated, and then Deleted
Talend not understand this situation and try to Update records with New data where all columns other than PK - NULLS and forget about NOT NULL constraints.
More - it crash not on UPDATE state, where it possible to manage, but crash on read state, because his own Metadata structure has same NOT NULL constraints.
I create testing Job for illustrate situation
Situation is realistic and my question is:
- what Talend think when design trigger based CDC without saving OLD and NEW data?
- or why in this case CDC not do the same as for UPDATE - fetch only delete if records deleted?


Re: MySQL CDC Capture - understanding logic

Hello vapukov,

We are supposing that you have read the online document about:TalendHelpCenter:CDC architectural overview

Could you please clarify in which Talend version/edition you are?

Best regards


Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.