CDC is not working in scheduling time

Five Stars

CDC is not working in scheduling time

Hi all,

 

I have created a CDC tables while it's working fine when ever i used run button in talend.

Same jobs after scheduled these build jobs using task scheduler for every hour am getting ZERO records from these CDC tables.

 

source                     destination 

mssql ----------------->HIVE

 

NOte: From client side the mssql database is dropping  and recreating every night 10 pm.IS this cause any issue am getting zero records in hive by CDC tables.

 

Can anyone help me for this issue to resolve..

Forteen Stars

Re: CDC is not working in scheduling time

@saibabu ,what was the scheduled time? is it same time dropping and creating database ? if yes it could be problem.

Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Five Stars

Re: CDC is not working in scheduling time

Scheduled time was started 17-04-2019 4.00.00 am and it will run every one hour as Repeat task and Duration : - INDEFINITELY
Forteen Stars

Re: CDC is not working in scheduling time

Hi @saibabu ,

 

of course it could be a problem

 

Talend use trigger based CDC, so when you drop the database, it must drop triggers on tables

Screen Shot 2019-04-20 at 1.23.45 AM.png

 

 

 

-----------
Five Stars

Re: CDC is not working in scheduling time

But they are re-creating triggers every day @10 pm dropping & creating entire db again

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now