DB Migration: Last table data is blank (Output DB)

Highlighted
Eight Stars

DB Migration: Last table data is blank (Output DB)

Hi,

 

Using the following job and settings, I was able to dynamically migrate a whole database from MSSQL to MySQL.

However, I noticed that the last table's data is blank (only the table structure and column is migrated).

Please refer to the screenshots and attached job file below.

 

**The table and its schema is migrated, but no data inside the table. // I'm using a dynamic schema.

 

Job Design and DBInput

dbinputdbinput

 

DBOutput

DBOutputDBOutput

 

When I tried to logged each component, I found that the last table's data is loaded and pass on from tDBInput to tMap and also from tMap to tDBOutput, as the last table's data showed up in the log.

 

However, when I check the actual output database, only the last table contains no data at all.

 

Output DB's 1st Table - Have data

tbl2.png

 

2nd (last) table - No Data

tbl1.png

 

==============================================================

 


Accepted Solutions
Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb 

 

I figured out the reason why the last table data is not inserted.

Now that I change tDBOutput_1's setting,, from use existing connection (tDBConnection_2) to manual setup in tDBOutput and it work now.

Do you happened to know why? Does it have something to do with the trigger 'OnComponentOK'?

 

dbinputdbinput

 

View solution in original post

Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

if that is the case built in of tdbouput and do not use the existing connection.
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.

View solution in original post


All Replies
Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

For the last table, is there any error from log?
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb
do you mean the whole run log??
If yes, only the tables' data are printed (both).
Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

What has printed for the last table?
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb 

 

This is the log for the job run...

log.png

 

 

and here is single table job run.

the table data is logged here, however, no data is imported to the database. (only schema and column headings)

 

 

single_table.png

Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

As per the log there is no error right and even it has printed to console that last table data right? If yes it bit confusing. Can you try to run the had last table hard-coded one for testing to see that why it is not inserting data?
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb 

yes, there is no error in the console.
can you elaborate more about 'hard-coded one'?

 

Here I tried to do a debug run

and somehow nothing shows up in row5?

debug.png

 

can you tried importing this job? please check the attached file.

Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb 

 

I figured out the reason why the last table data is not inserted.

Now that I change tDBOutput_1's setting,, from use existing connection (tDBConnection_2) to manual setup in tDBOutput and it work now.

Do you happened to know why? Does it have something to do with the trigger 'OnComponentOK'?

 

dbinputdbinput

 

View solution in original post

Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

one thing could be it might rollback? But not sure how it could have triggered.
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

use the on component trigger from tdboutput.
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb 

 

I tried to use on component trigger from tdboutput as suggested however there's an error

 

outputerror.png

Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

You are doing wrong your dbconnection component should trigger before tdbtablelist.
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb 

 

Here I tried connecting it like this and there's no error.

However, no data is inserted to the output database (not only the last one but all)

oncomponent.png

Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

If you used tdbconnection component you no need to connect to tdbouput, since you are using Torrin, which will execute first.
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.
Highlighted
Eight Stars

Re: DB Migration: Last table data is blank (Output DB)

@manodwhb

but when I use existing connection from tdbconnection component, the last table data is not inserted (that's the initial setting)
Highlighted
Moderator

Re: DB Migration: Last table data is blank (Output DB)

if that is the case built in of tdbouput and do not use the existing connection.
Manohar B
Don't forget to give kudos/accept the solution when a replay is helpful.

View solution in original post

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Self-service Talend Migration: Moving from On-Premises to the Cloud

Move from On-Premises to the Cloud by following the advice of experts

Read Now

How to deploy Talend Jobs as Docker images to Amazon, Azure and Google Cloud reg...

Learn how to deploy Talend Jobs as Docker images to Amazon, Azure and Google Cloud registries

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog