Configuring Talend to copy data from PostgreSQL to MS SQL Server

Highlighted
One Star

Configuring Talend to copy data from PostgreSQL to MS SQL Server

Ok here is my setup.

I have multiple tables in PostgreSQL database. I have those identical tables in a separate schema in MS SQL Server, which I've identified via a PostgreSQL connection and MS SQL Server connection (w/ specific default schema).

I need to somehow copy or sync the data across the two databases (diff. servers, by the way) throughout the day. I would prefer to run it every 5 minutes to keep it in-sync as much as possible. I'm 100% new to this but can pick up kind of quickly... but this tool looks like it is pretty powerful so some guidance would help BIG TIME.

I want to copy all data in all tables from source (PostgreSQL) to target (MS SQL Server). What steps can I take or toolbox items do I need to use to achieve this. As of now I setup in 1 loading job, the two connections, and a tPostgreSQLRow which appears to allow a custom SELECT against just one table at a time. I'm dealing with about 60 tables with low transactional rates but the volume of tables seems high to do individually.

Can anyone guide me on where to get going with this?
One Star

Re: Configuring Talend to copy data from PostgreSQL to MS SQL Server

Ok quick update. I FINALLY figured out how to get a postgresql input pushing data in Update or Insert mode to the jdbcoutput pointing to MS SQL Server.

Now - the big question - is there a way to replicate this for all tables in the source database with the identical table in the target database? I don't want to have to create 60 jobs or 60 steps in this process if at all possible.

Also, does anyone have tips on how to make this more autonomous and keep it in-sync either through a 5 minute scheduler or through some other type of automatic syncing fashion?

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Introduction to Talend Open Studio for Data Quality

Find out about Talend Open Studio for Data Quality

Watch Now

Enabling Data Governance

Learn how to enable Data Governance

Watch Now

The Definitive Guide to Government Data Quality

Take a peek at the definitive guide to Government Data Quality

Read