Talend Integration Cloud - Architecture

Overview

Talend Integration Cloud (TIC) is a fully-managed cloud option. It provides the great data integration, data quality, and Big Data features from Talend in a cloud environment that is managed, monitored, maintained, and secured by Talend. This article explains the different architecture models supported by Talend Integration Cloud.

 

SaaS & Cloud Platforms

This is the first kind of architecture, where Talend Studio is installed locally behind a firewall. Once the design activities are completed, the Jobs are published to the cloud. All the data routing between Studio and the cloud is performed over HTTPS and is very secure.

 

In this architecture, the integration flows and templates are run in the cloud. The Runtime engines are also in the cloud.

Saas_Cloud.jpg

 

Hybrid

This is the second type of architecture, which is very similar to the SaaS and cloud architecture. However, in this architecture, the runtime engine is behind the firewall - hence it is called a “remote engine”. So here Talend Studio, the remote engine, the apps, and the databases are behind the firewall.

 

In this architecture, the integration flows and templates are run in this remote engine, and when it comes to the runtime engines that actually perform the execution of these integration workflows, those are in the cloud. Here again the remote engine sends the status and the logs using HTTPS, and also leverages the on-prem apps and databases.

Hybrid.jpg

 

Virtual Private Cloud (VPC)

This is the third type of architecture, similar to the previous architecture with respect to Studio and cloud installation. The different is that the remote engine, the runtime engines, the on-prem apps, and the databases all lie within a virtual private cloud.

 

Here you could run the integration flows and templates in the remote engine in a VPC. The runtime engines would also be in the VPC.

VPC.jpg

 

Virtual Private Cloud - with Log Redirect

This architecture is an extension of VPC where the logs are redirected to a file and/or log services. The remote engine flows back only the status or approved log data to the Integration Cloud Service; the rest of the logs are redirected to a file on the remote engine or other log service within the virtual private cloud.

 

In this architecture, you could run the integration flows and templates in the remote engine in the VPC, leveraging both public and private sources.

VPC_Logging.jpg

 

Note: In all of these architectures, an additional capability is that you could leverage is the remote run/debug capability from Studio directly to the remote engine. The remote engine contains a JobServer, so if you need to do a remote run/debug from Studio to the remote engine, you need to configure the remote JobServer details in the Studio preferences to point to the remote engine.

 

For more information on configuring this capability, see the Remote run configuration (Talend > Run/Debug) page of the Talend Big Data Platform Studio User Guide.

Version history
Revision #:
11 of 11
Last update:
‎02-09-2018 12:07 PM
Updated by:
 
Labels (1)
Comments
Four Stars

Useful post to understand the TIC

Four Stars

Very useful post it help me to understand the TIC

Four Stars

Thanks for the post.

 

As a new Talend user, what do you mean when by "integration flows and templates are run", and what is the difference between the remote engine and a run-time engine? What do you mean when you refer to run-time engines as "that actually perform the execution of these integration workflows"?

 

Community Manager
Hi Ksharawi,

Thank you for giving a Kudos.

Integration flows are the jobs published from Studio and templates might be the ones imported from Talend Exchange. In hybrid architecture, Remote engine or the run time engine and the job servers placed behind the firewall (on prem) where the jobs are executed.

So lot of time, users ask if Talend cloud is running the job and if it is storing any of the data. However in reality the actual execution of the jobs happened in these remote engine (on-prem) and on-cloud engine(cloud).

Let me know if you have more questions

Thanks,
RekhaSree

Four Stars

Hello RekhaSree,

 

Thanks a lot, this answers my question.

 

Best regards,

Kareem