Newbie needs some guidance on architecture - Best Practices Question

One Star

Newbie needs some guidance on architecture - Best Practices Question

Best Practices Question
I just downloaded and starting using Talend Open Studio this week. I have a couple questions. First, let me describe what I am trying to accomplish:

We have a "Product Configurator" system that communicates with other systems using web services. The Product Configurator is both a consumer and Host for web services. Both consumer and host are asynchronous. We also have a legacy ERP system (using MSSQL server 2005). I am integrating these two systems.

Use Case:
1.Order is placed in Product Configurator, which sends a SOAP message (fireOrder) containing the Order ID.
2."Middleware" that hosts the web service reads the Order ID and connects to the sending server's (Configurator) web service and asks for the order details using a SOAP message (getOrder).
3. The Configurator's web service sends the response(getOrderReply) back to the "middleware".
4. The "Middleware" parses out the data, does some transformation, opens up a connection to the ERP's MSSQL database, saves the order data.
5. Once this task is completed, the "middleware" sends a "success" response back to the original web service call (fireOrderResponse).

There are other use cases such as updating Item master items in the configurator from the erp system, updating customers from the erp system, updating quote data from the configurator to the erp system.

So, here are my questions:

 1. I understand that to host web services using Talend, I need to host them using the Enterprise Service bus, using Open Studio for ESB to set up the web service. Is there any need for me to use the Open Studio for Data Integration?

 2. What is the best way to structure the use case example that I gave? Should I encapsulate steps 2-4 in another job and call that job from the web service job? How would a set of data cross between jobs? What is the general rule of thumb when you have a mix of database access, web service consumers, and web services that are all part of a longer process?

Thnx

Harold
Community Manager

Re: Newbie needs some guidance on architecture - Best Practices Question

Hi 
1) Talend Open Studio for ESB covers the basic functionality of Open Studio for Data Integration, so only Open Studio for ESB is [font=Verdana, Helvetica, Arial, sans-serif]sufficient for you to build the webservice.[/font]
[font=Verdana, Helvetica, Arial, sans-serif]2) You need to build 2 SOAP webservice, one for [/font]Product Configurator (step1) and another for Middleware (step2-4), refer to the component documentation and learn the following components:
tESBProviderRequest
tESBProviderResponse
tESBConsumer


[font=Verdana, Helvetica, Arial, sans-serif]Best regards[/font]
[font=Verdana, Helvetica, Arial, sans-serif]Shong[/font]
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Newbie needs some guidance on architecture - Best Practices Question

I think you will find that your use case is beyond the scope of what you can achieve solely in Jobs (Integration perspective). Although Jobs can be used for longer-running processes, support for asynchronous SOAP is non-existent. tESBConsumer (i.e. the component used to invoke SOAP services - which you would use to invoke getOrder) is designed for synchronous interactions.

You might want to take a look at implementing your solution using Routes (Mediation perspective). Particularly take a look at the cCXF component. This is based on the CXF component in Apache Camel, and gives much more configuration possibilities.

My initial thoughts would be to implement steps 2-5 as a Route which, after receiving the getOrderReply, invokes a Job (cTalendJob) to do the processing in step 4.

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Talend API Designer – Technical Overview

Take a look at this technical overview video of Talend API Designer

Watch Now

Getting Started with APIs

Find out how to get started with APIs

Read