Newbie questions

One Star

Newbie questions

Hi,
I am evaluating Talend Open Studio for my company. We are mainly a Microsoft shop and would prefer not to code in Java. I realize this is a Java code generator, we are perfectly fine as long as we can stay within gui development/configuration. My first two newbie/dumb questions are:
1. What is the difference between TOS Data Integration and ESB? I downloaded both and at a glance, didn't notice any apparent differences between the two. We are interested in a service bus, so I've been playing with TOS ESB for a couple of weeks now.
2. In a simple job with tMOMOutput followed by tMSSqlOutput, how do I make both components run within the same transaction? I would like to see both database commit and message acknowledgement happen within one atomic transaction. Either both happen or neither, under any circumstances.
Thanks in advance!
Moderator

Re: Newbie questions

Hi,
What is the difference between TOS Data Integration and ESB?

Only Talend ESB allows developers to develop, build, test and publish Java Web services, REST applications, data services and messaging routes ? all from a single user interface. http://www.talend.com/products/esb
In a simple job with tMOMOutput followed by tMSSqlOutput, how do I make both components run within the same transaction?

Do you want to get one output in Mssql DB and the other one is to ActiveMQ?
The workflow structure should be: input-->mian-->t<DB>output-->mian-->tMOMOutput
or input-->tReplicate-->mian-->t<DB>output
-->main-->tMOMOutput
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Newbie questions

Thanks Sabrina. Sorry, I've made a typo on my 2nd question above. The input is ActiveMQ and the output is SQL Server. So, my test flow looks like this:
tMOMInput --> tMap --> tMSSqlOutput
Can the ActiveMQ client acknowledgement and the database commit both happen within a single transaction? Is it possible that the server/process dies in between database commit and message acknowledgement so that a record is written to database but the message is not acknowledged? (<--this is undesirable for us)
Moderator

Re: Newbie questions

Hi,
Can the ActiveMQ client acknowledgement and the database commit both happen within a single transaction?

It can be achieved in a single transaction.
Such as:
tRowrenerator--mian-->tMomoutput-->oncomponentok-->tMOMInput --> tMap --> tMomOutput-->tMysqlOutput.
Can the ActiveMQ client acknowledgement and the database commit both happen within a single transaction? Is it possible that the server/process dies in between database commit and message acknowledgement so that a record is written to database but the message is not acknowledged? (<--this is undesirable for us)

It is possible that a record is written to database but the message is not acknowledged on ActiveMQ when the server/process crash suddenly.

Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Newbie questions

Thanks for the clarification Sabrina. Is there a way to implement this within Talend? I found some old posts here discussing about XA Transaction support. But those posts date back a couple of years. Has anything changed since then to support XA Trasnaction in Talend?
Moderator

Re: Newbie questions

Hi,
Perhaps this Talend ESB Mediation Developer Guide: TalendESBMediationDeveloperGuide53EN/3.24.2 Notesfor XA Transaction part will be helpful for you.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Newbie questions

Hi Sabrina,
That article seems to address messaging transaction only, it does not include database transaction. Please correct me if I am wrong.
In my example, for message acknowledgement and database insert, is it possible to make both happen, or neither happen under any circumstances? This is what I meant by "XA transaction" or "atomic transaction" across both messaging and database. This makes it impossible to have a database insert committed but message not acknowledged. Either both happen or neither happen under any circumstances, such as sudden server crash or network issues.
One Star

Re: Newbie questions

Hi again,
I found this article describing exactly what I was looking for (in the top section):
http://activemq.apache.org/should-i-use-xa.html
Is this supported by TOS ESB out of the box? Or do I have to write some Java code?
Moderator

Re: Newbie questions

Hi Chung Chang,
Sorry for the delay, so far talend don't support for XA transactions (two phase commit?) as the article mentioned both in Talend Open Studio for ESB and Talend Enterprise Subscription Version.
It is welcome to open a JIRA issue in the Talend ESB project of the JIRA bugtracker for a new feature.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Newbie questions

Thanks again Sabrina. I have opened a new issue as you have suggested: https://jira.talendforge.org/browse/TESB-10795#. I am surprised that this feature isn't more sought after. No infrastructure is 100% reliable. So, how do your users typically deal with inconsistent transaction states after an unexpected infrastructure crash?
Moderator

Re: Newbie questions

Hi,
Thanks for your feedback and your suggestion.
We plan to do XA transactions in the further job. So far, you can custom it using sprint.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: Newbie questions

Hi Sabrina,
I new to the Talend ETL tool i have a requirement as below
1. I want to do it a customized logging for all transactions which ever get fails.
2. When i will run job if founds any DB exception then its going to dis connect and not handling Exception/Error
I want to handle it
Please help me out i need your suggestion ASAP, much appreciated.
Moderator

Re: Newbie questions

Hi RizwanSharief,
Would you mind opening a new topic for your requirement and set an example with real working scenario in it.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Community Manager

Re: Newbie questions

Can you open the new topic in the following forum: http://www.talendforge.org/forum/viewforum.php?id=2
which is dedicated to ETL (data integration) rather than the ESB forum.
Many thanks