Talend and RabbitMQ

One Star

Talend and RabbitMQ

In our current setup (where we are now introducing Talend) we have RabbitMQ already in place as our central event broker. We now want Talend to start subscribing to queues in RabbitMQ, but this wasn't as easy as I'd initially hoped.
It seems that the ConnectionFactory in Talend is using AMQP 1.0, while RabbitMQ is using AMQP 0.9. There is an experimental plugin for RabbitMQ that introduces AMQP 1.0, but I would rather have Talend use AMQP 0.9 than having experimental plugins in production. For obvious reasons Smiley Happy
What is the best way to having Talend interacting with RabbitMQ?
Moderator

Re: Talend and RabbitMQ

Hi darkhorse,
We have redirected your issue to ESB experts then come back to you as soon as we can.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Employee

Re: Talend and RabbitMQ

Hi,
the best way in this case it to use the cMessageEndpoint component (Mediation Perspective in the Talend Studio) and to use the 'RabbitMQ'-component' (select 'rabbitmq' in the advanced tab of the cMessageEndpoint component - (add a new line to the table then you see a combobox which contains 'rabbitmq'.  ) In this case you can access RabbitMQ via AMQP within a Talend-Route (the camel-rabbitmq component uses the RabbitMQ AMQP client). 
The cAMQP component can only be used with an AMQP 1.0 compatible broker as you also mentioned / found out. 
More information on options of the camel-rabbitmq component can be found also here: http://camel.apache.org/rabbitmq.html 
Would be nice to get a short feedback if this has solved your questions and allowed you to progress on your tasks. 
Thanks,
Dietmar 
One Star

Re: Talend and RabbitMQ

Hello dietmar,
Am using Talend ESB to publish messages to Rabbit mq.
earlier i was using components like cConnectionFactory,Ctimer,CAMQP,Clog.
In your prev comment i read about cMessagingEndpoint. i enabled it and the rabbitmqcomponnet.
Can you tell me now what all componnets to use now, and how to use.
I order to connect to rabbit mq, ill need to pass url, exchange name, routing key, data
One Star

Re: Talend and RabbitMQ

Hi,
My ESB open studio does not have mdiation perspective option. Also i cannot find CmessageInput component. Any pointer?
Thanks
Thirteen Stars

Re: Talend and RabbitMQ

what version number?
in 6.2. it named - Routes, component will be available when You create (open) new route
-----------
One Star

Re: Talend and RabbitMQ

Hi vapukov,
Thanks a lot.
So I have the job in place now. My URI looks like "rabbitmq://XXXXXXX:5672/Myqueue?username=myuser&password=mypass".
But when I run I get
Starting job rmqnew at 15:12 07/11/2016.
connecting to socket on port 3356
connected
And it pretty much stays like that until I kill it. Any idea?
Thanks
Thirteen Stars

Re: Talend and RabbitMQ

I not great expert in RabbitMQ, but I use different settings, You can see from 2nd picture - You use not full format

    
-----------
One Star

Re: Talend and RabbitMQ

Hi Vapukov,
You have been a lot of help so far. My module is working now. Problem was with my URI. I had to pass more parameters to match exactly with the exchange set up.
Now can anyone suggest the best way to dump these messages to a database?
Thanks,
Thirteen Stars

Re: Talend and RabbitMQ

Add runJob, and parse header and body to Job, inside Job as usual - parse Json, do anything what You need    
  
-----------
One Star

Re: Talend and RabbitMQ

 
I have these 2 components now. I assume there is no need for a JSON parser as I want the whole payload to be loaded in Oracle.  Again I am a bit stuck. Nothing gets passed on to the oracle table. 


Thirteen Stars

Re: Talend and RabbitMQ

It work, how You pass body?
-----------
One Star

Re: Talend and RabbitMQ

Hey Vapukov,
Its working now . I am getting something like this in my oracle table

So now I have a tJsonextract component as I want all the tags in JSON payload to come as coumns in oracle table. Or atleast the ones which I define while creating the Oracle table. Can you advice what should be the idle configuration for this tjsonextract component. 
I am a bit confused as I saw in your previous screen shot that you used the tjsonextract component directly. However in  talend documentation for this component, there is a twritejsonfield before tjsonextract

Thanks
Six Stars

Re: Talend and RabbitMQ

Hello,

 

I am working on something quite similar but I don't manage to make it work ... I succeed in getting messages from rabbitmq, that means my cMessagingEndpoint component is well configured. But then i need to extract all informations contained in my message (Headers and Payload). I have built the same route and job as you but i still don't know how to configure these 3 components : cSetHeader, cProcessor, cExtractJsonFields.

 

You can find attached my current work.

 

Thanks.

Six Stars

Re: Talend and RabbitMQ

Hello guys,

 

I'm still stuck on that purpose... noone knows how to manage that ?

Seven Stars

Re: Talend and RabbitMQ