Using Apache Kafka for Message Oriented Middleware with Talend ESB

Highlighted
Six Stars

Using Apache Kafka for Message Oriented Middleware with Talend ESB

I just setup Apache Kafka 3 node experimental cluster and would like to use it as replacement for default ActiveMQ message broker for asynchronous message bus, inter-service communication is one of purposes, we will also connect the cluster into other messaging broker later.
Is there someone who tried this before and have some pros/cons/conclusions to state here base on your experience?
Thanks.
Thirteen Stars

Re: Using Apache Kafka for Message Oriented Middleware with Talend ESB

Not a huge experience, just some experience
1) Kafka - is extremely critical to minor versions changes, so depending from Your Talend version - You could be restricted to 0.9 or 0.8 and at the same time, other software manufacturer start support Only 1.0+ .. For example I several time change ActiveMQ from 5.3 to 5.14 and Job continue work, with Kafka You never could be sure at this time
2) Kafka support only topics. It not a issue, just difference between Brokers, but could be important
-----------
Six Stars

Re: Using Apache Kafka for Message Oriented Middleware with Talend ESB

Thanks for your reply.
Ok, I agree, to keep up with version dependency could be troublesome, especially when Kafka is still under wild dev when compare to ActiveMQ (even here was already dead Apollo and current Artemis) .
Regarding topics, yes, if required point-to-point, then only one consumer/subscriber to topic is required.
I will try to do some basic 2-3 services in chain and report results with loadtest to see throughput, issues, etc.
Thanks

Cloud Free Trial

Try Talend Cloud free for 30 days.

Tutorial

Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.