How to load message on Kafka Cluster by using Talend

Five Stars

How to load message on Kafka Cluster by using Talend

I would like to share that how can we produce messages on Kafka Cluster by using Talend.
How to do ?
1.It is pretty simple ,we have to   give Kafka Server detail  with port  in Kafkaconnection component and we have to write topic name in Kafka output component .You can see the screenshot as I have  subscribed to  "test" topic.
2.The datatype of your input should be byte as kafka output takes only "byte" datatype .If our input datatype  is a string then we have to use tjava or tjavarow component to perform the transformation (java code "output_row.serializedValue = input_row.users.getBytes()) (https://help.talend.com/search/all?query=tKafkaOutput&content-lang=en)
Job screenshot



Moderator

Re: How to load message on Kafka Cluster by using Talend

Hi harkirat,
Thanks for sharing your screenshots and experience on forum.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Six Stars

Re: How to load message on Kafka Cluster by using Talend

This is good information. I played around with this over the weekend and it opens up doors do performing "real-time" data provisioning.
Four Stars

Re: How to load message on Kafka Cluster by using Talend

Thanks for sharing.

I tried this and the Kafka output key is null, also the headers are empty as shown below.

Please advise how to fix this?  Thanks.

Screen Shot 2019-09-19 at 1.12.15 PM.png

One Star

Re: How to load message on Kafka Cluster by using Talend

May you leave the ETL listening all the time for new messages? if possible, how do you do?

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Put Massive Amounts of Data to Work

Learn how to make your data more available, reduce costs and cut your build time

Watch Now

How OTTO Utilizes Big Data to Deliver Personalized Experiences

Read about OTTO's experiences with Big Data and Personalized Experiences

Blog

Talend Integration with Databricks

Take a look at this video about Talend Integration with Databricks

Watch Now