Five Stars

How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

I have Talend job, where in I am using "tKafkaConnection", "tKafkaInput" and "tLogRow" components.
I am using Talend Open Studio for Big Data (6.3.1) Free Edition.

The value of Zookeeper Quorum list and Broker list for "tKafkaConnection" has been set properly for tKafkaConnection component.

Value of "Topic Name"has been set to "testKafkaTalend" and value of "Consumer Group Id" has been set to "Con_grp" for tKafkaInput component.
And then I have used "tLogRow" component.

I have built a Producer too in a Talend using tKafkaOutput component.

The problem that I am facing is the when I use the non-Talend Kafka Consumer (regular Kafka consumer running separately on the machine, and not use the tKafkaInput component), I can see the topics that are being produced via the Producer that I wrote.

But when I use the Talend based Kafka Consumer, I am not able to see any data being consumed.


Can anyone suggest what am I doing wrong with tKafkaInput? 
Attached are the configurations of the various components.

Pic1 - tKafkaConnection.jpgPic2 - Context Variables.jpgPic3 - tKakfaInput.jpgPic4 - tLogRow.jpg

 

Tags (2)
4 REPLIES
Moderator

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

Hello,

Is there any message printed on console when you use the Talend based Kafka Consumer(tKafkaInput component)?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

No, there is no message that is printed on console.

 

Ten Stars

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

version of Kafka same as You set in Talend?

Kafka is very sensitive for minor version changes

-----------
Five Stars

Re: How to use tKakfaInput component to consume the Topics generated via tKafkaOutput component

Yes, they are same in both producer and consumer.

For now, this has been resolved.

What I did was, created the whole job again, using the same components.

 

And it worked in the new job.

I am not able to see any difference in the old job and new job.
But it worked in the new job.