I have Talend job, where in I am using "tKafkaConnection", "tKafkaInput" and "tLogRow" components.
I am using Talend Open Studio for Big Data (6.3.1) Free Edition.
The value of Zookeeper Quorum list and Broker list for "tKafkaConnection" has been set properly for tKafkaConnection component.
Value of "Topic Name"has been set to "testKafkaTalend" and value of "Consumer Group Id" has been set to "Con_grp" for tKafkaInput component.
And then I have used "tLogRow" component.
I have built a Producer too in a Talend using tKafkaOutput component.
The problem that I am facing is the when I use the non-Talend Kafka Consumer (regular Kafka consumer running separately on the machine, and not use the tKafkaInput component), I can see the topics that are being produced via the Producer that I wrote.
But when I use the Talend based Kafka Consumer, I am not able to see any data being consumed.
Can anyone suggest what am I doing wrong with tKafkaInput?
Attached are the configurations of the various components.
Is there any message printed on console when you use the Talend based Kafka Consumer(tKafkaInput component)?
version of Kafka same as You set in Talend?
Kafka is very sensitive for minor version changes
Yes, they are same in both producer and consumer.
For now, this has been resolved.
What I did was, created the whole job again, using the same components.
And it worked in the new job.
I am not able to see any difference in the old job and new job.
But it worked in the new job.
Watch the recorded webinar!
Accelerate your data lake projects with an agile approach
Create systems and workflow to manage clean data ingestion and data transformation.
Introduction to Talend Open Studio for Data Integration.