Five Stars

Unable to publish or fetch data from Kafka

Hi

 

I have build job in Talend integration cloud hybrid edition, create kafka topic and publish data. Below mapping.

I am able to create topic in kafka but when i try to consume data in the terminal, it doesn't show any data.

 

Similar issue with kafka consumer tkafkainput as well.

 

Kafka job.PNG

 

terminal consumer.PNG

 

tkafkainput : always keep running inspite of setting stop after 6000ms.

 tkafkainput.PNG

 

 

Kafka versions in sync.

 

 

 

4 REPLIES
Moderator

Re: Unable to publish or fetch data from Kafka

Hello,

The topic you created in kafka is Ok with you? Could you please indicate on which build version you got this issue?

Here is a jira issue about tKafkaOutput component does not implement "use existing connection"

:https://jira.talendforge.org/browse/TBD-3435

Can you use tkafkaInput(Built-in)-->tlogrow to read your data without using tKafkaConnection component?

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Unable to publish or fetch data from Kafka

Hello,

 

I tried without tkafkaconnection but No luck job keeps running without fetching any data.

 

 

kafkainput wo connection.PNG

 

Talend version: Tested in 2 products

Talend Integration Cloud Hybrid Edition

Version: 6.4.1
Build id: 20170623_1246

----

Talend Open Studio for Big Data

Version: 6.3.1

 

 

 

 

Regards,

M

One Star

Re: Unable to publish or fetch data from Kafka

Hello,

What is the job you are building? Show me how you build specificially

Moderator

Re: Unable to publish or fetch data from Kafka

Hello,

Would you mind posting your tkafkainput component setting screenshot into forum?

The tKafkaInput is a streaming component, even in the DI palette.

Could you please try use tfileoutpudelimited component as output and check the option "use output stream" on the tFileOutputDelimited to correctly put the data into the file?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.