One Star

[resolved] How to use tkafkaInfput on talend open studio for big data v 6.0

Hi Guys, I am new to Talend and I want to use the tkafkainput component. Iam using talend open studio for big data v 6.0 free trial and there is no storm job on this talend studio. I found the example how to run tkafkainput but its use storm job to run it.
Can anyone give me a hint how to use the component?? Thanks
1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: [resolved] How to use tkafkaInfput on talend open studio for big data v 6.0

Hello!  The tKafkaInput can be used without Storm in version 6.0.
The serializer.encoding warning that you see is not a problem.
Since a Kafka topic doesn't have an "end", the DI job will never finish by itself, but you can specify the consumer timeout in the tKafkaInput to stop the job when the topic is "idle" after a length of time.
All my best, Ryan
5 REPLIES
Moderator

Re: [resolved] How to use tkafkaInfput on talend open studio for big data v 6.0

Hi,
TalendHelpCenter:tKafkaInput component will be available in the Palette of Talend Studio on the condition that you have subscribed to one of the Talend solutions with Big Data.
Best regards
Sabrina
--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
One Star

Re: [resolved] How to use tkafkaInfput on talend open studio for big data v 6.0

Hi Sabrina,
Yes the tkafkainput component is available in the palette but when I use this component there is warn coming out and its says : kafka.utils.VerifiableProperties - Property serializer.encoding is not valid . And I need to kill the job for stopping it. Do you have any idea about this problem?? I just want to try to take out some data from kafka to delimited file.
Thanks
Employee

Re: [resolved] How to use tkafkaInfput on talend open studio for big data v 6.0

Hello!  The tKafkaInput can be used without Storm in version 6.0.
The serializer.encoding warning that you see is not a problem.
Since a Kafka topic doesn't have an "end", the DI job will never finish by itself, but you can specify the consumer timeout in the tKafkaInput to stop the job when the topic is "idle" after a length of time.
All my best, Ryan
One Star

Re: [resolved] How to use tkafkaInfput on talend open studio for big data v 6.0

Hi Ryan,
I tried to produce delimited file from tkafkainput but it's only produce the file only. there is no context inside the file. 
At first I think this is a encoding problem. And where can I set the consumer timeout??
Thanks
One Star md
One Star

Re: [resolved] How to use tkafkaInfput on talend open studio for big data v 6.0

Hello 
I am  also  trying to consume Kafka messing queue by using tKafkaInput. How to extract data from Kafka queue using Talend Big Data OS. Is there any ideal example to understand this functionality in Ta-lend studio?
Thanks