Talend Connect
Virtual Summit
JOIN US!
And visit the Customer
& Community Lounge.
May 27-28, wherever you are.

Custom Component Rules

Highlighted
Five Stars

Custom Component Rules

Hi,

I'm considering developing a custom component for Data Integration. Are there any guidelines for custom components if I want to upload them to Talend Exchange?

 

Specifically, can I make a component which will send a file to a server where I can process it with a propietary algorithm and return the data? Or is including the algorithm in the component itself the only allowed method?


Accepted Solutions
Highlighted
Employee

Re: Custom Component Rules

Hi @siva227 ,

 

there is a http client in the kit out of the box ([1]) so you can send it directly or write an output component doing that (likely better).

 

[1] https://talend.github.io/component-runtime/main/1.1.13/services-built-in.html#httpclient_usage

 

 

Romain
Talend Component Kit Documentation: https://talend.github.io/component-runtime/

View solution in original post


All Replies
Highlighted
Moderator

Re: Custom Component Rules

Hello,

Are you trying to create one new component use the Component Kit starter and upload your custom component into Talend Exchange using Talend Component Starter?

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Five Stars

Re: Custom Component Rules

Hello,

Yes, I want to create a component and upload it to Talend Exchange. I started by following this document - https://help.talend.com/reader/q9sIDoqWg3R5cxZE38OKrQ/BiTmgCx_O7lH_OpFspAdrw

 

Thanks,

Siva

Highlighted
Moderator

Re: Custom Component Rules

Hello,

This documentation you provided is talking about how to create a custom component by javajet not component kit.

Please have a look at this documentation about:TalendHelpCenter:Uploading a component you created to Talend Exchange

From version 7.0 onwards, Talend is introducing a new component creation framework named Talend Component Kit. If you plan to create new components, it is recommended to upgrade to version 7.0 and use the new framework.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Five Stars

Re: Custom Component Rules

Thanks Sabrina, I will use the Component Kit. But getting back to my main question, are there any guidelines / rules or restrictions regarding what kind of components can be uploaded to the Talend Exchange?

 

Specifically, can I make a component which will send a file to a server where I can process it with a proprietary algorithm and return the data?

 

Highlighted
Moderator

Re: Custom Component Rules

Hello,

Could you please let us know if this related topic helps?

https://community.talend.com/t5/Component-Development/Exchange-Component-made-by-Talend-Component-Ki...

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Five Stars

Re: Custom Component Rules

Hi,

That's not what I am trying to find out, but it is useful too, thanks. That post is from last year, has a streamlined installation procedure been introduced since then? (for users installing custom components)

 

Regarding my main question - I just want to know whether I can create a custom component which will send a file to a private server. For comparison, the tGoogleDrivePut component can upload a file to Google Drive. Can I create a component which will upload a file to a particular IP address?

Highlighted
Moderator

Re: Custom Component Rules

Hello,

I will re-direct your issue to component development PMs and then come back to you as soon as we can.

Thanks for your time.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Highlighted
Employee

Re: Custom Component Rules

@siva227 Hi Siva,

 

Yes you can write a component reading a file, then your component does whatever you need and processes whatever you want to output the records to the next component. This sounds like it will be an input component (mapper in Talend Component Kit) but a processor (in the middle of a job) could fit too depending what where you read your configuration from (either from properties or incoming records).

 

Hope it helps,

 

Romain
Talend Component Kit Documentation: https://talend.github.io/component-runtime/
Highlighted
Five Stars

Re: Custom Component Rules

@rmannibucau 

Hi Romain,

I'm building it as a processor component. My question was what kind of  processing can be done, specifically - Can the component :

  1. read a csv file as a file object? and
  2. send the file object over a http POST request?

Thank you,

Siva

Highlighted
Employee

Re: Custom Component Rules

@siva227 in the "kit" we try to avoid any representation which can be "local" and we enforce to go through "Record" (or JsonObject) abstraction to try to guarantee that. It is also not possible to send a stream since the processing can be distributed and it wouldn't work. However, nothing prevents you to send a record describing the file (caricaturally {filename: /path/to/file}) and have a http component able to read from the file path in the record to do a post.

Romain
Talend Component Kit Documentation: https://talend.github.io/component-runtime/
Highlighted
Five Stars

Re: Custom Component Rules

Thanks @rmannibucau. I wanted to include the post part within the component so that users wouldn't have add any more components. To clarify, using a http component is the only way to send a file? It can't be done from within the custom component?

Highlighted
Employee

Re: Custom Component Rules

Hi @siva227 ,

 

there is a http client in the kit out of the box ([1]) so you can send it directly or write an output component doing that (likely better).

 

[1] https://talend.github.io/component-runtime/main/1.1.13/services-built-in.html#httpclient_usage

 

 

Romain
Talend Component Kit Documentation: https://talend.github.io/component-runtime/

View solution in original post

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog

Downloads and Trials

Test drive Talend's enterprise products.

Downloads