Embedded Talend?

One Star

Embedded Talend?

Is it possible to use Talend embedded inside another application (similar to how CloverETL allows embedded mode)? The goal would be to programmatically generate an ETL pipeline and then have Talend run it.
Thanks!
Community Manager

Re: Embedded Talend?

Hello
tSystem component executes one or more command as you do on CMD. This component can typically used for companies which already implemented other application that they integrate into their processing flow through Talend.
Best regards
shong
----------------------------------------------------------
Talend | Data Agility for Modern Business
One Star

Re: Embedded Talend?

Instead of having Talend call another process using tSystem, can the reverse be done? Can I have my system configure and run a Talend pipeline - so Talend is embedded into my application?
Thanks!
One Star

Re: Embedded Talend?

You can export your job as a Java POJO package (or as an Axis WebService). I'm using this packages for my projects (in java) everyday.
One Star

Re: Embedded Talend?

Exporting as a Java POJO sounds great for running the job, but doesn't that still require the UI to output the POJO?
Is there a way to have Talend programmatically generate the ETL pipeline, and then generate the runtime Java POJO? For example, suppose I know I have two sources of data and want to join them and output them to a file. Does Talend have an API for creating a new ETL pipeline, configuring the two sources and sink, adding the join, and exporting the pipeline - all without requiring user interaction?
Thanks!
Five Stars

Re: Embedded Talend?

Hi,
Talend Export Job Scripts feature allows you to deploy and execute a job on any server, regardless
Talend Open Studio.
The job scripts export adds to an archive all the files required to execute the job, including the .bat and
.sh along with the possible context-parameter files or relative files.
The export type include also Java POJO, Axis Webservice (WAR) and Axis,Webservice (Zip).
Best look
One Star

Re: Embedded Talend?

Perhaps my question is not clear. I understand that I can export an ETL pipeline I've created in the UI and then run that job embedded in another Java application.
My question:
Can I create a new job, configure it, and export it using *only* the Talend API, not using the Talend UI at all?
One Star

Re: Embedded Talend?

Is there a way to have Talend programmatically generate the ETL pipeline, and then generate the runtime Java POJO? For example, suppose I know I have two sources of data and want to join them and output them to a file. Does Talend have an API for creating a new ETL pipeline, configuring the two sources and sink, adding the join, and exporting the pipeline - all without requiring user interaction?

I mean you need own task generator, for recompute all in the space Smiley Happy ...
Create a job for a new data transfer, export the job to POJO package and embede generated package to your own app. If you need transfer something from somewhere to someplace, you need "somegenerator". But I mean, with well configured and prepared Talend job, you can use only one package and "something" define via properties and contexts.
One Star

Re: Embedded Talend?

mikem2005, did you ever get your question answered? I'm looking for the same thing. As far as I can tell the only way to create a new job is through the TalenD UI.
There doesn't seem to be an API exposed for other applications to create jobs.
One Star

Re: Embedded Talend?

Looks like Pentaho kettle uses an API. May please try to find out .
Please let us know in case it is useful.
Thanks and Regards,
Pravu Mishra.
One Star

Re: Embedded Talend?

Well, TOS is opensource, it'll be a pain, but you can try to look at the jar files & their API.
A talend job is basically a xml file (see .item files) serialized from an EMF model (see the eclipse project).
TIS (the non free version of TOS) has a console which can then compile the project.
One Star

Re: Embedded Talend?

Hi Morbo,
The question is not about design of Talend and using Eclipse modelling framework(EMF) or java classes without having intergace implementation.
The question is about whether there is/are APIs which can be used independently from out side of TOS. Please let us know in case you have some idea about the API/s exposed by talend.
There are many third party APIs those are used by talend. Might be there are APIs developed by talend on top of third party API. Do you have any idea about it?
Any way talend is an excellent software.
Thanks and Regards,
Pravu Mishra.
One Star

Re: Embedded Talend?

Hello,
I don't know much, I just did some digging on some area.
All I'm saying is that's it's already possible to manipulate the .item file in a regulated manner (thanks to EMF) and without TOS. I'm made a basic gui launcher wich uses this to read the item file, deduce the context parameters to expose.
See this about the EMF model of talend http://www.talendforge.org/forum/viewtopic.php?id=6606
And this was a start of discution about talend use of GEF : http://www.talendforge.org/forum/viewtopic.php?id=7237 but I didn't go further.
Feel free to share your investigations Smiley Tongue.
Regards

Re: Embedded Talend?

Hello,
I know this post has been asleep for a while but the question has never been properly answered.
So, once again, is there today a way to define programmatically talend Job and execute them from my own java application ?
I checked the documentation, there is a guide for components development but that is not what i want, there is (to my knowledge), there is no available API, nothing useful on the maven repositories so...
Did i miss something ? Is it even possible to do what i'm asking ? I saw some tutorials about running already defined Talend Job from java application, i can maybe turn aroud this to build my own application but i'd like to be sure...
Sorry for the inconvenience,
Best Regards, Cyprien.