Generate Java sources out of Talend sources

Six Stars

Generate Java sources out of Talend sources

Hello,

 

Using the Git Plugin, I was able to export my Talend Sources (with myJob.screenshot, myJob.item, myJob.properties in the process folder) into my git repository.

 

gitForTalend.PNG

 

Is it possible to generate the java sources out of that? And how?

 

Thanks for the help.

 

 

Eleven Stars

Re: Generate Java sources out of Talend sources

right click on the job and select build then you will generate code.


on a job, you can also click on the code tab to view the corresponding java code.

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Six Stars

Re: Generate Java sources out of Talend sources

So is it possible to put directly the java sources rather than the Talend sources into the git? I'am currently using the plugin Eclipse Egit to push Talend sources into my repository.

Forteen Stars TRF
Forteen Stars

Re: Generate Java sources out of Talend sources

For short, it is not.

You can try to generate xml files, but it's at your own risk.


TRF
Six Stars

Re: Generate Java sources out of Talend sources

It's not what the Talend CI Builder does, generating Java sources from Talend sources?
Eleven Stars

Re: Generate Java sources out of Talend sources

taled generate java code.

if you use git you want to reuse and upgrade your code.

so you need to save Talend code not java one.

 

 

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Six Stars

Re: Generate Java sources out of Talend sources

Thanks for the answers.

I want to be more clear on what I'am trying to achieve. Globally what I'am trying to do is to put my Talend sources into my git repository so then I can launch a Continuous Integration process using Jenkins and Maven. And I need to convert these Talend sources into Java sources so I can build them with Maven.

Is it possible to do so?
Eleven Stars

Re: Generate Java sources out of Talend sources

I think you can do that, but I did no do this before.

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Sixteen Stars

Re: Generate Java sources out of Talend sources

You will need command line for this (which is basically a headless Studio). The process is described here : https://help.talend.com/reader/HJ4q~FggK69ZfRYzb2Ftfg/CKGgNkuTaN8BB3xw~yCdWA (it is a long document and covers many different aspects of CI). 

 

From what I understand you will be able to generate your jobs using the MetaServlet API soon, but I don't believe this is available yet.

 

Before you go down the CI route, make sure you actually need it and that it is not just a buzzword that someone in your organisation has heard of. It is great when it is necessary, but can make your environment really convoluted if you don't need it and it is not managed well. 

Six Stars

Re: Generate Java sources out of Talend sources

Thank you rhall. What I don't understand yet is why the Talend CI Builder is needed in order to build java sources out of talend sources.Since the commandline is just a headless studio, why it can't transform natively the Talend sources into java sources just like a normal studio?

Sixteen Stars

Re: Generate Java sources out of Talend sources

You need to forget about the idea of generating Java sources unless you want to build the Open Source Edition from source and make changes to enable you to just build sources. Talend is not built to give you the Java source code. The Talend "source" is the XML, the Java source is an intermediary step between the XML and the compiled job. Continuous Integration is very possible, but it has to be done in the way described in the link I sent you.

Six Stars

Re: Generate Java sources out of Talend sources

In the link you sent me, there is this sentence : "Gets the sources (potentially at each commit) and converts them to Java classes with the Talend CI Builder". This is exaclty what I'am trying to achieve but I would like to know if it is possible to do this step only by using the CommandLine.

 

Like when I build a Job in the Talend Studio, I have a src/main/java/project_name/job_name containing the java source code in the zip file generated. Isn't that the same java code that would be generated by the CI Builder? If yes, isn't it possible to generate it only by using the CommandLine?

 

I'am very new to all this and I'am very thankful for any help I can get.

Sixteen Stars

Re: Generate Java sources out of Talend sources

The sources they are talking about are the Talend sources. If "source" is mentioned anywhere in Talend documentation it is talking about the very basic Talend source code (which is XML). From the XML it generates Java (which is in no way available to you unless you copy it from the Studio or retrieve it from a built zip file) and then it compiles. The Continuous Integration functionality requires the Command Line at the moment. You configure Jenkins to use the Command-Line. The Command-Line builds from the XML sources, it won't build from Java sources. While it obviously uses Java sources, it's purpose is to convert from the XML. There is absolutely no other supported way. Obviously you can hack it to get the Java source and hack it to compile from the Java source, but that will add a massive unsupported and unnecessary step into the process.

 

The best way of implementing CI (in my opinion) is to use what is described in that link. Essentially you use the Command-Line to compile your code into artefacts, store them in Nexus, deploy them and test, if the test passes promote from your "TEST" Nexus repository to your "PREPROD" or "PROD" Nexus repository, etc.

Can I ask what you think the benefits would be of trying to configure Jenkins to compile from Java Sources? It adds a lot of extra steps and I cannot see the benefit. If anything it adds a mass of configuration and maintenance when Talend patch and you have to reconfigure a load of Jars (at the very least).

Six Stars

Re: Generate Java sources out of Talend sources

"Can I ask what you think the benefits would be of trying to configure Jenkins to compile from Java Sources?"

 

This is not how it's done in the documentation, compiling from java sources?

 

https://help.talend.com/reader/mpVLyjeAbBh7eHWdz_0SlQ/HF8MMjUq3bllDlzOz2lqxw

 

Here you can read what the first Jenkins job does : "Gets the sources (potentially at each commit) and converts them to Java classes with the Talend CI Builder. This project calls the CommandLine application that generates all the sources of the Jobs and Tests you have created in your project."

 

So this is how I understood the workflow from the documentation :

 

Jenkins gets the Talend sources (the XML files) from git, then calls the commandline and the CI Builder to generate Java sources from those XML. Then with Maven we could run our test cases (at this point we need java sources to run test cases right?).

 

Am I mistaken at some point? If no, my purpose is to keep the same workflow but whitout the Talend CI Builder if possible. If it is not possible, I would like to know what the CI Builder does that the CommandLine can't do.

Sixteen Stars

Re: Generate Java sources out of Talend sources

 

The CI Builder is essentially a plugin for Maven. It works WITH the Command-Line. This is described in the link below.... 

 

https://community.talend.com/t5/Architecture-Best-Practices-and/Continuous-Integration-with-Talend-C...

 

With regard to testing, this video shows how this is intended to be handled (https://www.youtube.com/watch?v=Ygeg8Xt1x88). 

 

Why do you not want to use the CI Builder? Is this because you want to achieve continuous integration with the open source edition? If this is the case, the Command-Line will not be available to you either. 

 

You can probably hack this if you really want, but it won't be supported. You should find all of your Java sources in your Workspace. I believe these are up to date with each compilation (build) of each job. The folder is here (workspace\.Java\src\main\java\{project name}\{job_name}) in 6.2. As long as you have the related Jars available you will be able to compile from this code. However, as @fdenis said, you will need to make sure you update this code whenever you change a job in your Studio. You could save the job and this code would therefore be out of date with the saved Talend sources. It should be noted that CI Builder will also have a workspace folder where these files will be generated, but I do not have CI Builder installed so cannot check. I just want to reiterate that this is NOT the supported way of doing things.

 

Six Stars

Re: Generate Java sources out of Talend sources

"Why do you not want to use the CI Builder? Is this because you want to achieve continuous integration with the open source edition?"

If you already have a Talend license and a Command-Line, do you get automatically the CI Builder with it or do you have to pay for the additional plugin?
Sixteen Stars

Re: Generate Java sources out of Talend sources

That I am not sure about. Talend do change their pricing models quite regularly. I imagine you do have to pay extra for it, but you should check with your Talend contact

Six Stars

Re: Generate Java sources out of Talend sources

Ok. I already asked this question in another topic but I since I haven't received a convincing answer so I'am asking again here :

In the doc I can read "The CommandLine offers the same basic functionalities as Talend Studio and it is used to generate and export the Jobs developed with Talend Studio".

If it offers the same functionalities, why it can't transform Talend sources into Java sources and needs a a external plugin (Talend CI Builder) to do that?
Sixteen Stars

Re: Generate Java sources out of Talend sources

The Studio doesn't provide an option to just generate the Java Sources......that happens as a by-product of generating the built jobs (....if you know where to look). The Command-Line is exactly the same. Check out where it is installed and look at the folder structure. In fact you can even start it as a proper Studio......or at least you could when I last tried it. I no longer work with the Command-LIne and prefer to get my team to publish to our Nexus repository from their Studios manually. 

 

CI Builder is a Maven plugin built to work with Talend in the CI environment. From experience of playing around with it (not using it for very long I might add), it simply plugs the gaps between what Jenkins can do out of the box, what Command-Line offers out of the box and what Nexus (your Maven repository) can do. For a detailed breakdown of everything it does, you should speak to your Talend account manager.