Full Maven Integration

Five Stars

Full Maven Integration

Hi,

 

Currently, I work on Continuous Integration for Talend Project. I use tools supplied by Talend solution: CI Builder, CommandLine, TAC and Nexus. The main flow seems to work, but I have an issue about Maven integration in project. 

 

My context:

Some routine in my project uses external modules to work. These modules are added to Talend Studio through Talend studio module management. All these modules are finally installed in folder located here:

<TalendStdioPath>/configuration/.m2/org/talend/libraries/

All these libraries are available on official maven repositories or on my company repositories.

To work  with Talend Continuous Integration solution, I had to copy all the libraries in the same folder of CommandLine.

 

My need:

  • How can I integrate Maven dependency in my project to resolve all dependencies through Maven engine and not by CommandLine/Studio engine ?
  • How can I specify Maven dependencies for my routines instead of External Module ? How can I generate corresponding Maven pom for my Continuous Integration life-cycle? 

I hope my description is clear. 

 

Regards 

 

Tags (2)

Accepted Solutions
Highlighted
Four Stars

Re: Full Maven Integration

I found a solution, i modified the template pom file "pom_codes_template.xml" and added my routine jar dependencies, this file can be found inside a jar "org.talend.designer.maven_6.2.1.20160704_1411.jar", this jar is located in commandline-studio/plugins folder, i am using talend 6.2.1 version, if you are using different version you should have one with that version.


All Replies
Moderator

Re: Full Maven Integration

Hello,

Thanks for posting your issue here.

We have redirected your issue and concern to our product experts and will keep you posted.

Many thanks!

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Four Stars

Re: Full Maven Integration

I have similar need, do you have any updates?

Five Stars

Re: Full Maven Integration

Hi,

I didn't have any update about my issue. No expert came to me to suggest a solution.
I hope have an answer soon.

 

Regards

 

Gabriel

Moderator

Re: Full Maven Integration

Hello,

We are working on your issue and will keep you posted.

Thanks for your time.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Employee

Re: Full Maven Integration

Hello,

 

cf:

  • How can I integrate Maven dependency in my project to resolve all dependencies through Maven engine and not by CommandLine/Studio engine ?

=> Not possible actually, since the studio will take care of retrieve all the jars / ensure that all dependencies are available.

Even if it's really similar, since right now the nexus url / user / password, need to be setup in the config.ini, then commandline will retrieve all the jars. (you don't need to copy all the content of .m2 actually)

 

 

  • How can I specify Maven dependencies for my routines instead of External Module ? How can I generate corresponding Maven pom for my Continuous Integration life-cycle? 

=> Not possible right now, not even sure there is really some workaround (we can customize the pom, but the list of jars generated / included in final build will not change).

But we're working on a possibility right now to setup a custom maven uri directly instead of a fixed jar name in the routine dependency. We'll still retrieve for now the jar in the same nexus repository as before (for now), but at least you can set the jar even as snapshot if you want and retrieve automatically the jar later.

 

 

Highlighted
Four Stars

Re: Full Maven Integration

I found a solution, i modified the template pom file "pom_codes_template.xml" and added my routine jar dependencies, this file can be found inside a jar "org.talend.designer.maven_6.2.1.20160704_1411.jar", this jar is located in commandline-studio/plugins folder, i am using talend 6.2.1 version, if you are using different version you should have one with that version.

Five Stars

Re: Full Maven Integration

Hello,

 

Thanks for your answers. They don't simplify my work. 

  • I don't see my extra modules on my nexus. Is there a procedure to push added module to nexus repository ?
  • Where can I find the documentation to connect my CommandLine to nexus ?
  • Is there a way to generate manually an extra modules from a jar to facilitate my industrialisation ?
  • How can I synchronize added external modules with Nexus ?

Moreover, I tried to use the Maven project properties to modify the template but no effect on Maven generated project. It is just used for generate standalone POM for job and there is no configuration for Routine POM. I am not against maintain POM for Job or Routine.

  • What the best way to manage extra dependencies with your Continuous Integration solution ?

In my humble opinion, the full integration of Maven in Talend is the best solution to integrate Talend in continuous integration life-cycle. Waiting improvement of this part, I would to find a maintainable solution for my team.

 

Regards,

 

 

Eight Stars

Re: Full Maven Integration

Can I have an update for this situation on 2019? 

Moderator

Re: Full Maven Integration

Hello,

Here exists a new feture jira issue about:https://jira.talendforge.org/browse/TUP-18373

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Eight Stars

Re: Full Maven Integration

Thanks, Sabrina.
I see that the ticket has been closed, isn't it? Where should I learn how to use the new feature then?

Regards,
Moderator

Re: Full Maven Integration

Hello,

This issue is fixed in V 7.0.1. I don't find any documentation about it and will check it with our R&D and DOCT team later 

Have you tried to setup a custom maven uri directly instead of a fixed jar name in the routine dependency in V 7.0.1?

Best regards

Sabrina

 

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Full Maven Integration

Can you explain , what you have done. I am also facing same issue

Six Stars

Re: Full Maven Integration

Good :-)

The other option is to add depedencies on each end every routines by then you got very big build of every jobs because all is included.

I was looking for compile-time only dependencies on routines/beans level and up to 6.5.1 this is the best option

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Have you checked out Talend’s 2019 Summer release yet?

Find out about Talend's 2019 Summer release

Blog

Talend Summer 2019 – What’s New?

Talend continues to revolutionize how businesses leverage speed and manage scale

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog