Overview

This document describes the process for working with the third-party libraries (custom libraries) used within Talend.

 

Custom libraries are needed for Talend components to work properly. These libraries are not delivered by Talend for contractual and licensing reasons, so they must be downloaded by the Studio.

The process is as follows:

  1. Configure TAC with the location of Nexus as an artifact repository and as the custom libraries repository (not localhost).
  2. Set up a user in TAC with appropriate privileges and access credentials to the source repository (SVN/GIT).
  3. Set up a project in TAC, and assign the correct privileges to the user.
  4. Start Studio.
  5. Log in to Studio as a user.
  6. [Optional] Publish from Studio – set up artifact repository details in Preferences.

 

This document also describes what you can do if Talend is installed in an environment without an internet connection.

 

Configure TAC

Among other things, TAC needs the following configuration:

artifact_repo.pngSVN.pnguser_libraries.png

 

Note: Software Update is not part of the custom library process. It is the mechanism used to download patches and updates.

software_update.png

 

So you have a configuration for Nexus, the artifact repository, but this configuration is only the place for TAC to look for artifacts for deployment in the Job and ESB conductors. Studio has a separate configuration for the artifact repository to which it publishes, as detailed later.

 

Set up a user

The user needs to be set up with the appropriate credentials for the source code system.

users.png

 

Set up a project in TAC

Set up a project as shown below:

projects.png

 

Assign rights to the appropriate user(s)

Assign rights to users as follows:

proj_auths.png

 

Start Studio

Before Studio starts for the first time, the Maven .m2 repository has not been set up:

start_studio.png

 

When Studio starts for the first time, it sets up its local Maven repository. This is where it stores libraries delivered by Eclipse plugins that are not transferred to Nexus with the custom libraries.

config.png

 

At this point, you can see that the custom libraries directory – stored in org.talend.libraries – is still (almost) empty:

org_talend_libraries.png

 

and the custom libraries in Nexus are also still empty:

libs.png

 

Log in to Studio

studio.png

 

This is what happens behind the scenes:

behind_scenes.png

 

Here you can see why it is a bad idea to leave hostnames as localhost. When the Nexus location is passed to Studio, it would get localhost as the host name, and because Nexus is not local to Studio, Studio is unable to find Nexus to interact with it. As soon as the user has logged on to Studio, you see some libraries appearing in Nexus, but not the third-party libraries:

release.png

 

Now Studio will request that the custom libraries are downloaded, which need to have the licenses accepted. These are the custom libraries:

customlibs.png

 

Clicking Finish starts the download and you must now accept the licenses, which is the crucial part:

licenses.png

 

Note: the licenses will need to be explicitly accepted whether or not Studio downloads the libraries from the internet or internally from Nexus. Then you see the libraries downloading into the Studio’s Maven repository:

libs_download.png

 

And you see them appearing in the Nexus custom libraries repository:

libs_nexus.png

 

A note about the command line

Even though the command line is running, the custom libraries do not appear automatically in its Maven repository.

 

It will only bring these across from Nexus when it needs to build them into an artifact, such as when a Publisher or Job Conductor task is run. As mentioned earlier, try to avoid using these methods to publish or build artifacts and as much as possible, publish artifacts to Nexus and deploy those.

 

Publish from Studio

Before publishing from Studio, you must configure the Nexus artifact repository in the Preferences.

prefs.png

 

Installing in an environment without an internet connection

Many companies do not allow software such as Talend to have access to the internet. These are most typically financial institutions, but any company or organization handling sensitive data will probably impose internet access restrictions. This obviously poses a problem for a mechanism that relies on access to the internet to access third-party files. How can you solve this problem?

 

There are two main ways:

  • Install at least one Studio into an internet-connected environment.
  • Upload a backup of the Nexus custom libraries repository.

 

Install Studio into an internet-connected environment

As you have already seen, once the third-party libraries are uploaded into Nexus, there is no further requirement for access to the internet, even for Studio. So the easiest solution is to install one Studio on a machine and configure it with access to TAC and the Nexus installation. This will enable third-party libraries to be downloaded into Nexus and from there, the libraries are accessible from any Studio that connects to TAC, regardless of whether or not that Studio has internet access.

 

Upload a backup of the custom libraries

This involves having an “external” installation with access to the internet – often your own laptop - where the third-party libraries have already been downloaded into Nexus. Once the libraries are loaded into Nexus in the “external” installation, a backup of the libraries is made and then restored to the unconnected installation.

 

The process to make a backup

  1. By default, Nexus is installed with the TAC, in the TAC directory:

    nexus.png

     

  2. Within that, navigate to the storage for the Talend libraries in the custom libs repository:

    customlibs_repo.png

     

  3. Make a Zip backup of the entire directory.

 

The process to “restore” the backup to the target installation

  1. Log in to the Nexus in the target installation as an admin user: the default username is admin, the default password is Talend123.
  2. Click the Repositories menu item on the left:

    repositories.png

     

  3. Right-click the talend-custom-libs-release repository and select Put out of service:

    outofservice.png

     

  4. Navigate to the storage for the Talend libraries in the custom libs repository, and unzip the backed up libraries archive:

    unzip.png

     

  5. Right-click the talend-custom-libs-release repository and select Put in service:

    putinservice.png

     

  6. Right-click the talend-custom-libs-release repository, select Rebuild Metadata, then right-click and select Repair Index.

    rebuild.png

     

The libraries are now available for any Studio that logs in to the TAC.

Version history
Revision #:
7 of 7
Last update:
‎09-05-2017 10:18 AM
Updated by: