Talend Migrations: What you need to know before upgrading

Highlighted
Employee

Talend Migrations: What you need to know before upgrading

If you've been planning a Talend Migration or Upgrade, you've come to the right post.

 

I'm Arthur from the Support team. I got together with our Talend Support Engineers and our community content editors and discussed how we could make migrations/upgrades easier for everyone using Talend. Following this intro will be a list of URLs and topics that will hopefully make upgrading easier and smoother for all of our users, new or old. We'll continue to add to this list, so keep checking back! And without further ado...

 

Migration Guide Links:

7.1.1 Migration Guide

7.0.1 Migration Guide

6.5.1 Migration Guide

6.4.1 Migration Guide

6.3.1 Migration Guide

 

For all versions

Backup the database:Before you do anything, do not forget to backup the database. Even if you do accidentally go to the DB Config page, it will notice that you are on a lower version of Talend first. Stop here and backup all of your data before clicking on "Proceed to migrate".
 
Jar connectors: 

Be sure to check the version of the database that you are migrating from and take the appropriate version when placing it into the Tomcat: \tac\apache-tomcat\lib. 

 

Large databases:

Normally if the database has a lot of data, there is one particular table that you can truncate in order to allow the database to migrate. This table is the taskexecutionhistory. 

 

Cannot login to TAC:
After migrating, some users experience that they can no longer log into TAC because there are more users than allowed. As a workaround to this, what you can do is manually log into the database via the command prompt or terminal and do a "select * from user;". After viewing how many users you have and verifying your license, disable all extraneous users by using "update user set deleted = 1 where user = '<id>'". Please do this for as many users as you need to as is fit for your license. For example, if you have a 5 user named license, please disable ALL but 5 users in this way.
 
Migration from DSC to TDS

Per discussion in the community article (https://community.talend.com/t5/Talend-Tech-Internal/Data-Stewardship-Job-migration-on-Talend-7-0/m-...), The Data Stewardship Console has been retired in Talend 7.0, so the migration from data stewardship console to Talend data stewardship needs to be handled prior to Talend 7.0.

 

The migration methodology in the link below works fine for Talend 6.5 and older, but not 7.0

 

https://help.talend.com/reader/4dvqs4nNNNRnRFAjqeZCDQ/4ysSCD1JfndVMsKyoHfBrw

 
6.4.1 Specific
 
Database migration from 6.2.1 to 6.4.1 fails with Oracle 12c

This is a known issue with dirty records in the database regarding schema. You may get an "array out of bounds error"

The workaround is to truncate table "usergroup_user" and "user_usergroup" before proceed to migration

Talend's official suggestion is that if you are planning to migrate to 6.4.1: there is a patch available for the issue. Please open a Support Case and ask for patch TPS-2548.

 

6.5.1 Specific
 
Deprecated: SVN/GIT-based tasks in Job Conductor from Talend 6.5.1

Customer should avoid creating SVN/Git based tasks in job conductor as this feature is being deprecated. The best practices suggest that the customers should use CI builder to build and publish the artifacts and use these published artifacts to create the tasks in Job Conductor.

 

7.0.1 Specific
 
Recommended patch for Talend 7.0.1 ESB Studio/CommandLine

All customers planning to use Talend 7.0.1 ESB should apply the cumulative patch TPS-2462. Read more about it in the below documentation.

https://community.talend.com/t5/Migration-Configuration-and/Recommended-patch-for-Talend-7-0-1-ESB-S...

If you require this patch, please open a Support Case and request TPS-2462.

 

Publisher (in TAC) is being deprecated in Talend 7.0.1:

If a customer wants to use the "Publisher" in TAC they need to install Nexus 2 on their own as Publisher is not supported with Nexus 3 which comes with our installation package. The customer can either do a fresh install of Nexus 2 or can use a Nexus 2 that is working with their current Talend version.

 

You can find the list of new features and deprecations in the link below:

Talend 7.0.1 release notes

 

Build-aggregator.pom does not exist error after migrating to 7.0.1

This is a known issue.

To resolve the issue, go to File -> Edit project settings -> Build -> Maven -> Force full re-synchronization poms. This should resolve the Build-aggregator.pom does not exist errors after migration.

There is also a patch that was created for the issue.

After opening a Support Case please request: TPS-2571

 

All of the jobs on Studio 7.0 are not editable for LDAP users

This is a known issue where the job is always locked when LDAP user login to studio. (https://jira.talendforge.org/browse/TMC-13493)

The workaround is to use the value of the "Login field" configured on the TAC to login to Studio

 

LoginField.PNG

Login.PNGstudioConnection.PNG

 

If you run into the issue above, please open a Support Case and request for patch: TPS-2584

Third Party

Nexus

 

Clear steps from R&D is how to migrate the artifacts from Nexus 2.11 (Talend 6.3.1) to Nexus 3.9 (Talend 7)

Upgrade 2.11 to 2.14.9 which can be downloaded here: 
https://www.sonatype.com/download-oss-sonatype

overridethisfolder to upgrade.JPG

Just overwrite your folder in the example path with the new version folder and this will upgrade your nexus but you may need to fix the service:
ie. /talend/tac/nexus/nexus-2.14.3-02

Enable the upgrade agent capabilities

 

How to Enable the upgrade agent:


While both nexus 2.14.9 and 3.9 are running navigate to your nexus 3.x website making sure that both nexus sites do not have conflicting ports. Ports can be changed here:

In 3.x
ie. 7.0.1\tac\Artifact-Repository-Nexus-3.9.0-01-win64\sonatype-work\nexus3\etc\nexus.properties

In 2.x
ie. 6.4.1\tac\Artifact-Repository-Nexus-V2.14.3-02\nexus-2.14.3-02\conf\nexus.properties

In the UI of Nexus 3.9 go ahead and delete all of the repositories that you are wanting to import in. Go to the capabilities screen and activate the upgrade tab of the preferences screen. Then you will be able to point to the Endpoint and the access token that you get from Nexus 2.14.9 and complete the migration. Video from sonatype on the steps post upgrade to 2.14.9:

https://www.youtube.com/watch?v=q6qkj8UiSRY

 

Note: After migrating nexus from 2.11 - 2.14 - 3.9, the roles for talend-updates-admin in nexus 3.9 are very limited, include all the roles. In TAC, update the local username & password, it should clear all the errors from software update configuration section. 

 

Important Note Regarding Nexus 3

It is paramount that you move to Nexus 3.14 if you are planning to stay on Nexus 3. As mentioned above Nexus 3.9 has limitations and in particular has issues regarding performance. Do not move to higher versions of Nexus as 3.14 is the highest version supported by Talend as of Talend 7.0/7.1.

 

If you are using Nexus on versions later than 3.09, please review this link which is applicable for both 7.0 and 7.1: https://help.talend.com/reader/Od7BSSwZmOsHBIPRY760sA/PCXotGURcHH_eZEEHJiPpQ

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

APIs for Dummies

View this on-demand webinar about APIs....

Watch Now

6 Ways to Start Utilizing Machine Learning with Amazon We Services and Talend

Look at6 ways to start utilizing Machine Learning with Amazon We Services and Talend

Blog

Why Companies Move to the Cloud: 7 Success Stories

Learn how and why companies are moving to the Cloud

Read Now