Migrating your repository from Nexus to Artifactory in Talend 7.1+

Overview

This article explains the process of migrating an installed Nexus 2 or Nexus 3, configured for Talend, to Artifactory Pro on premises. The article also outlines how to migrate to Artifactory OSS, if you don't have a license for Artifactory Pro.

 

Environment

Windows Server 2016 (similar on Linux)

Talend 7.1

 

Prerequisites

  1. Talend 7.1 (or greater) installed on premises.
    1. Artifactory is not supported before 7.1.
  2. Nexus installed on premises.
  3. Artifactory Pro installed on premises.

    1. A valid Artifactory Pro license is required; this article relies on the REST API, which is not available in the OSS version.
    2. This article uses a standalone install rather than a service install.
    3. For information on installing Artifactory Pro, see the JFrog Artifactory User Guide, and scroll down to the Installing on Windows section.

      1. Install Artifactory in C:\Talend\ArtifactoryPro(8083).

        1. This article uses the 8083 port. However, any available port is valid; change the directory name as appropriate.
      2. Set the admin login password to Talend123 or use your Talend Nexus password.
      3. Set the JAVA_HOME and ARTIFACTORY_HOME environment variables.
      4. Configure Artifactory to match your Nexus settings, except for the repositories, users, groups, and authorities that are ported using this tooling.

        For best practice recommendations for configuring Artifactory for use in Talend, watch the Expert Session Learn About Repository Manager Migration on the Architect Sessions – Recorded Sessions web page.

 

Process

  1. Installing the NexusToArtifactory tool.
  2. Configuring the migration:

    1. Connecting to Nexus.

      • Nexus 2 differences

    2. Setting up the Repository.

    3. Setting Security.

    4. Verifying the configuration.

  3. Running the migration.

    1. Rerunning the migration.

 

Installing the NexusToArtifactory tool

For a brief introduction to the NexusToArtifactory tool, see Migrate from Nexus to Artifactory.

  1. Create a new directory: C:\Talend\ArtifactoryMigration\JFrog.
  2. Open your web browser and navigate to the GitHub jfrog/nexus2artifactory page.

    $1.pngGithub - NexusToArtifactory Tool

     

  3. Download and store the NexusToArtifactory.exe in the new directory. This is a directly executable file, not a Windows install file.

    $2.pngInstalled - NexusToArtifactory Tool

     

  4. Go back to the GitHub jfrog/nexus2artifactory page, and scroll down to the Usage section for instructions on how to run the tool.

  5. Still on the nexus2artifactory page, scroll down and read the Future Development section for the current unsupported items. Note that there are also notes in the Usage section.

 

Configuring the migration

  1. Open a command prompt window as an administrator in Windows, then using the cd command, change to the directory that you created earlier: C:\Talend\ArtifactoryMigration\JFrog.

  2. Run the NexusToArtifactory.exe.

    $5.pngNexusToArtifactory - Main Menu

    Note: The red ! after i indicates there is something to do.

 

Connecting to Nexus

  1. Enter i (Initial Setup) in the tool window. The following screen appears.

    $6.pngNexusToArtifactory - Initial Setup Menu

     

  2. Enter n (Nexus Data Directory).

    $7.pngNexusToArtifactory - Nexus Data Directory

     

  3. Enter the full path of the directory where your Talend Nexus is installed, in this case, C:\Talend\7.1.1\tac\Artifact-Repository-Nexus-3.9.0-01-win64\sonatype-work\nexus3, then press Enter.

    $8.pngNexusToArtifactory - Nexus 3 Details

     

  4. Enter x (Nexus URL) and provide the web address of your Talend Nexus, in this case, http://localhost:8081/, then press Enter.

  5. Enter s (Nexus Username) and provide the ID of the Talend Nexus admin user account, in this case, admin, then press Enter.

    $9.pngNexusToArtifactory - Nexus 3 - Incorrect Password

     

  6. Ignore the warning:

    Incorrect username and/or password.
  7. Enter c (Nexus Password) and provide the password of your Talend Nexus admin user account, in this case, Talend123, then press Enter. The text turns green when everything is correct.

    $10.pngNexusToArtifactory - Nexus 3 - Setup Complete (Green is good)

     

  8. Set up the Artifactory by performing the following steps:

    1. Enter a (Artifactory URL) and provide the web address of your Artifactory Pro install, in this case, http://localhost:8083/artifactory/, then press Enter.

    2. Enter u (Artifactory Username) and provide the admin ID of your Artifactory Pro install, in this case, admin, then press Enter.

    3. Enter p (Artifactory Password) and provide the admin password of your Artifactory Pro install, in this case, Talend123, then press Enter.

      $11.pngNexusToArtifactory - Artifactory - Setup Complete (Green is good)

       

  9. Enter q (Back) to go back, then enter s (Save Config JSON File), enter Nexus3, then press Enter.

    $12.pngNexusToArtifactory - Save the config file

     

Nexus 2 differences

  1. Enter i (Initial Setup) in the tool window. Enter n (Nexus Data Directory).

  2. Enter the full path of the directory where your Talend Nexus is installed, in this case, C:\Talend\Nexus2-651(8561)\sonatype-work\nexus.

    For Nexus 2 the ID and password are not needed. These are the only differences when connecting to Nexus 2.

    $13.pngNexusToArtifactory - Nexus 2 - Setup Complete (Green is good)

     

  3. Enter the Artifactory details as you did in the previous steps.

  4. In Step 9, replace Nexus3 with Nexus2, then press Enter.

 

Setting up the Repository

See the https://github.com/JFrog/nexus2artifactory page for some of the errors you may encounter with repositories.

 

The default Nexus 3 install of Talend does not appear to have any errors. However, on Nexus 2, the central-m1 repository cannot be migrated because it is a shadow, so it must be unmarked (m key to toggle). These commands are demonstrated below.

  1. You will transfer all the repositories, but only the following are needed for Talend by default:

    • releases
    • snapshots
    • talend-custom-libs-release
    • talend-custom-libs-snapshot
    • talend-updates

    You can choose to deselect the others if you wish.

  2. Enter r (Repository Migration Setup) for Talend Nexus 3 default. The likely output is shown below:

    $15.pngTalend Default Repositories - Screen 1

     

  3. If you don't see the repository you want, use the right arrow key > to go to the Next Page. Use the left arrow key < to go back to the Previous Page.

    $16.pngTalend Default Repositories - Screen 2

     

  4. Type a number (1-8) to toggle the migration of the respective repository on (a green +) or off.

  5. Enter e (Edit Repository), followed by 1. Pressing Enter on the second screen displays the following:

    $17.pngNexusToArtifactory - Edit a repository

    The commands on this screen work as you might expect, you do not need to make changes.

     

  6. Enter q (Back) to the main screen, then s (Save Config JSON File), type nexus3, then press Enter.

 

Setting up Security

  1. By default, there is a red ! warning on the Security Migration Setup. Enter u.

    $18.pngNexusToArtifactory - Security Setup - Warning

     

  2. Enter u again for Users Migration Setup.

    $19.pngNexusToArtifactory - User Migration - Warning

     

  3. Note that there isn't a green + sign next to admin and it is not going to be migrated. This is correct.

    $20.pngNexusToArtifactory - User Migration - List of users

     

  4. There is a red ! on Default Password, and you need to set this, but first a warning: Artifactory cannot copy the existing Nexus user passwords, so you must set new ones using one of the following strategies:

    1. One default password for all users (other than admin), using option P from the menu.
    2. A password for all user IDs using E # P (where # is a number from 1-8) repeated for all users.
    3. A mix of the above methods, but giving only certain users specific passwords.

    Use of the global Default Password is shown below:

    $21.pngNexusToArtifactory - User Migration - Set default password

     

  5. Make the changes you require for the users. You may wish to make changes to G (Groups) and P (Permissions), but there is no need to do this for the Talend Nexus 3 or Nexus 2 defaults.

  6. Enter q (Back) twice to go back to the main screen, and s (Save Config JSON File), Nexus3, then press Enter.

  7. Enter q (Exit). You are ready to verify and run the configuration.

 

Verifying the configuration

  1. In the admin command window, run the following command:

    NexusToArtifactory.exe –l logfile.log

    Notice the -l switch, which collects log messages to a file. More useful options are described in the GitHub repository for the code, for example, the -v (error, warning, info, debug) to set the logging level.

  2. Enter l (Load Config JSON File), enter nexus3, then press Enter.

    $23.pngNexusToArtifactory - Load a config file

     

  3. Enter v (Verify Configuration). If there are errors as noted by a red !, correct them. Enter s (Save Config JSON file) to save the file, and verify it again.

    $24.pngNexusToArtifactory - Verify configuration

 

Running the migration

  1. Enter x (Run Migration).

    $25.pngNexusToArtifactory - Run migration in progress

    $26.pngNexusToArtifactory - Run migration finished

     

  2. A sample log file, logfile.log, is attached to this article. Enter q to continue.

  3. Log in to Artifactory Pro, http://localhost:8083, as an Administrator.

  4. From the navigation menu on the left, select Repositories. Go back to the home page, then refresh the browser page. This will cause Artifactory to update the number of artifacts being served.

    $27.pngArtifactory - Home screen after migration

     

Resolving an error (Optional)

  1. Go back to the NexusToArtifactory.exe window, and enter x, then enter x again.

    $28.pngNexus2Artifactory - Running multiple migrations - looks wrong

     

  2. Notice the number of Artifacts has risen sharply; this is a concern. Go back into Artifactory and repeat the actions under Running the migration above, and you’ll see many more artifacts.

    $29.pngArtifactory - Home screen is this an error?

     

  3. Back in the Artifact Repository Browser view, select Trash Can. From the Actions pull-down menu select Empty Trash Can, then click Empty Trash button.

    $30.pngArtifactory - Empty the trash can

     

  4. Go back the home screen and refresh the page.

    $31.pngArtifactory - Home screen, all looks good again

     

  5. All is good. It seems that the migration tool deletes and inserts identical copies of some artifacts during a repeated migration, but this hasn’t caused any issues in testing. Test your migration results to ensure you have the same outcome.

 

Rerunning the migration

There are a number of ways to reset the Artifactory Pro install to empty, if you need to start again.

  1. Make a copy of the directory structure—before you start but after configuring Artifactory—then shut down Artifactory and copy the original directory structure back in.

  2. Do a backup in Artifactory when you start, and restore this (or Export/Import).

  3. Delete the repositories, users, groups, and others—remember to empty the trash. This is not difficult for a default Talend install, but it might be quite involved if this is your main repository for the company that covers far more than just the Talend repositories and users.

 

Checking the results

Simple counts are a good start, but you might want to run a full operational test of Jobs running in a test environment, to be sure.

 

Changing TAC/TMC over to Artifactory (passwords and config)

  1. Set the password of the user IDs.

  2. Update TAC (TMC). For more information on updating TMC see the Bookshelf references section of this article. For TAC you need to edit Settings > Configurations > Artifact Repository > Software Update.

    $32.pngTAC - Configuration for Repositories

     

Migrating to Artifactory OSS (Optional)

Artifactory OSS, the free version of Artifactory, does not support REST APIs, so you cannot use the NexusToArtifactory tool directly. If you don't have a licensed version of Artifactory Pro, download a trial version, then follow the instructions below to migrate to Artifactory OSS:

  1. Follow the article above through Running the migration.

  2. Install a copy of Artifactory OSS in another directory with a different port number.

  3. Run an Artifactory System Export from Artifactory Pro to the file system, then shut down Artifactory Pro.

  4. Update the ARTIFACTORY_HOME variable to point to the Artifactory OSS directory, then start Artifactory OSS.

  5. Log in to Artifactory OSS, then run System Import on the export from Artifactory Pro.

  6. You can now continue with Checking_the_results and Changing_TAC_TMC_over_to_Artifactory using Artifactory OSS.

 

Bookshelf references

For more information, see these pages on the Talend Help Center:

Configuring the Software Update repository in Talend Administration Center

Configuring Talend Artifact Repository in Talend Administration Center

Configuring the Artifact Repository

Comments
Forteen Stars

thank you for good article

 

just a question (it is new for me, want to understand what was mentioned)

A valid Artifactory Pro license is required; this article relies on the REST API, which is not available in the OSS version.

but:

OSSrepo.PNG

 

as it declared by vendor - REST API supported, what wrong from Talend point of view?

Community Manager

Hi vapukov,

 

Please ask your question on the appropriate Community discussion forum, where you will get help from Talend staff and other users. If you need further assistance, please contact Talend Support.

 

Thank you,

Donna

Four Stars

Hi Team,

 

We planning to migrate from nexus to jfrog artifactory pro. So the current version of nexus is 2.1.2 and artifactory version is Artifactory Professional
6.6.3 rev 60603900 . is this possible to migrate by using NexustoArtifactory.exe if yes how to migrate it . Could you please help me this .

Thanks in advance,

Raja

Moderator

Hello Raja,

Thanks for letting us know your issue. With your migration issue for talend subscription solution, would you mind creating a case on talend support portal? In this way, we could give you a remote assistance(webex session) through support cycle with priority.https://login.talend.com/support-login.php?

Let us know if it is OK with you.

Best regards

Sabrina