When trying to move a file from one machine in VPC1 to another machine in VPC2, a connection cannot be established between the two VPCs, as shown in the message below:
Servers located in two VPCs or two separate accounts are not able to communicate in AWS until you enable VPC Peering.
The platform architecture recommendation is to use a single Nexus across DEV/QA and PROD environments. However, organizations that leverage AWS as their platform often create one account for DEV/QA and another account for PROD, making it difficult for the DEV/QA Nexus to communicate with any machine in the PROD environment.
This article explains how to enable VPC Peering, and how to establish communication between two servers. In this example, you provision two Nexus, one in VPC1 and the other in VPC2, then move artifacts from the source to the destination Nexus.
Note: There is no connection between the two VPCs in the beginning, so you can extend this to VPCs in different accounts.
Select Peering Connections (highlighted in yellow).
The Peering Connections window opens displaying the Requestor and Acceptor settings.
If your VPC is in a different account, select Another account and enter the Account ID:
After you configure your settings, create the connection. Notice the connection status is Pending Acceptance.
Right-click Pending Acceptance, then select Accept Request.
Click Yes, Accept.
Notice that the VPC Peering Connection is now Active.
However, the connectivity is not yet established; you need to edit the route tables in both VPCs to ensure seamless connectivity. Start with the requestor VPC; refer to the highlighted entry of the route table below. You can restrict the connectivity to a specific server in the destination VPC.
Apply a similar route table entry for the second VPC, and select the server that you want to communicate with, as shown below:
Initiate a file transfer test to verify the connection. Type yes as shown below to add the remote host details to the known hosts file of the source server. This is an SSH protocol requirement on initial file transfers; once set, the message no longer appears.
Now that you have established communication between two servers in different VPCs, extend this to your Nexus use case. Install Nexus on both servers, then confirm the installations, as shown below:
Create a Job in Studio and publish it to Nexus, then confirm that the Job was published to the Snapshots repository of the source Nexus, as shown below:
Next, mimic a deployment process to the destination Nexus. Typically, you would do this by using Jenkins, but for this example, use SCP.
Note: This process is named promotion, so in theory, this should only happen with “release” type artifacts after promotion. It is not possible to determine where a snapshot may come from, since it is not unique. Confirm that the artifact was successfully transferred to the destination Nexus, as shown below:
The deployed artifacts are in the target Nexus (call it PROD NEXUS).
Now, you can extend this to a complete continuous integration/continuous deployment (CI/CD) process.
... View more
All, We are installing version 6.5.1 for a customer whose environment is locked down. It does not allow downloading third party jars from Studio. The customer did open up their firewall for Talend to download the software installer from the links provided in the license email and the link to download the jars seems to be the following: https://talend-update.talend.com/nexus/content/repositories/libraries/org/talend/libraries/ I'm assuming both use the same protocol and port. If not then please do let me know and potential reason why the jar downloads could be blocked while the installers can be downloaded without any issues. I do notice that installer is from opensourceetl.net so could it be that the firewall is actually open for this domain and not Talend? Would appreciate some insight. Thanks!!
... View more