Need help with Jenkins pipeline build

Seven Stars

Need help with Jenkins pipeline build

[ERROR] Plugin org.talend.ci:builder-maven-plugin:7.1.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor 
for org.talend.ci:builder-maven-plugin:jar:7.1.1: Could not transfer artifact org.talend.ci:builder-maven-plugin:pom:7.1.1
from/to plugins_snapshot (http://XXXXXXXXX:9081/repository/talend-custom-libs-snapshot/):
Failed to transfer file http://XXXXXXXX:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7.1.1...
with status code 400 -> [Help 1]

Am I missing  some plugins or missing some settings in my pipeline script on both?

 

More info...

 

Caused by: org.eclipse.aether.resolution.ArtifactResolutionException: Failure to transfer 
org.talend.ci:builder-maven-plugin:pom:7.1.1 from http://10.190.3.207:9081/repository/talend-custom-libs-snapshot/
was cached in the local repository, resolution will not be reattempted until the update interval of plugins_snapshot
has elapsed or updates are forced. Original error: Could not transfer artifact org.talend.ci:builder-maven-plugin:pom:7.1.1
from/to plugins_snapshot (http://10.190.3.207:9081/repository/talend-custom-libs-snapshot/):
Failed to transfer file http://10.190.3.207:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7... with status code 400

 

env.PROJECT_GIT_NAME = 'CICD_POC_PROJECT'
env.PROJECT_NAME = env.PROJECT_GIT_NAME.toLowerCase()
env.ARTIFACT = 'test_job'
env.VERSION = '0.1'
env.GIT_URL = 'https://github.com/YYYYYY/XXXXXX.git'
env.CLOUD_URL = 'https://tmc.us.cloud.talend.com/inventory/'
env.TYPE = "" // if big data = _mr
env.IMAGE_NAME = 'XXX'   
// Credentials IDs (Manage Jenkins => Credentials)
env.GIT_CREDENTIALS_ID = 'git_user'
env.TMC_CREDENTIALS_ID = 'tmc_user'

node {
 	// Clean workspace before doing anything
    try {
        def userInput
        def deployprod
        stage('Initialize') {
            sh '''
                echo "PATH = ${PATH}"
                echo "M2_HOME = ${M2_HOME}"
            ''' 
        }
        stage ('Git Checkout') {
            git(
                url: "${GIT_URL}",
                credentialsId: "${GIT_CREDENTIALS_ID}",
                branch: 'master'
            )       
            mvnHome = tool 'M3'
        }
        stage ('Build, Test and Publish artifacts to Cloud') {
            withCredentials([usernamePassword(credentialsId: "${TMC_CREDENTIALS_ID}", usernameVariable: 'Dusername', passwordVariable: 'Dpassword')]) {
					withMaven(
                            // Maven installation declared in the Jenkins "Global Tool Configuration"
                            maven: 'M3',
                            // Maven settings.xml file defined with the Jenkins Config File Provider Plugin
                            // Maven settings and global settings can also be defined in Jenkins Global Tools Configuration
                            mavenSettingsConfig: 'maven-settings-file',
                            mavenOpts: '-Dlicense.path=~/Documents/files/license -Dproduct.path=~/Documents/cmdline -Dgeneration.type=local -Dservice.url=${CLOUD_URL} -Dcloud.publisher.screenshot=true -Dcloud.publisher.skip=false -Dservice.accelerate=true -Dcloud.publisher.updateFlow=true -Dservice.username=$Dusername -Dservice.password=$Dpassword -Dcloud.publisher.environment=Development -Dcloud.publisher.workspace=YYYY -Xms1024m -Xmx3096m') 
                            {
                    
                        // Run the maven build
                        sh "mvn -X -f $PROJECT_GIT_NAME/poms/pom.xml -Pcloud-publisher clean deploy -fn -e -pl jobs/process${TYPE}/${ARTIFACT}_${VERSION} -am"
                    
                        }    
		        }
        }
        
    } catch (err) {
        currentBuild.result = 'FAILED'
        throw err
    }
}

 


Accepted Solutions
Seven Stars

Re: Need help with Jenkins pipeline build

Talend consultant was able to help us out. The problem was the permissions on cmdline folder(or whatever folder you have studio installed) were causing the failure. Resolution: change permission on cmdline folder so that Jenkins can create stuff in that folder.

 

Marking this as resolved.

View solution in original post


All Replies
Seven Stars

Re: Need help with Jenkins pipeline build

Replying to my own question:

I see issue with local studio unable to push libraries to Nexus talend-custom-libs-snapshot

 

here are some errors from my local studio log

 

org.talend.commons.exception.PersistenceException: Studio can't access the resource from server, please check the resource url.
	at org.talend.repository.remoteprovider.client.AbstractClient.getArtifactLocation(AbstractClient.java:822)
	at org.talend.designer.publish.di.preferences.PublishPreferences.getArtifactRepositoryFromServer(PublishPreferences.java:141)
	at org.talend.designer.publish.di.preferences.PublishPreferences.getArtifactRepository(PublishPreferences.java:75)
	at org.talend.designer.publish.di.preferences.PublishPreferences.getArtifactRepository(PublishPreferences.java:61)
	at org.talend.designer.publish.di.model.NexusService.getPublishNexusServerBean(NexusService.java:153)
	at org.talend.updates.runtime.nexus.component.NexusServerManager.getLocalNexusServer(NexusServerManager.java:65)
	at org.talend.updates.runtime.engine.factory.ComponentsNexusInstallFactory.getLocalNexusFeatures(ComponentsNexusInstallFactory.java:79)
	at org.talend.updates.engine.EditionComponentsNexusInstallFactory.getAllExtraFeatures(EditionComponentsNexusInstallFactory.java:41)
	at org.talend.updates.runtime.engine.factory.ComponentsNexusInstallFactory.retrieveAllExtraFeatures(ComponentsNexusInstallFactory.java:182)
	at org.talend.updates.runtime.engine.ExtraFeaturesUpdatesFactory.retrieveAllComponentFeatures(ExtraFeaturesUpdatesFactory.java:71)
	at org.talend.updates.runtime.feature.FeaturesManager.retrieveAllOfficalFeatures(FeaturesManager.java:156)
	at org.talend.updates.runtime.feature.FeaturesManager.getFeaturesCache(FeaturesManager.java:141)
	at org.talend.updates.runtime.feature.FeaturesManager.getUpdates(FeaturesManager.java:163)
	at org.talend.updates.runtime.feature.FeaturesManager.searchUpdates(FeaturesManager.java:214)
	at org.talend.updates.runtime.ui.feature.job.FeaturesCheckUpdateJob.run(FeaturesCheckUpdateJob.java:54)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: Studio can't access the resource from server, please check the resource url.

 

Seven Stars

Re: Need help with Jenkins pipeline build

@xdshi any help here?

Moderator

Re: Need help with Jenkins pipeline build

Hello,

Are you using talend open studio or subscription solution?

You can find the integration documentation here:
https://help.talend.com/reader/o2DMkp1_WVLFHSyNkjxn4w/lHXK~e0ZTKbbL6HDnQVofA

The missing CI builder jar , you showed in the question, is now part of the subscribed 7.1.1 Talend Studio. 

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Seven Stars

Re: Need help with Jenkins pipeline build

Subscription version 7.1.1

 

Error message says 

 

[ERROR] Plugin org.talend.ci:builder-maven-plugin:7.1.1 or one of its dependencies could not be resolved: Failed to read artifact descriptor for 
org.talend.ci:builder-maven-plugin:jar:7.1.1: Could not transfer artifact org.talend.ci:builder-maven-plugin:pom:7.1.1 from/to plugins_snapshot
(http://XXXXXX:9081/repository/talend-custom-libs-snapshot/): Failed to transfer file
http://XXXXXX:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7.1.1/b...
with status code 400 -> [Help 1]

Based on above message the location its looking for CI builder jar at http://XXXXXX:9081/repository/talend-custom-libs-snapshot/org/talend/ci/builder-maven-plugin/7.1.1/b...

but I I dont see org/talend/ci under talend-custom-libs-snapshot

But I do see it's available where the the cmdline is installed on server

/home/Documents/cmdline/configuration/.m2/repository/org/talend/ci

Where do you configure location to look for there libraries?

 

And if I change localRepository in maven settings.xml to where TalendStudio (cmdline) is installed

 

then I get this error message

[ERROR] Could not create local repository at /home/Documents/cmdline/configuration/.m2/repository -> [Help 1]org.apache.maven.repository.LocalRepositoryNotAccessibleException: Could not create local repository at /home/Documents/cmdline/configuration/.m2/repository

 

Seven Stars

Re: Need help with Jenkins pipeline build

Talend consultant was able to help us out. The problem was the permissions on cmdline folder(or whatever folder you have studio installed) were causing the failure. Resolution: change permission on cmdline folder so that Jenkins can create stuff in that folder.

 

Marking this as resolved.

View solution in original post

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Talend Cloud Developer Series - Introduction

The Talend Cloud Developer Series was created to give you a solid foundational understanding of Talend’s Cloud Integration Platform

Watch Now

Talend Cloud Available on Microsoft Azure

An integration platform-as-a-serviceto help enterprises collect, govern, transform, and share data from any data sources

Watch Now

Self-service Talend Migration: Moving from On-Premises to the Cloud

Move from On-Premises to the Cloud by following the advice of experts

Read Now