Six Stars

Jenkins CI build job, missing reference project

With Jenkins my build job (generate sources) successfully downloads a BitBucket repository, however eventually the job gives the following error when invoking CI Builder:

[INFO] exception : org.talend.commandline.client.command.CommandStatusWrapException: org.talend.commons.exception.LoginException: Missing the reference project for "SANDBOX", please make sure the reference projects must be existed in same workspace too.

 

What's the proper way to handle reference projects in my Jenkin's build job?  

 

Build

Talend 6.3.1

Jenkins 2.60.2

Maven 3.5.0

Nexus 2.11.3

 

Error stack

[INFO] ------------------------------------------------------------------------
[INFO] Building sandbox_artifact 1.0.1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- ci.builder:6.3.1:local-generate (default-cli) @ sandbox_artifact ---
[INFO] 
[INFO] *****************************************************
[INFO] List projects from:C:\jenkins\workspace\source
[INFO]   Not valid folder (no talend project) in:.git
[INFO]   Found project:SANDBOX
[INFO] 
[INFO] Creating the detected projects for Commandline workspace.
[INFO]  Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
[INFO]  Setting property: velocimacro.messages.on => 'false'.
[INFO]  Setting property: resource.loader => 'classpath'.
[INFO]  Setting property: resource.manager.logwhenfound => 'false'.
[INFO] Start to check local patches....
[INFO] COMPLETED at Mon Aug 07 10:41:52 CDT 2017
[INFO] 
[INFO] COMPLETED at Mon Aug 07 10:41:53 CDT 2017
[INFO] 
[INFO] COMPLETED at Mon Aug 07 10:41:53 CDT 2017
[INFO] 
[INFO] Time taken to dump: 110
[INFO] 
[INFO] *****************************************************
[INFO] Starting to generate sources for the detected projects
[INFO] 
[INFO] Preparing for the project: SANDBOX
[INFO]  Generating sources...
[INFO]  Setting property: classpath.resource.loader.class => 'org.codehaus.plexus.velocity.ContextClassLoaderResourceLoader'.
[INFO]  Setting property: velocimacro.messages.on => 'false'.
[INFO]  Setting property: resource.loader => 'classpath'.
[INFO]  Setting property: resource.manager.logwhenfound => 'false'.
[INFO] Start to check local patches....
[INFO] COMPLETED at Mon Aug 07 10:42:07 CDT 2017
[INFO] 
[INFO] FAILED at Mon Aug 07 10:42:07 CDT 2017
[INFO] exception : org.talend.commandline.client.command.CommandStatusWrapException: org.talend.commons.exception.LoginException: Missing the reference project for "SANDBOX", please make sure the reference projects must be existed in same workspace too.
[INFO] 	at org.talend.commandline.util.RepositoryHandler.logonProjectWithReferences(RepositoryHandler.java:102)
[INFO] 	at org.talend.commandline.util.RepositoryHandler.logonProject(RepositoryHandler.java:86)
[INFO] 	at org.talend.commandline.command.CommandProcessor.logonProject(CommandProcessor.java:128)
[INFO] 	at org.talend.commandline.command.CommandProcessorSwitch.caseLogonProjectCommand(CommandProcessorSwitch.java:128)
[INFO] 	at org.talend.commandline.client.util.CommandAbstractSwitch.doSwitch(CommandAbstractSwitch.java:54)
[INFO] 	at org.talend.commandline.command.CommandConsumer.executeCommand(CommandConsumer.java:57)
[INFO] 	at org.talend.commandline.command.CommandConsumer.execute(CommandConsumer.java:35)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.process(CommandLineHandler.java:280)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:210)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:197)
[INFO] 	at org.talend.commandline.command.CommandProcessor.processScriptFile(CommandProcessor.java:386)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.executeClientCommand(CommandLineHandler.java:732)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.process(CommandLineHandler.java:259)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:210)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:197)
[INFO] 	at org.talend.commandline.mode.NormalCommandLine.start(NormalCommandLine.java:32)
[INFO] 	at org.talend.commandline.CommandLineApplication.start(CommandLineApplication.java:248)
[INFO] 	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
[INFO] 	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
[INFO] 	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
[INFO] 	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
[INFO] 	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
[INFO] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[INFO] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[INFO] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[INFO] 	at java.lang.reflect.Method.invoke(Method.java:497)
[INFO] 	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
[INFO] 	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
[INFO] 	at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
[INFO] 	at org.eclipse.equinox.launcher.Main.main(Main.java:1438)
[INFO] 	at org.eclipse.core.launcher.Main.main(Main.java:34)
[INFO] 
[INFO] 
[INFO] 
[INFO] FAILED at Mon Aug 07 10:42:07 CDT 2017
[INFO] exception : org.talend.commandline.client.command.CommandStatusWrapException: java.lang.RuntimeException: java.lang.IllegalStateException: should be logged on a repository
[INFO] 	at org.talend.commandline.command.ExtensionCommandSwitch.caseExtensionServerCommand(ExtensionCommandSwitch.java:201)
[INFO] 	at org.talend.commandline.client.command.extension.AbstractExtensionCommandSwitch.doSwitch(AbstractExtensionCommandSwitch.java:31)
[INFO] 	at org.talend.commandline.command.CommandProcessorSwitch.caseExtensionCommand(CommandProcessorSwitch.java:161)
[INFO] 	at org.talend.commandline.client.util.CommandAbstractSwitch.doSwitch(CommandAbstractSwitch.java:70)
[INFO] 	at org.talend.commandline.command.CommandConsumer.executeCommand(CommandConsumer.java:57)
[INFO] 	at org.talend.commandline.command.CommandConsumer.execute(CommandConsumer.java:35)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.process(CommandLineHandler.java:280)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:210)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:197)
[INFO] 	at org.talend.commandline.command.CommandProcessor.processScriptFile(CommandProcessor.java:386)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.executeClientCommand(CommandLineHandler.java:732)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.process(CommandLineHandler.java:259)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:210)
[INFO] 	at org.talend.commandline.cli.CommandLineHandler.execute(CommandLineHandler.java:197)
[INFO] 	at org.talend.commandline.mode.NormalCommandLine.start(NormalCommandLine.java:32)
[INFO] 	at org.talend.commandline.CommandLineApplication.start(CommandLineApplication.java:248)
[INFO] 	at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
[INFO] 	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:134)
[INFO] 	at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:104)
[INFO] 	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:380)
[INFO] 	at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:235)
[INFO] 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[INFO] 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[INFO] 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[INFO] 	at java.lang.reflect.Method.invoke(Method.java:497)
[INFO] 	at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:648)
[INFO] 	at org.eclipse.equinox.launcher.Main.basicRun(Main.java:603)
[INFO] 	at org.eclipse.equinox.launcher.Main.run(Main.java:1465)
[INFO] 	at org.eclipse.equinox.launcher.Main.main(Main.java:1438)
[INFO] 	at org.eclipse.core.launcher.Main.main(Main.java:34)
[INFO] Caused by: java.lang.IllegalStateException: should be logged on a repository
[INFO] 	at org.talend.commandline.command.execute.AbstractExecuteCommand.checkLogged(AbstractExecuteCommand.java:125)
[INFO] 	at org.talend.commandline.command.execute.BuildProjectSourcesExecuteCommand.execute(BuildProjectSourcesExecuteCommand.java:94)
[INFO] 	at org.talend.commandline.command.ExtensionCommandSwitch.caseExtensionServerCommand(ExtensionCommandSwitch.java:199)
[INFO] 	... 29 more
[INFO] 
[INFO] 
[INFO] 
[INFO] COMPLETED at Mon Aug 07 10:42:07 CDT 2017
[INFO] 
[INFO] Time taken to dump: 104
[INFO]  Logoff project
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30.379 s
[INFO] Finished at: 2017-08-07T10:42:10-05:00
[INFO] Final Memory: 10M/205M
[INFO] ------------------------------------------------------------------------
Waiting for Jenkins to finish collecting data
[ERROR] Failed to execute goal org.talend:ci.builder:6.3.1:local-generate (default-cli) on project sandbox_artifact: Execution default-cli of goal org.talend:ci.builder:6.3.1:local-generate failed. NullPointerException -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[JENKINS] Archiving C:\jenkins\workspace\source\pom.xml to com/sandbox_artifact/1.0.1/sandbox_artifact-1.0.1.pom
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginExecutionException
channel stopped
Finished: FAILURE

 

1 ACCEPTED SOLUTION

Accepted Solutions
Employee

Re: Jenkins CI build job, missing reference project

Tested this, i can reproduce it and fix it. The issue here is in fact that when you have the project defined like below:

  • MAIN: 
    http://localhost:8181/git/r/MAIN.git
  • REFERENCE: 
    http://localhost:8181/git/r/REFERENCE.git

Even if you defined the two URL in your buildSource definition, see the screenshot attached. With that only one project
is checked, I had to manually copy paste the second project missing. To avoid that problem which is thing is related to 
jenkins i had to use one single repository for the two projects: 

http://localhost:8181/git/r/tac631.git
5 REPLIES
Employee

Re: Jenkins CI build job, missing reference project

Tested this, i can reproduce it and fix it. The issue here is in fact that when you have the project defined like below:

  • MAIN: 
    http://localhost:8181/git/r/MAIN.git
  • REFERENCE: 
    http://localhost:8181/git/r/REFERENCE.git

Even if you defined the two URL in your buildSource definition, see the screenshot attached. With that only one project
is checked, I had to manually copy paste the second project missing. To avoid that problem which is thing is related to 
jenkins i had to use one single repository for the two projects: 

http://localhost:8181/git/r/tac631.git
Six Stars

Re: Jenkins CI build job, missing reference project

Thank you for the response eTechman!

So when you say you had to use one single repo for 2 projects, does this mean you created 1 repository in BitBucket and referenced it from 2 different projects within TAC? Or does this mean there's some way within BitBucket to create a git URL reference that points to 2 different BitBucket repos?
Six Stars

Re: Jenkins CI build job, missing reference project

Submitted a support ticket and they gave some great advice.

 

Created By: Evan Teitelbaum (8/8/2017 11:27 AM)
Hello Al, 

Jenkins can only checkout one project at a time and this is why you are getting this issue. The solution here is to checkout the project so that under the workspace folder you have the project folder for the main and the reference project. Or you can do a git checkout anywhere else and then copy the project folder over into the CI's cmdline workspace folder. Then after you build again after this step please let us know if you still have this issue. 


I did a checkout in another directory and copied it to the other project.  You have to make sure you copy the reference project root to match the original talend project, the parent folder of all the talend-generated folders.  I ran my Jenkins job and it built successfully.

 

I will now attempt to automate this into a pre-build script.

 

Six Stars

Re: Jenkins CI build job, missing reference project

Accepted eTechman answer as solution since he had the same advice, to copy-paste the project.
Six Stars

Re: Jenkins CI build job, missing reference project

Successfully created a batch script that is called as a pre-build step that will

  • clone the reference project repository into a temp directory
  • move the folder into the workspace of the original Talend project
  • clean-up of temp directory