First off I would like to say that I am pretty new to all of this. I have only been learning Talend, Git, CI and such for about four months now. Our orginization is trying to follow what "Best Practices" are, but find that depending on who you talk to you get different "Best Practices". Below you will find what seem like the best fit for our orginization. Please feel free to correct me if I have any incorrect thought processes, or concepts.
From what I understand the lifecycle of a project follows this path:
00 - In TAC a new project is created, this creates a new master branch in the repository
01 - The master branch is checked out as "Development" creating a development branch on the git repo
02 - A developer checks out the development branch to their local system so they can work on bug fixes, new features and such
03 - The developer has some code ready to test in the development environment so they do a Pull and Merge to verify that their code will still function with any existing code on the development branch
04 - After verifying that all code still functions the developer will push their code to the development branch
* Due to the SCM Polling, Jenkins should detect that a change has occured on the development branch and fire off it's job to Generate, Build/Test the code, and deploy it to the Development environment
05 - The developer later on decides to tweak the code. so they first do a Pull and Merge so they have the current Development code to work with
06 - The commit from step 04 was approved to be moved to the testing environment and a Tag (qa/1013a) is created
* Jenkins should detect that a new tag has has been created on the development branch and fire off it's job to Generate, Build/Test the code, and deploy it to the QA environment
07 - The QA code gets the green light to be deployed to the Production environment and is commited to the Master branch
* Jenkins should detect that a change has occured on the master branch and fire off it's job to Generate, Build/Test the code, and deploy it to the Production environment
Master Branch (Remote) | 00 : New project is created generating a new repository and Maaster branch | | | Development Branch (Remote) |----->| 01 : Checkout Development branch | | | | | | Developer's Branch (On Local System) | |---------->| 02 : Checkout Development branch to local system | | | | | | | | | | |---------->| 03 : Pull and Merge | |<----------| 04 : Push updated code to Development branch | |---------->| 05 : Pull and Merge | | | | | | | | | | | : Tag | 06 : Tag qa/1013a created for Dev branch | | | | | | | | | |<-----| | 07 : Tag qa/1013a commited to Master branch | | | | | | V V V
If the above diagram isn't displaying gorrectly see the attached
Steps 00 to 05 I have working, but I have some questions/concerns about step 06. For security purposes we have to use SCM Polling instead of the git hook.
01 - I have found some information on using Jenkins to check tags, but it isn't working as expected. Here are the settings:
In the Source Code Management - Git section
* Refspec: +refs/tags/qa/*:refs/remotes/<DEVELOPMENT_BRANCH>/tags/qa/*
* Branches to build: */<DEVELOPMENT_BRANCH>/tags/qa/*
Everytime it runs it lists the new tag I created but is says "ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job."
02 - If a new tag is created and the jobe fires off will it only build the new tag?
03 - will the SCM polling detect both commits and tag creation as a change to the branch. If so is there a way to fix that?
04 - Would a better option for this be, to have a "Master, QA. Development" branch and instead of using the tag to mark QA just push it from the dev branch to the qa branch?
Thanks for posting your issue here.
First of all, could you please take a look at this online guide about:TalendHelpCenter:Talend Software Development Life Cycle Best Practices
and let us know if you have any question in it?
We have re-directed your issue to our product experts and will keep you posted.
For the initial setup I did use this document. However in the section after the CI for the development environment, it no longer is considered "Best Practices" it is saying that you should use "Repository Manager" for the moving data within the Nexus itself.
From discussions I have had with various Talend techs, they have said that we should not try and use it in that way and that the preferred way to do it the how I described above.
I've read your post with interest and was wondering if I could please pick your brain with questions we have as we start off our Talend Journey.
Would you be able to comment please? I'm mainly interested in know how people increment their job artifact version(s) on nexus and how you get any fine grained control over which job artifact version(s) you run in production...ie can you move forward and rollback job versions in a CI env or do you have to run all jobs at the same GIT branch/tag...so you never rollback?
Join us at the Community Lounge.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
Pick up some tips and tricks with Context Variables
Learn how media organizations have achieved success with Data Integration
Accelerate your data lake projects with an agile approach