Continuous Integration with Talend ESB Route Builder

 Overview

This article describes how to perform all the steps required for a continuous integration test of your Apache Camel-based routes designed within Talend ESB Studio using the Continuous Integration Testing Framework:

  1. Automated compilation of source artifacts
  2. Automated Unit-Test on each artifact
  3. Automated deployment into Test-Environment
  4. Run automated Integration-Tests

Environment

This procedure applies to the Talend solutions that include ESB starting from version 5.1.0.

Procedure

Automating the compilation of source artifacts

You cannot build directly your Camel Routes based on artifacts stored in your SVN repository. Mapping from Studio artifacts to Java Code is only possible in the Studio. Therefore you need to generate your Java Code in the Talend Studio first,  and then deploy these artifacts to Archiva. However, you can generate also your Java Code based on Studio artifacts using the Talend Commandline since version 5.2 onwards.

Automating the Unit-Test on each artefact

Usually an SVN Update triggers a recompilation of these sources in your Continuous Integration Framework and Maven will automatically run all attached JUnit Tests to your code. However, this does not work with ESB Routes designed in Talend Studio right now.

One reason is, that SVN commits happen quite often, each time someone makes changes to a route and tries to run it. Thus SVN Update events are not a proper trigger for running Unit-Test, since a SVN commit does not mean the development process is already completed. Therefore your Continuous Integration Framework should not listen for SVN updates, but rather for Archiva updates. Because if a developer is done with designing his route, he/she will publish his/her route to an Archiva snapshot release repository.

Since only the artifacts published to Archiva are "mavenized" but not the .Java project inside Studio, we can not simply append our Unit-Test to the .Java project. Code-changes within the .Java project could also get overridden easily by the code generator. Therefore it is recommended to create a separate Maven-Test-Project. This Maven-Test-Project needs to refer to the Maven Artefact generated by Studio. Just have a look at the attached Maven sample pom.xml file.

By default, Maven dependencies are only downloaded once from remote repositories, and then stored/cached in a local file system. Normally you can force Maven to download changed (snapshot) artifacts with same artefact-version each time from a remote repository, but for some reason Maven will not detect a newer version of the published route in the Archiva repository and thus use an old version stored locally. So changes in your route will not reflect in your next test-run. To avoid testing the old versions deployed first from Studio, you need to remove all matching directories or files from your local Maven repository. You can also use a Maven plugin to delete these folders each time you type "mvn clean". Refer to the sample pom.xml file for further details.

Since the Route deployment and Unit-Tests are separate Maven projects, you need to link these projects in your integration testing framework, so that your test will run each time an update is applied to your camel route project.

If you want to know more about Unit-Testing, have a look at the attached HelloWorldTest.java.

Handling 3rd Party dependencies

Unfortunately, the maven project file generated by Studio does not contain any dependencies (even though there are a couple). Hence you need to discover all dependencies used within the route by yourself and add these to your maven pom.xml Test project.

Automate the deployment into the Test-Environment

To make a real integration test, it is not sufficient to simply test each route individually. The goal of an integration test is to verify that routes are working in conjunction with each other in a production-like environment with real backend-systems.

Deploying routes to Talend ESB Runtime can be done through JMX or using Talend Commandline. Have a look at the attached deployment.script to see how this could be done. Remember to use a new Talend ESB runtime container for deployment each time, to ensure reproduceable results.

Run automated Integration Tests

After deployment of  your routes into Talend ESB Runtime, we need to run further Unit-Test focusing on integration tests.

There are several options to write Integration Tests. For example, you can deploy them into your runtime container and use JMX notifications or writing/checking log files, to evaluate your test results. Camel provides a rich Testing Framework that could also be used in a normal JUnit Test Case to send messages to JMS queues or creating sample files in a local test system.

Summary 

Through the description of steps above, the best practice can be summarized as follows:

  • Write your Test Cases in a separate Maven project linked to you RouteBuilder project.
  • Use Archiva Updates as triggers for your Tests instead of SVN commits.
  • Connect Unit-Test Project with Route Builder project
  • Clear your local Maven repository (from routing projects only)
  • Use Camel Mock components in your Unit-Tests
  • Test in production-like environments
  • Write Unit-Test for real-world integration testing

Running Hello World Sample

We provides a Hello world sample in this article, follow these steps below to run it:

  1. Download and import Hello World project into Studio.
  2. Deploy Studio Project to Archiva.
  3. Download and import HelloWorldTest Project (you can use a normal Eclipse or Java Perspective in Studio).
  4. Adjust pom.xml to match your deployed route artefact name.
  5. Run: "mvn clean compile" in your test project.
  6. Run: "mvn test" or "mvn install" to run your unit tests.

 Related Files

 

  File Modified
  ZIP Archive HelloWorld.zip Apr 12, 2013 byShicong Hong
  ZIP Archive HelloWorldTest.zip Apr 12, 2013 byShicong Hong
  Java Source HelloWorldTest.java Apr 12, 2013 byShicong Hong
  XML File pom.xml Apr 12, 2013 byShicong Hong
 

Labels

 
 
Version History
Revision #:
1 of 1
Last update:
‎04-17-2017 10:31 PM
Updated by:
 
Labels (1)
Contributors