'Exceeded the limit of deployment attempts: you have reached the limit of Flow deployments on the engine'

Problem Description

When trying to execute a Task, the execution fails with the following error:

Exceeded the limit of deployment attempts: you have reached the limit of Flow deployments on the engine.

 

Root Cause

You have reached the maximum number of allowable concurrent flows (running Jobs). The limit is set by license, or where the Remote Engine is being used to run the flows—your configuration.

 

A Cloud Engine can only run up to three flows at the same time (that is, three concurrent flows).

 

A Remote Engine can only run up to three flows at the same time by default.

 

So, when you look at your flow execution history in Talend Cloud, and you see this error message for some flows that have attempted to execute, it means that you already have three flows running. Thus, there are no open execution slots for the new flow to run. The flow that is not able to run is rescheduled and remains in the queue. When one of the currently-running Jobs finishes, the next flow in the queue is run.

 

If a flow is not able to be executed after several rescheduling attempts (that is, no open execution slots have opened up, or other flows were first/earlier in the queue), the flow moves into an error state and will no longer attempt to run.

 

Solution

You can modify or remove the limit by following the instructions and modifying the configuration of your Remote Engine.

 

Note: If you have already installed and paired your Remote Engine, you will have to reinstall it for this limit change to take effect. Read through the entirety of this page to get full instructions—it's relatively easy, but you have to do the steps in order.

  1. Install the Remote Engine, but do not enter the pairing key in the install wizard interface—leave this field blank. Do enter/select values for all other fields/drop-downs. If you already have a Remote Engine installed, stop the service, uninstall it, install a new Remote Engine (you can download the latest version if you don't have it yet, but this is not a requirement).

  2. Modify the following files, as shown below:
    1. REMOTE_ENGINE_INSTALL_DIR/etc/org.talend.ipaas.rt.deployment.agent.cfg

      Change this value to 0 (unlimited):

      max.deployed.flows=3
    2. REMOTE_ENGINE_INSTALL_DIR/etc/preauthorized.key.cfg

      Fill in these three fields:

      remote.engine.pre.authorized.key = <remote-engine-key-from-remote-engine-you-want-to-pain-in-Talend-Cloud>
      remote.engine.name = dev_remote_engine_1
      remote.engine.description = Cool remote engine for dev 1
      
  3. Restart the Remote Engine.

  4. Wait a few minutes, and the Remote Engine shows Available in Talend Cloud.

Version history
Revision #:
20 of 20
Last update:
‎04-14-2019 02:17 PM
Updated by: