TAC becomes unreachable and unresponsive, so no Jobs can be run

Talend Version          5.x, 6.x
Key words TAC performance unresponsive unreachable
Product Talend Data Integration
Component Talend Administration Center
Article Type Debugging
Problem Description Talend Administration Center (TAC) needs to be restarted frequently as it gets blocked after 1 hour.
Problem root cause

There could be many causes for this performance issue. It is important to know:

  • How many tasks are running at the same time?
  • How many tasks are running in a day on average?
  • For how long are they running?
  • When the tasks were created, was the "Generate on change" option checked?
  • How often are the related Jobs modified?

To get the TAC technical log, do a thread dump at the moment of the performance issue.

 

If using MySQL, changing the transaction isolation level can help.

 

If using Microsoft SQL Server 2008, read-committed isolation using row versioning must be activated.

Solution or Workaround

Per the online document (Recommendations about environment and configuration for the Talend Administration Center) you should look at the values of these three parameters:

  • hibernate.c3p0.max_size (set in the configuration.properties file)
  • org.quartz.threadPool.threadCount (set in the quartz.properties file)
  • org.quartz.dataSource.QRTZ_DS.maxConnections (set in the quartz.properties file)

Note: You can locate the configuration.properties and the quartz.properties files in the Tomcat8.0\webapps\org.talend.administrator-6.2.1\WEB-INF\classes folder.

 

Increasing these values can be done incrementally, and it depends on the value of MAX_CONCURRENT_TASK_EXECUTIONS + MAX_CONCURRENT_PLAN_EXECUTIONS.

 

If hibernate.c3p0.max_size < org.quartz.dataSource.QRTZ_DS.maxConnections you will have problems, so it is important to ensure that the hibernate.c3p0.max_size > org.quartz.dataSource.QRTZ_DS.maxConnections.

 

You will need to restart the Tomcat / TAC for the change to take effect. After changing the values, the TAC should behave well and stop crashing.

Default values

(small install)

# quartz.properties

org.quartz.threadPool.threadCount = 30

org.quartz.dataSource.QRTZ_DS.maxConnections = 30

 

# configuration.properties

hibernate.c3p0.max_size=32

 

Example values 1

(medium install)

# quartz.properties

org.quartz.threadPool.threadCount = 60

org.quartz.dataSource.QRTZ_DS.maxConnections = 70

 

# configuration.properties

hibernate.c3p0.max_size=80

 

Example values 2

(larger install)

# quartz.properties

org.quartz.threadPool.threadCount = 100

org.quartz.dataSource.QRTZ_DS.maxConnections = 110

 

# configuration.properties

hibernate.c3p0.max_size=120

 

Version history
Revision #:
9 of 9
Last update:
‎06-05-2018 02:54 PM
Updated by:
 
Labels (1)
Comments
DavidApodaca

Another good thing to check related to the memory settings is the setenv.bat (Windows) or the setenv.sh (Linux) to see if the relevant parameters are set correctly for their environment. If they are small or the default values, increase these. The parameters are below:

 

-Xms

-Xmx

-XX:MaxMetaspaceSize

 

It is found in the path: 

 

C:\Talend\6.x.x\tac\apache-tomcat\bin

OR

<TomcatPath>/bin

 

Documentation:

https://help.talend.com/reader/vuI_X~V6unFjTgNxRMPcLw/Ij~7tBlW8im63rAGnGHT3A