Understanding JobServer ratings in TAC

The goal of JobServer ratings (rank)

TAC manages a rating for each JobServer connected to it. This rating is also used in virtual server pools to dispatch Jobs, based on this rating server rates are based on indicators, whose bounds (such as free disk space limits) and weight are defined in the monitoring_client.properties file located in the <ApplicationPath>\WEB-INF\lib\org.talend.monitoring.client-A.B.C_rYYYYY.jar file.

 

The server rate updates every 90 seconds, which you can change by modifying the value of the notification.conf.checking.frequencyCheckJobServerState=90 parameter.

 

Rating values

The rating range is from 0 to 100:

  • 100: server availability is optimal, indicator values are within bounds
  • < 100: one or several indicators values are out of bounds

In TAC, this rating is represented by stars in the Rank item of the JobServer.

rank.png

 

The parameters used to establish this rating are:

  • free disk space (in kilobytes)
  • free physical memory (in bytes)
  • free swap memory (in bytes)
  • idle CPU usage (in percent between 0.0 and 1.0)
  • nice CPU usage (in percent between 0.0 and 1.0)
  • total CPU usage (in percent between 0.0 and 1.0)
  • minimum required number of CPU

Each of these parameters has a minimum and maximum value, and a weight that ranges from 0 to 5 (0 means the indicator is not taken into account, 5 means the indicator is really important) then the rating (multiplied by the weight given to the parameter) from all parameters is aggregated to obtain the rating of the server.

 

For more information on these calculations, see the Talend Big Data Installation Guide for Linux, JobServer rate computation page, available in the Talend Help Center.

 

Setup rating parameters

  1. Extract the monitoring_client.properties file, for example:

    jar xvf org.talend.monitoring.client-7.2.1.jar monitoring_client.properties

    The monitoring_client.properties file is located in the <ApplicationPath>/WEB-INF/lib/org.talend.monitoring.client-x.y.z.rabcd.jar file.

    • For example, in Talend 7.2.1:

      C:\talend\7.2.1\tac\apache-tomcat\webapps\org.talend.administrator\WEB-INF\lib\org.talend.monitoring.client-7.2.1.jar
    • For example, in Linux:

      /opt/talend_7.2.1/tac/apache-tomcat/webapps/org.talend.administrator/WEB-INF/lib/org.talend.monitoring.client-7.2.1.jar
  2. Edit the monitoring_client.properties file and modify the parameters as needed.

  3. Copy the edited file, to overwrite the one located in the JAR file, into the <ApplicationPath>/WEB-INF/classes directory.

    cp monitoring_client.properties   /opt/talend_7.2.1/tac/apache-tomcat/webapps/org.talend.administrator/WEB-INF/classes

 

Usage and examples

In this example, a single parameter is taken into account:

# Set the default NbCPULimit.WEIGHT
org.talend.monitoring.limits.NbCPULimit.WEIGHT=5

# Set the default minimum required number of CPU here
org.talend.monitoring.limits.NbCPULimit.LOWER_LIMIT=2

# Set the default maximum required number of CPU here
org.talend.monitoring.limits.NbCPULimit.HIGHER_LIMIT=2147483647
  1. Configure two JobServers as follows:

    1. Configure JS1 with 1 CPUs.

    2. Configure JS2 with 4 CPUs.

  2. Review the TAC rating:

    ex1.png

    This is not the expected outcome. Why?

    This is because the calculation uses:

    rel_offset{i} = offset{i} / [ Max{i} - Min{i} ]

    this example uses the default value for Max{i} = 2147483647

    The rel_offset is close to 0.

    Adjust the org.talend.monitoring.limits.NbCPULimit.HIGHER_LIMIT to a more suitable value, for example, if you have a pool of 5 servers, and at most, a server can have 4 CPUs, then select 4.

     

  3. Conduct the same test using the following parameters:

    org.talend.monitoring.limits.NbCPULimit.LOWER_LIMIT=2
    org.talend.monitoring.limits.NbCPULimit.HIGHER_LIMIT=4
    1. Configure JS1 with 1 CPUs.

    2. Configure JS2 with 4 CPUs.

  4. Review the TAC rating:

    ex2.png

     

  5. Conduct the test again using the following parameters:

    org.talend.monitoring.limits.NbCPULimit.LOWER_LIMIT=3
    org.talend.monitoring.limits.NbCPULimit.HIGHER_LIMIT=4
  6. Review the TAC rating:

    ex3.png

Notice that the minimum and maximum parameters are important and that a wrong value can have a significant impact on the server rate. Most, if not all, parameter values need to be tuned according to your configuration.

 

Also, consider the units used:

  • Disk space: kilobytes

  • Memory: bytes

  • CPU usage: percentage between 0.0 and 1.0

Version history
Revision #:
13 of 13
Last update:
‎01-08-2020 01:06 PM
Updated by:
 
Contributors
Comments
Community Manager

This article will be merged into the Talend Help Center per DOCT-13232.