Configuring Talend Administration Center with Filebeat

Overview

This is a continuation of the Talend Community Knowledge Base article, Talend logging using Filebeat.

 

Depending on your Talend Administration Center (TAC) installation (single component or Talend bundle component), TAC can be installed alone or with Filebeat.

 

If Filebeat is not part of the installation, copy a Filebeat installation folder from another installation folder (for example, the one installed with Log Server), and configure a dedicated service.

 

Configuring TAC portal

TAC uses two files to collect technical and business logs. Configure these files on TAC by performing the following steps.

  1. Log in to TAC.
  2. Change the location of the Technical and Business logs in the Logs section on the TAC Configuration page, as follows:
    • Technical file appender: %TAC_installation%/tac/logs/technical.log
    • Business log file path: %TAC_installation%/tac/logs/business.log

    03.png

    These files and folders should be created manually on the machine, and the TAC service user should have permission on it.

     

Configuring Filebeat

Configure Filebeat, on the installation location, to permit Log Server to redirect logs to a specific logstash instance.

  1. Open the %TAC_installation%/filebeat/filebeat.yml file and change output.logstash: > hosts: with the location of your Log Server, as shown below:

    output.logstash:
     # The Logstash hosts
     hosts: ["logserver_ip:5044"]
     #hosts: ["localhost:5044"]
  2. Change the filebeat.prospectors: section. By default, there are already two sub-sections, identified by fields > app_id: TAC and DQ (out of scope, not reported).

    filebeat.prospectors:
    - type: log 
      enabled: true 
      paths: 
        - %TAC_installation_path%/tac/logs/*.log 
      fields: 
        app_id: TAC 
      fields_under_root: true
  3. Ensure that the paths in the TAC section point to the same location of the logs you configured on TAC portal.

    Note: It is also possible to redirect the apache-tomcat logs by creating a new sub-section, as shown below:

    filebeat.prospectors:
    - type: log
      enabled: true
      paths:
        - %TAC_installation_path%/tac/logs/*.log
      fields:
        app_id: TAC
      fields_under_root: true
    - type: log
      enabled: true
      paths:
        - %TAC_installation_path%/tac/apache-tomcat/logs/*.log
      fields:
        app_id: TAC_APACHE_TOMCAT
      fields_under_root: true
  4. Restart the Filebeat service.

 

Kibana portal results

The following Kibana portal screenshots show two log entries related to logout and login:

(filter - application.keyword: “TAC”)

04.png05.png

 

For more information on configuring Talend components with Filebeat, see the following Talend Community Knowledge Base (KB) articles:

Version history
Revision #:
11 of 11
Last update:
‎07-02-2019 08:54 AM
Updated by: