How to Delete Elasticsearch Logs Automatically?

Talend Version          6.1.1

Summary

 
Additional Versions  
Key words Elasticsearch logserver
Product Talend Data Integration
Component Logserv/Kibana
Article Type Configuration
Problem Description Elasticsearch logs are generated in the Logserver/elasticsearch-1.5.2/log directory, so the disk space that contains those logs can become full if they are not moved or deleted. The question is how Elasticsearch log files can be purged automatically.
Problem root cause

The Elasticsearch logs configuration can be found in Logserver/elasticsearch-1.5.2/config/logging.yml. The default configuration is:

"
file:
   type: dailyRollingFile
    file: ${path.logs}/${cluster.name}.log
    datePattern: "'.'yyyy-MM-dd"
    layout:
      type: pattern
      conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
"

With this configuration, every day, a file will be created in order to save the previous day's log. As a result, the number of log files will increase, and it can lead to the disk full problem.

Solution or Workaround

One solution is to update the Elasticsearch log configuration, and to use the MaxBackupIndex option to determine how many backup files are kept before the oldest is erased. In this case, you can update the logging.yml configuration file as follows:

"
file:
type: rollingFile
file: ${path.logs}/${cluster.name}.log
maxFileSize : 200KB
maxBackupIndex: 4
layout:
type: pattern
conversionPattern: "[%d{ISO8601}][%-5p][%-25c] %m%n"
"
JIRA ticket number  
Version history
Revision #:
6 of 6
Last update:
‎09-29-2018 12:12 AM
Updated by:
 
Labels (2)