Migration, Configuration, and Administration

Remove -XX:+HeapDumpOnOutOfMemoryError, and increase the max heap size.
View full article
Reduce logging level from INFO to WARN/ERROR.
View full article
Overview By default, Elasticsearch only binds to loopback IPs. This article shows you how to configure Elasticsearch, in the initial setup, to allow event logging from a remote Talend Runtime server.   For more information, see the Elasticsearch, Important System Configuration page.   Configure Elasticsearch   Changing network settings for Elasticsearch This section shows you how to make configuration changes, to move Elasticsearch from development mode to production mode. After you make these changes, the Elasticsearch REST interface will start listening to calls on the network interface (not making these changes allows the REST API to be bound only on the loopback address, allowing only calls from the localhost).   Navigate to the Elasticsearch Installation directory on the server where it is installed. As the root user, edit the elasticsearch.yml file and change the following parameters: -bash-4.2$ vim elasticsearch.yml # ---------------------------------- Network ----------------------------------- # # Set the bind address to a specific IP (IPv4 or IPv6): # #network.host: 192.168.0.1 network.host: 0.0.0.0 # # Set a custom port for HTTP: # #http.port: 9200 http.port: 9200   Changing limits for the user that runs Elasticsearch Elasticsearch, when configured to run in production mode, enforces bootstrap checks. This section shows you how to overcome the most common bootstrap check, "number of open files".   If you experience additional bootstrap check fails, see the Elasticsearch, Bootstrap Checks page.   As the root user, open the /etc/security/limits.conf file and add the following lines:   [root@devlnxttca system]# vim /etc/security/limits.conf . . #Below lines are added by SBM for Elasticsearch hardening talenduser   soft    nofile          65536 talenduser   hard    nofile          65536 talenduser   memlock unlimited . .   Changing the service definition file Edit the Elasticsearch service definition file (/etc/systemd/system/talend-elastic-x.x.x.service), and change the permitted number of open files, by adding LimitNOFILE to the script, as shown below:   [root@devlnxttca system] # cat talend-elastic-7.1.1.service # SystemD descriptor file for ElasticSearch service (part of Talend LogServer) [Unit] Description=Talend ElasticSearch service Before=runlevel3.target runlevel5.target After=local-fs.target remote-fs.target network-online.target time-sync.target systemd-journald-dev-log.socket Wants=network-online.target Conflicts=shutdown.target [Service] Type=forking Restart=no TimeoutSec=1min KillMode=process Restart=no Environment=JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.191.b12-0.el7_5.x86_64/jre Environment=ES_JAVA_OPTS=-Dmapper.allow_dots_in_name=true ExecStart=/opt/talend/Talend-7.1.1/logserv/elasticsearch-6.1.2/bin/elasticsearch -d User=talenduser Group=talendgroup LimitNOFILE=65536 WorkingDirectory=/opt/talend/Talend-7.1.1/logserv SuccessExitStatus=143 SIGKILL [Install] WantedBy=multi-user.target   Restart Elasticsearch Restart Elasticsearch, ensure the restart is successful and that the logging/Kibana is working in TAC.   Starting all TESB features Make the following configuration changes using the values for your specific environment. If there are no specific MDC attributes, these changes aren't necessary.   All configurations @ {Talend ESB Container/runtime}/etc   org.talend.eventlogging.server.cfg:   The following exception, from the Elasticsearch logs, shows that the later version (6.x) of Elasticsearch does not allow the creation of multiple index types.   [2019-08-26T19:22:46,340][DEBUG][o.e.a.a.i.m.p.TransportPutMappingAction] [C9gJhaB] failed to put mappings on indices [[[talendesb-2019.08.26/07HCl5NFQD6ueaY9qGNHYg]]], type [ESB] java.lang.IllegalArgumentException: Rejecting mapping update to [talendesb-2019.08.26] as the final mapping would have more than 1 type: [ESB, doc] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:494) ~[elasticsearch-6.1.2.jar:6.1.2] at org.elasticsearch.index.mapper.MapperService.internalMerge(MapperService.java:350) ~[elasticsearch-6.1.2.jar:6.1.2] at org.elasticsearch.index.mapper.MapperService.merge(MapperService.java:282) ~[elasticsearch-6.1.2.jar:6.1.2] at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:311) ~[elasticsearch-6.1.2.jar:6.1.2] at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230) ~[elasticsearch-6.1.2.jar:6.1.2] at org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:640) ~[elasticsearch-6.1.2.jar:6.1.2] Resolve this issue by changing the indextype to doc, as shown below:   # Host address where elasticsearch instance is started #elasticsearch.host=localhost changed on 13-may-19 elasticsearch.host=remote.elasticsearch.net # Port of elasticsearch server instance elasticsearch.port=9200 # Elasticsearch is secured or not elasticsearch.secured=false . . # Optional, Certificate hostname accepted independent from the server URL (any -> no hostname check) elasticsearch.ssl.pass.hostname=any # Username and password for secured elasticsearch elasticsearch.username=elastic elasticsearch.password=Talend # Name of index that will be used in elasticsearch elasticsearch.indexname=talendesb # Type of the index #elasticsearch.indextype=ESB elasticsearch.indextype=doc For more information, see the Elasticsearch, Removal of mapping types page.   Starting all Talend ESB event logging features Start ESB features, as follows:   [root@devlnxtesa bin]# ./client Logging in as tadmin         ______     __            __        /_  __/__ _/ /__ ___  ___/ /         / / / _ `/ / -_) _ \/ _  /        /_/  \_,_/_/\__/_//_/\_,_/         (version 7.1.1)   Hit '<tab>' for a list of available commands and '[cmd] --help' for help on a specific command. Hit '<ctrl-d>' or 'system:shutdown' to shutdown the TRUN. tadmin@trun()> tesb:start-all . . tadmin@trun()> tesb:start-el-default . . tadmin@trun()> feature:install tesb-el-mdc-mapper     Check the logs @ {Talend ESB Container/runtime}/log, for example, /opt/talend/Talend-7.1.1/esb/container/log.
View full article
Top Contributors