One Star

[resolved] Error when starting Talend Log Server

Hello Team,

I am configuring Talend Administration Center and running into problem with Log Server. According to the documentation:
The Logstash host and port parameter will get green when you will start the Talend Log Server, by executing start_logserver.bat (Windows) or start_logserver.sh (Linux).
Unfortunately when I execute start_logserver.bat I get the following exception:


no known master node, scheduling a retry
failed to connect to master ]], retrying...
org.elasticsearch.transport.ConnectTransportException: ] connect_timeout
at org.elasticsearch.transport.netty.NettyTransport.connectToChannels(NettyTransport.java:797)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:731)
at org.elasticsearch.transport.netty.NettyTransport.connectToNode(NettyTransport.java:704)
at org.elasticsearch.transport.TransportService.connectToNode(TransportService.java:216)
at org.elasticsearch.discovery.zen.ZenDiscovery.joinElectedMaster(ZenDiscovery.java:464)
at org.elasticsearch.discovery.zen.ZenDiscovery.innerJoinCluster(ZenDiscovery.java:416)
at org.elasticsearch.discovery.zen.ZenDiscovery.access$6100(ZenDiscovery.java:85)
at org.elasticsearch.discovery.zen.ZenDiscovery$JoinThreadControl$1.run(ZenDiscovery.java:1373)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection timed out: no further information: /169.254.77.21:9301
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:152)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:337)
at org.elasticsearch.common.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
... 3 more

observer: timeout notification from cluster service. timeout setting , time since start


I tried to make changes to elasticsearch.yml file to define master node explicitly but no luck. My final goal here is to configure ESB dashboards, which currently don't work at all, and my assumption is that elasticsearch error above may be the cause of dashboards not working correctly. All other components were started successfully as described in Talend documentation (Starting Talend Runtime and Starting the Talend Administration Center: https://help.talend.com/search/all?query=Starting%252Bthe%252Bsoftware
We are using Talend 6.0.1 ESB Subscription.

Please advise.

Thank you,
Svetlana
1 ACCEPTED SOLUTION

Accepted Solutions
One Star

Re: [resolved] Error when starting Talend Log Server

it was fixed by adding more memory to the vm.
Cheers,
Svetlana
6 REPLIES
Employee

Re: [resolved] Error when starting Talend Log Server

Hi,
If you want to use cluster support, then add the same cluster name to every node, e.g
cluster => "talend-001"
in the Talend-LogServer/logstash-talend.conf
Or you can disable the cluster support by setting :
discovery.zen.ping.multicast.enabled: false
in the config/elasticsearch.yml
HTH
One Star

Re: [resolved] Error when starting Talend Log Server

Thank you very much for your help! I was able to start log server. Unfortunately it did not fix issues with dashboard. I have stopped and started all services as per documentation, then deployed test application with Use Service Activity Monitoring and Use Service Locator options enabled and called it several times, but still I am getting exceptions in log file and dashboard never goes live. I have undeployed all previous applications leaving only one running but still no luck. 
Please advise.
Thank you!
Here is exception in tesb.log file:
08:58:11,261 | WARN  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  161 | 560 - locator-monitor - 6.0.1 | Endpoint for Service {http://www.talend.org/rest/}testJob with Address http://10.166.32.190:8088/ is down since Jan 6, 2016 8:51:38 AM
08:58:11,264 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  171 | 560 - locator-monitor - 6.0.1 | 1 endpoints are registered for service {http://www.talend.org/rest/}testJob
08:58:11,264 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  175 | 560 - locator-monitor - 6.0.1 | 0 endpoints are active for service {http://www.talend.org/rest/}testJob
08:58:11,265 | WARN  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  181 | 560 - locator-monitor - 6.0.1 | 1 endpoints are offline for service {http://www.talend.org/rest/}testJob
08:58:11,269 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  158 | 560 - locator-monitor - 6.0.1 | Endpoint for Service {http://www.talend.org/test/}testJob with Address http://10.166.32.190:8089/ is alive since Jan 6, 2016 8:53:12 AM
08:58:11,269 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  171 | 560 - locator-monitor - 6.0.1 | 1 endpoints are registered for service {http://www.talend.org/test/}testJob
08:58:11,269 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  175 | 560 - locator-monitor - 6.0.1 | 1 endpoints are active for service {http://www.talend.org/test/}testJob
08:58:11,269 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  183 | 560 - locator-monitor - 6.0.1 | 0 endpoints are offline for service {http://www.talend.org/test/}testJob
08:58:11,275 | WARN  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  161 | 560 - locator-monitor - 6.0.1 | Endpoint for Service {http://www.talend.org/rest/}GeneralInformationService with Address http://localhost:8088/ is down since Jan 5, 2016 10:14:00 AM
08:58:11,276 | WARN  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  161 | 560 - locator-monitor - 6.0.1 | Endpoint for Service {http://www.talend.org/rest/}GeneralInformationService with Address http://10.166.32.190:8088/ is down since Jan 6, 2016 8:16:31 AM
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  171 | 560 - locator-monitor - 6.0.1 | 2 endpoints are registered for service {http://www.talend.org/rest/}GeneralInformationService
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  175 | 560 - locator-monitor - 6.0.1 | 0 endpoints are active for service {http://www.talend.org/rest/}GeneralInformationService
08:58:11,276 | WARN  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  181 | 560 - locator-monitor - 6.0.1 | 2 endpoints are offline for service {http://www.talend.org/rest/}GeneralInformationService
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  194 | 560 - locator-monitor - 6.0.1 | 3 services are registered at the ServiceLocator
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  198 | 560 - locator-monitor - 6.0.1 | 1 services are available and currently registered at the ServiceLocator
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  202 | 560 - locator-monitor - 6.0.1 | 2 services are currently registered at the ServiceLocator but are not available
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  208 | 560 - locator-monitor - 6.0.1 | 4 endpoints are registered at the ServiceLocator
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  212 | 560 - locator-monitor - 6.0.1 | 1 endpoints are available and currently registered at the ServiceLocator
08:58:11,276 | INFO  | ool-174-thread-1 | locator.monitor.LocatorMonitor$1  216 | 560 - locator-monitor - 6.0.1 | 3 endpoints are currently registered at the ServiceLocator but are not available
08:58:13,605 | WARN  | erver-additional | rg.apache.camel.util.CamelLogger  224 | 157 - org.apache.camel.camel-core - 2.15.3 | Error processing exchange. Exchange
eventUUID:
correlationId:
category:
eventType:
severity:
logSource:
logMessage:
signedLogMessage:
logTimestamp:
agentId:
agentTimestamp:
serverTimestamp:
audit:
auditSequenceNo:
subject:
customInfo:[]
]. Caused by:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
at sun.reflect.GeneratedMethodAccessor897.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
at com.sun.proxy.$Proxy543.receiveResponseHeader(Unknown Source)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:85)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:47)
at org.talend.esb.eventlogging.server.addon.ElasticsearchClient.sendData(ElasticsearchClient.java:26)
at org.talend.esb.eventlogging.server.addon.ElasticsearchSenderProcessor.process(ElasticsearchSenderProcessor.java:121)
at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:652)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:580)
at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:227)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.component.seda.SedaConsumer.sendToConsumers(SedaConsumer.java:291)
at org.apache.camel.component.seda.SedaConsumer.doRun(SedaConsumer.java:200)
at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:147)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
08:58:16,610 | WARN  | erver-additional | rg.apache.camel.util.CamelLogger  224 | 157 - org.apache.camel.camel-core - 2.15.3 | Error processing exchange. Exchange
eventUUID:
correlationId:
category:
eventType:
severity:
logSource:
logMessage:
signedLogMessage:
logTimestamp:
agentId:
agentTimestamp:
serverTimestamp:
audit:
auditSequenceNo:
subject:
customInfo:[]
]. Caused by:
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:170)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
at sun.reflect.GeneratedMethodAccessor897.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.http.impl.conn.CPoolProxy.invoke(CPoolProxy.java:138)
at com.sun.proxy.$Proxy543.receiveResponseHeader(Unknown Source)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:85)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:186)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:106)
at io.searchbox.client.http.JestHttpClient.execute(JestHttpClient.java:47)
at org.talend.esb.eventlogging.server.addon.ElasticsearchClient.sendData(ElasticsearchClient.java:26)
at org.talend.esb.eventlogging.server.addon.ElasticsearchSenderProcessor.process(ElasticsearchSenderProcessor.java:121)
at org.apache.camel.processor.DelegateSyncProcessor.process(DelegateSyncProcessor.java:63)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:652)
at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:580)
at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:227)
at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:190)
at org.apache.camel.component.seda.SedaConsumer.sendToConsumers(SedaConsumer.java:291)
at org.apache.camel.component.seda.SedaConsumer.doRun(SedaConsumer.java:200)
at org.apache.camel.component.seda.SedaConsumer.run(SedaConsumer.java:147)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
08:58:19,614 | WARN  | erver-additional | rg.apache.camel.util.CamelLogger  224 | 157 - org.apache.camel.camel-core - 2.15.3 | Error processing exchange. Exchange
Employee

Re: [resolved] Error when starting Talend Log Server

Hi,
Have you already started the EL agent and EL server with ElasticSearch as backend?
One Star

Re: [resolved] Error when starting Talend Log Server

Hi,
After your previous reply that helped me to resolve issues with elastic search, I stopped everything and started everything again following talend documentation 2.4.1 (Starting Talend Runtime), 2.4.2 (Starting the Talend Administration Center) and 2.4.4(Starting the ESB dashboards). Which means that after executing start_logserver.bat,  I executed following commands
tesb:start-el-server
tesb:stop-el-server
Change the configuration of the org.talend.eventlogging.server.cfg file
tesb:start-el-default
tesb:stop-sam
tesb:start-sam-retrieval-service
tesb:start-el-samlistener
tesb:start-el-locatorlistener
One Star

Re: [resolved] Error when starting Talend Log Server

Hello,
Anybody can help please? Exception is still happening Smiley Sad
Thanks!
Svetlana
One Star

Re: [resolved] Error when starting Talend Log Server

it was fixed by adding more memory to the vm.
Cheers,
Svetlana