This article shows you how to redirect logs to another target but still get the Job execution status in Talend Cloud. In this article, the target is ActiveMQ, a JMS message service.
This article covers, testing your configuration by creating a Talend Job to generate the logs. Running the Job and redirecting the logs to Talend Cloud Management Console, then changing the configuration to redirect the logs to the JMS.
The demo Job is available in the LogGenerator.zip file attached to this article.
Install and pair a remote engine
For more information on installing and pairing a remote engine, see the following Talend Help Center documentation:
Install a JMS broker such as ActiveMQ
Create a Job using a tWarn component to display the log4j message, and a tRowGenerator component with a tLogrow component to display data directly in the console.
Run the Job to display the logs.
Publish the Job to Talend Cloud, by right-clicking the Job, then select Publish to Cloud.
If this isn't the first version, select the version you want from the Publish With Version field, then select Personal from the Workspace drop-down menu. Click Finish.
When the Job has finished uploading, click Open Job Task.
Notice the Task doesn't have an assigned Runtime.
On the Engine tab, you should see your engine, in this case, demo-mga.
Select the Task and click the pencil icon to edit the configuration.
In step 2, Go Live, select the Runtime and the Run type.
Click Go Live. You will be sent back to the Task View. Notice that the Job Status is Running.
Once the Task is complete, select the execution and click View Logs.
Notice that the logs are redirected to Talend Cloud. In the next steps, you will change the configuration to modify this behavior.
Stop the Remote Engine service.
Edit the remote-engine-installation/etc/org.talend.eventlogging.sender.jms.cfg file in your Remote Engine installation folder.
The configuration should look similar to this:
# JMS connection url sender.destination.jms.url = failover:(https://log.eu.cloud.talend.com:443) # Name of JMS queue sender.destination.jms.queue=event.logging.server # JMS Username sender.destination.jms.username = ca677a20-086b-472e-8811-6x48fbcd13d5_kIt4srr50GAyZ9pwr # JMS Password sender.destination.jms.password = WxLgy6pBHadz4xq0HvASe74 sender.destination.jms.pool.maxConnections = 8
Update the sender.destination.jms.url parameter to your JMS broker, in this case, localhost.
# JMS connection url sender.destination.jms.url = failover:(tcp://localhost:61616) # Name of JMS queue sender.destination.jms.queue=event.logging.server # JMS Username sender.destination.jms.username = ca677a20-086b-472e-8s811-6b48dbcd1q3qad5_kIt4rr50GAyZ9pwr # JMS Password sender.destination.jms.password = WxLgy6pBH4xq0HvASe74 sender.destination.jms.pool.maxConnections = 8
Restart the Remote Engine service.
Log in to the ActiveMQ admin console; you should see a new queue, event.logging.server.
Go back to Talend Cloud Management Console and rerun the Task.
Once it is successfully executed, check the logs to validate no logs are displayed.
In the ActiveMQ console, you should see several messages enqueued.
Open one to validate that it is available.
In this example, you learned how to edit the Remote Engine configuration to redirect Task logs in to a JMS broker. Once logs are available in the JMS, you can have a pipeline to read and process them to another system.