So it would appear that the Remote Engines do not out of the box clean-up after themselves leaving large amounts of data in <install_dir>/TalendJobServersFiles/repository/, including the temp directory.
There are 2 major problems with this:
1) If your jobs stage data to disk you rapidly run out of space.
2) Data, possibly including sensitive data, is left "at rest" with all the GDPR, etc. implications of this.
So is there a switch or setting that I can change to get the Remote Engine to properly clean up after itself or will I have to work out a way to do it using CRON, etc?
Do you want to delete the files in tmp folder (TalendRemoteEngine\data\tmp) without affecting the operation of Talend Integration?
We can delete the content when the Remote Engine is ON but we would better suggest to stop the Remote Engine, DELETE the content in tmp folder and Restart the Remote Engine.
Talend named a Leader.
Kickstart your first data integration and ETL projects.
Watch the recorded webinar!
This video will show you how to add context parameters to a job in Talend Cloud
This video will show you how to run a job in Studio and then publish that job to Talend Cloud
This video will help someone new to using Talend Studio get started by connecting to Talend Cloud and fetching the Studio License