Four Stars

Change working directory when running a job in TOS, in order to use relative paths

Hello,

 

I have to test jobs in a project where relative paths are used in file components, like "./sqlfolder/exec.sql". When the job is exported and run in a production setting using a UNIX-based OS, the paths are relative to the shell script, but when I press the Run button in TOS on Windows, relative paths start from "D:\Talend\TOS_DI-<version>\", and I would like to change that to something like "D:\Workspaces\Myproject\Project_One\Files\".

 

The solution is not to use a context variable with a root path, because I would have to append that in front of the relative paths used in every component.

Using "system" components for "cd" commands also seems to cause more problems for portability that it solves (the same code should run in Windows and UNIX environments).

 

Is there a way to achieve that ?
Many thanks in advance

  • Data Integration
4 REPLIES
Nine Stars TRF
Nine Stars

Re: Change working directory when running a job in TOS, in order to use relative paths

Not a good idea (in my opinion). Also think the relative start path will change when you'll run jobs from outside of TOS.

Using context variables is the solution as you can concatenate their values with the partial filename in any component where you need a full pathname simply using 

context.yourRootPath + "yourPath/yourFile"

 

or

context.yourRootPath + (String)globalMap.get("yourGlobalContiningFileName")

TRF
Four Stars

Re: Change working directory when running a job in TOS, in order to use relative paths

Perhaps I wasn't clear enough, I am working on legacy code that I am not allowed to modify, the code cannot be tested in TOS as it is, only when exported, unless I move all the input files to Talend's installation folder, which would be a bad practice because we have multiple projects and multiple machines where Talend is installed.

 

It is either possible or not possible to change that folder before clicking the run button (without having to edit Talend's source code), since this folder is not randomly defined nor arbitrary, I believe someone knows at a technical level how this folder is defined (the run button calls talend.exe which calls the jar from the location of the exe maybe).

Nine Stars TRF
Nine Stars

Re: Change working directory when running a job in TOS, in order to use relative paths

You don't have to move anything to Talend's installation folder.

Once again, concerning the input files (and files generaly speaking), the best practice is to define as many context variable as input folders with appropriate value depending on the context (DEV, UAT, PROD for example).

Please, explain the case for which it cannot works for you.

 


TRF
Four Stars

Re: Change working directory when running a job in TOS, in order to use relative paths

I'm testing, not developing.
All the paths are hardcoded string literals containing relative paths, and once I'm done testing, I will commit the source code, which will be rejected if I made modifications to jobs. I could edit and revert, but this would mean doing that work each and every time I have to test. I can however make modifications to the environment variables, TOS settings, the jvm arguments in the settings, etc.
I believe that few people have that information because I did not find it on the forums nor in the help, if you don't have it either, no worries, this topic will wait for an answer and in the meantime, I'll implement temporary solutions, like context variables, or only testing after exporting, that I already know to be working. But that is bad practice and I'm looking for a better solution in the long run. But I agree, those who coded the jobs I'm working on should have used absolute paths, using a concatenation of at least one root_path and some relative paths, in string literals or in variables.