Skipping subjobs if file not present

Highlighted
One Star

Skipping subjobs if file not present

I have a mapping that has 5 sub jobs. Each sub job reads an Excel file and outputs a text file after doing some transformations. Currently they are linked with "if subjob OK" link so the jobs are executed one after other and the execution stops if one of the jobs abends. The order is not important. All 5 excel files need to be present for the job to run succesfully.
I want to change this job so that if an excel file is not present, I want that subjob to skip gracefully and go to the next job. So I want to test for presence of a file, execute the job if the file is present. If not, go to the next job and so on.
If above is not possible, I want the job where the excel file is not present to fail, but still the next job to run (so keep the sequence of jobs but not stop the execution for any error in the earlier job).
What is the way to do this? I have coded this in TOS 2.3/PERL.
Any tips will be appreciated.
Regards,
Sean
Tags (1)
One Star

Re: Skipping subjobs if file not present

OK. I do not know how I missed the tFileExist component. That will work to check the file. But I have 5 jobs and I still want them to fire one after the other.
So for each file processing, I want to check the file existence. Run the job if the file exists. If not, skip that file processing.
Then go to the next file. Check its existence, run the job if the 2nd file exists and so on.
I am still not clear how I can link these jobs together. I want them in one mapping/job as I need to create one script that I can compile with perlapp.
Thank you in advance for any advice.
Regards,
Sean
Employee

Re: Skipping subjobs if file not present

As tFileExist doesn't exist in TOS 2.3.x, I have directly used TOS 3.0.0. I can use a tPerl if you really want the job to be done in TOS 2.3.x.
In my input_dir, I have 2 files: {a, c} but not b. In the screenshot job, tFileExist_1 checks for "a" file existence, tFileExist_2 checks for "b" existence and tFileExist_3 checks for "c" existence.
One Star

Re: Skipping subjobs if file not present

Thank you for your reply. This will help me with some future projects. This also explains how I missed this component earlier when I was working with 2.3. Would you be kind enough to explain how I can use tPerl to check the existence and implement something similar to what you have shown me with 3.0? My original job is in 2.3 and I would rather keep this minor enhancement in the same version rather than risk upgrading and come up with other issues.
Thank you again for your help.
Regards,
Sean
Employee

Re: Skipping subjobs if file not present

The perl code in tFileExist is so simple that it doesn't cost much to recode in a tPerl. An even better solution would be to make tFileExist available in the ecosystem compatible with TOS 2.3.3.
In tPerl_1, the code is:
if (-e $_context{input_dir}.'/a') {
$_globals{tFileExist_1}{EXISTS} = 1;
}
else {
$_globals{tFileExist_1}{EXISTS} = 0;
}
One Star

Re: Skipping subjobs if file not present

Thanks plegall. I had not looked inside the tFileExists code. This will help me nicely.
Regards,
Sean