Run of the job stopped by an FTP error (~"Die on error" unchecked)

Five Stars

Run of the job stopped by an FTP error (~"Die on error" unchecked)

Good Morning,

 

the purpose of my job is transferring files connecting to multiple ftp whose connection parameters are retrieved from a database.

The directory containing the files to be delivered is scanned and the files are sent to the expected ftp.

 

There are two main loops in my job as can be seen on the screenshot.

 

I have some trouble on the FTP part of my job as described below :

 

While the two FTP components have their "Die on error" parameter unchecked, my job isn't reliable if an ftp server is not available or is returning an error code.

 

The error is not caught as expected.

 

Indeed my job fails at the first error, for example a bad password, and my job will quit while I would like it to continue and process the waiting files for the remaining ftp.

 

    => How can I make the ftp components truly "fail-proof" (ignore the rotten apple) ?

 

Thank you for your answer.

 

screenshot_LIV_OCTOPUS_FTP_v2_20180130.pngglobal screenshot of the jobtFTPPut_Die_on_error_OFF.pngfirst FTP component parameterstFTPRename_Die_on_error_OFF.pngsecond FTP component parameters

Moderator

Re: Run of the job stopped by an FTP error (~"Die on error" unchecked)

Hello,

You can use tContextLoad or implicit tContextLoad to retrieve your FTP connection parameters from a database.

The first part can be triggered the next subjob by using "OnSubJobOk" connection type on the condition that the main subjob completed without error. This connection is to be used only from the start component of the Job. 

The workflow looks like:

t<DB>Input-->tContextLoad

|

onsubjobOK

|

FTP part.

Best regards

Sabrina

--
Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Run of the job stopped by an FTP error (~"Die on error" unchecked)

Good Morning,

 

retrieving the ftp parameters and creating a successful connection was OK, I managed to do the first loop, thank you.

 

My problem is actually on the second loop, handling errors in ftp transfers : 

 

what I observe is that my job fails if there's any connection error on a ftp, for example a timeout, a bad password, a wrong host, etc, whereas I want my job to continue and process the next files and their related ftps remaining in the queue.

 

Please note that while the "Die on error" parameter of the ftp components is uncheked my job still "dies".

 

A design with a job for the first loop and a subjob for the ftp part (second loop) would make it "fail-proof" ?

 

Thank you.

 

Best Regards,

Jacky Siveton.

Six Stars

Re: Run of the job stopped by an FTP error (~"Die on error" unchecked)

Hi all,

I've got the same issue... and i'm really interested in the solution.

as well if the connections settings are embedded in a tFTPPut or if they are inherited from an existing tFTPConnection

 

Is there a solution arround "run if" and the "__DIE_ON_ERROR__" variable ?

 

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download