Run of the job stopped by an FTP error (~"Die on error" unchecked)

Five Stars

Run of the job stopped by an FTP error (~"Die on error" unchecked)

Good Morning,


the purpose of my job is transferring files connecting to multiple ftp whose connection parameters are retrieved from a database.

The directory containing the files to be delivered is scanned and the files are sent to the expected ftp.


There are two main loops in my job as can be seen on the screenshot.


I have some trouble on the FTP part of my job as described below :


While the two FTP components have their "Die on error" parameter unchecked, my job isn't reliable if an ftp server is not available or is returning an error code.


The error is not caught as expected.


Indeed my job fails at the first error, for example a bad password, and my job will quit while I would like it to continue and process the waiting files for the remaining ftp.


    => How can I make the ftp components truly "fail-proof" (ignore the rotten apple) ?


Thank you for your answer.


screenshot_LIV_OCTOPUS_FTP_v2_20180130.pngglobal screenshot of the jobtFTPPut_Die_on_error_OFF.pngfirst FTP component parameterstFTPRename_Die_on_error_OFF.pngsecond FTP component parameters


Re: Run of the job stopped by an FTP error (~"Die on error" unchecked)


You can use tContextLoad or implicit tContextLoad to retrieve your FTP connection parameters from a database.

The first part can be triggered the next subjob by using "OnSubJobOk" connection type on the condition that the main subjob completed without error. This connection is to be used only from the start component of the Job. 

The workflow looks like:





FTP part.

Best regards


Don't forget to give kudos when a reply is helpful and click Accept the solution when you think you're good with it.
Five Stars

Re: Run of the job stopped by an FTP error (~"Die on error" unchecked)

Good Morning,


retrieving the ftp parameters and creating a successful connection was OK, I managed to do the first loop, thank you.


My problem is actually on the second loop, handling errors in ftp transfers : 


what I observe is that my job fails if there's any connection error on a ftp, for example a timeout, a bad password, a wrong host, etc, whereas I want my job to continue and process the next files and their related ftps remaining in the queue.


Please note that while the "Die on error" parameter of the ftp components is uncheked my job still "dies".


A design with a job for the first loop and a subjob for the ftp part (second loop) would make it "fail-proof" ?


Thank you.


Best Regards,

Jacky Siveton.


Introduction to Talend Open Studio for Data Integration.

Definitive Guide to Data Integration

Practical steps to developing your data integration strategy.

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.