FTP problem. Getting 0KB files

Highlighted
Six Stars

FTP problem. Getting 0KB files

I have a simple job to get XML files from a sFTP server to a local directory. My job is working perfectly but sometimes im getting empty files and i cant figure out whats the problem. The job is something like tFTPConnection -onComponentOK-> tFTPFileList -iterate-> tFTPGet. I close the connection with an onSubjobOk from the tFTPFileList. In my Connection i have to use the SFTP Support.

 

Did somebody have the same problem or know how this is happening?

Twelve Stars

Re: FTP problem. Getting 0KB files

your file may be empty when you get it for the first time! (it happen when you create and fill file into the ftp folder).
Solution 1 is to have a ready flag file who is add to ftp folder when file is generated.
Solution 2 is to download file 2 time (with delay) to check than the size is the same.

Regards

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Six Stars

Re: FTP problem. Getting 0KB files

Can you explain a little bit more the first solution? I dont really know if this will help me but just in case.

The 2nd one is not possible for me because when the i download the file it gets deleted from the SFTP server so cant download it again. The thing is that the file in the server has size (5KB) but when i get it to my local directory, the file is empty (0KB).

 

Thanks so much for your reply.

Twelve Stars

Re: FTP problem. Getting 0KB files

when you build a file: you create the address into the allocation table then you fill datas int the file.
sometime when file is created on an ftp folder, you can download file between this steps so you download an empty file.
the best way is to create file in a work folder, then copy it into ftp folder.

Solution 1: ask the file builder to add a semaphore. it's an empty file who say file is ready to be download. when the data file is completely present into ftp folder create an file "nameOfDataFile.ok".

Solution 2: is used when you cannot add semaphore. it's a simple wait (how many time do you need to fill this file?).

good luck.

Francois Denis

Tag as "solved" for others! Kudos to thanks!

Six Stars

Re: FTP problem. Getting 0KB files

I just need to fill each file one time. So if i add a wait when im doing the tFTPGet will give the server time to fill the file completly? 

 

 

Twelve Stars

Re: FTP problem. Getting 0KB files

a wait is just a delay. You can get list of file to download after the download job. if you run your job each 5 min your delay will be 5 min…

get the list --- wait --- download ------get the list --- wait --- download ------…..
regards,

Francois Denis

Tag as "solved" for others! Kudos to thanks!

What’s New for Talend Spring ’19

Watch the recorded webinar!

Watch Now

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download

Tutorial

Introduction to Talend Open Studio for Data Integration.

Watch

Downloads and Trials

Test drive Talend's enterprise products.

Downloads