Seven Stars

SFTP multiple files using tFTPPut

Trying to build a solution to transfer files to an FTP server from local on-premise system. Currently the process is only transferring the last file of the lot found from the directory. Tried debugging the job but couldn't understand where the actual problem is. 

What I have tried so far:

Used tFileList to get the list of files for a given directory, given "*.txt" as FileMask.

Used tIterateToFlow to iterate through the list of files. The Schema is of a single column "fileName" with the value set as - ((String)globalMap.get("tFileList_1_CURRENT_FILE"))

Finally used tFTPPut to SFTP the files to remote directory. Local Directory is set as - ((String)globalMap.get("tFileList_1_CURRENT_FILEDIRECTORY")) & FileMask as - ((String)globalMap.get("tFileList_1_CURRENT_FILE")). Overwrite is the option chosen.

Screenshot of the job -

tFTP-1.JPG

Thanks in advance for any kind of help and let me know if more information is required for this.

Platform: Talend Open Studio for Data Integration Version: 6.5.1

2 ACCEPTED SOLUTIONS

Accepted Solutions
Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishekUntitled.png

Manohar B
Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishek,since you need to design job as i shared in screenshot, other wise in your design first file will be placed and for second file tFTPPUT will fail that connection alreday closed.that you have used on component ok of tFTPPUt to tFTPCLOSE

Manohar B
11 REPLIES
Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishek,you no need to use tIteratetoFlow,since you can directly connect Iterate flow from tFilelist to tFTPPut and in between you cna use tJava componet to print the current file name.

Manohar B
Seven Stars

Re: SFTP multiple files using tFTPPut

@manodwhb Have tried that too. If we don't use tIterateFlow and directly pass the iteration from tFileList to tFTPPut then only the first file would be retrieved and sent and for the second file there would be en error - "No File Present".

This is the output when debugged using tJava - 

[statistics] connecting to socket on port 3583
[statistics] connected
[trace] connecting to socket on port 5031
[trace] connected
Output.txt
C:/FTP/dir
Sample.txt
C:/FTP/dir
No File Present
[statistics] disconnected
[trace] disconnected

 

Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishek,i have used below way and working for me.check your desing based on this.

 

Untitled.pngUntitled.png

Manohar B
Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishek,below settings also working as expected.Untitled.png

Manohar B
Seven Stars

Re: SFTP multiple files using tFTPPut

@manodwhb Thank you for the solution. 

Noticed that you have used "*.txt" as FileMask in the tFTPPut which ensures that it will pull all the files from the CURRENT_FILEDIRECTORY. In that case we don't even need to iterate the files in the directory and we don't need tFileList too.

But the desired approach should have been like get the files using tFileList and iterate over the list and using tFTPPut transfer the files. The FileMask should be used in tFileList and whatever the filename is passed in that iteration tFTPPut should use that preferably using - CURRENT_FILE  

Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishek,you can find my other sceernshot i have used ((String)globalMap.get("tFileList_1_CURRENT_FILE")) as file mask in tFTPPut and working fine.

Manohar B
Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishekUntitled.png

Manohar B
Seven Stars

Re: SFTP multiple files using tFTPPut

Thank you @manodwhb, sorry I might have overlooked your reply.

Yes, your solution is working perfectly for the list of files in local directory much appreciated.

One thing I noticed here is and the reason for my process not working was -

I was using tFTPPut --> (on component ok) --> tFTPClose , wherein you have used

tFileList --> (on subjob ok) --> tFTPClose which has clicked. 

 

Any idea on this.

Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishek,since you need to design job as i shared in screenshot, other wise in your design first file will be placed and for second file tFTPPUT will fail that connection alreday closed.that you have used on component ok of tFTPPUt to tFTPCLOSE

Manohar B
Twelve Stars

Re: SFTP multiple files using tFTPPut

@iamabhishek,please accept solution.

Manohar B
Seven Stars

Re: SFTP multiple files using tFTPPut

Thanks for the explanation, makes sense now.