Is it possible to use the output from tFTPGet to feed tFileInputDelimited?

Four Stars

Is it possible to use the output from tFTPGet to feed tFileInputDelimited?

Greetings:

 

I have a process that needs to download a file from an FTP server. Parse it and add the rows into a PostgreSQL database table.

 

Currently, my job looks like this:

tFTPGet_1 - OnSubjobOk -> tFileList_1 - iterate -> tFileInputDelimited_1 - row 1 (Main) -> tMap_1 - row 1 (Main) -> tpostgresqlOutput1Screen Shot 2018-05-25 at 4.22.13 PM.pngft

tFTPGet_1 selects a file (or files) from a remove folder using a mask: "cmgcfscommfileout-*.csv" there may be more than one (but there is usually only one and we don't the the detail for the glob '*' for certian; It's a date but it might be any date). It then downloads the file into a specific folder "Downloads/Commissions/CUNA/CFS".

 

Currently, tFIleList_1 gets any files named "*.csv" in "Downloads/Commissions/CUNA/CFS". While this works it's not really how I'd like to do it. I'd love to send the output of tFTPGet_1 into tFileDelimited_1 directly but I don't see any global variables that I could use like in tFileList_1 (I'm using ((String)globalMap.get("tFileList_1_CURRENT_FILEPATH")) to extract the filename in tFileDelimited_1 from tFileList_1. But as I said I don't see any output globals that I could use for tFTPGet_1.

 

Is there any way to do this?

 

Any help is gratefully appreciated.

 

Paul Carrick

Forteen Stars

Re: Is it possible to use the output from tFTPGet to feed tFileInputDelimited?

as You describe - it not supported

as I understand - main idea proceed each file sequently, without waiting while all downloaded

If Your server support not only FTP/SFTP, You can use tSCPFileList for list remote folder, and then use tSCPGet (or continue tFTPGet with file mask from variable)

-----------
Four Stars

Re: Is it possible to use the output from tFTPGet to feed tFileInputDelimited?

Thanks vapuok that is what I was afraid of. I'll try your solution.

15TH OCTOBER, COUNTY HALL, LONDON

Join us at the Community Lounge.

Register Now

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Definitive Guide to Data Quality

Create systems and workflow to manage clean data ingestion and data transformation.

Download