[resolved] Seeding tFileList a list of files from database

One Star

[resolved] Seeding tFileList a list of files from database

I am trying to seed tFileList a list of files that I pulled from a table using Talend java. How can I dynamically set this up? It looks like I have to manually add in the file names to tFileList. Can I feed in a csv file?
In example:
Grab file names from a database >> store it in a csv file >> feed it to tFileList

Accepted Solutions
Community Manager

Re: [resolved] Seeding tFileList a list of files from database

Hi
If you pull the file name (or file path) from database, not the file itself, you can iterate each file name directly without tFileList component. For example:
tMysqlInput--row1--tFlowToIterate---iterate-->the rest of job.
On the rest of job, you can access the current file name (or file path), use this expression:
(String)globalMap.get("row1.filename")
//filename is the column name;
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

View solution in original post


All Replies
One Star

Re: [resolved] Seeding tFileList a list of files from database

Hi
Just store the csv file into an empty directory. Then use tFileList to locate this directory and set Filemask "*".
Regards,
Pedro
One Star

Re: [resolved] Seeding tFileList a list of files from database

This doesnt actually process the raw data files. It only processes the csv file which holds the name of each file to be processed.
I am trying to feed tFileList a csv which contains targeted set of files for processing.
One Star

Re: [resolved] Seeding tFileList a list of files from database

give this a try:
tFileInput--row-->flowToIterate-->tFileInput---row--->(rest of job)
the first file input should contain your filenames. you retrieve them from the iterate with a call to the globalMap.
One Star

Re: [resolved] Seeding tFileList a list of files from database

Yeah this is a lot better solution.
Community Manager

Re: [resolved] Seeding tFileList a list of files from database

Hi
If you pull the file name (or file path) from database, not the file itself, you can iterate each file name directly without tFileList component. For example:
tMysqlInput--row1--tFlowToIterate---iterate-->the rest of job.
On the rest of job, you can access the current file name (or file path), use this expression:
(String)globalMap.get("row1.filename")
//filename is the column name;
Best regards
Shong
----------------------------------------------------------
Talend | Data Agility for Modern Business

View solution in original post

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog