iterating files using tFileList only processes last file

Highlighted
Six Stars

iterating files using tFileList only processes last file

I have the following config:

 

tFileList > iterate > tFileInputPositional > Schemacheck > tMap > tFileOutputDelimited

 

There are 2 files in the file list. when running the job it states '2 execs finished' on the iterate connector. However, from tFileInputPositional, only the last file is processed.

 

What am I missing?

Fifteen Stars TRF
Fifteen Stars

Re: iterating files using tFileList only processes last file

Can you share tFileInputPositional and tFileInputPositional settings?

TRF
Six Stars

Re: iterating files using tFileList only processes last file

 

Like this?

Screenshot_20191002_135016.png

 

 

Screenshot_20191002_134756.png

Fifteen Stars TRF
Fifteen Stars

Re: iterating files using tFileList only processes last file

Nothing special in the settings.
Are you sure both files have the same schema?

TRF
Six Stars

Re: iterating files using tFileList only processes last file

I found the issue(s):

 

- The row count after the tFileInputxx is the rowcount of the last file, not the total

- I had been playing with the 'append' option of the tFileOutputxxx. It was greyed out for me and said that the property was managed by the Repository (where I could not find that option at ll). When I switched the Property Type from Repository to Built-In *and back*, now I could select Append, and I find all my records in the outputfile.

 

Thanks for the (moral) support.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog