merge data from multiple files into one

Six Stars

merge data from multiple files into one

Hi,
I have plenty of spreadsheets with same columns but different data (the old database extracted the data grouped in small fractions by separate files). I need to merge all those files into one only. Is there an easy way to do this? Many thanks for help
Five Stars

Re: merge data from multiple files into one

If you have same columns in all the files then Use tFileList component.
https://help.talend.com/search/all?query=tFileList&content-lang=en
Six Stars

Re: merge data from multiple files into one

Hi,
I thought it was working as suggested but I have actually just noticed that the tFileList component iterates through 12 folders where I have my files, but the output is taken from the very last folder only. What am I doing wrong? See attached screenshots to see the component settings, all files in subdirectories are named c5_contact. Many thanks for help.

Five Stars

Re: merge data from multiple files into one

You're iterating files, so make sure you append to your spreadsheet - "Append existing file"
Check out the other options to make sure you get what you're after.
Highlighted
Six Stars

Re: merge data from multiple files into one

Good point thanks but if I use 'Apend Existing File', the output is same.. and if I add 'Apend Existing Sheet' (dont really know what the difference is) I get the error attached.. Im not code oriented so I dont understand it.
Five Stars

Re: merge data from multiple files into one

How many rows are you attempting to put in this spreadsheet?
Six Stars

Re: merge data from multiple files into one

In this particular example I was trying with 12 files having around 20 000 rows in total, but Im going to actually need 2x more later. Thanks

Calling Talend Open Studio Users

The first 100 community members completing the Open Studio survey win a $10 gift voucher.

Start the survey

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog

How Media Organizations Achieved Success with Data Integration

Learn how media organizations have achieved success with Data Integration

Read

Agile Data lakes & Analytics

Accelerate your data lake projects with an agile approach

Watch