Load multiple excel files into Redshift

Highlighted
Nine Stars

Load multiple excel files into Redshift

Hi experts

 

Hope you are well.

 

I have got the scenario where I am looping in all 3 excel files stored locally and then loading them into Redshift.

 

I have designed my job like this:

 

Screen Shot 2019-12-18 at 1.59.32 pm.png

 

This is my setup in tfileinputexcel.

Screen Shot 2019-12-18 at 2.02.37 pm.png

It actually shows 2 exection finished but it actually loads only one file in redshift. Not sure what am I doing wrong.

 

Please help. 

 

Thanks in advance.

 

Thanks

Harshal.


Accepted Solutions
Nine Stars

Re: Load multiple excel files into Redshift

Hi Nikhil and all

 

Thanks a lot for your reply.

 

I ended up resolving this by using existing connection instead of defining as new.

 

Screen Shot 2019-12-19 at 9.45.48 am.png

 

Thanks

Harshal.

View solution in original post


All Replies
Highlighted
Community Manager

Re: Load multiple excel files into Redshift

Try to make a testing, add a tLogRow after tMap to the print the data on the console, to check if each file has data and all data are read from the file.

----------------------------------------------------------
Talend | Data Agility for Modern Business
Highlighted
Nine Stars

Re: Load multiple excel files into Redshift

Hi @shong

I added tlogrow and it works fine in there. Not sure what's the problem with tdbout component.
Highlighted
Community Manager

Re: Load multiple excel files into Redshift

Do you create the table on tRedshiftOutput? What is the option in Action on table list?

----------------------------------------------------------
Talend | Data Agility for Modern Business
Highlighted
Employee

Re: Load multiple excel files into Redshift

Hi,

 

    Could you please add a tRedshiftCommit to tRedshiftOutput using On Component Ok? Please do not close the connection since you need to repeat the iterations. You can close the connection in tPostJob section once the entire job is complete.

 

image.png

 

     At the same time, add Die on error option in tRedshiftOutput component to make sure that data is indeed getting loaded without any error.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved

Nine Stars

Re: Load multiple excel files into Redshift

Hi Nikhil and all

 

Thanks a lot for your reply.

 

I ended up resolving this by using existing connection instead of defining as new.

 

Screen Shot 2019-12-19 at 9.45.48 am.png

 

Thanks

Harshal.

View solution in original post

2019 GARTNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog