How to get CSV Files from S3 and transform to XML using mediation?

Highlighted
Five Stars

How to get CSV Files from S3 and transform to XML using mediation?

Hi

I am creating a route for the below scenario.

1. Connect to AWSS3

2. Download CSV files from an S3 bucket and save the file to the directory

3. Read the CSV file from a directory and transform CSV rows into XML

 

I tried to do the first two-point, but unfortunately, there is no component to close the connection. Also not sure on how to pass the CSV file to cTaldnJob for transformation.

 

Can anyone please help?

 

Thanks

Employee

Re: How to get CSV Files from S3 and transform to XML using mediation?

Hi,

 

   Below is a skeleton flow for the task.

image.png

 

If you are facing any issue, I would recommend to share the job flow screen shot along with error and problematic component screenshots for further analysis.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Five Stars

Re: How to get CSV Files from S3 and transform to XML using mediation?

Hello Nikhil,

 

Thank you for your response and sorry delay in my reply. I have managed to replicate the job and successfully retrieved the file. 

 

I am struck with the below scenario can you please shed some light on this?

When any CSV row failed the validation I do not want to process the CSV file further and push the entire input file to a different folder. I've tried but I am getting right, can you please?

 

 

S3.png

Employee

Re: How to get CSV Files from S3 and transform to XML using mediation?

Hi,

 

     You can easily do it by adding a flag variable. At the starting of the job, assign assign reject_flag="N"  using a tjava in Prejob section.

 

The next step is to verify whether any records are getting rejected by using tschemacompliancecheck. You can add a tjavarow  after that and assign value to a context variable assign reject_flag="Y" . Then use a if condition to verify whether the value is "Y" or "N".

 

    If the value is "Y" send the control to move the input file to reject area and if its "N" send the control to process the record further.

 

    I hope this will resolve your issue. Please spare a second to mark the topic as resolved since your original query has been answered.

 

Warm Regards,
Nikhil Thampi

Please appreciate our Talend community members by giving Kudos for sharing their time for your query. If your query is answered, please mark the topic as resolved :-)

Five Stars

Re: How to get CSV Files from S3 and transform to XML using mediation?

Hi Nikhil,

 

Thank you for your response. I will try the suggested approach and get back to you soon, please.

 

Thanks

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

An API-First Approach to Modernizing Applications

Learn how to use an API-First Approach to Modernize your Applications

Watch Now

Best Practices for Using Context Variables with Talend – Part 3

Read about some useful Context Variable ideas

Blog

Talend Studio Improvements for API Services

Take a look at the Talend Studio improvements for API Services

Watch Now