How to trigger different components based on data in Redshift table ?

Highlighted
Five Stars

How to trigger different components based on data in Redshift table ?

I have a table in redshift with a few columns. The data in column A defines the execution order.

If A = Red then I need to write the output to a file. If A = Yellow then I need to load the data into another table.

 

How do I implement this requirement ?

Which component can I use which sends the action to the required path according to the data in the table?


Accepted Solutions
Community Manager

Re: How to trigger different components based on data in Redshift table ?

OK. Then all you need to do is connect your database component (or whatever brings your data in) to a tFlowToIterate component. This will, for every row, create globalMap variables for every column. So if you have a row with columns called "col1", "col2" and "col3" and your row feeding the tFlowToIterate is called "row1", your globalMap variables will be available using the following code.....

 

Assuming "col1" is a String

((String)globalMap.get("row1.col1"))

Assuming "col2" is an Integer

((Integer)globalMap.get("row1.col2"))

Assuming "col3" is a Long

((Long)globalMap.get("row1.col1"))

 

Now, connect your tFlowToIterate to a tJava using an iterate link. Then from the tJava use RunIf links and use the globalMap variables as the values for the IF condition logic. The RunIfs will be used to point to the components that you want run conditionally.

View solution in original post


All Replies
Community Manager

Re: How to trigger different components based on data in Redshift table ?

You can use a tMap for this. You can have an output for each scenario and use the filtering functionality of the tMap. This page here might help.....

https://help.talend.com/reader/QgrwjIQJDI2TJ1pa2caRQA/rvIDyyz_tAaUI42YybAZ6w

Five Stars

Re: How to trigger different components based on data in Redshift table ?

tMap will help me filter the data and send different streams of the data.

I don't want to filter out and separate the data and then use it.

I just want to trigger different components based upon the content of the
data. I don't need to carry forward the data.

For example, if a certain value exists in the table. I need to send an
email. Else I need to write generate a file. Else ... Else...
And similarity more conditions and their subsequent actions.

Can this be achieved through tMap ?
Community Manager

Re: How to trigger different components based on data in Redshift table ?

OK. Then all you need to do is connect your database component (or whatever brings your data in) to a tFlowToIterate component. This will, for every row, create globalMap variables for every column. So if you have a row with columns called "col1", "col2" and "col3" and your row feeding the tFlowToIterate is called "row1", your globalMap variables will be available using the following code.....

 

Assuming "col1" is a String

((String)globalMap.get("row1.col1"))

Assuming "col2" is an Integer

((Integer)globalMap.get("row1.col2"))

Assuming "col3" is a Long

((Long)globalMap.get("row1.col1"))

 

Now, connect your tFlowToIterate to a tJava using an iterate link. Then from the tJava use RunIf links and use the globalMap variables as the values for the IF condition logic. The RunIfs will be used to point to the components that you want run conditionally.

View solution in original post

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 2

Part 2 of a series on Context Variables

Blog

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog