Pass schema column from input to output unchanged

Five Stars

Pass schema column from input to output unchanged

Hi all

 

Cannot figured out how to accomplish simple thing.

 

I have tWriteJSONfield component and input row connection that contains data that need for JSON generation as well as data that need for nex components. So I need just to transfer some column ("document_id") to output schema of JSON component. I defined outout column with same name as input column that I need to send forward. But it column becomes null.

 

How can I achieve this?

talend1.JPGtalend2.JPG

 


Accepted Solutions
Community Manager

Re: Pass schema column from input to output unchanged

Did this resolve your question?


All Replies
Community Manager

Re: Pass schema column from input to output unchanged

I'm not entirely sure what you are trying do here, but I suspect you may get further by having the Document_Id field first in your output. The order of your columns is important when passing data through components.

Five Stars

Re: Pass schema column from input to output unchanged

Unfortunately  didnt help...

I trying to do simple thing... I have 3 components in job:

1st - some REST call that return me a JSON with two values, say {a:"aaa", b:"bbb"}

2nd - tWriteJSONfield, I need value of "a" for it

3rd - some other component, I need value of "b" for it and also output of tWriteJSONfield

 

So I need somehow deliver "a" to 3rd component. But it becomes Null in output row of the 2nd.

 

 

Five Stars

Re: Pass schema column from input to output unchanged

> So I need somehow deliver "a" to 3rd component. But it becomes Null in output row of the 2nd.

Sorry, deliver "b"
Community Manager

Re: Pass schema column from input to output unchanged

Ah, I think you want to use a tExtractJsonFields component for that (https://help.talend.com/reader/g8zdjVE7fWNUh3u4ztO6Dw/Je6AV_TMnYHC_MpEtaMBaA)

Highlighted
Five Stars

Re: Pass schema column from input to output unchanged

I use tExtractJsonField component, between 1st and 2nd (forgot to mention it but it does not actually matter for my question). Ok, I have

 

1- some REST call that return me a JSON with two values, say {a:"aaa", b:"bbb"}

1a - tExtractJsonField to extract a and b and send it to next component by row connection

2 - tWriteJSONfield, I need value of "a" for it (i.e. I need to form some new JSON,  {a1 : a + "blablabla"})

3 - some other component, I need value of "b" for it and also output of tWriteJSONfield (i.e. I need to write this new JSON to a file with name b)

 

And the question still - how to deliver "b" to 3rd component

Community Manager

Re: Pass schema column from input to output unchanged

You seem to need to use the "b" value in multiple places. As this is the case, you may want to try this....

 

Connect a component which returns all of the values you wish to use to a tFlowToIterate component. This will store all of the column values in an object called the globalMap. This is essentially a HashMap. The "key" to these values will be the name of the row feeding the data (automatically something like "row1") and the column name. Now you should connect the tFlowToIterate to a tFixedFlowInput using an Iterate link. What this does is ensure that the rest of the subjob is processed before the next row is started. 

Now that you have put the above into your job, you can make use of your "b" value wherever you want in the job, using the below code (assuming the row is "row1" and the column name is "bValue"...and it is a String).....

 

((String)globalMap.get("row1.bValue"))

That can be used in any component needing the value of "b". You need to use the above code (for your different columns) when outputting your row using the tFixedFlowInput  component.

Five Stars

Re: Pass schema column from input to output unchanged

Thank rhall, it should work but make job much complicated. And one question, what should I provide in the tFixedFlowInput in Mode table? Or it doesnt matter in this case?

Community Manager

Re: Pass schema column from input to output unchanged

Since you are iterating the tFixedFlowInput component, you will only be triggering 1 row per iteration. As such, use the Single table option.

Community Manager

Re: Pass schema column from input to output unchanged

Did this resolve your question?

Five Stars

Re: Pass schema column from input to output unchanged

But while there is a workaround iI still think that this is an issue in tWriteJSONField component. And it should be fixed. From my point the fix is very easy, just need to update output scheme with values from the same columns of input scheme.

2019 GARNER MAGIC QUADRANT FOR DATA INTEGRATION TOOL

Talend named a Leader.

Get your copy

OPEN STUDIO FOR DATA INTEGRATION

Kickstart your first data integration and ETL projects.

Download now

What’s New for Talend Summer ’19

Watch the recorded webinar!

Watch Now

Best Practices for Using Context Variables with Talend – Part 1

Learn how to do cool things with Context Variables

Blog

Migrate Data from one Database to another with one Job using the Dynamic Schema

Find out how to migrate from one database to another using the Dynamic schema

Blog

Best Practices for Using Context Variables with Talend – Part 4

Pick up some tips and tricks with Context Variables

Blog