How to join data from multiple tExtractJson ?

Five Stars

How to join data from multiple tExtractJson ?

I have 2 tExtractJson and want to join both result set together before importing into a database.     

 

I tried this method:

tExtractJson1   _____>>

                                           tMap  (  trying to join both tExtractJson output )

tExtractJson2   -------->> 

 

but it doesn't allow becasue tExtractJson is not an input datasource.  Any suggestion/workaround ?  Thanks.

 

 

 


Accepted Solutions
Forteen Stars TRF
Forteen Stars

Re: How to join data from multiple tExtractJson ?

It seems both tExtractJSONFields are issued from the same input (tXMLMap?)
If so, you cannot join them in the same subjob where they have been created.
Store the result of each in a dedicated tHashOutput then on the next subjob get the rows from the corresponding tHashInput and make join in a tMap.

TRF
Thirteen Stars

Re: How to join data from multiple tExtractJson ?


TRF wrote:
It seems both tExtractJSONFields are issued from the same input (tXMLMap?)
If so, you cannot join them in the same subjob where they have been created.
Store the result of each in a dedicated tHashOutput then on the next subjob get the rows from the corresponding tHashInput and make join in a tMap.

as variant also - store to csv for avoid "out of memory" problems with huge json

-----------

All Replies
Forteen Stars TRF
Forteen Stars

Re: How to join data from multiple tExtractJson ?

Have look to tUnite if both tExtractJSONFields have the same schema.
If they don't, rearrange them using tMap or tFilterColumn.

TRF
Five Stars

Re: How to join data from multiple tExtractJson ?

I should have be more clear.   What I want to do is not to union the data.  I wanted to do a left join from tExtractJson1 to tExtractJson2 on the id field.   Is there any way ?

 

I tried using tMap to join but it doesn't allow because tExtractJson is not an input source. 

 

Thanks.

Forteen Stars TRF
Forteen Stars

Re: How to join data from multiple tExtractJson ?

Are you speaking about tExtractJSONFields?
Can you share your job design?

TRF
Five Stars

Re: How to join data from multiple tExtractJson ?

Yes.  I am trying to do something like that setting multiple tExtractJsonFields as an input to tMap so I can do a left join from 1sr tExtractJsonFields to the 2nd tExtractJsonFields.    Is there a workaround ?  Thanks.

 

 

Capture.JPG

Forteen Stars TRF
Forteen Stars

Re: How to join data from multiple tExtractJson ?

It seems both tExtractJSONFields are issued from the same input (tXMLMap?)
If so, you cannot join them in the same subjob where they have been created.
Store the result of each in a dedicated tHashOutput then on the next subjob get the rows from the corresponding tHashInput and make join in a tMap.

TRF
Forteen Stars TRF
Forteen Stars

Re: How to join data from multiple tExtractJson ?

Thirteen Stars

Re: How to join data from multiple tExtractJson ?


TRF wrote:
It seems both tExtractJSONFields are issued from the same input (tXMLMap?)
If so, you cannot join them in the same subjob where they have been created.
Store the result of each in a dedicated tHashOutput then on the next subjob get the rows from the corresponding tHashInput and make join in a tMap.

as variant also - store to csv for avoid "out of memory" problems with huge json

-----------
Five Stars

Re: How to join data from multiple tExtractJson ?

 

 

I tried using tHashOut / tHashInput, but somehow still wouldn't allow me to connect, so I used the file method.  If I have to export a HUGE file and then read the file right after,  will the "SubJobOK" step make sure that the file is "done" exporting before the process reads it ?   Below is a sample design, please feel free to comment if there is a better way of doing things.  Thanks.

 

 

 

Capture.JPG